吃瓜大本营

In today鈥檚 world, computerized algorithms are everywhere: They can decide whether you get a , how much you access, and what you see. And, increasingly, it鈥檚 not just private companies that use algorithms. The , is turning to proprietary algorithms to make profound decisions about your life, from what level of health benefits you receive to whether or not you get .

This isn鈥檛 necessarily good or bad. At their core, 鈥渁lgorithms鈥� are just instructions, like a recipe or user manual, that use raw inputs to determine outcomes in all kinds of decision making. But it becomes a serious problem when the government keeps those algorithms 鈥� including the source code that executes the programs and the raw data that constitutes their inputs 鈥� secret from the public.

And that鈥檚 exactly what is happening in criminal trials around the country.

Take, for example, the case of Billy Ray Johnson, who was sentenced to life in prison without parole for a series of burglaries and sexual assaults he says he did not commit, largely based on the results of a proprietary algorithm called TrueAllele. TrueAllele claims to identify the perpetrator of a crime from a tiny, degraded DNA sample swimming in a larger soup of multiple individuals鈥� DNA. It鈥檚 an experimental technology, not at all like the DNA tests that have developed over the past two decades, which also have serious flaws. At Mr. Johnson鈥檚 trial, the court denied the defense team access to TrueAllele鈥檚 source code 鈥� information crucial to the defense case 鈥� all because the company that owns it cried, 鈥淭rade secret!鈥�

As we explained in an amicus brief we filed in the case on Wednesday, this is unconstitutional in a number of ways. Our Constitution gives a defendant the right to confront the witnesses against him, and it provides him with the right to a fundamentally fair trial that includes a meaningful opportunity to present a complete defense. It also gives the public a right of access to criminal proceedings, including evidence, so that we can serve as a check upon the judicial process.

Access to the source code of algorithms used in the criminal justice system is critical to ensure fairness and justice. Algorithms are human constructs that are , which can plague them throughout their design and use. For example, at the building stage, something as simple as a misplaced ampersand can have profound implications. A coding error in another DNA algorithm was recently found to have produced in Australia, altering its reported statistics by a factor of 10 and forcing prosecutors to replace 24 expert statements.

Beyond random mistakes, people hold that can materially affect the variables they include in an algorithm, as well as how they interpret the results. Racial bias also often creeps into algorithms, both because the underlying data reflects existing racial disparities and because inaccurate results for smaller minority groups may be hidden in overall results.

And, of course, there鈥檚 the possibility that financial incentives will pervert the goals of companies that build these algorithms. In the context of DNA typing, the prosecution, backed by the substantial resources of the state, is a company鈥檚 most likely customer 鈥� and that customer is likely to be most satisfied with an algorithm that delivers a match. So companies may build programs to skew toward matches over the truth.

In Mr. Johnson鈥檚 case, the trial court decided to ignore these potential pitfalls 鈥� and, more significantly, the defendant鈥檚 constitutional rights 鈥� ruling in favor of TrueAllele鈥檚 argument for secrecy. This is legally wrong and has troubling practical implications. Research shows that juries put too much trust in uncontested algorithms. Prosecutors and their expert witnesses present their results as infallible truth, which go 鈥�.鈥� And juries, when given no other option, generally do not question them.

But the results need to be questioned, and this case demonstrates why.

TrueAllele鈥檚 parent company, Cybergenetics, and a government lab that bought the algorithm to run in-house got wildly different results 鈥� both from themselves on different test runs and from each other overall. Indeed, TrueAlelle鈥檚 creator testified that he expected the government鈥檚 results, generated by running the same data through the same program, to be 鈥渨ithin two zeros,鈥� or a magnitude of 100, of his results. Yet even though he expected a significant discrepancy, he was able to offer his results as unquestioned evidence. All while the defense was given no meaningful opportunity to challenge his testimony.

Access to similar DNA algorithms has revealed serious errors in them. Much like the example from Australia, a recent case in New York revealed that another DNA algorithm 鈥� from its calculations, in ways . . . that could unpredictably affect the likelihood assigned to the defendant鈥檚 DNA being in the mixture.鈥� This was only discovered after the trial court correctly ordered that the algorithmic source code be disclosed to the defense, prompting the prosecution to withdraw the evidence. Yet courts continue to admit the results of other DNA algorithms, like TrueAllele, without disclosure to the defense or the public.

This isn鈥檛 the first time we鈥檝e been down this road with technology in criminal courts. There is a long history of junk science being used under the guise of technological advance. Public access to such evidence was a prerequisite to establishing its invalidity.

In the 1990s, 鈥渁 series of high-profile legal challenges鈥� and 鈥� of forensic evidence鈥� caused various long-standing methods 鈥� from bite-mark analysis to ballistics testing and from fingerprinting to microscopic-hair-comparison 鈥� to get 鈥渄eflated or outright debunked.鈥� Similarly, after a New Yorker exposed a flawed case based on arson science, the state responsible not only 鈥渞econsider[ed] old cases that had been by the original investigators,鈥� but also 鈥渞einvented itself as a leader in arson science and investigation.鈥�

Scientific errors in the criminal justice system are a serious problem. But the examples above also reveal the power of adversarial testing and public scrutiny to correct those errors and create better science.

We hope the California appellate court agrees with us and orders disclosure of the algorithmic source code. An adversarial testing process is crucial to ensure that Mr. Johnson鈥檚 constitutional rights are enforced.