Beyond Reasonable Doubt? - Richard Dawkins [link]
A new article looking at the jury system rationally and scientifically.
Excerpt:
Courtroom dramas accurately portray the suspense that hangs in the air when the jury returns and delivers its verdict. All, including the lawyers on both sides and the judge, are on tenterhooks and hold their breath while they wait to hear the foreman of the jury pronounce the words, “Guilty” or “Not guilty”. However, if the phrase “beyond reasonable doubt” means what it says, there should be no doubt of the outcome in the mind of anybody who has sat through the same trial as the jury. That includes the judge who, as soon as the jury has delivered its verdict, is prepared to give the order for execution — or release the prisoner without a stain on his character.
And yet, before the jury returned, there was enough “reasonable doubt” in that same judge’s mind to keep him on tenterhooks waiting for the verdict.
You cannot have it both ways. Either the verdict is beyond reasonable doubt, in which case there should be no suspense while the jury is out. Or there is real, nail-biting suspense, in which case you cannot claim that the case has been proved “beyond reasonable doubt”.
This really struck me as something that could have been on LW’s front page.
There’s a mathematical law about this. If you split something into groups, there will always be a dividing line. Move it within epsilon, and it crosses the line.
Whether or not they’re guilty may be beyond resonable doubt without whether or not its beyond resonable doubt being beyond reasonable doubt.
For example, if we define “reasonable doubt” to be < 99% chance of guilt, then if you think there’s a 99% chance of them being guilty, you’re pretty sure they’re guilty, but there’s about a 50% chance of them being convicted, based on whether the jury considers it slightly more likely or slightly less likely.
It seems that the reasonable assessment of P(guilty) should only rarely fall so close to the cut-off line that there could be serious doubt about the jury’s verdict. So if the suspense is common, that still demonstrates that the assessments of probability held by different case participants are all over the place.
It only has to be different enough that there’s a significant chance of one person saying it isn’t likely enough. Given how bad people are with extreme values of probability, this wouldn’t be that surprising.
Also, nobody has ever said where the cutoff is. Two jury members could both think there’s a 97% chance of guilt, and one thinks the cutoff is 95%, while the other thinks it’s 99%, and they’ll disagree on whether the defendant should be considered innocent or guilty.
You are right. The suspense isn’t about the probability, but about the probability times value. Where the value is the sentence or the pardon. Something big is at stake here.
When the maximal sentence is $100, then there is no big suspense no matter how far beyond reasonable doubt the case is.
This is such an important counter-point that I am disappointed that Dawkins failed to see it and that none of the posts (at least on the first page) of the original article mention it. On the plus side, this gives me evidence (although slight, since there is selection bias) that LW can go beyond traditional rationalist movements.
Why is it a counterpoint? What (implicit) conclusion made by Dawkins it contradicts?
Dawkins starts from the premise that there is high uncertainty about the outcome of the case, and concludes that there is high uncertainty about the guilt, which does not follow. Even if it is obvious to everyone that the defendant is very probably guilty, it may be far from obvious exactly how high the jury will estimate the probability of innocence, and where they will set the bar for reasonable doubt.*
*It has never been clear to me where this should be. If I put the credence of guilt at g, should I convict when g>.9? .99? .999? Should I say “to hell with the idea of reasonable doubt anyways, I’m going to estimate myself appropriate relative weights to attach to the outcomes 1) innocent man spends a lifetime in prison, and 2) serial murderer is unleashed upon the public?” I suppose because the lawyers and judge are unlikely to provide me with a credence threshold to use, the most sensible thing to do would be to derive one myself.
(This sort of renormalization problem shows up a lot when trying to set up decision problems where baselines are unknown.)
Even if the judge is sure beyond a reasonable doubt that Jane Doe committed the murder, the judge is not sure beyond a reasonable doubt that the jury will return a guilty verdict.
I love Dawkins’ pop-sci works on biology and evolution, but like a lot of very smart and articulate people, he can be lazy when he goes outside the realm of his expertise. He uses “courtroom dramas” as evidence of what happens in real-life courtrooms. I wonder if he thinks Hollywood’s portrayal of what scientists do is similarly accurate? Later, he relies on the O.J. Simpson criminal trial, which is famous partly because it was so incredibly weird and wildly outside the mainstream. He says the professionals in the courtroom are “on tenterhooks and hold their breath” when it’s time for the verdict. Are they? My impression is that in the overwhelming majority of cases, experienced trial lawyers are not surprised by juries very often.
In any case, judges and juries see a trial from very, very different perspectives. Judges see defendants in chains and jail uniforms, juries see the defendant in a borrowed business suit. Judges know the when a defendant has tentatively agreed to a plea and then pulled out at the last minute. Moreover, it is the job of a trial judge to make sure that juries do not always see all the evidence. A defendant may have made a full confession and led the cops to the secret spot where the body was hidden, but if the police induced this confession by unconstitutional means, it can’t go before the jury. The list of legal reasons to exclude evidence from a jury is very long. Many, many times, judges preside over criminal trials with the full knowledge that the defendant did in fact do what he (or she) was accused of. However, the trial goes forward, because one major purpose of the trial and the jury system is not just to determine the facts accurately, but to restrain the police and the government from becoming dangerous.
After each side in a criminal case rests, a judge will instruct a jury of laypeople (mostly not mathematicians and philosophers) that the defendant must be found not guilty unless the prosecution has proved its case beyond a reasonable doubt. Dawkins picks on the phrase “beyond a reasonable doubt.” I would agree with him that this is a very vague and un-mathematical phrase. But he misses the point that the really important words are “unless the prosecution has proved its case.”
A lot of people focus on the fact that a judge can disallow evidence provided by the prosecution… but they seem unaware that judges can, and do, also disallow evidence provided by the defense. It is an unfortunate fact of our trial system in the U.S. that judges have a great deal of power to influence the verdict to produce the outcome they want, by what evidence they allow and disallow.
In A Devil’s Chaplain, page 40, Dawkins mentions “the three juries that it has been my misfortune to serve on”.
He mentioned that in the linked article as well. Good for him for performing his civic duty.
I hate to brag but, in high school I not only took chemistry, but biology as well. With my own hands, I dissected not only a worm, and a frog, but even a fetal pig. If memory serves, my final grade was no less than an A- in either semester of a full year course.
With that in mind, I am happy to praise my colleague Richard Dawkins as an accomplished biologist. And yet, I’m afraid this article is not his best work.
The prosecution are only going to bring a case if they think they can get a conviction, i.e. prove it beyond the jury’s reasonable doubt. In effect, the prosecution are another jury, but one subject to several biases compared with the real jury:
self-serving assessment of how good their case is
illusion of transparency
priming by evidence they have which for one reason or another cannot be used in court
regression to the mean.
So in general we should expect jurors to have a lower estimate of the prosecution’s case than the prosecution themselves do.
In addition, cases only go to trial that are contestible—where the prosecution thinks it can be proved beyond reasonable doubt but the defence does not. When the case is overwhelming and the accused is actually guilty, the accused is more likely to plead guilty without a trial. Jurors will never see those cases, only those that the accused finds worth contesting.
So it is not surprising that in a substantial proportion of cases, nobody can be sure of the jury’s verdict until it is announced.
Or in cases where the defendant is, you know, actually innocent. Some people find that they can’t stomach the idea of pleading guilty to a crime they didn’t commit, whether or not they can prevail in court.
And cases when the penalty after going to trial is likely to be the same as the penalty after a guilty plea. If the sentence is mandatory life, might as well roll the die and hope you get the jury that doesn’t believe confessions, DNA, ballistics, or eyewitnesses.
And it’s worth noting that the fraction of cases that go to trial is quite small. The best statistics I could find was the table in EXAMINING THE WORK OF STATE COURTS, 1998 page 72. Out of all criminal cases, the prosecutor or court dismisses 18.4%, the accused pleads guilty in 63.4%, and only the middle 3.7% go to trial. (There is also a mysterious 14.5% “other” category...).
Of course, these percentages do not directly reflect the state of the evidence: because of plea bargaining there is an expected utility calculation involved as well.
With regards to the comments here, a great deal of personal belief in morality of the world you live in, the fairness of the world, etc. relies on belief in good justice system, and those beliefs force you to believe justice system works better than it is and construct elaborate explanations of how it may be working well. It is not because the case is on dividing line between say 99% certainty, that you don’t know what decision will court decide. The average people don’t do probabilities that well as to estimate their uncertainty like that! We all know this.
Imagine you were dealing with a panel of chimpanzees or something else that’s obviously dumber than you, deciding on the verdict while being demonstrably unable to comprehend much of the basic logic*, so that both sides HAVE to appeal to fallacies, and then sit nervous unknowing which fallacies latched onto the minds of jury better. The right side in principle could stick to logic alone but in any remotely complex case that is likely to fail.
*it suffices that average member of the jury has measured IQ of 100, with failure rate on the logic section of IQ test such that it implies inability to do much of even most basic reasoning. Nevermind probabilities. They can’t do probabilities. Even for something as trivial as monty hall.
And you know what’s worst? You can make a jury of PhDs, and the jury is still barely intelligent (even if more intelligent than average). Homo sapiens is barely intelligent, that’s how things work—we evolved to be barely intelligent, at which point very rapid technological progress happened, and that was (in biological terms) very recently, and we are still barely intelligent.
Even worse still, there could be an utterly rational, effective agency at work as well. Selfish one. This agency determines that 1: thought-time expenditure on the process is not worth it (which leads to apparent stupidity well beyond one’s stupidity on a test that might influence one’s personal future), 2: if there is even 1% chance that the person is guilty of a major crime that can affect you, within even a large city, the personally rational, selfish action for you is to convict as the personal cost of convicting innocent is much smaller than personal cost of setting free the guilty (the social cost is however other way around). Even the perfect rational intelligence needs not arrive at a good judgement, nor to carefully listen and follow the argumentation, if that rational intelligence is selfish and optimizing it’s use of computing time.
One should do scientific study on this. Do some experimental trials of the trial system, see if they can decide on correctness of proofs of theorems—two mathematicians represented by 2 barristers, one premise to a simple theorem, one mathematician trying to prove some fallacy, other trying to prove correct answer.
For me, lack of routine testing of the trial system in this fashion is evidence enough that it is probably not very effective—nobody tried to see what in practice does ‘reasonable doubt’ equate to (some juries may think its unreasonable to set free someone when there’s doubts as to their innocence, some juries may get manipulated into setting free a certain murderer). It may be the best that we can do, but still be very bad.
Mind if I use that idea for my Research Methods class?
I would prefer professional jurors now that transparency can be maintained via technological means. These would be people who give up their right to privacy and train in weighing evidence correctly in exchange for high social status and good pay.
Almost like Supreme Court Justices only jurors. Only the would be numerous enough to be rather more likened to congressmen or senators, and the impartiality of those two groups is no clear matter.