Strange thing about this is, if I’ve calculated it right, the average probability estimate of Guede’s guilt is only ~87%. It seems to me that if this were your real probability estimate of his guilt, and you were on the jury at the guy’s trial, you would be obligated to vote innocent. If you operate on the basis that a 13% chance of innocence is not a reasonable doubt, about thirteen out of every hundred people who go to jail will be innocent. That is (let me check) more than one in ten, which strikes me as rather a lot. I think my own estimate of Guede’s guilt is above 99%, so I would vote guilty, but I’m surprised the average here is so low.
If you operate on the basis that a 13% chance of innocence is not a reasonable doubt, about thirteen out of every hundred people who go to jail will be innocent.
That’s if everyone who went to jail had a 13% chance of innocence. Presumably much of the time it would be lower.
Not 13 out of every 100 people in jail, but still 13 out of every 100 people sentenced by the jury as guilty in the case of a probability estimate of only ~87%. ….The argument still works to show that the probability of guilt at 87% is too low to vote guilty.
Which argument? I meant the argument loosely defined as the one where you count which fraction of innocent people are jailed to determine if the probability of guilt at 87% is appropriate. Steven0641 correctly pointed out that the target space for the fraction isn’t all people in jail, but then you modify the target space to all people judged guilty with probability 87% and the argument ‘works’.
The argument works if adding a 13% innocent population to jail is clearly wrong even though sending an individual with 13% probability of innocence to jail is not clearly wrong. Peter’s point, I think, is that we don’t have that “if”.
I thought that “13% innocent population in jail is wrong” was a premise, and “individual with 13% probability of innocence in jail in wrong” was the conclusion.
Which seems perfectly reasonable to me: if you have an 87% certainty threshold for conviction, it means you’re willing to tolerate up to 13% of convicts being innocent, an unacceptably high number by my lights.
I agree if you mean that the damage from an irrational bias is higher when the stakes are higher, but disagree if you mean that rational marginal certainty levels needed for conviction would be higher for severe crimes. The risks from letting a thief go free (more thefts) seem lower than the risks from letting a murderer go free (more murders) even compared to the damage done to a potential convicted innocent (assuming no death penalty, and also assuming higher conviction rates would actually result in fewer of the real culprits going free, which often does not seem to be the case).
So it sounds like you’re saying we do have the “if”. But are you sure the number is not just unacceptably high because in any realistic example of a 13% innocent population of convicts, many of them would have to have been seen as having substantially greater than 13% chances of innocence? If not for some biasing effect like that, it’s hard for me to see why the moral question would suddenly be clear once it was stated in population frequencies rather than in individual probabilities.
So it sounds like you’re saying we do have the “if”.
Actually, no, because the equivalence of the two formulations is obvious to me.
But it might not be for everyone; it’s well known that many people find thinking in terms of frequencies more intuitive than thinking in terms of bare probabilities. For such people, a statement about probabilities may simply not have any moral force unless and until it is translated into a statement about frequencies.
But are you sure the number is not just unacceptably high because [...] many of them would have to have been seen as having substantially greater than 13% chances of innocence? If not for some biasing effect like that, it’s hard for me to see why the moral question would suddenly be clear once it was stated in population frequencies rather than in individual probabilities.
Well, people’s intuitions about justice aren’t all that consistent, so I don’t think this particular moral question is going to suddenly become clear to all observers no matter how it’s stated. That being said, though, I don’t think we have any particular reason to think that Guede was convicted on unusually shaky evidence, so it seems reasonable—given certain assumptions—to take our estimates of his case as representative of murder cases in general.
A 13% innocence threshold for each particular case won’t give you a 13% innocent prison population (assuming good estimates, which is probably generous in this context), but if we adopt that criterion and Guede’s in the middle of the probability distribution for murder defendants, it seems likely that the resulting population-level incidence would still land on the bad side of 8 or 10%. Which doesn’t look much better.
By the way, I should probably clarify that I don’t think the LW average of 87% probability of guilt for Guede at all means that he should have been acquitted. I attribute the low number to a lack of confidence due to not having delved sufficiently deeply into the case, as per the third point in my earlier comment.
One should only believe that a miscarriage of justice has occurred in his case if one believes that the jury should not have had more than (say) 87% confidence. But in order to believe that one would presumably have to be highly confident in one’s belief about what information the jury had.
I would say that sending an individual with 13% probability of innocence to jail is clearly wrong, because 1 out of 10 of them would be innocent.
One wonders how many of those are people the jury correctly thinks have done other crimes, or subjectively think deserve more punishment for past crimes. That would be a different malfunction from the expressed intent of the system and would imply the system otherwise does much better than the 87⁄13 ratio.
I see. It’s a little more obvious to spell out “more than 1 out of 10 innocent” instead of “only 87% probablity of guilt” but if you see them as immediately equivalent then indeed the argument will do nothing for you.
In a situation like a trial, where I would be limited to just those “facts” presented by the lawyers, it would be extraordinarily unlikely for me to give better than 90% probability of anything.
Well, I haven’t looked at those estimates for a few months, but I’d imagine that a lot of the margin in the outside view of Guede’s case comes from uncertainties introduced by the media handling of the case or by an imperfect view of the evidence. Neither of those factors would, presumably, apply to a jury.
That being said, I wouldn’t be all that surprised if thirteen out of a hundred prisoners in Europe and the US were innocent of some of the charges that put them in jail. It’s higher than my own estimate would be, but within the same order of magnitude.
I would think that hypothetical juror judgments of guilt or innocence may be a lot more prone to bias than a more “dispassionate” look at the evidence generating a probabilty estimate. Even if one should count one’s own hypothetical guilt/innocence judgment as a small bit of evidence in the right direction, explicitly trying to calibrate this judgment with one’s prior probability estimate is going to make one over-correct one’s estimate.
Strange thing about this is, if I’ve calculated it right, the average probability estimate of Guede’s guilt is only ~87%. It seems to me that if this were your real probability estimate of his guilt, and you were on the jury at the guy’s trial, you would be obligated to vote innocent. If you operate on the basis that a 13% chance of innocence is not a reasonable doubt, about thirteen out of every hundred people who go to jail will be innocent. That is (let me check) more than one in ten, which strikes me as rather a lot. I think my own estimate of Guede’s guilt is above 99%, so I would vote guilty, but I’m surprised the average here is so low.
That’s if everyone who went to jail had a 13% chance of innocence. Presumably much of the time it would be lower.
Yes indeed. My mistake.
Not 13 out of every 100 people in jail, but still 13 out of every 100 people sentenced by the jury as guilty in the case of a probability estimate of only ~87%. ….The argument still works to show that the probability of guilt at 87% is too low to vote guilty.
This argument does not show that.
Which argument? I meant the argument loosely defined as the one where you count which fraction of innocent people are jailed to determine if the probability of guilt at 87% is appropriate. Steven0641 correctly pointed out that the target space for the fraction isn’t all people in jail, but then you modify the target space to all people judged guilty with probability 87% and the argument ‘works’.
The argument works if adding a 13% innocent population to jail is clearly wrong even though sending an individual with 13% probability of innocence to jail is not clearly wrong. Peter’s point, I think, is that we don’t have that “if”.
I thought that “13% innocent population in jail is wrong” was a premise, and “individual with 13% probability of innocence in jail in wrong” was the conclusion.
Which seems perfectly reasonable to me: if you have an 87% certainty threshold for conviction, it means you’re willing to tolerate up to 13% of convicts being innocent, an unacceptably high number by my lights.
It gets worse—the most severe crimes face the strongest pro-conviction biases.
...which is of course exactly the opposite of how it should work.
I agree if you mean that the damage from an irrational bias is higher when the stakes are higher, but disagree if you mean that rational marginal certainty levels needed for conviction would be higher for severe crimes. The risks from letting a thief go free (more thefts) seem lower than the risks from letting a murderer go free (more murders) even compared to the damage done to a potential convicted innocent (assuming no death penalty, and also assuming higher conviction rates would actually result in fewer of the real culprits going free, which often does not seem to be the case).
So it sounds like you’re saying we do have the “if”. But are you sure the number is not just unacceptably high because in any realistic example of a 13% innocent population of convicts, many of them would have to have been seen as having substantially greater than 13% chances of innocence? If not for some biasing effect like that, it’s hard for me to see why the moral question would suddenly be clear once it was stated in population frequencies rather than in individual probabilities.
Actually, no, because the equivalence of the two formulations is obvious to me.
But it might not be for everyone; it’s well known that many people find thinking in terms of frequencies more intuitive than thinking in terms of bare probabilities. For such people, a statement about probabilities may simply not have any moral force unless and until it is translated into a statement about frequencies.
Well, people’s intuitions about justice aren’t all that consistent, so I don’t think this particular moral question is going to suddenly become clear to all observers no matter how it’s stated. That being said, though, I don’t think we have any particular reason to think that Guede was convicted on unusually shaky evidence, so it seems reasonable—given certain assumptions—to take our estimates of his case as representative of murder cases in general.
A 13% innocence threshold for each particular case won’t give you a 13% innocent prison population (assuming good estimates, which is probably generous in this context), but if we adopt that criterion and Guede’s in the middle of the probability distribution for murder defendants, it seems likely that the resulting population-level incidence would still land on the bad side of 8 or 10%. Which doesn’t look much better.
By the way, I should probably clarify that I don’t think the LW average of 87% probability of guilt for Guede at all means that he should have been acquitted. I attribute the low number to a lack of confidence due to not having delved sufficiently deeply into the case, as per the third point in my earlier comment.
One should only believe that a miscarriage of justice has occurred in his case if one believes that the jury should not have had more than (say) 87% confidence. But in order to believe that one would presumably have to be highly confident in one’s belief about what information the jury had.
I agree, for reasons outlined here. Like you, I’m speaking hypothetically.
I would say that sending an individual with 13% probability of innocence to jail is clearly wrong, because 1 out of 10 of them would be innocent.
So the premise instead is: adding a 13% innocent population of any subset or category of individuals to jail is clearly wrong
leading to the conclusion: sending an individual with only 87% probability of guilt to jail is wrong
One wonders how many of those are people the jury correctly thinks have done other crimes, or subjectively think deserve more punishment for past crimes. That would be a different malfunction from the expressed intent of the system and would imply the system otherwise does much better than the 87⁄13 ratio.
Yes, that’s what I meant by what I said. But the problem is that, at least to me, the premise is no more obvious than the conclusion.
I see. It’s a little more obvious to spell out “more than 1 out of 10 innocent” instead of “only 87% probablity of guilt” but if you see them as immediately equivalent then indeed the argument will do nothing for you.
In a situation like a trial, where I would be limited to just those “facts” presented by the lawyers, it would be extraordinarily unlikely for me to give better than 90% probability of anything.
Well, I haven’t looked at those estimates for a few months, but I’d imagine that a lot of the margin in the outside view of Guede’s case comes from uncertainties introduced by the media handling of the case or by an imperfect view of the evidence. Neither of those factors would, presumably, apply to a jury.
That being said, I wouldn’t be all that surprised if thirteen out of a hundred prisoners in Europe and the US were innocent of some of the charges that put them in jail. It’s higher than my own estimate would be, but within the same order of magnitude.
I agree that it wouldn’t be hugely surprising. I meant it strikes me as higher than acceptable.
I would think that hypothetical juror judgments of guilt or innocence may be a lot more prone to bias than a more “dispassionate” look at the evidence generating a probabilty estimate. Even if one should count one’s own hypothetical guilt/innocence judgment as a small bit of evidence in the right direction, explicitly trying to calibrate this judgment with one’s prior probability estimate is going to make one over-correct one’s estimate.