I wouldn’t go that far. There are many cases where the legal system explicitly deviates from Bayesianism. Some examples:
Despite the fact that Demographic Group X is more/less likely to have committed crime Y, neither side can introduce this as evidence, e.g. “Since my client is a woman, you should reduce the odds you assign to her having committed a murder by a factor of 4.” (Obviously, the jury will notice the race/gender of the defendant, but you can’t argue that this is informative about the odds of guilt.)
Prohibition on many types of prejudicial evidence that is informative about the probability of guilt (like whether the defendant is a felon). (This can be justified on grounds of cognitive bias maybe, but not Bayesian grounds.)
In the US, the Constitutional prohibition on using the defendant’s silence as evidence, despite its informativeness, e.g., “If he’s really innocent, why doesn’t he just tell his side of the story? What’s the big deal? Why did he wait hours before even saying what happened? Did he need to get his story straight first?” (Again, the jury will notice that the defendant didn’t take the stand, but you can’t draw their attention to this as the prosecution.)
The exclusionary rule. The impact of illegally-collected physical evidence (i.e. not forced confessions but e.g. warrantless searches) has a small to non-existent impact on the evidence’s strength. The policy on excluding illegally-obtained evidence may be justified on decision-theoretic grounds, but not on Bayesian grounds.
Outside of trials, the fact that you have to wait years before you hear a judge’s binding opinion on whether or not a law actually can be enforced (i.e. is Constitutional).
But notice that these are examples of restrictions on evidence of guilt. The assumption (very reasonable, it seems to me) is that human irrationality tends in the direction of false positives, i.e. wrongful convictions. (Possibly along with the assumption that our values require a lower tolerance for false positives than false negatives.)
If juries are capable of convicting on the sort of evidence presented at the Knox/Sollecito trial (and they are, whether in Italy, the U.S., or anywhere else)...well, can you imagine all the false convictions we would have if such rules as you listed were relaxed?
The bias toward false positives is probably especially strong in criminal cases. The archetypal criminal offense is such that it unambiguously happened (not quite like the Willingham case), and in the ancestral human environment there were far fewer people around who could have done it. That makes the priors for everyone higher, which means that for whatever level of probability you’re asking for it takes less additional evidence to get there. That a person is acting strangely might well be enough—especially since you’d have enough familiarity with that person to establish a valid baseline, which doesn’t and can’t happen in any modern trial system.
Now add in the effects of other cognitive biases: we tend to magnify the importance of evidence against people we don’t like and excessively discount evidence against people we do. That’s strictly noise when dealing with modern criminal defendants, but ancestral humans actually knew the people in question, and had better reason for liking or disliking them. That might count as weak evidence by itself, and a perfect Bayesian would count it while also giving due consideration to the other evidence. But these weren’t just suspects, but your personal allies or rivals. Misweighing evidence could be a convenient way of strengthening your position in the tribe, and having a cognitive bias let you do that in all good conscience. We can’t just turn that off when we’re dealing with strangers, especially when the media creates a bogus familiarity.
But notice that these are examples of restrictions on evidence of guilt.
No, they’re not. The first one I listed can go either way.
“Since my client is a woman, you should reduce the odds you assign to her having committed a murder by a factor of 4.”
The second one can go either way too; it just as much excludes e.g. hearsay evidence that implicates someone else.
The assumption (very reasonable, it seems to me) is that human irrationality tends in the direction of false positives, i.e. wrongful convictions.
Sure, but that needs to be accounted for via the guilt probability threshold, not by reducing the accuracy of the evidence. Favoring acquittal through a high burden and biasing evidence in favor of the defendant is “double-dipping”.
If juries are capable of convicting on the sort of evidence presented at the Knox/Sollecito trial (and they are, whether in Italy, the U.S., or anywhere else)...well, can you imagine all the false convictions we would have if such rules as you listed were relaxed?
I only listed a few examples off the top of my head. The appropriate comparison is to the general policy of, per Bayesianism, incorporating all informative evidence. This would probably lead to more accurate assessments of guilt. In particularly egregious cases like K/S, it would have been a tremendous boon to them to allow them to have an explicit guilt threshold and count up the (log) likelihood ratio of all the evidence.
In any case, remember that there’s a cost to false negatives as well. Although that’s heavily muddled by the fundamental injustice of so many laws for which such a cost is non-existent.
Let me take a step back here, because despite the fact that it sounds like we’re arguing, I find myself in total agreement with other comments of yours in this thread, in particular your description of how trials should work; I could scarcely have said it better myself.
Here’s what I claim: the rules of evidence constitute crude attempts to impose some degree of rationality on jurors and prosecutors who are otherwise not particularly inclined to be rational. These hacks are not always successful, and occasionally even backfire; and they would not be necessary or useful for Bayesian juries who could be counted on to evaluate evidence properly. However, removing such rules without improving the rationality of jurors would be a disaster.
(Let’s not forget, after all, that there were people here on LW who reacted with indignation at my dismissal of certain discredited evidence in the Knox case, protesting that legal rules of admissibility don’t apply to Bayesian calculations—as if I had been trying to pass off some kind of legal loophole as a Bayesian argument. Such people were apparently taking it for granted that this evidence was significant, which suggests to me that it is very difficult for people—even aspiring rationalists—to discount information they come across. This provides support for the necessity of rules that exclude certain kinds of information from courtrooms, given the population currently doing the judging.)
Okay, then I think we’re in agreement. I guess I had interpreted your earlier comment as a much stronger claim about the mapping between pure Bayesianism and existing legal systems, but I definitely agree with what you’ve said here. I would just note that it would probably be more accurate to say that the rules of evidence are hacks to approximate Bayes and correct for predictable cognitive biases, though perhaps in this context those aren’t quite separate categories.
I think that is an incomplete description of the justification of the rules of evidence—some of these rules are also introduced to discourage particular abuses of the system, such as unreasonable searches. Otherwise, agreed.
The policy on excluding illegally-obtained evidence may be justified on decision-theoretic grounds, but not on Bayesian grounds.
In that case, why should we design the system on Bayesian grounds?
I think that’s really why I concur with komponisto—our system may not be optimal, but optimal for a system has to work as a system, including resistance to gaming. Aside from what you suggest about constitutionality, on which I have no comment, your changes are generally unlikely to improve the ability of a legal system to prosecute the guilty and acquit the innocent.
I think the proper response to illegally obtained evidence, is to allow it to be presented as evidence, but charge those who obtained it with whatever crimes made its obtainment illegal.
The problem with implementing this in the current system is that the government has a monopoly on prosecuting criminal charges, so that agents of the government can get away with criminal acts. If ordinary citizens had the same power as district attorneys to seek indictments and prosecute criminal charges, it would provide a huge disincentive for illegally obtaining evidence, and many other government abuses.
In that case, why should we design the system on Bayesian grounds?
Maybe we shouldn’t; I was just disputing komponisto’s insinuation that there’s some unappreciated, general mapping between Bayesianism and the existing justice system.
I think that’s really why I concur with komponisto—our system may not be optimal, but optimal for a system has to work as a system, including resistance to gaming.
Even when it allows so much relative weight to be given to sociological “evidence” (“she had a wild sex life”) compared to physical evidence?
Maybe we shouldn’t; I was just disputing komponisto’s insinuation that there’s some unappreciated, general mapping between Bayesianism and the existing justice system.
I agree that the necessity of a mapping has not been shown, although that’s not what I read into komponisto’s comment.
Even when it allows so much relative weight to be given to sociological “evidence” (“she had a wild sex life”) compared to physical evidence?
No. But that would be best corrected by sanity and education, not by changing the law. A jury of people interested primarily in the physical evidence would not be distracted by trivia about countercultural tendencies on the parts of relevant persons.
that would be best corrected by sanity and education, not by changing the law. A jury of people interested primarily in the physical evidence would not be distracted by trivia about countercultural tendencies on the parts of relevant persons.
But I think it would make a big (positive) difference if everything had to be phrased in terms of likelihood ratios against a prior and guilt threshold.
Individual pieces of evidence are not independent. If Mortimer Q. Snodgrass is shown to have left his home at 11:50, arrived at the scene of the crime at midnight, and returned home fifteen minutes later is damning if the victim died at midnight and exculpatory if the victim died three hours later. There’s a combinatorial explosion trying to describe the effects of every piece of evidence separately.
Sure, but at least each side can draw its theorized causal diagram, how the evidence fits in, how the likelihood ratios interplay (per Pearl’s method of separating inferential and causal evidence flows), and what probability that justifies. It would still lend a clarity of thought not currently present among the mouthbreathers on juries that haven’t been exposed to any of this, even if you had to train them in it first.
(And that would be easy if the trainers really understood [at Level 2 at least] causal diagrams and read my forthcoming article on guidelines for explaining...)
Thanks, but I don’t see how much the points being discussed here hinge on it.
Are you saying that you’re skeptical that Pearl’s networks and Bayesian inference can be quickly (e.g. over a day or so) explained to random people selected for jury duty, but might be convinced of the ease of such training after seeing my exposition of how to enhance your explanatory abilities?
Hm, now that I think about it, that by itself should be evidence I have some abnormally high explanatory mojo—if I could explain your position to you better than you could explain it to yourself. :-P
I wouldn’t go that far. There are many cases where the legal system explicitly deviates from Bayesianism. Some examples:
Despite the fact that Demographic Group X is more/less likely to have committed crime Y, neither side can introduce this as evidence, e.g. “Since my client is a woman, you should reduce the odds you assign to her having committed a murder by a factor of 4.” (Obviously, the jury will notice the race/gender of the defendant, but you can’t argue that this is informative about the odds of guilt.)
Prohibition on many types of prejudicial evidence that is informative about the probability of guilt (like whether the defendant is a felon). (This can be justified on grounds of cognitive bias maybe, but not Bayesian grounds.)
In the US, the Constitutional prohibition on using the defendant’s silence as evidence, despite its informativeness, e.g., “If he’s really innocent, why doesn’t he just tell his side of the story? What’s the big deal? Why did he wait hours before even saying what happened? Did he need to get his story straight first?” (Again, the jury will notice that the defendant didn’t take the stand, but you can’t draw their attention to this as the prosecution.)
The exclusionary rule. The impact of illegally-collected physical evidence (i.e. not forced confessions but e.g. warrantless searches) has a small to non-existent impact on the evidence’s strength. The policy on excluding illegally-obtained evidence may be justified on decision-theoretic grounds, but not on Bayesian grounds.
Outside of trials, the fact that you have to wait years before you hear a judge’s binding opinion on whether or not a law actually can be enforced (i.e. is Constitutional).
You give the legal system way too much credit.
But notice that these are examples of restrictions on evidence of guilt. The assumption (very reasonable, it seems to me) is that human irrationality tends in the direction of false positives, i.e. wrongful convictions. (Possibly along with the assumption that our values require a lower tolerance for false positives than false negatives.)
If juries are capable of convicting on the sort of evidence presented at the Knox/Sollecito trial (and they are, whether in Italy, the U.S., or anywhere else)...well, can you imagine all the false convictions we would have if such rules as you listed were relaxed?
The bias toward false positives is probably especially strong in criminal cases. The archetypal criminal offense is such that it unambiguously happened (not quite like the Willingham case), and in the ancestral human environment there were far fewer people around who could have done it. That makes the priors for everyone higher, which means that for whatever level of probability you’re asking for it takes less additional evidence to get there. That a person is acting strangely might well be enough—especially since you’d have enough familiarity with that person to establish a valid baseline, which doesn’t and can’t happen in any modern trial system.
Now add in the effects of other cognitive biases: we tend to magnify the importance of evidence against people we don’t like and excessively discount evidence against people we do. That’s strictly noise when dealing with modern criminal defendants, but ancestral humans actually knew the people in question, and had better reason for liking or disliking them. That might count as weak evidence by itself, and a perfect Bayesian would count it while also giving due consideration to the other evidence. But these weren’t just suspects, but your personal allies or rivals. Misweighing evidence could be a convenient way of strengthening your position in the tribe, and having a cognitive bias let you do that in all good conscience. We can’t just turn that off when we’re dealing with strangers, especially when the media creates a bogus familiarity.
No, they’re not. The first one I listed can go either way.
The second one can go either way too; it just as much excludes e.g. hearsay evidence that implicates someone else.
Sure, but that needs to be accounted for via the guilt probability threshold, not by reducing the accuracy of the evidence. Favoring acquittal through a high burden and biasing evidence in favor of the defendant is “double-dipping”.
I only listed a few examples off the top of my head. The appropriate comparison is to the general policy of, per Bayesianism, incorporating all informative evidence. This would probably lead to more accurate assessments of guilt. In particularly egregious cases like K/S, it would have been a tremendous boon to them to allow them to have an explicit guilt threshold and count up the (log) likelihood ratio of all the evidence.
In any case, remember that there’s a cost to false negatives as well. Although that’s heavily muddled by the fundamental injustice of so many laws for which such a cost is non-existent.
Let me take a step back here, because despite the fact that it sounds like we’re arguing, I find myself in total agreement with other comments of yours in this thread, in particular your description of how trials should work; I could scarcely have said it better myself.
Here’s what I claim: the rules of evidence constitute crude attempts to impose some degree of rationality on jurors and prosecutors who are otherwise not particularly inclined to be rational. These hacks are not always successful, and occasionally even backfire; and they would not be necessary or useful for Bayesian juries who could be counted on to evaluate evidence properly. However, removing such rules without improving the rationality of jurors would be a disaster.
(Let’s not forget, after all, that there were people here on LW who reacted with indignation at my dismissal of certain discredited evidence in the Knox case, protesting that legal rules of admissibility don’t apply to Bayesian calculations—as if I had been trying to pass off some kind of legal loophole as a Bayesian argument. Such people were apparently taking it for granted that this evidence was significant, which suggests to me that it is very difficult for people—even aspiring rationalists—to discount information they come across. This provides support for the necessity of rules that exclude certain kinds of information from courtrooms, given the population currently doing the judging.)
Okay, then I think we’re in agreement. I guess I had interpreted your earlier comment as a much stronger claim about the mapping between pure Bayesianism and existing legal systems, but I definitely agree with what you’ve said here. I would just note that it would probably be more accurate to say that the rules of evidence are hacks to approximate Bayes and correct for predictable cognitive biases, though perhaps in this context those aren’t quite separate categories.
I think that is an incomplete description of the justification of the rules of evidence—some of these rules are also introduced to discourage particular abuses of the system, such as unreasonable searches. Otherwise, agreed.
In that case, why should we design the system on Bayesian grounds?
I think that’s really why I concur with komponisto—our system may not be optimal, but optimal for a system has to work as a system, including resistance to gaming. Aside from what you suggest about constitutionality, on which I have no comment, your changes are generally unlikely to improve the ability of a legal system to prosecute the guilty and acquit the innocent.
I think the proper response to illegally obtained evidence, is to allow it to be presented as evidence, but charge those who obtained it with whatever crimes made its obtainment illegal.
The problem with implementing this in the current system is that the government has a monopoly on prosecuting criminal charges, so that agents of the government can get away with criminal acts. If ordinary citizens had the same power as district attorneys to seek indictments and prosecute criminal charges, it would provide a huge disincentive for illegally obtaining evidence, and many other government abuses.
Maybe we shouldn’t; I was just disputing komponisto’s insinuation that there’s some unappreciated, general mapping between Bayesianism and the existing justice system.
Even when it allows so much relative weight to be given to sociological “evidence” (“she had a wild sex life”) compared to physical evidence?
I agree that the necessity of a mapping has not been shown, although that’s not what I read into komponisto’s comment.
No. But that would be best corrected by sanity and education, not by changing the law. A jury of people interested primarily in the physical evidence would not be distracted by trivia about countercultural tendencies on the parts of relevant persons.
But I think it would make a big (positive) difference if everything had to be phrased in terms of likelihood ratios against a prior and guilt threshold.
Individual pieces of evidence are not independent. If Mortimer Q. Snodgrass is shown to have left his home at 11:50, arrived at the scene of the crime at midnight, and returned home fifteen minutes later is damning if the victim died at midnight and exculpatory if the victim died three hours later. There’s a combinatorial explosion trying to describe the effects of every piece of evidence separately.
Sure, but at least each side can draw its theorized causal diagram, how the evidence fits in, how the likelihood ratios interplay (per Pearl’s method of separating inferential and causal evidence flows), and what probability that justifies. It would still lend a clarity of thought not currently present among the mouthbreathers on juries that haven’t been exposed to any of this, even if you had to train them in it first.
(And that would be easy if the trainers really understood [at Level 2 at least] causal diagrams and read my forthcoming article on guidelines for explaining...)
Please make this come forth promptly. I plan to explain some pretty complicated stuff to a bunch of people soon, and could use the help!
I’ll do my best.
I’ll put off judgment until after your article, then.
Thanks, but I don’t see how much the points being discussed here hinge on it.
Are you saying that you’re skeptical that Pearl’s networks and Bayesian inference can be quickly (e.g. over a day or so) explained to random people selected for jury duty, but might be convinced of the ease of such training after seeing my exposition of how to enhance your explanatory abilities?
Related: maybe you just suck at explaining
LOL, you have no idea how many times I’ve thought that about people who claim something’s hard to explain …
Yes. Edit: That’s probably a better summary of my thoughts than I could give at the moment, even.
Can I call ’em or what? ;-)
I aim to be predictable. (-:
Hm, now that I think about it, that by itself should be evidence I have some abnormally high explanatory mojo—if I could explain your position to you better than you could explain it to yourself. :-P
Don’t promote the hypothesis excessively—you’re comparing yourself to The Worst Debater In The World with sleep deprivation. (-;
Damn I’m good B-)