they grant about 65% after meal breaks, and about 10% right before breaks and at the end of the day
While the idea that judges are more lenient shortly after eating is plausible, this effect size does not pass sanity checking. I would like to see this reproduced. The judgments to be made should break down into some one-sided decisions and some borderline decisions, but these numbers imply not only that one-sided decisions are uncommon, but also that borderline decisions aren’t influenced much by other things like the eloquence of the lawyers.
I agree the effects are dramatic and that replication would be ideal. But if you look at the study, it really does say what I said (actually, it drops close to 0%), and it seems like they control for severity of offense (which would hardly be time-of-day correlated anyhow). The N is over a thousand, with 8 different judges.
It wouldn’t indicate that lawyer eloquence is irrelevant, only that lawyer eloquence is much less significant when the judge is hungry or tired. That actually isn’t such a big stretch.
The true effect may be less noticeable than the effect observed in this study, but the study is nonetheless rather strong evidence that some meaningful effect exists. I bet there will be a lot of followups to this; it’s rather staggering.
It wouldn’t indicate that lawyer eloquence is irrelevant, only that lawyer eloquence is much less significant when the judge is hungry or tired. That actually isn’t such a big stretch.
Or possibly that lawyers and judges meal schedules tend to be synchronized, and lawyer eloquence correlates with lawyer hunger. Which would not be too surprising, really, though arguably it’s just as problematic as the interpretation it replaces.
One question worth considering is how parole hearings get scheduled in the first place.
If there’s a general understanding among lawyers that their odds are adjusted even slightly based on judges meals, that might drive a systematic tendency to schedule the most compelling cases right after meals (to “put all of my wood behind one arrow,” as a former boss used to put it., and the least compelling cases right before them.
The study states that no one in the profession they talked to (judges or lawyers) expected this bias. So such a deliberate scheduling explanation seems unlikely.
How likely is it that lawyers never noticed an effect of such magnitude? I find that rather implausible, assuming that there is some minimal competitiveness in the profession.
Very easily. You wouldn’t see a lot of data points, and you wouldn’t be looking for it. You’d have on parole hearing maybe every few weeks. It’d be extremely hard to piece together that the time of day had a significant effect on the outcome, especially when you want credit for good results (and generally want to blame the judge for bad results). You probably don’t have many lawyers in a position to observe enough parole hearings to really form a suspicion.
I can imagine that individuals might be unable to spot the correlation. But if the lawyering industry is competitive to any significant degree, it would be strikingly implausible if none of the competitors ever stumbled onto it and proceeded to take advantage of it. Especially considering that larger law firms, and even individual lawyers with long experience, could easily data-mine their past record for such correlations. Are they all really too stupid to think of that?
Whenever social scientists—a phrase I’m always tempted to put into scare quotes—claim that they’ve found something that indicates unexploited profit opportunities if true, it is likely that they’re either talking nonsense or that they’ve reinvented some wheel that is however infeasible or forbidden to use in practice (at least openly) for other reasons. Otherwise, it would violate the weak efficient markets hypothesis, in which I certainly have far more confidence than in “social science.”
“But the market should fix everything!” is something you only get to say when you actually have a decent notion of the market in question. It is apparent that you do not, at least with respect to lawyers doing parole hearings (which may often not involve lawyers in the States).
There is no money to be made here. Most of these attorneys are being paid by the state. The ones that are not do not generally disclose their past records in an indexed, searchable database. There is no scalability—there’s a finite number of after-meal time slots. There are no (or virtually no) large law firms working in this field. Lawyers as a group are not scientists and are generally relatively innumerate. I could probably go on for pages on how lawyer’s interests really aren’t that closely aligned with positive outcomes; a thousand times more so when their clients aren’t the ones paying the bill.
It’s one thing to say, “But if you figured this out, you could dominate the market!” It’s another thing to articulate how to actually dominate the market with this knowledge. If you can’t do that, the EMH doesn’t really do much for you.
EDIT: You’ve edited your comment substantially since the time I posted this reply, so now it looks like I’m writing with disregard to your arguments, and I’m thus editing my reply too.
I certainly don’t have an idealized view of the legal system in general, or criminal defense lawyers in particular—on the contrary. If there really is a complete disconnect between the lawyers’ incentives and the destiny of their clients, then I can imagine that they wouldn’t notice correlations like these (though I’d still find it implausible, given the purported magnitude of the effect).
However, if such incentives do exist, then I find it absurdly implausible that lawyers would be unable to figure out these correlations and take advantage of them. (Even if they’re theoretically unable to influence the scheduling, no bureaucratic system is immune to manipulation if there are smart and determined people with the right incentives.) As innumerate as they might be, there’s no way lawyers could be so incompetent that a few social scientists could just show up and point out that they’ve been doing their job suboptimally because they’re unable to put two and two together about such a simple issue.
On the whole, when you argue that the interests of lawyers and their clients are misaligned, I find that a plausible claim. But when you argue that they’d be unable to figure this out even if they had the incentive, that’s a far more extreme and implausible assertion.
It could also be that those who notice it, have attributed it to other factors (for example, that easier/simpler cases are generally scheduled first, as mentioned by some other posters). It could even be that these other factors are real, but there’s still a pattern after adjusting for that.
That said, I can’t imagine the correlation being true unless there’s some other factor significantly influencing things as well.
I think psycho’s point is that not only would they have to notice it, they’d have to care.
And since (1) We’re talking about people already adjudicated guilty, and (2) almost always NOT the party paying the lawyer there is no real incentive for the lawyer to win, just to present a credible (Professional) case. Most convicts have little money, and even if they had a private lawyer for the court case, they gave him most of their money and their now-ex wife spent the rest on booze and pool boys before the divorce, so now he’s got a public defender who gets the same check win or lose.
And since (1) We’re talking about people already adjudicated guilty, and (2) almost always NOT the party paying the lawyer there is no real incentive for the lawyer to win, just to present a credible (Professional) case.
Even if only a small percentage of prisoners seeking parole pay for their own lawyers, that still constitutes a market whose participants have the incentive to figure out the informal intricacies of parole hearings and adapt to them.
There is no money to be made here. Most of these attorneys are being paid by the state. The ones that are not do not generally disclose their past records in an indexed, searchable database.
OK, but why haven’t private law firms looked into the scheduling of hearings as a possible determinant of outcome? Surely they have an incentive to; or do these results only hold for paroles, in which “there is no money to be made”? If the hunger theory is right, then shouldn’t judges of any type of case be under its effect?
Very few judicial decisions are actually made entirely during a hearing; despite what you see on television, most major issues are going to be briefed and the judge (or his staff) will have already read the briefs and come to a not-too-tentative decision about how they are going to rule. For issues that are so small as not to be briefed, lawyers have pretty much no control over when these will be heard by the judge, and the stakes tend to be relatively small anyways. Moreover, where there are two parties involved, it seems impossible to predict which direction this effect would take—would the judge be wiser, less wise, less agreeable, lazier? Even if someone were paying careful attention to the data, it seems unlikely they could discern a clear trend, and no one’s paying such attention because (A) no one really has an incentive to and (B) the payoff is likely very close to 0 anyways.
The Israeli parole result was for a short single high-stakes decision; most hearings are not like that, I think.
… is the exact response I wanted to make.
Most legal choices are either incredibly short term—like an objection that a judge must often respond to immediately—or medium to long term—like a motion that a judge will ask for parties to provide briefs (written legal arguments) on. Parole hearing like this are one a few legal decisions where there really is a quick decision made—another area would be bail hearings, but there the outcome isn’t binary, it’s a dollar amount. There isn’t much money to be made in gaming either,.
No. The study was specifically on parole decisions, which often are made at the time of the hearing, although other judicial decisions generally aren’t.
I agree, but you should consider that this is about to happen (in a widespread way) for the first time.
Also, some people keep such knowledge to themselves—weighing the advantage they get from being one of the few to have and use it as more than they’d get by gaining fame in sharing it. (But it’s true that such powerful secrets tend to spread rapidly, if shared at all.)
This would probably split randomly. Also, lawyers are probably not scheduling blocks of hearings (if they even have any say in scheduling). You schedule for your client, so if you knew, all you would do is schedule ALL your cases for right after lunch, if possible.
Even if that weren’t the case, there’s no particular reason why you’d want a more favorable judge for an easy win than for a tough one. Indeed, there are many reasons you’d want a more favorable judge for a tougher-to-win case—higher marginal benefit, mostly. In short, it’d be very hard for underlying factors to create meaningful systematic bias on such an obvious characteristic in a sample this size. Unless there’s some odd organizational principle, like, “More serious offenses are put off until the end of the day,” and such was not observed.
We also interviewed three attorneys, a parole panel judge, and five personnel at Israeli Prison Services and Court Management, learning that case ordering is not random and that several factors contribute to the downward trend in prisoner success between meal breaks. The most important is that the board tries to complete all cases from one prison before it takes a break and to start with another prison after the break. Within each session, unrepresented prisoners usually go last and are less likely to be granted parole than prisoners with attorneys. Using the same decision rules as Danziger et al., our data indicate that unrepresented prisoners account for about one-third of all cases, but they prevail only 15% of the time, whereas prisoners with counsel prevail at a 35% rate
Glöckner’s analysis doesn’t prove that extraneous factors weren’t influencing the judges, but he shows how the same effect could be produced by entirely rational judges interacting with the protocols required by the legal system.
The main analysis works like this: we know that favourable rulings take longer than unfavourable ones (~7 mins vs ~5 mins), and we assume that judges are able to guess how long a case will take to rule on before they begin it (from clues like the thickness of the file, the types of request made, the representation the prisoner has and so on). Finally, we assume judges have a time limit in mind for each of the three sessions of the day, and will avoid starting cases which they estimate will overrun the time limit for the current session.
It turns out that this kind of rational time-management is sufficient to generate the drops in favourable outcomes.
While the idea that judges are more lenient shortly after eating is plausible, this effect size does not pass sanity checking. I would like to see this reproduced. The judgments to be made should break down into some one-sided decisions and some borderline decisions, but these numbers imply not only that one-sided decisions are uncommon, but also that borderline decisions aren’t influenced much by other things like the eloquence of the lawyers.
I agree the effects are dramatic and that replication would be ideal. But if you look at the study, it really does say what I said (actually, it drops close to 0%), and it seems like they control for severity of offense (which would hardly be time-of-day correlated anyhow). The N is over a thousand, with 8 different judges.
It wouldn’t indicate that lawyer eloquence is irrelevant, only that lawyer eloquence is much less significant when the judge is hungry or tired. That actually isn’t such a big stretch.
The true effect may be less noticeable than the effect observed in this study, but the study is nonetheless rather strong evidence that some meaningful effect exists. I bet there will be a lot of followups to this; it’s rather staggering.
Or possibly that lawyers and judges meal schedules tend to be synchronized, and lawyer eloquence correlates with lawyer hunger. Which would not be too surprising, really, though arguably it’s just as problematic as the interpretation it replaces.
One question worth considering is how parole hearings get scheduled in the first place.
If there’s a general understanding among lawyers that their odds are adjusted even slightly based on judges meals, that might drive a systematic tendency to schedule the most compelling cases right after meals (to “put all of my wood behind one arrow,” as a former boss used to put it., and the least compelling cases right before them.
The study states that no one in the profession they talked to (judges or lawyers) expected this bias. So such a deliberate scheduling explanation seems unlikely.
How likely is it that lawyers never noticed an effect of such magnitude? I find that rather implausible, assuming that there is some minimal competitiveness in the profession.
Very easily. You wouldn’t see a lot of data points, and you wouldn’t be looking for it. You’d have on parole hearing maybe every few weeks. It’d be extremely hard to piece together that the time of day had a significant effect on the outcome, especially when you want credit for good results (and generally want to blame the judge for bad results). You probably don’t have many lawyers in a position to observe enough parole hearings to really form a suspicion.
I can imagine that individuals might be unable to spot the correlation. But if the lawyering industry is competitive to any significant degree, it would be strikingly implausible if none of the competitors ever stumbled onto it and proceeded to take advantage of it. Especially considering that larger law firms, and even individual lawyers with long experience, could easily data-mine their past record for such correlations. Are they all really too stupid to think of that?
Whenever social scientists—a phrase I’m always tempted to put into scare quotes—claim that they’ve found something that indicates unexploited profit opportunities if true, it is likely that they’re either talking nonsense or that they’ve reinvented some wheel that is however infeasible or forbidden to use in practice (at least openly) for other reasons. Otherwise, it would violate the weak efficient markets hypothesis, in which I certainly have far more confidence than in “social science.”
“But the market should fix everything!” is something you only get to say when you actually have a decent notion of the market in question. It is apparent that you do not, at least with respect to lawyers doing parole hearings (which may often not involve lawyers in the States).
There is no money to be made here. Most of these attorneys are being paid by the state. The ones that are not do not generally disclose their past records in an indexed, searchable database. There is no scalability—there’s a finite number of after-meal time slots. There are no (or virtually no) large law firms working in this field. Lawyers as a group are not scientists and are generally relatively innumerate. I could probably go on for pages on how lawyer’s interests really aren’t that closely aligned with positive outcomes; a thousand times more so when their clients aren’t the ones paying the bill.
It’s one thing to say, “But if you figured this out, you could dominate the market!” It’s another thing to articulate how to actually dominate the market with this knowledge. If you can’t do that, the EMH doesn’t really do much for you.
EDIT: You’ve edited your comment substantially since the time I posted this reply, so now it looks like I’m writing with disregard to your arguments, and I’m thus editing my reply too.
I certainly don’t have an idealized view of the legal system in general, or criminal defense lawyers in particular—on the contrary. If there really is a complete disconnect between the lawyers’ incentives and the destiny of their clients, then I can imagine that they wouldn’t notice correlations like these (though I’d still find it implausible, given the purported magnitude of the effect).
However, if such incentives do exist, then I find it absurdly implausible that lawyers would be unable to figure out these correlations and take advantage of them. (Even if they’re theoretically unable to influence the scheduling, no bureaucratic system is immune to manipulation if there are smart and determined people with the right incentives.) As innumerate as they might be, there’s no way lawyers could be so incompetent that a few social scientists could just show up and point out that they’ve been doing their job suboptimally because they’re unable to put two and two together about such a simple issue.
On the whole, when you argue that the interests of lawyers and their clients are misaligned, I find that a plausible claim. But when you argue that they’d be unable to figure this out even if they had the incentive, that’s a far more extreme and implausible assertion.
It could also be that those who notice it, have attributed it to other factors (for example, that easier/simpler cases are generally scheduled first, as mentioned by some other posters). It could even be that these other factors are real, but there’s still a pattern after adjusting for that.
That said, I can’t imagine the correlation being true unless there’s some other factor significantly influencing things as well.
I think psycho’s point is that not only would they have to notice it, they’d have to care.
And since (1) We’re talking about people already adjudicated guilty, and (2) almost always NOT the party paying the lawyer there is no real incentive for the lawyer to win, just to present a credible (Professional) case. Most convicts have little money, and even if they had a private lawyer for the court case, they gave him most of their money and their now-ex wife spent the rest on booze and pool boys before the divorce, so now he’s got a public defender who gets the same check win or lose.
Even if only a small percentage of prisoners seeking parole pay for their own lawyers, that still constitutes a market whose participants have the incentive to figure out the informal intricacies of parole hearings and adapt to them.
OK, but why haven’t private law firms looked into the scheduling of hearings as a possible determinant of outcome? Surely they have an incentive to; or do these results only hold for paroles, in which “there is no money to be made”? If the hunger theory is right, then shouldn’t judges of any type of case be under its effect?
Very few judicial decisions are actually made entirely during a hearing; despite what you see on television, most major issues are going to be briefed and the judge (or his staff) will have already read the briefs and come to a not-too-tentative decision about how they are going to rule. For issues that are so small as not to be briefed, lawyers have pretty much no control over when these will be heard by the judge, and the stakes tend to be relatively small anyways. Moreover, where there are two parties involved, it seems impossible to predict which direction this effect would take—would the judge be wiser, less wise, less agreeable, lazier? Even if someone were paying careful attention to the data, it seems unlikely they could discern a clear trend, and no one’s paying such attention because (A) no one really has an incentive to and (B) the payoff is likely very close to 0 anyways.
Doesn’t this strongly cut against the theory that the degree of hunger at the time of the hearing influences the decision?
… is the exact response I wanted to make.
Most legal choices are either incredibly short term—like an objection that a judge must often respond to immediately—or medium to long term—like a motion that a judge will ask for parties to provide briefs (written legal arguments) on. Parole hearing like this are one a few legal decisions where there really is a quick decision made—another area would be bail hearings, but there the outcome isn’t binary, it’s a dollar amount. There isn’t much money to be made in gaming either,.
The Israeli parole result was for a short single high-stakes decision; most hearings are not like that, I think.
No. The study was specifically on parole decisions, which often are made at the time of the hearing, although other judicial decisions generally aren’t.
I agree, but you should consider that this is about to happen (in a widespread way) for the first time.
Also, some people keep such knowledge to themselves—weighing the advantage they get from being one of the few to have and use it as more than they’d get by gaining fame in sharing it. (But it’s true that such powerful secrets tend to spread rapidly, if shared at all.)
That seems like a really really large correlation to miss. One lawyer doesn’t see 20 hearings?
Fair point. Not conclusive, but compelling.
This would probably split randomly. Also, lawyers are probably not scheduling blocks of hearings (if they even have any say in scheduling). You schedule for your client, so if you knew, all you would do is schedule ALL your cases for right after lunch, if possible.
Even if that weren’t the case, there’s no particular reason why you’d want a more favorable judge for an easy win than for a tough one. Indeed, there are many reasons you’d want a more favorable judge for a tougher-to-win case—higher marginal benefit, mostly. In short, it’d be very hard for underlying factors to create meaningful systematic bias on such an obvious characteristic in a sample this size. Unless there’s some odd organizational principle, like, “More serious offenses are put off until the end of the day,” and such was not observed.
I’ve found two critiques of this study:
1. https://www.pnas.org/content/pnas/108/42/E833.full.pdf
2. https://mindhacks.com/2016/12/08/rational-judges-not-extraneous-factors-in-decisions/