How would prosecutors be trained and incentivized to make and report accurate predictions? Who would be funding the defense attorneys, especially overworked public defenders, to learn to properly evaluate prosecutor’s claims?
The main thing a prosecutor cares about is getting reelect as a prosecutor. Writing the metric that measures whether they are good at reporting accurate predictions and writing it on the ballot is one of the strongest ways to incentives prosecutor.
Even if the prosecutor wants to go to another office after being a prosecutor, doing badly at the metric for prosecutors that gets written on their ballot, will be hold against the prosecutor.
Shifting the incentives on which prosecutors get evaluated is part of what this proposal is about.
Also, I understand that one of the big issues with plea deals is that it is often in a defendent’s individual interest to plead guilty to a lesser charge they’re innocent of, even if they’re pretty sure they wouldn’t be found guilty at trial, either because they’re risk averse (not necessarily unreasonable when years of your life are on the line), or because the agreed-on punishment will be less than the time in jail, and cost in fees, they’d incur just by going to trial even if exonerated. Data on likelihood of conviction wouldn’t help with that.
Let’s say a defendent Joe correctly believes that his chance of being found guilty at trial is 10%. This means that a prosecutor that’s well calibrated because that’s in his interest for getting reelected would also tell him that his chance of being found guilty is 10%.
If I’m a risk adverse person then knowing that not only I believe that my chances of being found guilty is 10% but that the prosecutor whose has access to evidence that I don’t see also believes the chance is 10% is very helpful.
Then when I’m in the court room my lawyer can ask the prosecutor “Why did you bring that case when you only think that there’s a 10% chance that my client will be found guilty?”. Journalists could write story “Prosecutor Smith brings a case against Joe to trial even so he only believes that there’s a 10% chance that Joe would be convicted”.
That’s not good for the public reputation of prosecutor Smith and thus won’t charge Joe in the first place and has no leverage to get Joe to accept the plea deal.
Who would be funding the defense attorneys, especially overworked public defenders, to learn to properly evaluate prosecutor’s claims?
The same way they are currently learning to evaluate the charges that the prosecutor makes. If the defense completley ignores the numbers nothing is lost over the status quo.
At a more basic level, I think this solves the wrong problem.
Rejecting a proposal because it doesn’t solve the problem you most care about it is bad. I agree that it would be good if the penal code would be changed and sentencing guidelines be reduced but that’s not the issue my post is addressing.
Sorry, then I think I made a bad assumption about why you were trying to solve the problem of overcharging. I added that at the end of my comment and probably shouldn’t have. Still, it’s not the reason I initially rejected the proposal. I honestly don’t think your proposal actually solves the overcharging problem or would make plea deals much fairer.
Then when I’m in the court room my lawyer can ask the prosecutor “Why did you bring that case when you only think that there’s a 10% chance that my client will be found guilty?” Journalists could write story “Prosecutor Smith brings a case against Joe to trial even so he only believes that there’s a 10% chance that Joe would be convicted”.
Journalists could write that, true. But if we’re relying on juries and voters to properly evaluate that kind of statistic as a reflection on the prosecutor’s job performance or the case’s merits, then hopefully they’ll also understand that the data point is immaterial to the question of whether the defendant is guilty. I can be very confident a policeman is guilty of murder, or a politician is guilty of accepting a bribe, or a frat boy is guilty of rape, and still correctly believe the probability of conviction is low. I’m not sure how a court would regard such an argument, but to this layman it seems that to whatever extent the percentage is based on the evidence, giving the number as an additional data point is double-counting evidence (or a way of introducing information based on evidence not presented at trial or inadmissible at trial, which is just as bad or worse), and to whatever degree it is based on assumptions about the judge and jury’s behavior and thinking, it’s speculation.
Overall I’m very skeptical that the enforcement mechanism you proposed to incentivize prosecutors to be honest is anywhere near strong enough. It might be better to scale the prosecutor’s pay to how well calibrated their estimates are, for those cases that make it to trial, with additional penalties like removal from office for being too far off. Or better yet, do a literal randomized trial where the prosecutor who offers the plea deal and makes the estimate is different from the prosecutor who goes to trial, and evaluate the former by accuracy and the latter by conviction rate.
Overall I’m very skeptical that the enforcement mechanism you proposed to incentivize prosecutors to be honest is anywhere near strong enough.
The incentive it’s as strong as the desire for people who evaluate the prosecutors want it to be. You can also make it stronger by linking it to bonus payments if you want. If you do pay prosecutors any bonuses it would make sense to do that, but I think it’s generally better to pay prosecutors a fixed salary.
In any case, even if many prosecutors give crapy numbers those numbers can just be ignored by the defendent and produce no disadvantage over the status quo.
I’m not sure how a court would regard such an argument, but to this layman it seems that to whatever extent the percentage is based on the evidence, giving the number as an additional data point is double-counting evidence
Evidence is by it’s nature very complex. If a journalist writes an article then a reader doesn’t know whether the journalist gives a fair representation of the issue at hand.
It takes a lot of effort to understand all the evidence that goes into all the cases where a prosecutor charges people to evaluate the prosecutor. If you have the prosecutor summarize the strength of the evidence that evaluation gets easier.
Or better yet, do a literal randomized trial where the prosecutor who offers the plea deal and makes the estimate is different from the prosecutor who goes to trial
I don’t think you can easily do a trial because it’s a systematic intervention that needs to run a few years for people to adept to the new system before it leads it’s provides most of it’s benefits.
It takes a lot of effort to understand all the evidence that goes into all the cases where a prosecutor charges people to evaluate the prosecutor. If you have the prosecutor summarize the strength of the evidence that evaluation gets easier.
That’s true, but I think you’re being very optimistic, both in the ability of defendants and defense council to ignore or evaluate information the other side in an adversarial system claims is their true opinion, and in the ability and interest of the public in properly evaluating the job performance of prosecutors in local elections based on actual data. I think both are possible, and would be very valuable, but can’t be achieved without much deeper and broader reforms to make the underlying justice system more open, transparent, and trustworthy.
I don’t think you can easily do a trial because it’s a systematic intervention that needs to run a few years for people to adept to the new system before it leads it’s provides most of it’s benefits.
Sorry, I didn’t mean a trial as an experiment, I meant literally running legal trials this way, where in general the prosecutor that tries a case is not the one that produces the conviction probability estimate. Then, grade each both on the accuracy of their assessments, and separately on their conviction rates in trials they prosecute. I’d say either the one trying the case or a separate third prosecutor should have final say on which charges to bring. I think this would eliminate a lot of the potential for perverse incentives.
That’s true, but I think you’re being very optimistic, both in the ability of defendants and defense council to ignore or evaluate information the other side in an adversarial system claims is their true opinion
At would expect that in the beginning after the reform defendants and defense council would not trust the probability at all.
I would expect that trust in the numbers will only come when the system works well that they provide valid information.
I think both are possible, and would be very valuable, but can’t be achieved without much deeper and broader reforms to make the underlying justice system more open, transparent, and trustworthy.
Good political reforms aren’t about doing one thing but multiple things. There’s a reason why Obamacare is 906 pages.
The main thing a prosecutor cares about is getting reelect as a prosecutor. Writing the metric that measures whether they are good at reporting accurate predictions and writing it on the ballot is one of the strongest ways to incentives prosecutor.
Even if the prosecutor wants to go to another office after being a prosecutor, doing badly at the metric for prosecutors that gets written on their ballot, will be hold against the prosecutor.
Shifting the incentives on which prosecutors get evaluated is part of what this proposal is about.
Let’s say a defendent Joe correctly believes that his chance of being found guilty at trial is 10%. This means that a prosecutor that’s well calibrated because that’s in his interest for getting reelected would also tell him that his chance of being found guilty is 10%.
If I’m a risk adverse person then knowing that not only I believe that my chances of being found guilty is 10% but that the prosecutor whose has access to evidence that I don’t see also believes the chance is 10% is very helpful.
Then when I’m in the court room my lawyer can ask the prosecutor “Why did you bring that case when you only think that there’s a 10% chance that my client will be found guilty?”. Journalists could write story “Prosecutor Smith brings a case against Joe to trial even so he only believes that there’s a 10% chance that Joe would be convicted”.
That’s not good for the public reputation of prosecutor Smith and thus won’t charge Joe in the first place and has no leverage to get Joe to accept the plea deal.
The same way they are currently learning to evaluate the charges that the prosecutor makes. If the defense completley ignores the numbers nothing is lost over the status quo.
Rejecting a proposal because it doesn’t solve the problem you most care about it is bad. I agree that it would be good if the penal code would be changed and sentencing guidelines be reduced but that’s not the issue my post is addressing.
Sorry, then I think I made a bad assumption about why you were trying to solve the problem of overcharging. I added that at the end of my comment and probably shouldn’t have. Still, it’s not the reason I initially rejected the proposal. I honestly don’t think your proposal actually solves the overcharging problem or would make plea deals much fairer.
Journalists could write that, true. But if we’re relying on juries and voters to properly evaluate that kind of statistic as a reflection on the prosecutor’s job performance or the case’s merits, then hopefully they’ll also understand that the data point is immaterial to the question of whether the defendant is guilty. I can be very confident a policeman is guilty of murder, or a politician is guilty of accepting a bribe, or a frat boy is guilty of rape, and still correctly believe the probability of conviction is low. I’m not sure how a court would regard such an argument, but to this layman it seems that to whatever extent the percentage is based on the evidence, giving the number as an additional data point is double-counting evidence (or a way of introducing information based on evidence not presented at trial or inadmissible at trial, which is just as bad or worse), and to whatever degree it is based on assumptions about the judge and jury’s behavior and thinking, it’s speculation.
Overall I’m very skeptical that the enforcement mechanism you proposed to incentivize prosecutors to be honest is anywhere near strong enough. It might be better to scale the prosecutor’s pay to how well calibrated their estimates are, for those cases that make it to trial, with additional penalties like removal from office for being too far off. Or better yet, do a literal randomized trial where the prosecutor who offers the plea deal and makes the estimate is different from the prosecutor who goes to trial, and evaluate the former by accuracy and the latter by conviction rate.
The incentive it’s as strong as the desire for people who evaluate the prosecutors want it to be. You can also make it stronger by linking it to bonus payments if you want. If you do pay prosecutors any bonuses it would make sense to do that, but I think it’s generally better to pay prosecutors a fixed salary.
In any case, even if many prosecutors give crapy numbers those numbers can just be ignored by the defendent and produce no disadvantage over the status quo.
Evidence is by it’s nature very complex. If a journalist writes an article then a reader doesn’t know whether the journalist gives a fair representation of the issue at hand.
It takes a lot of effort to understand all the evidence that goes into all the cases where a prosecutor charges people to evaluate the prosecutor. If you have the prosecutor summarize the strength of the evidence that evaluation gets easier.
I don’t think you can easily do a trial because it’s a systematic intervention that needs to run a few years for people to adept to the new system before it leads it’s provides most of it’s benefits.
That’s true, but I think you’re being very optimistic, both in the ability of defendants and defense council to ignore or evaluate information the other side in an adversarial system claims is their true opinion, and in the ability and interest of the public in properly evaluating the job performance of prosecutors in local elections based on actual data. I think both are possible, and would be very valuable, but can’t be achieved without much deeper and broader reforms to make the underlying justice system more open, transparent, and trustworthy.
Sorry, I didn’t mean a trial as an experiment, I meant literally running legal trials this way, where in general the prosecutor that tries a case is not the one that produces the conviction probability estimate. Then, grade each both on the accuracy of their assessments, and separately on their conviction rates in trials they prosecute. I’d say either the one trying the case or a separate third prosecutor should have final say on which charges to bring. I think this would eliminate a lot of the potential for perverse incentives.
At would expect that in the beginning after the reform defendants and defense council would not trust the probability at all.
I would expect that trust in the numbers will only come when the system works well that they provide valid information.
Good political reforms aren’t about doing one thing but multiple things. There’s a reason why Obamacare is 906 pages.
On those points I completely agree.