Your argument assumes that the algorithm and the prisons have access to the same data. This need not be the case—in particular, if a prison bribes a judge to over-convict, the algorithm will be (incorrectly) relying on said conviction as data, skewing the predicted recidivism measure.
That said, the perverse incentive you mentioned is absolutely in play as well.
Yes, I glossed over the possibility of prisons bribing judges to screw up the data set. That’s because the extremely small influence of marginal data points and the cost of bribing judges would make such a strategy incredibly expensive.
Your argument assumes that the algorithm and the prisons have access to the same data. This need not be the case—in particular, if a prison bribes a judge to over-convict, the algorithm will be (incorrectly) relying on said conviction as data, skewing the predicted recidivism measure.
That said, the perverse incentive you mentioned is absolutely in play as well.
Yes, I glossed over the possibility of prisons bribing judges to screw up the data set. That’s because the extremely small influence of marginal data points and the cost of bribing judges would make such a strategy incredibly expensive.