This is actually a pretty abysmal failure of an epistemic spot check.
For people not paying close attention: I believe magfrump is saying that they failed to check the claims as thoroughly as intended, not that the claims were false; those numbers do broadly agree with the post. (I misread it the first time through and was very confused about why the claims were supposedly failing an epistemic spot check when the numbers were generally consistent with the post’s claims.)
That is not what I intended, and I think we disagree here, though probably I don’t endorse calling it an abysmal failure.
in the original post you claim that total prison populations were about 5k, though they were noticably more for a long time. This is a small point.
you claim that prison populations ballooned in the 1960s. According to the sources linked, prison populations grew consistently from 1945ish-1968ish. Certainly they grew to a larger size than ever before, but it’s not at all clear where in that range the escape from dunbar’s number would occur (why 25k, why not 10k? What are individual prison populations is something I didn’t see). Maybe there was a more gradual process of displacement or a series of specific incidences of failures of the code system? But this seems like an important missing piece. I went in looking for a specific phase change in prison population and what I found was a phase change in prison growth, starting 25 years earlier.
you say prison populations reached 25k in 1970, then that they are currently 170k. These numbers are reasonably accurate, but misleading, suggesting smooth growth from 1970 to the present. 25k was the high water mark and prisons then dropped in population for a decade before REALLY exploding in the 1980s. That seems like the point where I’d expect a phase change in systems to be necessary, or at least to be a serious challenge of how well the gang based conflict resolution methods scale.
Overall I went into the search expecting to look at a graph and see “here’s the obvious phase change” and have that timeline match up, and for that change to be over a short period of time. I saw a clear phase change at a completely different set of dates from those outlined in the post, and that makes me skeptical. Looking at these in detail (instead of trying to summarize a slow and somewhat frustrating search before going to bed) I don’t think the points are terribly damaging to the narrative of the article but they do leave me with more questions.
I’ll edit my original comment to remove the “abysmal failure” part, now that it accurately describes my spot check.
Ah, I see. To draw an analogy: if you boil water, there is not a big upward jump in temperature at the point where the water turns to steam. Likewise here: the prison population is like temperature. I would not particularly expect a large upward jump in the population, right around the time of the phase change.
If there were a large upward jump in population during some time, then that would be a more-than-usually-likely time to cross phase-change thresholds, just as a large jump in temperature is likely to cross phase-change thresholds. But the converse does not apply: a phase change can occur even when the underlying parameter is changing slowly. So while this is Bayesian evidence against the model on net, it is quite weak.
(I do think the big jump in prison population in the 80′s is indicative of a phase change in a different system somewhere upstream of imprisonment, but I think that’s largely a separate phenomenon from that discussed in the post. Continuing the analogy: if we see a sharp jump in temperature, that’s probably indicative of something sharply changing in the system, but not necessarily water boiling.)
Anyway, I did just go back and re-check the relevant chapter of Legal Systems Very Different From Ours, and I do think my summary was wrong on the timeline: it says that there has been a dramatic increase in gangs and gang members “since the 60′s”, with the “Code era” covering roughly the first half of the twentieth century. So I will definitely edit that. Thanks again for the check!
(Note that the timeline from the book is a bit more consistent with what you expected, though I still maintain that we shouldn’t put particularly high prior on a big population jump around the time of the phase change. Weak evidence, either way.)
Here’s the graph on that wikipedia page:
Thanks for finding these!
Forpeople not paying close attention: I believe magfrump is saying that they failed to check the claims as thoroughly as intended, not that the claims were false; those numbers do broadly agree with the post. (I misread it the first time through and was very confused about why the claims were supposedly failing an epistemic spot check when the numbers were generally consistent with the post’s claims.)That is not what I intended, and I think we disagree here, though probably I don’t endorse calling it an abysmal failure.
in the original post you claim that total prison populations were about 5k, though they were noticably more for a long time. This is a small point.
you claim that prison populations ballooned in the 1960s. According to the sources linked, prison populations grew consistently from 1945ish-1968ish. Certainly they grew to a larger size than ever before, but it’s not at all clear where in that range the escape from dunbar’s number would occur (why 25k, why not 10k? What are individual prison populations is something I didn’t see). Maybe there was a more gradual process of displacement or a series of specific incidences of failures of the code system? But this seems like an important missing piece. I went in looking for a specific phase change in prison population and what I found was a phase change in prison growth, starting 25 years earlier.
you say prison populations reached 25k in 1970, then that they are currently 170k. These numbers are reasonably accurate, but misleading, suggesting smooth growth from 1970 to the present. 25k was the high water mark and prisons then dropped in population for a decade before REALLY exploding in the 1980s. That seems like the point where I’d expect a phase change in systems to be necessary, or at least to be a serious challenge of how well the gang based conflict resolution methods scale.
Overall I went into the search expecting to look at a graph and see “here’s the obvious phase change” and have that timeline match up, and for that change to be over a short period of time. I saw a clear phase change at a completely different set of dates from those outlined in the post, and that makes me skeptical. Looking at these in detail (instead of trying to summarize a slow and somewhat frustrating search before going to bed) I don’t think the points are terribly damaging to the narrative of the article but they do leave me with more questions.
I’ll edit my original comment to remove the “abysmal failure” part, now that it accurately describes my spot check.
Ah, I see. To draw an analogy: if you boil water, there is not a big upward jump in temperature at the point where the water turns to steam. Likewise here: the prison population is like temperature. I would not particularly expect a large upward jump in the population, right around the time of the phase change.
If there were a large upward jump in population during some time, then that would be a more-than-usually-likely time to cross phase-change thresholds, just as a large jump in temperature is likely to cross phase-change thresholds. But the converse does not apply: a phase change can occur even when the underlying parameter is changing slowly. So while this is Bayesian evidence against the model on net, it is quite weak.
(I do think the big jump in prison population in the 80′s is indicative of a phase change in a different system somewhere upstream of imprisonment, but I think that’s largely a separate phenomenon from that discussed in the post. Continuing the analogy: if we see a sharp jump in temperature, that’s probably indicative of something sharply changing in the system, but not necessarily water boiling.)
Anyway, I did just go back and re-check the relevant chapter of Legal Systems Very Different From Ours, and I do think my summary was wrong on the timeline: it says that there has been a dramatic increase in gangs and gang members “since the 60′s”, with the “Code era” covering roughly the first half of the twentieth century. So I will definitely edit that. Thanks again for the check!
(Note that the timeline from the book is a bit more consistent with what you expected, though I still maintain that we shouldn’t put particularly high prior on a big population jump around the time of the phase change. Weak evidence, either way.)
(That is a good cite of Sarah’s blogpost. I might use it like that in future.)