There are reasons to be skeptical of any claim based on correlations between such widely
separated variables as lead exposure (the cause) and crime (the effect). Consuming lead does
not instantly turn someone into a criminal in the way that consuming vitamin C cures scurvy. It
affects the child’s developing brain, which makes the child duller and more impulsive, which, in
some children, and under the right circumstances, leads them to grow up to make short-sighted
and risky choices, which, in some children and under the right circumstances, leads them to
commit crimes, which, if enough young people act in the same way and at the same time,
affects the crime rate. The lead hypothesis correlates the first and last link in this chain, but it
would be more convincing if there were evidence about the intervening links. Such correlations
should be far stronger than the one they report: presumably most kids with lead are more
impulsive, whereas only a minority of impulsive young adults commit crimes. If they are right
we should see very strong changes in IQ, school achievement, impulsiveness, childhood
aggressiveness, lack of conscientiousness (one of the “Big Five” personality traits) that mirror
the trends in lead exposure, with a suitable time delay. Those trends should be much stronger
than the time-lagged correlation of lead with crime itself, which is only indirectly related to
impulsiveness, an effect that is necessarily diluted by other causes such as policing and
incarceration. I am skeptical that such trends exist, though I may not be aware of such studies.
...
Also, the parallelism in curves for lead and time-shifted crime seem too good to be true, since
the lead hypothesis assumes that the effects of lead exposure are greatest in childhood. But 23
years after the first lower-lead cohort, only a small fraction of the crime-prone cohort should be
lead-free; there are still all those lead-laden young adults who have many years of crime ahead
of them. Only gradually should the crime-prone demographic sector be increasingly populated
by lead-free kids. The time-shifted curve for crime should be an attenuated, smeared version of
the curve for lead, not a perfect copy of it. Also, the effects of age on crime are not sharply
peaked, with a spike around the 23rd birthday, and a sharp falloff—it’s a very gentle bulge
spread out over the 15-30 age range. So you would not expect such a perfect time-shifted
overlap as you might, for example, for first-grade reading performance, where the
measurement is so restricted in time.
Finally, the most general reason for skepticism about a causal hypothesis based on
epidemiological correlations between a widely separated cause and effect is that across times
and places, many things tend to go together. Neighborhoods next to smoggy freeways also
tend to be poorer, more poorly policed, more poorly schooled, less stable, more dependent on
contraband economies, and so on. It’s all too easy to find spurious correlations in this tangle –
which is why so many epidemiological studies of the cause and prevention of disease (this gives
you cancer; that prevents it) fail to replicate.
It sounds like Pinker is unfamiliar with most of the research on the topic. He refers to this graph, but not any of the research on the various other predictions that you could derive from the hypothesis that lead caused much of the hump in crime over the past 60 years.
The things that he says about priors, and about the sorts of research that he’d like to see, sound plausible, but I wouldn’t put much stock in what he says about the state of the research.
He refers to this graph, but not any of the research on the various other predictions that you could derive from the hypothesis that lead caused much of the hump in crime over the past 60 years.
Can you name some of these predictions? Can you link to some of the research? What exactly are you referring to?
3 predictions that I came up with, when I heard about the hypothesis:
The lead hypothesis predicts a cohort effect: the crime rate for 35-year-olds should drop 10 years after the crime rate drops for 25-year-olds. Many competing hypotheses (like new policing tactics) predict a cross-sectional effect: the crime rate for 35-year-olds drops at the same time that the crime rate drops for 25-year-olds. (Although there may be feedback effects which cause some smudging of this sharp distinction, e.g. more crime among a subgroup means that police are spread thinner which makes crime more attractive for everyone.)
The lead hypothesis makes pretty specific predictions about differences in the timing of the crime drop across different regions. If one jurisdiction removes lead 8 years after another jurisdiction, then their crime rate should drop 8 years later.
The lead hypothesis predicts differences in the size of the crime increase & drop across different regions. If one region had more environmental lead than another, then it should have both a larger increase in crime during the “increasing crime rates” time period and a larger drop in crime during the “declining crime rates” time period.
I looked at one of Nevin’s papers shortly after the Drum piece originally came out and it had some evidence for all 3 predictions, though not with as much rigor/precision/detail as I would’ve liked. For example, on prediction #3 it compared larger cities (which had more driving per unit area, and thus more environmental lead due to gasoline) to smaller cities and showed that their crime rates matched this pattern.
There is also research on other steps in the long chain (e.g., measuring blood levels of lead), and on other outcomes attributed to lead (e.g., teen pregnancy rates), some of which is mentioned in Drum’s original piece. I haven’t looked into that research beyond what I’ve seen in the popular press articles.
I think the paper that I looked at was The Answer is Lead Poisoning. Mainly just looking at the graphs & tables.
The city size pattern is not a unique prediction of the lead hypothesis (there are various other differences between large & small cities which could account for it, though nothing that strikes me as overwhelmingly obvious), but it is a relatively unambiguous prediction (especially if there’s high quality data on city size vs. environmental lead levels—I’m not sure how good those data are). If large vs. small cities turned out not to have this difference in crime trends then that would be pretty strong evidence against the lead hypothesis, so the fact that the comparison did come out this way must be at least some evidence in favor of the lead hypothesis.
Also, the parallelism in curves for lead and time-shifted crime seem too good to be true, since the lead hypothesis assumes that the effects of lead exposure are greatest in childhood. But 23 years after the first lower-lead cohort, only a small fraction of the crime-prone cohort should be lead-free; there are still all those lead-laden young adults who have many years of crime ahead of them. Only gradually should the crime-prone demographic sector be increasingly populated by lead-free kids. The time-shifted curve for crime should be an attenuated, smeared version of the curve for lead, not a perfect copy of it. Also, the effects of age on crime are not sharply peaked, with a spike around the 23rd birthday, and a sharp falloff—it’s a very gentle bulge spread out over the 15-30 age range. So you would not expect such a perfect time-shifted overlap as you might, for example, for first-grade reading performance, where the measurement is so restricted in time.
Am I misreading this, or is this suggesting that the fact that the time-shifted correlation is unusually strong should be taken as evidence -against- the correlation?
What Pinker is saying is that P(data | lead causes crime) is not as high as you’d think, because if lead really does cause crime, we should not expect the crime curve to be a time-shifted version of the lead curve. It’s probably still true that P(data | lead causes crime) > P(data), so that you should update in the direction of lead causes crime, but this update should probably be smaller than you thought before reading that paragraph.
Has anyone figured out what crime curve you would expect based on the lead curve (presumably a version that is shifted & smeared out based on the age distribution of criminals), and checked how well it fits the actual crime data? It’s not obvious to me, from looking at the pictures that I’ve seen with the shifted curves, that adding the smearing would make the fit worse. For instance, the graph I linked earlier shows that the recent drop in crime is more gradual than the drop in lead that happened 20-30 years ago, which seems to fit the more rigorous “time-shifted and smeared out” prediction better than it fits the simplistic time-shifted curve approach that Nevin used.
Meh, probably not:
http://stevenpinker.com/files/pinker/files/pinker_comments_on_lead_removal_and_declining_crime.pdf
...
It sounds like Pinker is unfamiliar with most of the research on the topic. He refers to this graph, but not any of the research on the various other predictions that you could derive from the hypothesis that lead caused much of the hump in crime over the past 60 years.
The things that he says about priors, and about the sorts of research that he’d like to see, sound plausible, but I wouldn’t put much stock in what he says about the state of the research.
Can you name some of these predictions? Can you link to some of the research? What exactly are you referring to?
3 predictions that I came up with, when I heard about the hypothesis:
The lead hypothesis predicts a cohort effect: the crime rate for 35-year-olds should drop 10 years after the crime rate drops for 25-year-olds. Many competing hypotheses (like new policing tactics) predict a cross-sectional effect: the crime rate for 35-year-olds drops at the same time that the crime rate drops for 25-year-olds. (Although there may be feedback effects which cause some smudging of this sharp distinction, e.g. more crime among a subgroup means that police are spread thinner which makes crime more attractive for everyone.)
The lead hypothesis makes pretty specific predictions about differences in the timing of the crime drop across different regions. If one jurisdiction removes lead 8 years after another jurisdiction, then their crime rate should drop 8 years later.
The lead hypothesis predicts differences in the size of the crime increase & drop across different regions. If one region had more environmental lead than another, then it should have both a larger increase in crime during the “increasing crime rates” time period and a larger drop in crime during the “declining crime rates” time period.
I looked at one of Nevin’s papers shortly after the Drum piece originally came out and it had some evidence for all 3 predictions, though not with as much rigor/precision/detail as I would’ve liked. For example, on prediction #3 it compared larger cities (which had more driving per unit area, and thus more environmental lead due to gasoline) to smaller cities and showed that their crime rates matched this pattern.
There is also research on other steps in the long chain (e.g., measuring blood levels of lead), and on other outcomes attributed to lead (e.g., teen pregnancy rates), some of which is mentioned in Drum’s original piece. I haven’t looked into that research beyond what I’ve seen in the popular press articles.
Do you think you could link to that paper?
A Larger vs. smaller cities comparison sounds like it has ample room for confounding factors, no?
The other outcomes attributed to lead sound like they correlate with crime rates, so this isn’t independent evidence for the lead hypothesis.
I think the paper that I looked at was The Answer is Lead Poisoning. Mainly just looking at the graphs & tables.
The city size pattern is not a unique prediction of the lead hypothesis (there are various other differences between large & small cities which could account for it, though nothing that strikes me as overwhelmingly obvious), but it is a relatively unambiguous prediction (especially if there’s high quality data on city size vs. environmental lead levels—I’m not sure how good those data are). If large vs. small cities turned out not to have this difference in crime trends then that would be pretty strong evidence against the lead hypothesis, so the fact that the comparison did come out this way must be at least some evidence in favor of the lead hypothesis.
Am I misreading this, or is this suggesting that the fact that the time-shifted correlation is unusually strong should be taken as evidence -against- the correlation?
Suppose you’re Bayesian, and you’re calculating
P(lead causes crime | data) = P(data | lead causes crime) * P(lead causes crime) / P(data).
What Pinker is saying is that P(data | lead causes crime) is not as high as you’d think, because if lead really does cause crime, we should not expect the crime curve to be a time-shifted version of the lead curve. It’s probably still true that P(data | lead causes crime) > P(data), so that you should update in the direction of lead causes crime, but this update should probably be smaller than you thought before reading that paragraph.
Has anyone figured out what crime curve you would expect based on the lead curve (presumably a version that is shifted & smeared out based on the age distribution of criminals), and checked how well it fits the actual crime data? It’s not obvious to me, from looking at the pictures that I’ve seen with the shifted curves, that adding the smearing would make the fit worse. For instance, the graph I linked earlier shows that the recent drop in crime is more gradual than the drop in lead that happened 20-30 years ago, which seems to fit the more rigorous “time-shifted and smeared out” prediction better than it fits the simplistic time-shifted curve approach that Nevin used.