Alzheimer’s Disease (AD) is truly, unduly cruel, and truly, unduly common. A huge amount of effort goes into curing it, which I think is a true credit to our civilization. This is in the form of both money, and the efforts of many of the brightest researchers.
But it hasn’t worked.
Since AD is characterised by amyloid plaques, the “amyloid hypothesis” that these were the causative agent has been popular for a while. Mutations to genes which encode the amyloid beta protein can cause AD. Putting lots of amyloid into the brain causes brain damage in mice. So for many years, drugs were screened by testing them in mutant mice which were predisposed to AD. If the plaques disappeared, they were considered good candidates.
So why didn’t it work?
Lots of things can affect amyloid plaques as it turns out, right up to the latest FDA approved drug, which is just antibodies which target amyloid protein. While this does reduce amyloid, it has no effect on cognitive decline.
Goodhart’s law has reared its head: amyloid plaque buildup is a metric for AD progression, but selecting for drugs which reduce it causes the relationship between AD and plaques to fall apart.
Equally, amyloid plaques are very easy to measure in mouse (and human) brains. It can be done by MRI scan, or by dissection. Memory loss and mood changes are harder to measure, and even harder in mice. The methods for measuring amyloid plaques also feel better in many ways. There’s less variation in potential methods, they can be compared across species, they’re qualitative, and they’re also more in line with what the average biologist/chemist will be used to.
Understanding these, we can see how looking for drugs which decrease amyloid plaques in mice just really feels like productive research. We can also understand, now, why it wasn’t.
Avoiding Wasted Effort
Pointing out biases is fairly useless. Pointing out specific examples is better. But the best way to help others is to point out how it feels from the inside to be making these mistakes.
So what does it feel like to be on the inside of these biases? Unfortunately as someone who has not been intimately involved in AD research I can’t say exactly. But as someone involved with research in general I can make a guess:
Research will feel mostly productive. It may feel like you are becoming how you imagine a researcher to be. Papers will be published. This is because you’re in the streetlight.
What you won’t feel is a sense of building understanding. Learning to notice a lack of understanding is one of the most important skills, and it is sadly not an easy thing to explain.
Think about the possible results of your experiments. Do you expect something you’ve not seen before? Or do you expect a result with a clear path to success? Creative work usually passes the first. Well-established and effective protocols pass the second. Mouse AD models do not pass either (anymore).
A positive experimental result will be much easier than a “true” success. This has the benefit (for researchers) of allowing you to seem successful without actually doing good. The ratio of AD papers to AD cures is 1:0 (“Alzheimer’s Disease Treatment” returns 714,000 results in Google Scholar)
Beyond this I do not know. Perhaps it is a nameless virtue. But it might be useful to try to identify more cases. I hereby precommit to posting a follow-up with at least five examples of this within the next seven days.
Amyloid Plaques: Chemical Streetlight, Medical Goodhart
Alzheimer’s Disease (AD) is truly, unduly cruel, and truly, unduly common. A huge amount of effort goes into curing it, which I think is a true credit to our civilization. This is in the form of both money, and the efforts of many of the brightest researchers.
But it hasn’t worked.
Since AD is characterised by amyloid plaques, the “amyloid hypothesis” that these were the causative agent has been popular for a while. Mutations to genes which encode the amyloid beta protein can cause AD. Putting lots of amyloid into the brain causes brain damage in mice. So for many years, drugs were screened by testing them in mutant mice which were predisposed to AD. If the plaques disappeared, they were considered good candidates.
So why didn’t it work?
Lots of things can affect amyloid plaques as it turns out, right up to the latest FDA approved drug, which is just antibodies which target amyloid protein. While this does reduce amyloid, it has no effect on cognitive decline.
Goodhart’s law has reared its head: amyloid plaque buildup is a metric for AD progression, but selecting for drugs which reduce it causes the relationship between AD and plaques to fall apart.
Equally, amyloid plaques are very easy to measure in mouse (and human) brains. It can be done by MRI scan, or by dissection. Memory loss and mood changes are harder to measure, and even harder in mice. The methods for measuring amyloid plaques also feel better in many ways. There’s less variation in potential methods, they can be compared across species, they’re qualitative, and they’re also more in line with what the average biologist/chemist will be used to.
Understanding these, we can see how looking for drugs which decrease amyloid plaques in mice just really feels like productive research. We can also understand, now, why it wasn’t.
Avoiding Wasted Effort
Pointing out biases is fairly useless. Pointing out specific examples is better. But the best way to help others is to point out how it feels from the inside to be making these mistakes.
So what does it feel like to be on the inside of these biases? Unfortunately as someone who has not been intimately involved in AD research I can’t say exactly. But as someone involved with research in general I can make a guess:
Research will feel mostly productive. It may feel like you are becoming how you imagine a researcher to be. Papers will be published. This is because you’re in the streetlight.
What you won’t feel is a sense of building understanding. Learning to notice a lack of understanding is one of the most important skills, and it is sadly not an easy thing to explain.
Think about the possible results of your experiments. Do you expect something you’ve not seen before? Or do you expect a result with a clear path to success? Creative work usually passes the first. Well-established and effective protocols pass the second. Mouse AD models do not pass either (anymore).
A positive experimental result will be much easier than a “true” success. This has the benefit (for researchers) of allowing you to seem successful without actually doing good. The ratio of AD papers to AD cures is 1:0 (“Alzheimer’s Disease Treatment” returns 714,000 results in Google Scholar)
Beyond this I do not know. Perhaps it is a nameless virtue. But it might be useful to try to identify more cases. I hereby precommit to posting a follow-up with at least five examples of this within the next seven days.