Is there anything a non-famous non-billionaire person do to meaningfully impact medical research? It seems like the barriers to innovation are insurmountable to everyone with the will to try, and the very few organizations and people who might be able to aren’t dedicated to it.
Sure. I am not an MD, but this is my view as an outsider: the medical profession is quite conservative, and people that publish medical papers have an incentive to publish “fast and loose,” and not necessarily be very careful (because hey you can just write another paper later if a better method comes along!)
Because medicine deals with people, you often can’t do random treatment assignment in your studies (for ethical reasons), so you often have evidence in the form of observational data, where you have a ton of confounding of various types. Causal inference can help people make use of observational data properly. This is important—most data we have is observational. And if you are not careful, you are going to get garbage from it. For example, there was a fairly recent paper by Robins et al. that basically showed that their way of adjusting for confounding in observational data was correct because they reproduced a result found in an RCT.
There is room both for people to go to graduate school and work on new methods for properly dealing with observational data for drawing causal conclusions, and for popularizers.
Popularizing this stuff is a lot of what Pearl does, and is also partly the purpose of Hernan’s and Robin’s new book:
Here on Less Wrong there are a significant number of mathematically inclined software engineers who know some probability theory, meaning they’ve read/worked through at least one of Jaynes and Pearl but may not have gone to graduate school. How could someone with this background contribute to making causal inference more accessible to researchers? Any tools that are particularly under-developed or missing?
I am not sure I know what the most impactful thing to do is, by edu level. Let me think about it.
My intuition is the best thing for “raising the sanity waterline” is what the LW community would do with any other bias: just preaching association/causation to the masses that would otherwise read bad scientific reporting and conclude garbage about e.g. nutrition. Scientists will generally not outright lie, but are incentivized to overstate a bit, and reporters are incentivized to overstate a bit more. In general, we trust scientific output too much, so much of it is contingent on modeling assumptions, etc.
Explaining good clear examples of gotchas in observational data is good: e.g. doctors give sicker people a pill, so it might look like the pill is making people sick. It’s like the causality version of the “rare cancer ⇒ likely you have a false positive by Bayes theorem”. Unlike Bayes theorem, this is the kind of thing people immediately grasp if you point it out, because we have good causal processing natively, unlike our native probability processing which is terrible. Association/causation is just another type of bias to be aware of, it just happens to come up a lot when we read scientific literature.
If you are looking for some specific stuff to do as a programmer, email me :). There is plenty to do.
Yes depending on how you define “meaningful”. If you can speed up a cure for cancer by 1 day, or increase the chance of us getting a cure next year by 1 in a million then the expected value of this increase in terms of human welfare is huge compared to what most people manage to accomplish in their life.
It seems like the barriers to innovation are insurmountable to everyone with the will to try
Why?
When it comes to running studies, you could run a study for the effect of taking Vitamin D in the morning vs. in the evening. Various QS people have found an effect for themselves. The effect should be there. As far as I know there no study establishing the effect. Doesn’t seem to be difficult for someone without resources but with a decent amount of time on their hand.
As far as diagnostic tools go, we today have a lot of smart scales and blood pressure measurement devices.
As far as I know nobody produced a similarly smart peak flow meter (/ FEV1). Such a device would allow the gathering of more data for a lot of people. More and better data means more opportunities for scientific insight.
On a more theoretical level I think there could be advances in organizing the large pile of biological and medical knowledge we have. I don’t think that textbooks and journal articles are good mediums for transferring knowledge between scientists. Protein databases like uniprot contain a lot of information in a way that’s easy to query. I think that finding ways to organize less structured biological insights and the evidence for them is an area with a high potential impact.
Is there anything a non-famous non-billionaire person do to meaningfully impact medical research? It seems like the barriers to innovation are insurmountable to everyone with the will to try, and the very few organizations and people who might be able to aren’t dedicated to it.
Causal inference research.
Is the limiting factor here heads or funding?
Could you elaborate?
Sure. I am not an MD, but this is my view as an outsider: the medical profession is quite conservative, and people that publish medical papers have an incentive to publish “fast and loose,” and not necessarily be very careful (because hey you can just write another paper later if a better method comes along!)
Because medicine deals with people, you often can’t do random treatment assignment in your studies (for ethical reasons), so you often have evidence in the form of observational data, where you have a ton of confounding of various types. Causal inference can help people make use of observational data properly. This is important—most data we have is observational. And if you are not careful, you are going to get garbage from it. For example, there was a fairly recent paper by Robins et al. that basically showed that their way of adjusting for confounding in observational data was correct because they reproduced a result found in an RCT.
There is room both for people to go to graduate school and work on new methods for properly dealing with observational data for drawing causal conclusions, and for popularizers.
Popularizing this stuff is a lot of what Pearl does, and is also partly the purpose of Hernan’s and Robin’s new book:
https://www.facebook.com/causalinference
Full disclosure: obviously this is my area, so I am going to say that. So don’t take my word for it :).
Here on Less Wrong there are a significant number of mathematically inclined software engineers who know some probability theory, meaning they’ve read/worked through at least one of Jaynes and Pearl but may not have gone to graduate school. How could someone with this background contribute to making causal inference more accessible to researchers? Any tools that are particularly under-developed or missing?
I am not sure I know what the most impactful thing to do is, by edu level. Let me think about it.
My intuition is the best thing for “raising the sanity waterline” is what the LW community would do with any other bias: just preaching association/causation to the masses that would otherwise read bad scientific reporting and conclude garbage about e.g. nutrition. Scientists will generally not outright lie, but are incentivized to overstate a bit, and reporters are incentivized to overstate a bit more. In general, we trust scientific output too much, so much of it is contingent on modeling assumptions, etc.
Explaining good clear examples of gotchas in observational data is good: e.g. doctors give sicker people a pill, so it might look like the pill is making people sick. It’s like the causality version of the “rare cancer ⇒ likely you have a false positive by Bayes theorem”. Unlike Bayes theorem, this is the kind of thing people immediately grasp if you point it out, because we have good causal processing natively, unlike our native probability processing which is terrible. Association/causation is just another type of bias to be aware of, it just happens to come up a lot when we read scientific literature.
If you are looking for some specific stuff to do as a programmer, email me :). There is plenty to do.
Yes depending on how you define “meaningful”. If you can speed up a cure for cancer by 1 day, or increase the chance of us getting a cure next year by 1 in a million then the expected value of this increase in terms of human welfare is huge compared to what most people manage to accomplish in their life.
Become a famous billionaire?
Or just a billionaire, or (in a pinch) just very famous.
Why?
When it comes to running studies, you could run a study for the effect of taking Vitamin D in the morning vs. in the evening. Various QS people have found an effect for themselves. The effect should be there. As far as I know there no study establishing the effect. Doesn’t seem to be difficult for someone without resources but with a decent amount of time on their hand.
As far as diagnostic tools go, we today have a lot of smart scales and blood pressure measurement devices. As far as I know nobody produced a similarly smart peak flow meter (/ FEV1). Such a device would allow the gathering of more data for a lot of people. More and better data means more opportunities for scientific insight.
On a more theoretical level I think there could be advances in organizing the large pile of biological and medical knowledge we have. I don’t think that textbooks and journal articles are good mediums for transferring knowledge between scientists. Protein databases like uniprot contain a lot of information in a way that’s easy to query. I think that finding ways to organize less structured biological insights and the evidence for them is an area with a high potential impact.