I agree that the highest-leverage place to start is probably the paradigm of encouraging people with obvious long-lasting chronic problems to look for immediate obvious effects by doing maximal spray n pray. Equivalents of “have nausea from cancer/chemo daily → medical marijuana → no more problems.”
Once we get away from that, I think that systems for collection, analysis, and self-blinding become important. There are a lot of details and trivial inconveniences in any research project, and most people just aren’t equipped to work them out on their own. There’s a lot you can do to smooth the path.
For example, I can imagine a nootropics test kit. It would come with:
A standardized questionnaire that you fill out for every nootropic you try
A sample of nootropics to try along with placebos. Placebos would mimic the appearance of various drugs so it’s impossible to tell which is which without deliberately unblinding yourself. the supply would be large enough to give you adequate power given the number of drugs you’re trying.
An analytic framework that takes multiple comparisons etc. into account and lets you see if any correlations are statistically significant.
Perhaps packaging drugs in different ways so that you can order more of the things that work, but with a different appearance, to do a more focused experiment on the likeliest candidates.
There’s a lot of detail to work out in designing such a kit, but it’s easy for me to see that it could convert an intractable problem into a do-able puzzle for a motivated and reasonably intelligent user.
An analytic framework that takes multiple comparisons etc. into account and lets you see if any correlations are statistically significant.
Blinding.
Two issues, one of which I did not think of, out of like 20.
EDIT: I suspect, including from my own experience, that many problems can be solved without resorting to advanced statistics. Often by using through experimental procedure instead. Like eliminating a food type for a month then not doing an intervention for a month. Repeat. Trying out medications sounds like it should be done safely. This safety can only be achieved by monitoring vital signs and analyzing them using advanced statistics.
I agree that the highest-leverage place to start is probably the paradigm of encouraging people with obvious long-lasting chronic problems to look for immediate obvious effects by doing maximal spray n pray. Equivalents of “have nausea from cancer/chemo daily → medical marijuana → no more problems.”
Once we get away from that, I think that systems for collection, analysis, and self-blinding become important. There are a lot of details and trivial inconveniences in any research project, and most people just aren’t equipped to work them out on their own. There’s a lot you can do to smooth the path.
For example, I can imagine a nootropics test kit. It would come with:
A standardized questionnaire that you fill out for every nootropic you try
A sample of nootropics to try along with placebos. Placebos would mimic the appearance of various drugs so it’s impossible to tell which is which without deliberately unblinding yourself. the supply would be large enough to give you adequate power given the number of drugs you’re trying.
An analytic framework that takes multiple comparisons etc. into account and lets you see if any correlations are statistically significant.
Perhaps packaging drugs in different ways so that you can order more of the things that work, but with a different appearance, to do a more focused experiment on the likeliest candidates.
There’s a lot of detail to work out in designing such a kit, but it’s easy for me to see that it could convert an intractable problem into a do-able puzzle for a motivated and reasonably intelligent user.
Two issues, one of which I did not think of, out of like 20.
EDIT: I suspect, including from my own experience, that many problems can be solved without resorting to advanced statistics. Often by using through experimental procedure instead. Like eliminating a food type for a month then not doing an intervention for a month. Repeat. Trying out medications sounds like it should be done safely. This safety can only be achieved by monitoring vital signs and analyzing them using advanced statistics.