At some point, I’d like to consider whether there’s value in developing a protocol + software + device for facilitating personalized randomly controlled trials beyond whatever is the Pareto frontier for current enthusiasts
I know someone who’s working on this and I love the idea, although thus far his app is absolutely useless for something that doesn’t kick in and wear off the same day and I haven’t verified it for that use case either. I also have an entrepreneur client whose very interested in this and would plausibly provide support to someone working on it, but it’s not his own top priority.
Let’s say your problem is reliable and obvious. Get a sample pack of pills and remedies. Take 1 pill at a time. If it’s a miracle cure, keep taking it. If not, try the next one. No fancy stuff required. Does anyone sell a sample pack like this?
Tl;dr
If your mystery malady is obvious, reliable, and rare in the general population, then you can simply try pills one at a time, one pill at a time, to see if they provide a reliable treatment. If so, then this will become apparent right away, and you can keep taking it a few more times to confirm it works reliably. It should only take a few trials to show the pill works for you. If the pill doesn’t work, you can set it aside after one try and try the next one.
This would not help you if the problem is a deficiency or excess of some compound that takes time to build up. It would only help if the pill had an immediate, obvious effect for your condition—a “miracle cure.”
This approach requires no fancy statistics, devices, or complexities of taking multiple pills at once. The easiest way to overcome the bottleneck would be if a company sold “sample packs” containing 1 pill of eat type. You’d take them one at a time, see if they fixed the problem, and if they did, you’d buy a larger quantity of that specific pill and keep experimenting to confirm the result.
Long:
I could imagine a fruitful collaboration between a mechanical engineer, a statistician, a programmer, and a biomedical researcher/doctor. The approach could be Bayes’ law-based. Using your problem as an example, here’s how it might work:
Does eating meat (cooked, store-bought, fresh, 50g) make me nauseous?
P(H): You specify how common it is in the human population for eating meat to cause nausea
P(E): You also specify how common it is for a member of the human population to eat meat and get nauseous, whether or not there’s a causal relationship
P(E|H): Finally, you specify how likely it would be for you to get nauseous after eating meat if eating meat makes you nauseous.
If you don’t have a good way to estimate these likelihoods, the app would give you suggestions for ways you could find reasonable numbers to plug in.
Then, every time you ate meat and got nauseous, you could enter another data point and it would update the likelihood that eating meat causes nausea.
Once you’re adequately confident that eating meat causes nausea, you could then repeat the exercise for a range of herbs.
Given that eating meat causes nausea, does taking Boswelia (BosPro) before eating prevent nausea from meat consumption?
P(H): You specify how common it is for people who get nauseous after eating meat to not get nauseous if they take Boswelia first.
P(E): You also specify how common it is for people who get nauseous from eating meat to take Boswelia, then eat meat, and not get nauseous.
P(E|H): Finally, you specify how likely it would be for you to not get nauseous after taking Boswelia, then eating meat, if taking Boswelia prevents nausea from meat consumption.
With a very reliable and unusual effect, as in your case, it’s possible to update a very low prior to a very confident prior in just a few trials.
I think the statistics would be doing most of the heavy lifting here. Being able to do a blinded placebo-controlled trial is important if you’re testing a relatively subtle/sporadic effect or testing a single compound.
But if you’re testing an obvious, reliable effect and experimenting with a range of compounds, I don’t think the self-blinding aspect is as important.
Here’s my attempt (not trained in this, correct me if I’m going about it wrong) to apply Bayesian inference to the problem.
For example, let’s say:
P(H), the chance that meat gives you nausea, is one in a million (.000001)
P(E), the likelihood of you just randomly feeling nauseous after eating meat, is one in a thousand (.001)
P(E|H), the likelihood you feel nauseous after eating meat, given that the meat causes you nausea, is 0.98
Then P(H|E), the chance that eating meat causes your nausea, given you ate meat and felt nauseous afterward, is 0.00098 after one time eating meat and getting nauseous. Feeding this back in as the new P(H), the chance that meat causes your nausea, P(H|E) becomes .9604, or 96.04%, the second time this happens (assuming you don’t eat meat and not feel nauseous). Realistically, we’d want to check if there’s something about the way you cooked the meat, the animal it comes from, the store you bought it from, the knowledge that you’re consuming meat, etc, that is confounding the result.
To deal with some of the confounds, you could repeat the experiment using different types of meat from different stores, and figure out how robust the effect is. It seems difficult to self-blind eating meat, since 50 g is about a sandwich’s worth of deli meat. Possibly you could try incorporating it into another more flavorful dish and eating the food with your eyes closed. But this is just to rule out the possibility that the result is from the perception of eating meat, as opposed to something primarily physiological about meat-material being in your digestive tract. If you weren’t concerned about this possibility, you could skip it.
But if the user is self-testing to find a treatment for a very reliable, obvious effect (as with your meat->nausea problem), and if our prior on any random pill just randomly working to prevent the nausea is low, then these results show you can probably figure out whether any random supplement works or not in just a few tries. That sort of obviates the need for fancy statistics or an app. You might be able to get much of the same result by just directing the user to “Try taking this pill, then eating 50 g meat. If the pill doesn’t work on the first try, move on to the next pill. If it does work, continue taking it. If it helps but doesn’t entirely eliminate the problem, explore dosing, timing, and manufacturer, to see if you can achieve significant improvements.”
I’m not sure if it makes sense to try multiple pills at once. Balancing the possibility that you stumble upon an effective treatment faster is the possibility that an effective treatment is disguished by a drug interaction, as well as the possibility that the average user just struggles to manage the added complexity, gets confused, and quits.
But this then becomes very simple.
If you have a reliable, obvious mystery malady that no doctor has treated effectively, systematically take 1 pill at a time until you find one with an effect. Then keep taking it, exploring dosage, timing, and manufacturer, until you maximize the effect. If satisfied, problem solved. If not fully satisfied, continue exploring other pills (unless it’s possible to devise a mechanistic hypothesis as to why the effective pill works, then use that to find likely alternatives more efficiently).
TBC: the client is interested scaleable software that people intuitively find useful for experimenting on themselves and can combine selfishly motivated individual data into useful aggregate data. But there’s a point here I want to argue.
I’m not sure if it makes sense to try multiple pills at once. Balancing the possibility that you stumble upon an effective treatment faster is the possibility that an effective treatment is disguished by a drug interaction, as well as the possibility that the average user just struggles to manage the added complexity, gets confused, and quits.
It seems like you’re assuming all drug interactions are bad. What if you need three things in combination to see the effect, and then it works really well?
That’s entirely possible. However, it introduces a level of risk that isn’t as present when you take one at a time. It’s unlikely that a given supplement will kill you at normal doses. Such a severe side effect would have been detected and the supplement most likely wouldn’t be sold.
But it’s not impossible that certain rare combinations of drugs might have lethal side effects in combination, and that you might stumble across such a combination by chance. The risk of this happening seems in my judgment to outweigh the potential benefit of finding the right drug for a chronic condition faster.
I’m not certain if it’s plausible an interaction between supplements could kill you in a few days. Even the examples here seem to take a while to work. The only potentially instantly lethal drugs I’ve seen are things like alcohol and heroine. But I would shy away from taking combinations—that’s just where my risk budget is at. I could be persuaded otherwise.
If you’re concerned about supplement interactions, you should be concerned about supplements effects at all. People should seek based on possible luck, but should know (and be able to tell doctors) what they’re putting into their bodies, and take contextual advice from relatives, friends, and community.
Collecting individuals susceptible to effective treatment with intervention X and having them ready for researchers to talk to, test more directly—is this already done? I suppose around individual notable conditions: celiac disease and gluten intolerance, or n=1 genetic issue self-diagnosis.
Some conditions have an intermittency, that makes it hard to assess interventions with unknown timing.
Perhaps blinded timing studies, self-studies, after something is found to work. Perhaps helping people to log and journal symptoms and effects during the blinded periods, as well as analyze, interpret, and share them—especially in ways that make it easier for others to trust.
I know someone who’s working on this and I love the idea, although thus far his app is absolutely useless for something that doesn’t kick in and wear off the same day and I haven’t verified it for that use case either. I also have an entrepreneur client whose very interested in this and would plausibly provide support to someone working on it, but it’s not his own top priority.
Tl;dr is tl;dr
Let’s say your problem is reliable and obvious. Get a sample pack of pills and remedies. Take 1 pill at a time. If it’s a miracle cure, keep taking it. If not, try the next one. No fancy stuff required. Does anyone sell a sample pack like this?
Tl;dr
If your mystery malady is obvious, reliable, and rare in the general population, then you can simply try pills one at a time, one pill at a time, to see if they provide a reliable treatment. If so, then this will become apparent right away, and you can keep taking it a few more times to confirm it works reliably. It should only take a few trials to show the pill works for you. If the pill doesn’t work, you can set it aside after one try and try the next one.
This would not help you if the problem is a deficiency or excess of some compound that takes time to build up. It would only help if the pill had an immediate, obvious effect for your condition—a “miracle cure.”
This approach requires no fancy statistics, devices, or complexities of taking multiple pills at once. The easiest way to overcome the bottleneck would be if a company sold “sample packs” containing 1 pill of eat type. You’d take them one at a time, see if they fixed the problem, and if they did, you’d buy a larger quantity of that specific pill and keep experimenting to confirm the result.
Long:
I could imagine a fruitful collaboration between a mechanical engineer, a statistician, a programmer, and a biomedical researcher/doctor. The approach could be Bayes’ law-based. Using your problem as an example, here’s how it might work:
Does eating meat (cooked, store-bought, fresh, 50g) make me nauseous?
P(H): You specify how common it is in the human population for eating meat to cause nausea
P(E): You also specify how common it is for a member of the human population to eat meat and get nauseous, whether or not there’s a causal relationship
P(E|H): Finally, you specify how likely it would be for you to get nauseous after eating meat if eating meat makes you nauseous.
If you don’t have a good way to estimate these likelihoods, the app would give you suggestions for ways you could find reasonable numbers to plug in.
Then, every time you ate meat and got nauseous, you could enter another data point and it would update the likelihood that eating meat causes nausea.
Once you’re adequately confident that eating meat causes nausea, you could then repeat the exercise for a range of herbs.
Given that eating meat causes nausea, does taking Boswelia (BosPro) before eating prevent nausea from meat consumption?
P(H): You specify how common it is for people who get nauseous after eating meat to not get nauseous if they take Boswelia first.
P(E): You also specify how common it is for people who get nauseous from eating meat to take Boswelia, then eat meat, and not get nauseous.
P(E|H): Finally, you specify how likely it would be for you to not get nauseous after taking Boswelia, then eating meat, if taking Boswelia prevents nausea from meat consumption.
With a very reliable and unusual effect, as in your case, it’s possible to update a very low prior to a very confident prior in just a few trials.
I think the statistics would be doing most of the heavy lifting here. Being able to do a blinded placebo-controlled trial is important if you’re testing a relatively subtle/sporadic effect or testing a single compound.
But if you’re testing an obvious, reliable effect and experimenting with a range of compounds, I don’t think the self-blinding aspect is as important.
Here’s my attempt (not trained in this, correct me if I’m going about it wrong) to apply Bayesian inference to the problem.
For example, let’s say:
P(H), the chance that meat gives you nausea, is one in a million (.000001)
P(E), the likelihood of you just randomly feeling nauseous after eating meat, is one in a thousand (.001)
P(E|H), the likelihood you feel nauseous after eating meat, given that the meat causes you nausea, is 0.98
Then P(H|E), the chance that eating meat causes your nausea, given you ate meat and felt nauseous afterward, is 0.00098 after one time eating meat and getting nauseous. Feeding this back in as the new P(H), the chance that meat causes your nausea, P(H|E) becomes .9604, or 96.04%, the second time this happens (assuming you don’t eat meat and not feel nauseous). Realistically, we’d want to check if there’s something about the way you cooked the meat, the animal it comes from, the store you bought it from, the knowledge that you’re consuming meat, etc, that is confounding the result.
To deal with some of the confounds, you could repeat the experiment using different types of meat from different stores, and figure out how robust the effect is. It seems difficult to self-blind eating meat, since 50 g is about a sandwich’s worth of deli meat. Possibly you could try incorporating it into another more flavorful dish and eating the food with your eyes closed. But this is just to rule out the possibility that the result is from the perception of eating meat, as opposed to something primarily physiological about meat-material being in your digestive tract. If you weren’t concerned about this possibility, you could skip it.
But if the user is self-testing to find a treatment for a very reliable, obvious effect (as with your meat->nausea problem), and if our prior on any random pill just randomly working to prevent the nausea is low, then these results show you can probably figure out whether any random supplement works or not in just a few tries. That sort of obviates the need for fancy statistics or an app. You might be able to get much of the same result by just directing the user to “Try taking this pill, then eating 50 g meat. If the pill doesn’t work on the first try, move on to the next pill. If it does work, continue taking it. If it helps but doesn’t entirely eliminate the problem, explore dosing, timing, and manufacturer, to see if you can achieve significant improvements.”
I’m not sure if it makes sense to try multiple pills at once. Balancing the possibility that you stumble upon an effective treatment faster is the possibility that an effective treatment is disguished by a drug interaction, as well as the possibility that the average user just struggles to manage the added complexity, gets confused, and quits.
But this then becomes very simple.
If you have a reliable, obvious mystery malady that no doctor has treated effectively, systematically take 1 pill at a time until you find one with an effect. Then keep taking it, exploring dosage, timing, and manufacturer, until you maximize the effect. If satisfied, problem solved. If not fully satisfied, continue exploring other pills (unless it’s possible to devise a mechanistic hypothesis as to why the effective pill works, then use that to find likely alternatives more efficiently).
TBC: the client is interested scaleable software that people intuitively find useful for experimenting on themselves and can combine selfishly motivated individual data into useful aggregate data. But there’s a point here I want to argue.
It seems like you’re assuming all drug interactions are bad. What if you need three things in combination to see the effect, and then it works really well?
That’s entirely possible. However, it introduces a level of risk that isn’t as present when you take one at a time. It’s unlikely that a given supplement will kill you at normal doses. Such a severe side effect would have been detected and the supplement most likely wouldn’t be sold.
But it’s not impossible that certain rare combinations of drugs might have lethal side effects in combination, and that you might stumble across such a combination by chance. The risk of this happening seems in my judgment to outweigh the potential benefit of finding the right drug for a chronic condition faster.
Examples: https://www.google.com/amp/s/www.news24.com/amp/health24/medical/backache/news/7-medication-combinations-that-could-be-deadly-20160829
I’m not certain if it’s plausible an interaction between supplements could kill you in a few days. Even the examples here seem to take a while to work. The only potentially instantly lethal drugs I’ve seen are things like alcohol and heroine. But I would shy away from taking combinations—that’s just where my risk budget is at. I could be persuaded otherwise.
The blinded aspect is hard.
If you’re concerned about supplement interactions, you should be concerned about supplements effects at all. People should seek based on possible luck, but should know (and be able to tell doctors) what they’re putting into their bodies, and take contextual advice from relatives, friends, and community.
Collecting individuals susceptible to effective treatment with intervention X and having them ready for researchers to talk to, test more directly—is this already done? I suppose around individual notable conditions: celiac disease and gluten intolerance, or n=1 genetic issue self-diagnosis.
Some conditions have an intermittency, that makes it hard to assess interventions with unknown timing.
Perhaps blinded timing studies, self-studies, after something is found to work. Perhaps helping people to log and journal symptoms and effects during the blinded periods, as well as analyze, interpret, and share them—especially in ways that make it easier for others to trust.