Just because he hasn’t personally done much in the way of personal experimentation doesn’t mean he’s not riding on a large volume of experimental evidence. Just using RDA values and all known micronutrients (not that his history has exactly been stellar in actually doing so) should get him almost all the way to where he wants his product to be.
(I’m really dubious of the carbohydrate he uses, though.)
And provided people are willing to eat a meal or two outside the system, their cravings should guide them to whatever nutrients it is they’re lacking in.
So… I’m not sure I approve of the level of enthusiasm for the project that exists. But by the same token, I don’t think the extreme pessimism is warranted, either.
The concept is good, but the methodology could have been significantly better. It has lots of potential, and the real danger is limited to those that will be consuming ONLY Soylent for extended periods. Using it to replace a meal or two a day, and having a complete meal every day, shouldn’t be dangerous (I think).
What confuses me about the negativity is, what’s so bad about the current situation? The earliest of adopters will serve as a giant trial, and if there are problems they’ll come up there.
Also: people who intend to switch to JUST soylent should be monitored by a doctor or a nutritionist, at least for the first while. And post it either here or on the Solyent board. I am very interested to hear some anecdata.
What confuses me about the negativity is, what’s so bad about the current situation? The earliest of adopters will serve as a giant trial, and if there are problems they’ll come up there.
No, they won’t. Or, if they are interpretable as a trial, it’ll be as the worst epidemiological survey ever run—no blinding, no followup, response bias out the wazoo, attrition, expectancy and Hawthorne effects already built in etc etc. You name a bias, this (‘hand out goodies and hope someone will report problems’) will have it. You ever wonder why we have things like ‘evidence-based medicine’? It’s because when we hand out goodies and hope people will tell us how well it works, we get people grinding up tiger penises because nothing works better for fixing your virility problems! Everyone says so! And how could they be wrong, right?
To quote myself again from my G+ thread:
For [How many people will get sick/die?], there’s no way to tell. People get sick and die all the time. No one will be reporting systematically, which means that no matter how many datapoints you collect, your results will still be worthless because increased sample size only reduces random error, it doesn’t reduce systematic error which sets a floor on your result’s quality (see the ‘emperor of China’ bit in http://www.gwern.net/DNB%20FAQ#flaws-in-mainstream-science-and-psychology ) - and with a random release to enthusiasts like this, the systematic error/bias is going to be huge.
What do you expect will happen? Do you think lots of people are going to get very sick by going on a Soylent-only diet immediately, not monitoring their health closely, and ending up with serious nutritional deficiencies? That’s one of the more negative scenario, but I honestly don’t know how likely that is. I think people are likely to do at least one of three things:
Monitor their health more closely (especially on a soylent-only diet),
Only replace a few meals with Soylent (not more than, say, 75%),
Return to normal food or see a doctor if a serious deficiency occurs.
Then again, I may have too much confidence in people’s common sense. Rob is definitely marketing it as a finished product and a miracle solution.
I think there will be a range of issues from a few diehards hitting serious issues to people just having low-grade issues which they don’t notice because they won’t be randomizing blocks, effects similar to the hedonic treadmill will make it hard to compare over time, they’ll get initial benefits from the usual placebo/Hawthorne/overjustification effects, and subjective self-rating has many known loopholes where you can think you’re getting better even as you’re actually getting worse—but regardless of the exact distribution or what the worst-cases look like, we won’t know for the reasons I list above.
Instead, we’ll get another internet circle-jerk about how Soylent is awesome and the critics are wrong.
What would you like to see done differently? You mentioned the more thorough self-experimentation he could have done (really should have done), but there’s still someone else who could step up to the plate and do some self-testing.
Thorough studies? Those might also be done some time in the future, whether or not they’re funded by Rob (not sure about this point, there might not be an incentive to do so once it’s being sold).
Sure, Rob jumped the gun and hyped it up. But most of the internet is already a giant circle-jerk. Doesn’t stop people from generating real information, right?
Doesn’t stop people from generating real information, right?
That’s the damnable thing about these sorts of biases, it’s not clear to me whether one can compensate for the biases. If you pay attention to the ‘real information’, you may wind up learning what we might call anti-information—information that predictably and systematically makes your beliefs worse than your defaults.
This is the problem of the clever arguer: how do you, and can you, adjust for the fact that all the reports coming out about Soylent are so deeply error-prone (and now with the kickstarter, we get a delicious cherry on top of conflicts of interest)?
I would much rather have a handful of randomized self-experiments from some obscure blogger interested in a weird recipe he came up with than a forum of Soylent enthusiasts raving to each other that the latest formulation on sale is the greatest ever and telling each other that if you feel bad you should be eating your Soylent twice a day and not three times a day, don’t you know about intermittent fasting?, and also I ran 20 seconds faster today, so Soylent must be working for me!
Ok, I see what your concern is, with the hype around Soylent everyone’s opinion is skewed (even if they’re not among the fanboys).
You decided above that it wasn’t worth your time to try your own self-experiments with it. What if someone else were to take the time to do it? I like the concept but agree with the major troubles you listed above, and I have no experience with designing self-experiments. But maybe I’ll take the time to try and do it properly, long-term, with regular blood tests, noting what I’ve been eating for a couple months before starting, taking data about my fitness levels, etc. Of course, I would need to analyze the risk to myself beforehand.
What if someone else were to take the time to do it?
If they actually go through with it and write it up, that’s better than the status quo, yes. But if they don’t determine to go through with it and may give up, it’s another selection bias, specifically, publication bias (person A does a self-experiment but halfway through runs out of spare effort and abandons it; person B, by chance, gets better results and blogs about it etc).
Just because he hasn’t personally done much in the way of personal experimentation doesn’t mean he’s not riding on a large volume of experimental evidence. Just using RDA values and all known micronutrients (not that his history has exactly been stellar in actually doing so) should get him almost all the way to where he wants his product to be.
(I’m really dubious of the carbohydrate he uses, though.)
And provided people are willing to eat a meal or two outside the system, their cravings should guide them to whatever nutrients it is they’re lacking in.
So… I’m not sure I approve of the level of enthusiasm for the project that exists. But by the same token, I don’t think the extreme pessimism is warranted, either.
The concept is good, but the methodology could have been significantly better. It has lots of potential, and the real danger is limited to those that will be consuming ONLY Soylent for extended periods. Using it to replace a meal or two a day, and having a complete meal every day, shouldn’t be dangerous (I think).
What confuses me about the negativity is, what’s so bad about the current situation? The earliest of adopters will serve as a giant trial, and if there are problems they’ll come up there.
Also: people who intend to switch to JUST soylent should be monitored by a doctor or a nutritionist, at least for the first while. And post it either here or on the Solyent board. I am very interested to hear some anecdata.
No, they won’t. Or, if they are interpretable as a trial, it’ll be as the worst epidemiological survey ever run—no blinding, no followup, response bias out the wazoo, attrition, expectancy and Hawthorne effects already built in etc etc. You name a bias, this (‘hand out goodies and hope someone will report problems’) will have it. You ever wonder why we have things like ‘evidence-based medicine’? It’s because when we hand out goodies and hope people will tell us how well it works, we get people grinding up tiger penises because nothing works better for fixing your virility problems! Everyone says so! And how could they be wrong, right?
To quote myself again from my G+ thread:
What do you expect will happen? Do you think lots of people are going to get very sick by going on a Soylent-only diet immediately, not monitoring their health closely, and ending up with serious nutritional deficiencies? That’s one of the more negative scenario, but I honestly don’t know how likely that is. I think people are likely to do at least one of three things:
Monitor their health more closely (especially on a soylent-only diet),
Only replace a few meals with Soylent (not more than, say, 75%),
Return to normal food or see a doctor if a serious deficiency occurs.
Then again, I may have too much confidence in people’s common sense. Rob is definitely marketing it as a finished product and a miracle solution.
I think there will be a range of issues from a few diehards hitting serious issues to people just having low-grade issues which they don’t notice because they won’t be randomizing blocks, effects similar to the hedonic treadmill will make it hard to compare over time, they’ll get initial benefits from the usual placebo/Hawthorne/overjustification effects, and subjective self-rating has many known loopholes where you can think you’re getting better even as you’re actually getting worse—but regardless of the exact distribution or what the worst-cases look like, we won’t know for the reasons I list above.
Instead, we’ll get another internet circle-jerk about how Soylent is awesome and the critics are wrong.
What would you like to see done differently? You mentioned the more thorough self-experimentation he could have done (really should have done), but there’s still someone else who could step up to the plate and do some self-testing.
Thorough studies? Those might also be done some time in the future, whether or not they’re funded by Rob (not sure about this point, there might not be an incentive to do so once it’s being sold).
Sure, Rob jumped the gun and hyped it up. But most of the internet is already a giant circle-jerk. Doesn’t stop people from generating real information, right?
That’s the damnable thing about these sorts of biases, it’s not clear to me whether one can compensate for the biases. If you pay attention to the ‘real information’, you may wind up learning what we might call anti-information—information that predictably and systematically makes your beliefs worse than your defaults.
This is the problem of the clever arguer: how do you, and can you, adjust for the fact that all the reports coming out about Soylent are so deeply error-prone (and now with the kickstarter, we get a delicious cherry on top of conflicts of interest)?
I would much rather have a handful of randomized self-experiments from some obscure blogger interested in a weird recipe he came up with than a forum of Soylent enthusiasts raving to each other that the latest formulation on sale is the greatest ever and telling each other that if you feel bad you should be eating your Soylent twice a day and not three times a day, don’t you know about intermittent fasting?, and also I ran 20 seconds faster today, so Soylent must be working for me!
Ok, I see what your concern is, with the hype around Soylent everyone’s opinion is skewed (even if they’re not among the fanboys).
You decided above that it wasn’t worth your time to try your own self-experiments with it. What if someone else were to take the time to do it? I like the concept but agree with the major troubles you listed above, and I have no experience with designing self-experiments. But maybe I’ll take the time to try and do it properly, long-term, with regular blood tests, noting what I’ve been eating for a couple months before starting, taking data about my fitness levels, etc. Of course, I would need to analyze the risk to myself beforehand.
If they actually go through with it and write it up, that’s better than the status quo, yes. But if they don’t determine to go through with it and may give up, it’s another selection bias, specifically, publication bias (person A does a self-experiment but halfway through runs out of spare effort and abandons it; person B, by chance, gets better results and blogs about it etc).