for me, the question is “what should vegan activist’s best guess be right now”
This is fair and a completely reasonable question to ask, even if we agree that the self-reported data is unreliable. I agree the self-reported data could be a useful first step towards answering this question if we had almost no other information. I also haven’t looked deeply into the RCTs and scientific data and so I don’t have a confident view on the health value of vegan diets.
On the other hand, personally, my understanding is that multiple mainstream scientific institutions say that vegan diets are generally healthy (in the vast majority of cases) unless you don’t take the proper supplementation regularly. If you put a gun to my head right now and asked me to submit my beliefs about this question, I would defer heavily to (my perception of) the mainstream consensus, rather than the self-reported data. That’s not because I think mainstream consensus isn’t sometimes wrong or biased, but—in the spirit of your question—that’s just what I think is most reliable out of all the easily accessible facts that I have available right now, including the self-reported data.
To be clear, I didn’t really want to take this line, and talk about how The Experts disagree with Elizabeth, and so the burden of proof is on her rather than me, because that’s often a conversation stopper and not helpful for fruitful discussion, especially given my relative ignorance about the empirical data. But if we’re interested in what a good “best guess” should be, then yes, I think mainstream scientific institutions are generally reliable on questions they have strong opinions about. That’s not my response to everything in this discussion, but it’s my response to your specific point about what we should believe in the meantime.
In this case I don’t think the claim you’re ascribing to the experts are Elizabeth are actually in conflict. You say:
vegan diets are generally healthy (in the vast majority of cases) unless you don’t take the proper supplementation regularly.
And I think Elizabeth said several times “If you actually are taking the supplementation, it’s healthy, but I know many people who aren’t taking that supplementation. I think EA vegan activists should put more effort into providing good recommendations to people they convince to go vegan.” So I’m not sure why you’re thinking of the expert consensus here as saying a different thing.
I feel a bit confused about what the argument is about here. I think the local point of “hey, you should be quite skeptical of self-reports” is a good, important point (thanks for bringing it up. I don’t think I agree with you on how much I should discount this data, but I wasn’t modeling all the possible failure modes you’re pointing out). But it feels from your phrasing like there’s something else going on, or the thread is overall getting into a cycle of arguing-for-the-sake-of-arguing, or something. (Maybe it’s just that Elizabeth’s post is long and it’s easy to lose track of the various disclaimers she made? Maybe it’s more of a “how much are you supposed to even have an opinion if all your evidence is weak?” frame clash)
Could you (or Natalie) say more about what this thread is about from your perspective?
So I’m not sure why you’re thinking of the expert consensus here as saying a different thing.
As far as I can tell, I didn’t directly assert that expert consensus disagreed with Elizabeth in this thread. Indeed I mentioned that I “didn’t really” want to make claims about that. I only brought up expert consensus to reply to a narrow question that you asked about what we should rely on as a best guess. I didn’t mention expert consensus in any of my comments prior to that one, at least in this thread.
My primary point in this thread was to talk about the unreliability of self-reported data and the pitfalls of relying on it. Secondarily, I commented that most of the people she’s critiquing in this post don’t seem obviously guilty of the allegation in the title. I think it’s important to push back against accusations that a bunch of people (or in this case, a whole sub-community) is “not truthseeking” on the basis of weak evidence. And my general reply here is that if indeed vegan diets are generally healthy as long as one takes the standard precautions, then I think it is reasonable for others to complain about someone emphasizing health tradeoffs excessively (which is what I interpreted many of the quoted people in the post as doing).
(At the very least, if you think these people are being unreasonable, I would maintain that the sweeping accusation in the title requires stronger evidence than what was presented. I am putting this in parentheses though to emphasize that this is not my main point.)
Also, I think it’s possible that Elizabeth doesn’t agree with the scientific consensus, or thinks it’s at least slightly wrong. I don’t want to put words in her mouth, though. Partly I think the scientific consensus is important to mention at some point because I don’t fully know what she believes, and I think that bringing up expert consensus is a good way to ground our discussion and make our premises more transparent. However, if she agrees with the consensus, then I’m still OK saying what I said, because I think almost all of it stands or falls independently of whether she agrees with the consensus.
But it feels from your phrasing like there’s something else going on, or the thread is overall getting into a cycle of arguing-for-the-sake-of-arguing, or something.
That’s possible too. I do think I might be getting too deep into this over what is mostly a few pointless quibbles about what type of data is reliable and what isn’t. You’re right to raise the possibility that things are going off the rails in an unintended way.
I think the original post was a bit confusing in what it claimed the Faunalytics study was useful for.
For example, the section
The ideal study is a longitudinal RCT where diet is randomly assigned, cost (across all dimensions, not just money) is held constant, and participants are studied over multiple years to track cumulative effects. I assume that doesn’t exist, but the closer we can get the better.
I’ve spent several hours looking for good studies on vegan nutrition, of which the only one that was even passable was the Faunalytics study.
[...]
A non-exhaustive list of common flaws:
Studies rarely control for supplements. [...]
makes it sound like the author is interested on the effects of vegan diets on health, both with and without supplementation, and that they’re claiming that the Faunalytics study is the best study we have to answer that question. This is what I and Matthew would strongly disagree with.
This post uses the Faunalytics study in a different (and IMO more reasonable) way, to show which proportion of veg*ans report negative health effects and quit in practice. This is a different question because it can loosely track how much veg*ans follow dietary guidelines. For example, vitamin B12 deficiency should affect close to 100% of vegans who don’t supplement and have been vegan for long enough, and, on the other side of the spectrum, it likely affects close to 0% of those who supplement, monitor their B12 levels and take B12 infusions when necessary.
A “longitudinal RCT where diet is randomly assigned” and that controls for supplements would not be useful for answering the second question, and neither would the RCTs and systematic reviews I brought up. But they would be more useful than the Faunalytcis survey for answering the first question.
Elizabeth has put at least dozens of hours into seeking good RCTs on vegan nutrition, and has come up nearly empty. At this point, if you want to say there is an expert consensus that disagrees with her, you need to find a particular study that you are willing to stand behind, so that we can discuss it. This is why Elizabeth wrote a post on the Adventist study—because that was the best that people were throwing at her.
Elizabeth has put at least dozens of hours into seeking good RCTs on vegan nutrition, and has come up nearly empty.
Given that I haven’t looked deeply into the RCTs and observational studies, I fully admit that I can’t completely address this comment and defend the merits of the scientific data. That said, I find it very unlikely that the RCTs and observational studies are so flawed that the self-reported survey data is more reliable. Although I find it quite plausible that the diet studies are flawed, why would the self-reported data be better?
The scientific studies might be bad, but that doesn’t mean we should anchor to an even more unreliable source of information.
At this point, if you want to say there is an expert consensus that disagrees with her, you need to find a particular study that you are willing to stand behind, so that we can discuss it.
I was very careful in my comment to say that I was only bringing up expert consensus to respond purely to a narrow point about what the “best guess” of vegan activists should be in the absence of a thorough investigation. Moreover, expert consensus is not generally revealed via studies, and so I don’t think I need to bring one up in order to make this point. Expert consensus is usually revealed by statements from mainstream institutions and prominent scientists, and sometimes survey data from scientists. If you’re asking me to show expert consensus, then I’d refer you to statements from the American Dietetic Association and the British Dietetic Association as a start. But I also want to emphasize that I really do not see expert consensus as the primary point of contention here.
This is fair and a completely reasonable question to ask, even if we agree that the self-reported data is unreliable. I agree the self-reported data could be a useful first step towards answering this question if we had almost no other information. I also haven’t looked deeply into the RCTs and scientific data and so I don’t have a confident view on the health value of vegan diets.
On the other hand, personally, my understanding is that multiple mainstream scientific institutions say that vegan diets are generally healthy (in the vast majority of cases) unless you don’t take the proper supplementation regularly. If you put a gun to my head right now and asked me to submit my beliefs about this question, I would defer heavily to (my perception of) the mainstream consensus, rather than the self-reported data. That’s not because I think mainstream consensus isn’t sometimes wrong or biased, but—in the spirit of your question—that’s just what I think is most reliable out of all the easily accessible facts that I have available right now, including the self-reported data.
To be clear, I didn’t really want to take this line, and talk about how The Experts disagree with Elizabeth, and so the burden of proof is on her rather than me, because that’s often a conversation stopper and not helpful for fruitful discussion, especially given my relative ignorance about the empirical data. But if we’re interested in what a good “best guess” should be, then yes, I think mainstream scientific institutions are generally reliable on questions they have strong opinions about. That’s not my response to everything in this discussion, but it’s my response to your specific point about what we should believe in the meantime.
Nod.
In this case I don’t think the claim you’re ascribing to the experts are Elizabeth are actually in conflict. You say:
And I think Elizabeth said several times “If you actually are taking the supplementation, it’s healthy, but I know many people who aren’t taking that supplementation. I think EA vegan activists should put more effort into providing good recommendations to people they convince to go vegan.” So I’m not sure why you’re thinking of the expert consensus here as saying a different thing.
I feel a bit confused about what the argument is about here. I think the local point of “hey, you should be quite skeptical of self-reports” is a good, important point (thanks for bringing it up. I don’t think I agree with you on how much I should discount this data, but I wasn’t modeling all the possible failure modes you’re pointing out). But it feels from your phrasing like there’s something else going on, or the thread is overall getting into a cycle of arguing-for-the-sake-of-arguing, or something. (Maybe it’s just that Elizabeth’s post is long and it’s easy to lose track of the various disclaimers she made? Maybe it’s more of a “how much are you supposed to even have an opinion if all your evidence is weak?” frame clash)
Could you (or Natalie) say more about what this thread is about from your perspective?
As far as I can tell, I didn’t directly assert that expert consensus disagreed with Elizabeth in this thread. Indeed I mentioned that I “didn’t really” want to make claims about that. I only brought up expert consensus to reply to a narrow question that you asked about what we should rely on as a best guess. I didn’t mention expert consensus in any of my comments prior to that one, at least in this thread.
My primary point in this thread was to talk about the unreliability of self-reported data and the pitfalls of relying on it. Secondarily, I commented that most of the people she’s critiquing in this post don’t seem obviously guilty of the allegation in the title. I think it’s important to push back against accusations that a bunch of people (or in this case, a whole sub-community) is “not truthseeking” on the basis of weak evidence. And my general reply here is that if indeed vegan diets are generally healthy as long as one takes the standard precautions, then I think it is reasonable for others to complain about someone emphasizing health tradeoffs excessively (which is what I interpreted many of the quoted people in the post as doing).
(At the very least, if you think these people are being unreasonable, I would maintain that the sweeping accusation in the title requires stronger evidence than what was presented. I am putting this in parentheses though to emphasize that this is not my main point.)
Also, I think it’s possible that Elizabeth doesn’t agree with the scientific consensus, or thinks it’s at least slightly wrong. I don’t want to put words in her mouth, though. Partly I think the scientific consensus is important to mention at some point because I don’t fully know what she believes, and I think that bringing up expert consensus is a good way to ground our discussion and make our premises more transparent. However, if she agrees with the consensus, then I’m still OK saying what I said, because I think almost all of it stands or falls independently of whether she agrees with the consensus.
That’s possible too. I do think I might be getting too deep into this over what is mostly a few pointless quibbles about what type of data is reliable and what isn’t. You’re right to raise the possibility that things are going off the rails in an unintended way.
I think the original post was a bit confusing in what it claimed the Faunalytics study was useful for.
For example, the section
makes it sound like the author is interested on the effects of vegan diets on health, both with and without supplementation, and that they’re claiming that the Faunalytics study is the best study we have to answer that question. This is what I and Matthew would strongly disagree with.
This post uses the Faunalytics study in a different (and IMO more reasonable) way, to show which proportion of veg*ans report negative health effects and quit in practice. This is a different question because it can loosely track how much veg*ans follow dietary guidelines. For example, vitamin B12 deficiency should affect close to 100% of vegans who don’t supplement and have been vegan for long enough, and, on the other side of the spectrum, it likely affects close to 0% of those who supplement, monitor their B12 levels and take B12 infusions when necessary.
A “longitudinal RCT where diet is randomly assigned” and that controls for supplements would not be useful for answering the second question, and neither would the RCTs and systematic reviews I brought up. But they would be more useful than the Faunalytcis survey for answering the first question.
Elizabeth has put at least dozens of hours into seeking good RCTs on vegan nutrition, and has come up nearly empty. At this point, if you want to say there is an expert consensus that disagrees with her, you need to find a particular study that you are willing to stand behind, so that we can discuss it. This is why Elizabeth wrote a post on the Adventist study—because that was the best that people were throwing at her.
Given that I haven’t looked deeply into the RCTs and observational studies, I fully admit that I can’t completely address this comment and defend the merits of the scientific data. That said, I find it very unlikely that the RCTs and observational studies are so flawed that the self-reported survey data is more reliable. Although I find it quite plausible that the diet studies are flawed, why would the self-reported data be better?
The scientific studies might be bad, but that doesn’t mean we should anchor to an even more unreliable source of information.
I was very careful in my comment to say that I was only bringing up expert consensus to respond purely to a narrow point about what the “best guess” of vegan activists should be in the absence of a thorough investigation. Moreover, expert consensus is not generally revealed via studies, and so I don’t think I need to bring one up in order to make this point. Expert consensus is usually revealed by statements from mainstream institutions and prominent scientists, and sometimes survey data from scientists. If you’re asking me to show expert consensus, then I’d refer you to statements from the American Dietetic Association and the British Dietetic Association as a start. But I also want to emphasize that I really do not see expert consensus as the primary point of contention here.