I’m not sure how to square the fact that you agreed with “all of [my comment]” with this post.
In a basic sense, I implied that the self-report data appears consistent with no health difference between veg*n diets and non-veg*n diets. In practice, I expect there to be some differences because many veg*ns don’t take adequate supplementation, and also there are probably some subtle differences between the diets that are hard to detect. But, assuming the standard recommended supplementation, it seems plausible to me that health isn’t a non-trivial tradeoff for the vast majority of people when deciding to adopt a veg*n diet, and the self-report data doesn’t move me much on this question at all.
This comment is probably better suited as a response to your post on the health downsides of a vegan diet, rather than this post. However, in this post you critique multiple people for seeking to dismiss or suppress discussion about the health downsides of a vegan diet, or for attempting to reframe the discussion. When I read the comments you cite in this post under a background assumption that these health downsides are tiny or non-existent, most of the comments don’t seem very unreasonable to me anymore. If they’re right that the health downsides are small, then I don’t think it’s fair to allege that they weren’t being truth-seeking, in most cases you cited. It sounds more like they simply think your claims are frivolous and misleading.
If someone was writing a post about the safety or health downsides of nuclear energy production, I would probably similarly argue that focusing too much on this element of the discussion can be distracting and irrelevant, since nuclear energy is not significantly more unsafe or unhealthy than other forms of energy production if managed appropriately. I don’t think that means I’m a denialist about the tradeoffs. An open discussion of tradeoffs is important, but it’s equally important to emphasize an honest appraisal of whether the tradeoffs imply anything significant about what we should actually do.
I took your original comment to be saying “self-report is of limited value”, so I’m surprised that you’re confused by Elizabeth’s response. In your second comment, you seem to be treating your initial comment to have said something closer to “self-report is so low value that it should not materially alter your beliefs.” Those seem like very different statements to me.
In the original comment I said “I’m highly skeptical that you can get reliable information about the causal impacts of diets by asking people about their self-reported health after trying the diets”. It’s subjective whether that means I’m saying self-report data has “limited value” vs. “very little value” but I assumed Elizabeth had interpreted me as saying the latter, and that’s what I meant.
If this had come from a meat industry mouthpiece or even a provably neutral party, I would have tossed it. It would have been too easy to create those results. But it’s from Faunalytics, an org focused on reducing animal suffering through data analytics. Additionally, they seemed to accept the number as is. In fact my number is lower than the one they give, because I only include people who reported remission after resuming meat consumption, where Faunalytics reports “listed health issues as a reason they quit”. Given that, using their number seemed better than everyone guessing.
I hedged a little less about this after wilkox, a doctor who was not at all happy with the Change My Mind post, said he thought it was if anything an underestimate.
Edit: seems quite possible he meant “this is small given the self-reporting bias”, not “this is small given my estimate of the problem”
I think the interesting question here is “what % attrition to health issues do you think is okay?” If it’s 19%, I think it’s reasonable for you to decide this isn’t worth your time[1]. If it’s 2%, then you’d need to show the various factors were inflating estimates by a full order of magnitude.
Additionally, they seemed to accept the number as is.
I don’t think that’s fair to say given this disclaimer in the faunalytics study:
Note: Some caution
is needed in
considering these
results. It is possible
that former
vegetarians/vegans
may have
exaggerated their
difficulties given that
they provide a
justification for their
current behavior.
.
where Faunalytics reports “listed health issues as a reason they quit”.
This isn’t a quote from the faunalytics data, nor is it an accurate description of the data they gathered.
The survey asked people who are no longer veg*/n if they experienced certain health issues while they were veg*/n. Not whether they attributed those health issues to their diet, or whether they quit because of those health issues.
Someone who experienced depression/anxiety while they were vegan for example, who then quit being vegan because they broke up with their vegan partner, would be included in the survey data you’re talking about.
It’s possible I’m confused or missing something.
I hedged a little less about this after wilkox, a doctor who was not at all happy with the Change My Mind post, said he thought it was if anything an underestimate.
I’m much less confident about my issue with this part because im not totally sure what they meant, but I don’t interpret their comment as saying that in their professional opinion they think the number of people experiencing health issues from veg*nism is higher.
I interpret their surprise at the numbers being due to the fact that it’s a self reported survey. Given that people can say whatever they want, and that it’s surveying ex veg*ns, they’re surprised more people didn’t use health as a rationalization (is my impression).
Given that people can say whatever they want, and that it’s surveying ex veg*ns, they’re surprised more people didn’t use health as a rationalization
That seems reasonable.
Your quote from Faunalytics also seems reasonable, and a counter to my claim. I remembered another line that implied they accepted the number but thought it didn’t matter because it was small. It seems plausible they also were applying heavy discounting for self-reporting bias and expressing surprise about that.
you’re right, my summary in this post was wrong. Thank you for catching that and persisting in pointing it out when I missed it the first time. I’m fixing it now.
I agree with you that self-reports are inherently noisy, and I wish they’d included things like “what percentage of people develop an issue on that list after leaving veg*nism?”, “what percentage of veg*ns recover from said issues without adding in animal products?”, and “how prevalent are these issues in veg*ns, relative to omnivores” However I think self-reporting on the presence of specific issues is a stronger metric than self-reporting on something like “did you leave veganism for medical reasons?”.
Thanks I appreciate this! (What follows doesn’t include any further critical feedback about what you wrote)
One thing I also thought was missing in the survey is something that would touch on a general sense of loss of energy.
Its my impression that many people attempting veganism (perhaps more specifically a whole foods plant based diet, but also veg*nism generally) report a generalized loss of energy. Often this is cited as a reason for stopping the diet.
It’s also my impression (opinion?) that this is largely due to a difference in the intuitive sense of whether you’re getting enough calories, since vegan food is often less caloricaly dense. (You could eat 2Lb of mushrooms, feel super full, and only have eaten 250 calories)
If someone is used to eating a certain volume of food until they feel full, that same heuristic without changing may leave them at a major caloric defecit if eating healthy vegan foods.
This can also be a potential risk factor for any sort of deficiency. The food you’re eating might have 100% of the nutrients you need, but if you’re eating 70% of the food you need, you’ll not be getting enough nutrients.
Yeah I think this is a very important question and I’d love to get more data on it.
My very milquetoast guess is that some vegans or aspiring vegans do just need to eat more, and others are correct that they can’t just eat more, so it doesn’t matter if eating more would help (some of whom could adapt given more time and perhaps a gentler transition, and some of whom can’t).[1] A comment on a previous post talked about all the ways plant-based foods are more filling per unit calorie, and that may be true as far it goes, but it also means those foods are harder to digest, and not everyone considers that a feature.
My gut says that the former group (who just need to literally put more of the same food in their mouth) should be small, because surely they would eventually stumble on the “eat more” plan? The big reason not to would be if they’re calorie-restricting, but that’s independent of veganism. But who knows, people can be really disembodied and there are so many cultural messages tell us to eat less.
I talked in some earlier posts about digestive privilege, where veganism is just easier for some people than others. I think some portion of those people typical mind that everyone else faces the exact same challenge level, and this is the cause of a lot of inflammatory discussion. I have a hunch that people who find veganism more challenging than other vegans, but still less than the population average, or particular people they’re to, are the worst offenders because they did make some sacrifice.
I’m not claiming that the self-reported data is unreliable because it comes from a dubious institutional source. I’m claiming it’s unreliable because people are unreliable when they self-report facts about their health, especially when it comes to causal attribution. I’m sure Faunalytics is honest and the survey was designed reasonably well, but it’s absurd to take self-reported health data at face value for the reasons I stated: confirmation bias, selective memories, and motivated reasoning etc.
I think the interesting question here is “what % attrition to health issues do you think is okay?” If it’s 19%, I think it’s reasonable for you to decide this isn’t worth your time[1]. If it’s 2%, then you’d need to show the various factors were inflating estimates by a full order of magnitude.
It would be shockingly bad if we treated self-reported health data similarly in other circumstances. For example, I found one article that reported, “available epidemiological data points to a relatively high prevalence of perceived [electromagnetic hypersensitivity] in the general population, reaching 1,6% in Finland and 2,7% in Sweden, 3,5% in Austria, 4,6% in Taiwan, 5% in Switzerland and 10.3% in Germany”. Are you comfortable saying that the number of people who have electromagnetic hypersensitivity in Germany is more than one order of magnitude lower than this estimate of 10.3%? Because I am. In this case, I think the whole phenomenon is probably bullshit from top to bottom, and therefore the figure itself is bunk, not merely inflated.
Of course, the fact that self-reported health data is unreliable doesn’t actually imply that vegan diets are healthy. And as I noted, we already know that there are many vegans who don’t take adequate supplementation. So, I’d be very surprised if the number of people who suffer from ill-health as a result of becoming veg*n is literally zero. But I reject the framework that we should anchor to the self-report data and then try to figure out how much it’s inflated. As a datapoint about the health downsides of veg*nism, I think it simply provides very little value. And that seems especially true if we’re talking about veg*ns who take the standard recommended supplements regularly.
From what I remember of Elizabeth’s posts on the subject, her opinion is the literature surrounding this topic is abysmal. To resolve the question of why some veg*ns desist, we would need one that records objective clinical outcomes of health and veg*n/non-veg*n diet compliance. What I recall from Elizabeth’s posts was that no study even approaches this bar, and so she used other less reliable metrics.
I’m aware that people have written scientific papers that include the word vegan in the text, including the people at Cochrane. I’m confused why you thought that would be helpful. Does a study that relates health outcomes in vegans with vegan desistance exist, such that we can actually answer the question “At what rate do vegans desist for health reasons?”
Does a study that relates health outcomes in vegans with vegan desistance exist, such that we can actually answer the question “At what rate do vegans desist for health reasons?”
I don’t think that’s the central question here. We were mostly talking about whether vegan diets are healthy. I argued that self-reported data is not reliable for answering this question. The self-reported data might provide reliable evidence regarding people’s motives for abandoning vegan diets, but it doesn’t reliably inform us whether vegan diets are healthy.
Analogously, a survey of healing crystal buyers doesn’t reliably tell us whether healing crystals improve health. Even if such a survey is useful for explaining motives, it’s clearly less valuable than an RCT when it comes to the important question of whether they actually work.
So far as I can tell, the central question Elizabeth has been trying to answer is “Do the people who convert to veganism because they get involved in EA have systemic health problems?” Those health problems might be easily solvable with supplementation (Great!), systemic to having a fully vegan diet but only requires some modest amount of animal product, or something more complicated. She has several self-reported people coming to her saying they tried veganism, had health problems, and stopped. So, “At what rate do vegans desist for health reasons?” seems like an important question to me. It will tell you at least some of what you are missing when surveying current vegans only.
Analogously, a survey of healing crystal buyers doesn’t reliably tell us whether healing crystals improve health. Even if such a survey is useful for explaining motives, it’s clearly less valuable than an RCT when it comes to the important question of whether they actually work.
I agree that if your prior probability of something being true is near 0, you need very strong evidence to update. Was your prior probability that someone would desist from the vegan diet for health reasons actually that low? If not, why is the crystal healing metaphor analogous?
So, “At what rate do vegans desist for health reasons?” seems like an important question to me. It will tell you at least some of what you are missing when surveying current vegans only.
As I argued in my original comment, self-reported data is unreliable for answering this question. I simply do not trust people’s ability to attribute the causal impact of diets on their health. Separately, I think people frequently misreport their motives. Even if vegan diets caused no health effects, a substantial fraction of people could still report desisting from veganism for health reasons.
I’m honestly not sure why think that self-reported data is more reliable than proper scientific studies like RTCs when trying to shed light on this question. The RCTs should be better able to tell us the actual health effects of adopting veganism, which is key to understanding how many people would be forced to abandon the diet for health reasons.
As I argued in my original comment, self-reported data is unreliable for answering this question. I simply do not trust people’s ability to attribute the causal impact of diets on their health.
It seems to me like even if this isn’t relevant to our beliefs about underlying reality (“teachers think criticism works because they’re not correcting for regression to the mean, and so we can’t rely on teacher’s beliefs”) it should be relevant to our beliefs about individual decision-making (“when people desist from vegan diets, 20% of the time it’s because they think their health is worse”), which should direct our views on advocacy (if we want fewer people to desist from vegan diets (and possibly then dissuade others from trying them), we should try to make sure they don’t think their health is worse on them).
I’m honestly not sure why think that self-reported data is more reliable than proper scientific studies like RTCs when trying to shed light on this question.
Often there’s question-substitution going on. If the proper scientific studies are measuring easily quantifiable things like blood pressure and what people care about more / make decisions based off of is the difficult-to-quantify “how much pep is in my step”, then the improper survey may point more directly at the thing that’s relevant, even if it does so less precisely.
The question I assumed Stephen was asking (and at least my question for myself) here is, “okay, but what do we believe in the meanwhile?”.
Natalia responded with a process that might find some good evidence (but, might not, and looks like at least several hours of skilled-labor search to find out). I agree someone should do that labor and find out if better evidence exists.
I also realize Vaniver did explicitly ask “what alternate framework you prefer?” and it makes sense that your framework is interested in different questions than mine or Elizabeths or Stephens or whatnot. But, for me, the question is “what should vegan activist’s best guess be right now”, not “what might it turn out to be after doing a bunch more research that maybe turns out to have good data and maybe doesn’t.”
for me, the question is “what should vegan activist’s best guess be right now”
This is fair and a completely reasonable question to ask, even if we agree that the self-reported data is unreliable. I agree the self-reported data could be a useful first step towards answering this question if we had almost no other information. I also haven’t looked deeply into the RCTs and scientific data and so I don’t have a confident view on the health value of vegan diets.
On the other hand, personally, my understanding is that multiple mainstream scientific institutions say that vegan diets are generally healthy (in the vast majority of cases) unless you don’t take the proper supplementation regularly. If you put a gun to my head right now and asked me to submit my beliefs about this question, I would defer heavily to (my perception of) the mainstream consensus, rather than the self-reported data. That’s not because I think mainstream consensus isn’t sometimes wrong or biased, but—in the spirit of your question—that’s just what I think is most reliable out of all the easily accessible facts that I have available right now, including the self-reported data.
To be clear, I didn’t really want to take this line, and talk about how The Experts disagree with Elizabeth, and so the burden of proof is on her rather than me, because that’s often a conversation stopper and not helpful for fruitful discussion, especially given my relative ignorance about the empirical data. But if we’re interested in what a good “best guess” should be, then yes, I think mainstream scientific institutions are generally reliable on questions they have strong opinions about. That’s not my response to everything in this discussion, but it’s my response to your specific point about what we should believe in the meantime.
In this case I don’t think the claim you’re ascribing to the experts are Elizabeth are actually in conflict. You say:
vegan diets are generally healthy (in the vast majority of cases) unless you don’t take the proper supplementation regularly.
And I think Elizabeth said several times “If you actually are taking the supplementation, it’s healthy, but I know many people who aren’t taking that supplementation. I think EA vegan activists should put more effort into providing good recommendations to people they convince to go vegan.” So I’m not sure why you’re thinking of the expert consensus here as saying a different thing.
I feel a bit confused about what the argument is about here. I think the local point of “hey, you should be quite skeptical of self-reports” is a good, important point (thanks for bringing it up. I don’t think I agree with you on how much I should discount this data, but I wasn’t modeling all the possible failure modes you’re pointing out). But it feels from your phrasing like there’s something else going on, or the thread is overall getting into a cycle of arguing-for-the-sake-of-arguing, or something. (Maybe it’s just that Elizabeth’s post is long and it’s easy to lose track of the various disclaimers she made? Maybe it’s more of a “how much are you supposed to even have an opinion if all your evidence is weak?” frame clash)
Could you (or Natalie) say more about what this thread is about from your perspective?
So I’m not sure why you’re thinking of the expert consensus here as saying a different thing.
As far as I can tell, I didn’t directly assert that expert consensus disagreed with Elizabeth in this thread. Indeed I mentioned that I “didn’t really” want to make claims about that. I only brought up expert consensus to reply to a narrow question that you asked about what we should rely on as a best guess. I didn’t mention expert consensus in any of my comments prior to that one, at least in this thread.
My primary point in this thread was to talk about the unreliability of self-reported data and the pitfalls of relying on it. Secondarily, I commented that most of the people she’s critiquing in this post don’t seem obviously guilty of the allegation in the title. I think it’s important to push back against accusations that a bunch of people (or in this case, a whole sub-community) is “not truthseeking” on the basis of weak evidence. And my general reply here is that if indeed vegan diets are generally healthy as long as one takes the standard precautions, then I think it is reasonable for others to complain about someone emphasizing health tradeoffs excessively (which is what I interpreted many of the quoted people in the post as doing).
(At the very least, if you think these people are being unreasonable, I would maintain that the sweeping accusation in the title requires stronger evidence than what was presented. I am putting this in parentheses though to emphasize that this is not my main point.)
Also, I think it’s possible that Elizabeth doesn’t agree with the scientific consensus, or thinks it’s at least slightly wrong. I don’t want to put words in her mouth, though. Partly I think the scientific consensus is important to mention at some point because I don’t fully know what she believes, and I think that bringing up expert consensus is a good way to ground our discussion and make our premises more transparent. However, if she agrees with the consensus, then I’m still OK saying what I said, because I think almost all of it stands or falls independently of whether she agrees with the consensus.
But it feels from your phrasing like there’s something else going on, or the thread is overall getting into a cycle of arguing-for-the-sake-of-arguing, or something.
That’s possible too. I do think I might be getting too deep into this over what is mostly a few pointless quibbles about what type of data is reliable and what isn’t. You’re right to raise the possibility that things are going off the rails in an unintended way.
I think the original post was a bit confusing in what it claimed the Faunalytics study was useful for.
For example, the section
The ideal study is a longitudinal RCT where diet is randomly assigned, cost (across all dimensions, not just money) is held constant, and participants are studied over multiple years to track cumulative effects. I assume that doesn’t exist, but the closer we can get the better.
I’ve spent several hours looking for good studies on vegan nutrition, of which the only one that was even passable was the Faunalytics study.
[...]
A non-exhaustive list of common flaws:
Studies rarely control for supplements. [...]
makes it sound like the author is interested on the effects of vegan diets on health, both with and without supplementation, and that they’re claiming that the Faunalytics study is the best study we have to answer that question. This is what I and Matthew would strongly disagree with.
This post uses the Faunalytics study in a different (and IMO more reasonable) way, to show which proportion of veg*ans report negative health effects and quit in practice. This is a different question because it can loosely track how much veg*ans follow dietary guidelines. For example, vitamin B12 deficiency should affect close to 100% of vegans who don’t supplement and have been vegan for long enough, and, on the other side of the spectrum, it likely affects close to 0% of those who supplement, monitor their B12 levels and take B12 infusions when necessary.
A “longitudinal RCT where diet is randomly assigned” and that controls for supplements would not be useful for answering the second question, and neither would the RCTs and systematic reviews I brought up. But they would be more useful than the Faunalytcis survey for answering the first question.
Elizabeth has put at least dozens of hours into seeking good RCTs on vegan nutrition, and has come up nearly empty. At this point, if you want to say there is an expert consensus that disagrees with her, you need to find a particular study that you are willing to stand behind, so that we can discuss it. This is why Elizabeth wrote a post on the Adventist study—because that was the best that people were throwing at her.
Elizabeth has put at least dozens of hours into seeking good RCTs on vegan nutrition, and has come up nearly empty.
Given that I haven’t looked deeply into the RCTs and observational studies, I fully admit that I can’t completely address this comment and defend the merits of the scientific data. That said, I find it very unlikely that the RCTs and observational studies are so flawed that the self-reported survey data is more reliable. Although I find it quite plausible that the diet studies are flawed, why would the self-reported data be better?
The scientific studies might be bad, but that doesn’t mean we should anchor to an even more unreliable source of information.
At this point, if you want to say there is an expert consensus that disagrees with her, you need to find a particular study that you are willing to stand behind, so that we can discuss it.
I was very careful in my comment to say that I was only bringing up expert consensus to respond purely to a narrow point about what the “best guess” of vegan activists should be in the absence of a thorough investigation. Moreover, expert consensus is not generally revealed via studies, and so I don’t think I need to bring one up in order to make this point. Expert consensus is usually revealed by statements from mainstream institutions and prominent scientists, and sometimes survey data from scientists. If you’re asking me to show expert consensus, then I’d refer you to statements from the American Dietetic Association and the British Dietetic Association as a start. But I also want to emphasize that I really do not see expert consensus as the primary point of contention here.
This is a pretty transparent isolated demand for rigor. Can you tell me you’ve never uncritically cited surveys of self-reported data that make veg*n diets look good?
This is a pretty transparent isolated demand for rigor.
I don’t see what you mean. I thought an isolated demand for rigor is when you demand rigor selectively. But I am similarly skeptical of health claims about diets in almost every circumstance. Can you explain what you think I’m being selective about?
Can you tell me you’ve never uncritically cited surveys of self-reported data that make veg*n diets look good?
It’s hard to go back and check, and maybe I said some things when I was e.g. 16 that I wouldn’t stand by today, but I honestly don’t think I’ve taken this strategy at all. In my life I mostly remember being highly skeptical of self-reported health data, especially when it comes to asking people about causal effects. That’s not a vegan thing. That’s just my take on the value of the scientific method over anecdotes, self-reported data, and personal speculation. Do you have any evidence otherwise?
I’m not sure how to square the fact that you agreed with “all of [my comment]” with this post.
In a basic sense, I implied that the self-report data appears consistent with no health difference between veg*n diets and non-veg*n diets. In practice, I expect there to be some differences because many veg*ns don’t take adequate supplementation, and also there are probably some subtle differences between the diets that are hard to detect. But, assuming the standard recommended supplementation, it seems plausible to me that health isn’t a non-trivial tradeoff for the vast majority of people when deciding to adopt a veg*n diet, and the self-report data doesn’t move me much on this question at all.
This comment is probably better suited as a response to your post on the health downsides of a vegan diet, rather than this post. However, in this post you critique multiple people for seeking to dismiss or suppress discussion about the health downsides of a vegan diet, or for attempting to reframe the discussion. When I read the comments you cite in this post under a background assumption that these health downsides are tiny or non-existent, most of the comments don’t seem very unreasonable to me anymore. If they’re right that the health downsides are small, then I don’t think it’s fair to allege that they weren’t being truth-seeking, in most cases you cited. It sounds more like they simply think your claims are frivolous and misleading.
If someone was writing a post about the safety or health downsides of nuclear energy production, I would probably similarly argue that focusing too much on this element of the discussion can be distracting and irrelevant, since nuclear energy is not significantly more unsafe or unhealthy than other forms of energy production if managed appropriately. I don’t think that means I’m a denialist about the tradeoffs. An open discussion of tradeoffs is important, but it’s equally important to emphasize an honest appraisal of whether the tradeoffs imply anything significant about what we should actually do.
I took your original comment to be saying “self-report is of limited value”, so I’m surprised that you’re confused by Elizabeth’s response. In your second comment, you seem to be treating your initial comment to have said something closer to “self-report is so low value that it should not materially alter your beliefs.” Those seem like very different statements to me.
In the original comment I said “I’m highly skeptical that you can get reliable information about the causal impacts of diets by asking people about their self-reported health after trying the diets”. It’s subjective whether that means I’m saying self-report data has “limited value” vs. “very little value” but I assumed Elizabeth had interpreted me as saying the latter, and that’s what I meant.
If this had come from a meat industry mouthpiece or even a provably neutral party, I would have tossed it. It would have been too easy to create those results. But it’s from Faunalytics, an org focused on reducing animal suffering through data analytics. Additionally, they seemed to accept the number as is. In fact my number is lower than the one they give, because I only include people who reported remission after resuming meat consumption, where Faunalytics reports “listed health issues as a reason they quit”. Given that, using their number seemed better than everyone guessing.
I hedged a little less about this after wilkox, a doctor who was not at all happy with the Change My Mind post, said he thought it was if anything anunderestimate.Edit: seems quite possible he meant “this is small given the self-reporting bias”, not “this is small given my estimate of the problem”
I think the interesting question here is “what % attrition to health issues do you think is okay?” If it’s 19%, I think it’s reasonable for you to decide this isn’t worth your time[1]. If it’s 2%, then you’d need to show the various factors were inflating estimates by a full order of magnitude.
Although even then, I believe veganism will have many more issues than vegetarianism, and Faunalytics’s sample is overwhelmingly vegetarian.
I don’t think that’s fair to say given this disclaimer in the faunalytics study:
.
This isn’t a quote from the faunalytics data, nor is it an accurate description of the data they gathered.
The survey asked people who are no longer veg*/n if they experienced certain health issues while they were veg*/n. Not whether they attributed those health issues to their diet, or whether they quit because of those health issues.
Someone who experienced depression/anxiety while they were vegan for example, who then quit being vegan because they broke up with their vegan partner, would be included in the survey data you’re talking about.
It’s possible I’m confused or missing something.
I’m much less confident about my issue with this part because im not totally sure what they meant, but I don’t interpret their comment as saying that in their professional opinion they think the number of people experiencing health issues from veg*nism is higher.
I interpret their surprise at the numbers being due to the fact that it’s a self reported survey. Given that people can say whatever they want, and that it’s surveying ex veg*ns, they’re surprised more people didn’t use health as a rationalization (is my impression).
That seems reasonable.
Your quote from Faunalytics also seems reasonable, and a counter to my claim. I remembered another line that implied they accepted the number but thought it didn’t matter because it was small. It seems plausible they also were applying heavy discounting for self-reporting bias and expressing surprise about that.
I appreciate the response
Though I’m mostly concerned that you seem to be falsely quoting the faunalytics study:
This isn’t in the study and it’s not something they surveyed. They surveyed something meaningfully different, as I outlined in my comment.
you’re right, my summary in this post was wrong. Thank you for catching that and persisting in pointing it out when I missed it the first time. I’m fixing it now.
I agree with you that self-reports are inherently noisy, and I wish they’d included things like “what percentage of people develop an issue on that list after leaving veg*nism?”, “what percentage of veg*ns recover from said issues without adding in animal products?”, and “how prevalent are these issues in veg*ns, relative to omnivores” However I think self-reporting on the presence of specific issues is a stronger metric than self-reporting on something like “did you leave veganism for medical reasons?”.
Thanks I appreciate this! (What follows doesn’t include any further critical feedback about what you wrote)
One thing I also thought was missing in the survey is something that would touch on a general sense of loss of energy.
Its my impression that many people attempting veganism (perhaps more specifically a whole foods plant based diet, but also veg*nism generally) report a generalized loss of energy. Often this is cited as a reason for stopping the diet.
It’s also my impression (opinion?) that this is largely due to a difference in the intuitive sense of whether you’re getting enough calories, since vegan food is often less caloricaly dense. (You could eat 2Lb of mushrooms, feel super full, and only have eaten 250 calories)
If someone is used to eating a certain volume of food until they feel full, that same heuristic without changing may leave them at a major caloric defecit if eating healthy vegan foods.
This can also be a potential risk factor for any sort of deficiency. The food you’re eating might have 100% of the nutrients you need, but if you’re eating 70% of the food you need, you’ll not be getting enough nutrients.
Yeah I think this is a very important question and I’d love to get more data on it.
My very milquetoast guess is that some vegans or aspiring vegans do just need to eat more, and others are correct that they can’t just eat more, so it doesn’t matter if eating more would help (some of whom could adapt given more time and perhaps a gentler transition, and some of whom can’t).[1] A comment on a previous post talked about all the ways plant-based foods are more filling per unit calorie, and that may be true as far it goes, but it also means those foods are harder to digest, and not everyone considers that a feature.
My gut says that the former group (who just need to literally put more of the same food in their mouth) should be small, because surely they would eventually stumble on the “eat more” plan? The big reason not to would be if they’re calorie-restricting, but that’s independent of veganism. But who knows, people can be really disembodied and there are so many cultural messages tell us to eat less.
I talked in some earlier posts about digestive privilege, where veganism is just easier for some people than others. I think some portion of those people typical mind that everyone else faces the exact same challenge level, and this is the cause of a lot of inflammatory discussion. I have a hunch that people who find veganism more challenging than other vegans, but still less than the population average, or particular people they’re to, are the worst offenders because they did make some sacrifice.
And others have other reasons they can’t be vegan, but not focusing on those right now.
I’m not claiming that the self-reported data is unreliable because it comes from a dubious institutional source. I’m claiming it’s unreliable because people are unreliable when they self-report facts about their health, especially when it comes to causal attribution. I’m sure Faunalytics is honest and the survey was designed reasonably well, but it’s absurd to take self-reported health data at face value for the reasons I stated: confirmation bias, selective memories, and motivated reasoning etc.
It would be shockingly bad if we treated self-reported health data similarly in other circumstances. For example, I found one article that reported, “available epidemiological data points to a relatively high prevalence of perceived [electromagnetic hypersensitivity] in the general population, reaching 1,6% in Finland and 2,7% in Sweden, 3,5% in Austria, 4,6% in Taiwan, 5% in Switzerland and 10.3% in Germany”. Are you comfortable saying that the number of people who have electromagnetic hypersensitivity in Germany is more than one order of magnitude lower than this estimate of 10.3%? Because I am. In this case, I think the whole phenomenon is probably bullshit from top to bottom, and therefore the figure itself is bunk, not merely inflated.
Of course, the fact that self-reported health data is unreliable doesn’t actually imply that vegan diets are healthy. And as I noted, we already know that there are many vegans who don’t take adequate supplementation. So, I’d be very surprised if the number of people who suffer from ill-health as a result of becoming veg*n is literally zero. But I reject the framework that we should anchor to the self-report data and then try to figure out how much it’s inflated. As a datapoint about the health downsides of veg*nism, I think it simply provides very little value. And that seems especially true if we’re talking about veg*ns who take the standard recommended supplements regularly.
Do you have another framework you prefer, or do you just think that we should not speak about this because we can’t know anything about this?
[deleted]
Does such a study exist?
From what I remember of Elizabeth’s posts on the subject, her opinion is the literature surrounding this topic is abysmal. To resolve the question of why some veg*ns desist, we would need one that records objective clinical outcomes of health and veg*n/non-veg*n diet compliance. What I recall from Elizabeth’s posts was that no study even approaches this bar, and so she used other less reliable metrics.
[deleted]
I’m aware that people have written scientific papers that include the word vegan in the text, including the people at Cochrane. I’m confused why you thought that would be helpful. Does a study that relates health outcomes in vegans with vegan desistance exist, such that we can actually answer the question “At what rate do vegans desist for health reasons?”
I don’t think that’s the central question here. We were mostly talking about whether vegan diets are healthy. I argued that self-reported data is not reliable for answering this question. The self-reported data might provide reliable evidence regarding people’s motives for abandoning vegan diets, but it doesn’t reliably inform us whether vegan diets are healthy.
Analogously, a survey of healing crystal buyers doesn’t reliably tell us whether healing crystals improve health. Even if such a survey is useful for explaining motives, it’s clearly less valuable than an RCT when it comes to the important question of whether they actually work.
So far as I can tell, the central question Elizabeth has been trying to answer is “Do the people who convert to veganism because they get involved in EA have systemic health problems?” Those health problems might be easily solvable with supplementation (Great!), systemic to having a fully vegan diet but only requires some modest amount of animal product, or something more complicated. She has several self-reported people coming to her saying they tried veganism, had health problems, and stopped. So, “At what rate do vegans desist for health reasons?” seems like an important question to me. It will tell you at least some of what you are missing when surveying current vegans only.
I agree that if your prior probability of something being true is near 0, you need very strong evidence to update. Was your prior probability that someone would desist from the vegan diet for health reasons actually that low? If not, why is the crystal healing metaphor analogous?
As I argued in my original comment, self-reported data is unreliable for answering this question. I simply do not trust people’s ability to attribute the causal impact of diets on their health. Separately, I think people frequently misreport their motives. Even if vegan diets caused no health effects, a substantial fraction of people could still report desisting from veganism for health reasons.
I’m honestly not sure why think that self-reported data is more reliable than proper scientific studies like RTCs when trying to shed light on this question. The RCTs should be better able to tell us the actual health effects of adopting veganism, which is key to understanding how many people would be forced to abandon the diet for health reasons.
It seems to me like even if this isn’t relevant to our beliefs about underlying reality (“teachers think criticism works because they’re not correcting for regression to the mean, and so we can’t rely on teacher’s beliefs”) it should be relevant to our beliefs about individual decision-making (“when people desist from vegan diets, 20% of the time it’s because they think their health is worse”), which should direct our views on advocacy (if we want fewer people to desist from vegan diets (and possibly then dissuade others from trying them), we should try to make sure they don’t think their health is worse on them).
Often there’s question-substitution going on. If the proper scientific studies are measuring easily quantifiable things like blood pressure and what people care about more / make decisions based off of is the difficult-to-quantify “how much pep is in my step”, then the improper survey may point more directly at the thing that’s relevant, even if it does so less precisely.
The question I assumed Stephen was asking (and at least my question for myself) here is, “okay, but what do we believe in the meanwhile?”.
Natalia responded with a process that might find some good evidence (but, might not, and looks like at least several hours of skilled-labor search to find out). I agree someone should do that labor and find out if better evidence exists.
I also realize Vaniver did explicitly ask “what alternate framework you prefer?” and it makes sense that your framework is interested in different questions than mine or Elizabeths or Stephens or whatnot. But, for me, the question is “what should vegan activist’s best guess be right now”, not “what might it turn out to be after doing a bunch more research that maybe turns out to have good data and maybe doesn’t.”
Best guess of what, specifically?
This is fair and a completely reasonable question to ask, even if we agree that the self-reported data is unreliable. I agree the self-reported data could be a useful first step towards answering this question if we had almost no other information. I also haven’t looked deeply into the RCTs and scientific data and so I don’t have a confident view on the health value of vegan diets.
On the other hand, personally, my understanding is that multiple mainstream scientific institutions say that vegan diets are generally healthy (in the vast majority of cases) unless you don’t take the proper supplementation regularly. If you put a gun to my head right now and asked me to submit my beliefs about this question, I would defer heavily to (my perception of) the mainstream consensus, rather than the self-reported data. That’s not because I think mainstream consensus isn’t sometimes wrong or biased, but—in the spirit of your question—that’s just what I think is most reliable out of all the easily accessible facts that I have available right now, including the self-reported data.
To be clear, I didn’t really want to take this line, and talk about how The Experts disagree with Elizabeth, and so the burden of proof is on her rather than me, because that’s often a conversation stopper and not helpful for fruitful discussion, especially given my relative ignorance about the empirical data. But if we’re interested in what a good “best guess” should be, then yes, I think mainstream scientific institutions are generally reliable on questions they have strong opinions about. That’s not my response to everything in this discussion, but it’s my response to your specific point about what we should believe in the meantime.
Nod.
In this case I don’t think the claim you’re ascribing to the experts are Elizabeth are actually in conflict. You say:
And I think Elizabeth said several times “If you actually are taking the supplementation, it’s healthy, but I know many people who aren’t taking that supplementation. I think EA vegan activists should put more effort into providing good recommendations to people they convince to go vegan.” So I’m not sure why you’re thinking of the expert consensus here as saying a different thing.
I feel a bit confused about what the argument is about here. I think the local point of “hey, you should be quite skeptical of self-reports” is a good, important point (thanks for bringing it up. I don’t think I agree with you on how much I should discount this data, but I wasn’t modeling all the possible failure modes you’re pointing out). But it feels from your phrasing like there’s something else going on, or the thread is overall getting into a cycle of arguing-for-the-sake-of-arguing, or something. (Maybe it’s just that Elizabeth’s post is long and it’s easy to lose track of the various disclaimers she made? Maybe it’s more of a “how much are you supposed to even have an opinion if all your evidence is weak?” frame clash)
Could you (or Natalie) say more about what this thread is about from your perspective?
As far as I can tell, I didn’t directly assert that expert consensus disagreed with Elizabeth in this thread. Indeed I mentioned that I “didn’t really” want to make claims about that. I only brought up expert consensus to reply to a narrow question that you asked about what we should rely on as a best guess. I didn’t mention expert consensus in any of my comments prior to that one, at least in this thread.
My primary point in this thread was to talk about the unreliability of self-reported data and the pitfalls of relying on it. Secondarily, I commented that most of the people she’s critiquing in this post don’t seem obviously guilty of the allegation in the title. I think it’s important to push back against accusations that a bunch of people (or in this case, a whole sub-community) is “not truthseeking” on the basis of weak evidence. And my general reply here is that if indeed vegan diets are generally healthy as long as one takes the standard precautions, then I think it is reasonable for others to complain about someone emphasizing health tradeoffs excessively (which is what I interpreted many of the quoted people in the post as doing).
(At the very least, if you think these people are being unreasonable, I would maintain that the sweeping accusation in the title requires stronger evidence than what was presented. I am putting this in parentheses though to emphasize that this is not my main point.)
Also, I think it’s possible that Elizabeth doesn’t agree with the scientific consensus, or thinks it’s at least slightly wrong. I don’t want to put words in her mouth, though. Partly I think the scientific consensus is important to mention at some point because I don’t fully know what she believes, and I think that bringing up expert consensus is a good way to ground our discussion and make our premises more transparent. However, if she agrees with the consensus, then I’m still OK saying what I said, because I think almost all of it stands or falls independently of whether she agrees with the consensus.
That’s possible too. I do think I might be getting too deep into this over what is mostly a few pointless quibbles about what type of data is reliable and what isn’t. You’re right to raise the possibility that things are going off the rails in an unintended way.
I think the original post was a bit confusing in what it claimed the Faunalytics study was useful for.
For example, the section
makes it sound like the author is interested on the effects of vegan diets on health, both with and without supplementation, and that they’re claiming that the Faunalytics study is the best study we have to answer that question. This is what I and Matthew would strongly disagree with.
This post uses the Faunalytics study in a different (and IMO more reasonable) way, to show which proportion of veg*ans report negative health effects and quit in practice. This is a different question because it can loosely track how much veg*ans follow dietary guidelines. For example, vitamin B12 deficiency should affect close to 100% of vegans who don’t supplement and have been vegan for long enough, and, on the other side of the spectrum, it likely affects close to 0% of those who supplement, monitor their B12 levels and take B12 infusions when necessary.
A “longitudinal RCT where diet is randomly assigned” and that controls for supplements would not be useful for answering the second question, and neither would the RCTs and systematic reviews I brought up. But they would be more useful than the Faunalytcis survey for answering the first question.
Elizabeth has put at least dozens of hours into seeking good RCTs on vegan nutrition, and has come up nearly empty. At this point, if you want to say there is an expert consensus that disagrees with her, you need to find a particular study that you are willing to stand behind, so that we can discuss it. This is why Elizabeth wrote a post on the Adventist study—because that was the best that people were throwing at her.
Given that I haven’t looked deeply into the RCTs and observational studies, I fully admit that I can’t completely address this comment and defend the merits of the scientific data. That said, I find it very unlikely that the RCTs and observational studies are so flawed that the self-reported survey data is more reliable. Although I find it quite plausible that the diet studies are flawed, why would the self-reported data be better?
The scientific studies might be bad, but that doesn’t mean we should anchor to an even more unreliable source of information.
I was very careful in my comment to say that I was only bringing up expert consensus to respond purely to a narrow point about what the “best guess” of vegan activists should be in the absence of a thorough investigation. Moreover, expert consensus is not generally revealed via studies, and so I don’t think I need to bring one up in order to make this point. Expert consensus is usually revealed by statements from mainstream institutions and prominent scientists, and sometimes survey data from scientists. If you’re asking me to show expert consensus, then I’d refer you to statements from the American Dietetic Association and the British Dietetic Association as a start. But I also want to emphasize that I really do not see expert consensus as the primary point of contention here.
Yeah, that does sound nicer; have those already been done or are we going to have to wait for them?
This is a pretty transparent isolated demand for rigor. Can you tell me you’ve never uncritically cited surveys of self-reported data that make veg*n diets look good?
I don’t see what you mean. I thought an isolated demand for rigor is when you demand rigor selectively. But I am similarly skeptical of health claims about diets in almost every circumstance. Can you explain what you think I’m being selective about?
It’s hard to go back and check, and maybe I said some things when I was e.g. 16 that I wouldn’t stand by today, but I honestly don’t think I’ve taken this strategy at all. In my life I mostly remember being highly skeptical of self-reported health data, especially when it comes to asking people about causal effects. That’s not a vegan thing. That’s just my take on the value of the scientific method over anecdotes, self-reported data, and personal speculation. Do you have any evidence otherwise?