[note: this comment was edited after people’s replies.]
I think it’s useful to look at the confidence intervals, rather than only the point estimates.
The 95% confidence intervals of the adjusted hazard ratios for overall mortality, for men, were [0.56, 0.92] and [0.57, 0.93] for vegan and pescatarian diets, respectively, and for women the CIs are [0.72, 1.07] and [0.78, 1.20], respectively. For women, the confidence intervals for all diets are [0.78, 1.2], [0.83, 1.07], [0.72, 1.07] and [0.7, 1.22].
What these CIs indicate is that there was likely no difference between pescatarian and vegan diets for men, both of which are better than omnivorism, and likely no difference between any of the diets for women.
The CIs for women specifically look so similar that you could pretend that all of those CIs came from different studies examining the exact same diet, and write a meta-analysis with them, and readers of the meta-analysis would think, “oh, cool, there’s no heterogeneity among the studies!”
In fact, we can go ahead and run a meta-analysis of those aHRs (the ones for women), pretending they’re all for the same diet, and quantitatively check the heterogeneity we get. Doing so, with a random-effects meta-analysis, we find that the I2 is exactly 0%, as is the τ2. The p-value for heterogeneity is 0.92. Whereas this study should update us a little bit on pescatarian diets being better than vegan diets for women, these differences are almost certainly due to chance. No one would suspect that these are actually different diets if you had a meta-analysis with those numbers.
Since the total meta-analytic aHR is also very close to 1, it also looks like none of the diets are meaningfully associated with increased or decreased mortality for women, though there was a slight trend towards lower mortality compared to the nonvegetarian reference diet (p-value: 0.11).
Code:
library(metafor)
# This is a file with the point estimate and 95% confidence interval for the aHR of each AHS-2 diet.
# We're looking at data from women specifically.
diet_dataset <- read.csv("diets.csv")
# Calculate the SEs from the 95% CIs
diet_dataset$logHR <- log(diet_dataset$HR)
diet_dataset$SElogHR <- (log(diet_dataset$CI_high) - log(diet_dataset$CI_low)) / 3.92
# Run the meta-analysis
res <- rma(yi = logHR, sei = SElogHR, data = diet_dataset, method = "REML")
# Print the results
summary(res)
Let me make sure I understand correctly: you view AHS-2 as supporting the belief that vegan diets can be healthy, for some people, but not anything larger than that?
I’d also like to point out that the sentences you describe as “slightly misleading” come immediately after I said “if you take the data at face value (which you shouldn’t)”. So as far as I can tell we’re in agreement here, and the area of contention is that I thought you were using AHS2 to endorse much larger claims than you are.
it seems slightly misleading to claim “Adventist Health Study-2 supports pescetarianism more than veganism,” or that vegan diets were “the worst choice besides unconstrained meat consumption” for women. What these CIs indicate is that there was likely no difference between pescatarian and vegan diets for men, and likely no difference between any of the diets for women
I’d also like to point out that the sentences you describe as “slightly misleading” come immediately after I said “if you take the data at face value (which you shouldn’t)”. So as far as I can tell we’re in agreement here
The first sentence is the title of the post, which has no hedging. The post also claims that“[i]f you’re going to conclude anything from these papers, it’s that fish are great,” which doesn’t seem to be the correct takeaway of this specific study given how wide and similar so many of the aHR 95% CIs are. You also claim “[o]utcomes for veganism are [...] worse than everything except for omnivorism in women” in another post without hedging.
In any case, taking the data at face value doesn’t imply “ignoring the confidence intervals.” I don’t see why it would imply that.
I thought you were using AHS2 to endorse much larger claims than you are.
My original comment concluded that there was a “substantial probability” that vegan diets are healthier, and quoted someone (correctly) showing that vegans have lower mortality than omnivores in the AHS-2. I didn’t mean to claim it was the healthiest option for everyone; the comment is perfectly compatible with this null result for women. I also claimed that it wasn’t obvious that vegan diets can be expected to make you less healthy ex ante, modulo things like B12. I apologize if it wasn’t clear.
I think everything I said makes sense in a context where people (including the study authors) are using this paper to argue for veg*nism. I’m arguing that the study is both weaker than reported, and (weakly) pointing in a different direction than reported. I agree that if a fish farm lobbying group was using this paper to claim that fish were the cure for aging, that would be very misleading and I’d argue against them too.
What you originally said was “say it’s not at all obvious that a vegan diet has health tradeoffs ex-ante”. I think what you meant here was “it’s not clear a vegan diet is net negative.” A vegan diet leading to lower energy levels but longer lifespan is the definition of a trade-off.
It would be helpful if you clarified the population you are talking about. I’ve already said I think some people’s optimal diet is vegan, and for some other people vegan is the best out of the options they can realistically achieve. So unless you mean a substantial probability everyone’s optimal diet is vegan, and there is no such thing as a prohibitive health issue, we’re not disagreeing.
I also feel like saying “modulo things like B12” is burying the lede. A lot of my point is that vegan advocates are recruiting people without providing the necessary nutritional education, and are in some cases fighting that education even when it’s done in a vegan-compatible way.
What you originally said was “say it’s not at all obvious that a vegan diet has health tradeoffs ex-ante”. I think what you meant here was “it’s not clear a vegan diet is net negative.” A vegan diet leading to lower energy levels but longer lifespan is the definition of a trade-off.
This might be semantics, but when you said “Change my mind: Veganism entails trade-offs, and health is one of the axes” I (until now) interpreted the claim as vegans needing to trade off health (writ large) against other desirable properties (taste, cost, convenience, etc), not a tradeoff within different components of health.
I don’t have a sense of how common my reading was, however, and I don’t want to put words in Natalia’s mouth.
[note: this comment was edited after people’s replies.]
I think it’s useful to look at the confidence intervals, rather than only the point estimates.
The 95% confidence intervals of the adjusted hazard ratios for overall mortality, for men, were [0.56, 0.92] and [0.57, 0.93] for vegan and pescatarian diets, respectively, and for women the CIs are [0.72, 1.07] and [0.78, 1.20], respectively. For women, the confidence intervals for all diets are [0.78, 1.2], [0.83, 1.07], [0.72, 1.07] and [0.7, 1.22].
What these CIs indicate is that there was likely no difference between pescatarian and vegan diets for men, both of which are better than omnivorism, and likely no difference between any of the diets for women.
The CIs for women specifically look so similar that you could pretend that all of those CIs came from different studies examining the exact same diet, and write a meta-analysis with them, and readers of the meta-analysis would think, “oh, cool, there’s no heterogeneity among the studies!”
In fact, we can go ahead and run a meta-analysis of those aHRs (the ones for women), pretending they’re all for the same diet, and quantitatively check the heterogeneity we get. Doing so, with a random-effects meta-analysis, we find that the I2 is exactly 0%, as is the τ2. The p-value for heterogeneity is 0.92. Whereas this study should update us a little bit on pescatarian diets being better than vegan diets for women, these differences are almost certainly due to chance. No one would suspect that these are actually different diets if you had a meta-analysis with those numbers.
Since the total meta-analytic aHR is also very close to 1, it also looks like none of the diets are meaningfully associated with increased or decreased mortality for women, though there was a slight trend towards lower mortality compared to the nonvegetarian reference diet (p-value: 0.11).
Code:
Let me make sure I understand correctly: you view AHS-2 as supporting the belief that vegan diets can be healthy, for some people, but not anything larger than that?
I’d also like to point out that the sentences you describe as “slightly misleading” come immediately after I said “if you take the data at face value (which you shouldn’t)”. So as far as I can tell we’re in agreement here, and the area of contention is that I thought you were using AHS2 to endorse much larger claims than you are.
The first sentence is the title of the post, which has no hedging. The post also claims that“[i]f you’re going to conclude anything from these papers, it’s that fish are great,” which doesn’t seem to be the correct takeaway of this specific study given how wide and similar so many of the aHR 95% CIs are. You also claim “[o]utcomes for veganism are [...] worse than everything except for omnivorism in women” in another post without hedging.
In any case, taking the data at face value doesn’t imply “ignoring the confidence intervals.” I don’t see why it would imply that.
My original comment concluded that there was a “substantial probability” that vegan diets are healthier, and quoted someone (correctly) showing that vegans have lower mortality than omnivores in the AHS-2. I didn’t mean to claim it was the healthiest option for everyone; the comment is perfectly compatible with this null result for women. I also claimed that it wasn’t obvious that vegan diets can be expected to make you less healthy ex ante, modulo things like B12. I apologize if it wasn’t clear.
I think everything I said makes sense in a context where people (including the study authors) are using this paper to argue for veg*nism. I’m arguing that the study is both weaker than reported, and (weakly) pointing in a different direction than reported. I agree that if a fish farm lobbying group was using this paper to claim that fish were the cure for aging, that would be very misleading and I’d argue against them too.
What you originally said was “say it’s not at all obvious that a vegan diet has health tradeoffs ex-ante”. I think what you meant here was “it’s not clear a vegan diet is net negative.” A vegan diet leading to lower energy levels but longer lifespan is the definition of a trade-off.
It would be helpful if you clarified the population you are talking about. I’ve already said I think some people’s optimal diet is vegan, and for some other people vegan is the best out of the options they can realistically achieve. So unless you mean a substantial probability everyone’s optimal diet is vegan, and there is no such thing as a prohibitive health issue, we’re not disagreeing.
I also feel like saying “modulo things like B12” is burying the lede. A lot of my point is that vegan advocates are recruiting people without providing the necessary nutritional education, and are in some cases fighting that education even when it’s done in a vegan-compatible way.
This might be semantics, but when you said “Change my mind: Veganism entails trade-offs, and health is one of the axes” I (until now) interpreted the claim as vegans needing to trade off health (writ large) against other desirable properties (taste, cost, convenience, etc), not a tradeoff within different components of health.
I don’t have a sense of how common my reading was, however, and I don’t want to put words in Natalia’s mouth.
Life is trade-offs all the way down.