Thanks Jeff. I’m curious how you feel about my original phrasing, versus what’s here.
I agree that the CIs heavily overlap, and drawing strong conclusions from this would be unjustified. I nonetheless think it’s relevant that even if you treat the results as meaningful, the summaries given by the abstract and some commenters are inaccurate (Natalia has since said she was making a more limited claim). That means they’re making two errors (overstating effect, and effect in wrong direction) rather than just one (overstating effect).
I can see how my phrasing here wouldn’t convey all that if you don’t click through the link, and that seems worth fixing, but I’m curious what you think of the underlying point.
That means they’re making two errors (overstating effect, and effect in wrong direction) rather than just one (overstating effect).
Froolow’s comment claimed that “there’s somewhere between a small signal and no signal that veganism is better with respect to all-cause mortality than omnivorism.” How is that a misleading way of summarizing the adjusted hazard ratio 0.85 (95% CI, 0.73–1.01), in either magnitude or direction? Should he have said that veganism is associated with higher mortality instead?
None of the comments you mentioned in that section claimed that veganism was associated with lower mortality in all subgroups (e.g. women). But even if they had, the hazard ratio for veganism among women was still in the “right” direction (below 1, though just slightly and not meaningfully). Other diets were (just slightly and not meaningfully) better among women, but none of the commenters claimed that veganism was better than all diets either.
Unrelatedly, I noticed that in this comment (and in other comments you’ve made regarding my points about confidence intervals) you don’t seem to argue that the sentence “[o]utcomes for veganism are [...] worse than everything except for omnivorism in women” is not misleading.
I think one issue here is that my phrasing was bad. I meant “support” as in “doesn’t support claims of superiority”, but reading it now it’s obvious that it could be read as “doesn’t support claims as fine”, which is not what I meant. I agree that veganism works for some people. I want to fix that, although am going to wait until I’m sure there aren’t other changes, because all changes need to be made in triplicate.
I appreciate you pointing out that potential reading so I could fix it. However I am extremely frustrated with this conversation overall. You are holding me to an exacting standard while making basic errors yourself, such as: * Quoted me as saying “X is great”, when what I said was “If you believe Y, which you shouldn’t, X is great” * implying that when I wrote this post I ignored a response from you, when the response was made a day after this was posted posted, after I pointed out your non-response in comments. (This was somewhat fixed after another commenter pointed it out) * Your original comment on Change My Mind referred to this study as “missing context”, as if it was something I should have known and deliberately left out. That’s a loaded implication in general, but outright unfair when the post is titled “change my mind” and its entire point was asking for exactly that kind of information.
The second point here was not intended and I fixed it within 2 minutes of orthonormal pointing it out, so it doesn’t seem charitable to bring that up. (Though I just re-edited that comment to make this clearer).
I’m not sure what to say regarding the third point other than that I didn’t mean to imply that you “should have known and deliberately left out” that study. I just thought it was (literally) useful context. Just edited that comment.
All of this also seems unrelated to this discussion. I’m not sure why me addressing your arguments is being construed as “holding [you] to an exacting standard.”
Thanks Jeff. I’m curious how you feel about my original phrasing, versus what’s here.
I agree that the CIs heavily overlap, and drawing strong conclusions from this would be unjustified. I nonetheless think it’s relevant that even if you treat the results as meaningful, the summaries given by the abstract and some commenters are inaccurate (Natalia has since said she was making a more limited claim). That means they’re making two errors (overstating effect, and effect in wrong direction) rather than just one (overstating effect).
I can see how my phrasing here wouldn’t convey all that if you don’t click through the link, and that seems worth fixing, but I’m curious what you think of the underlying point.
Froolow’s comment claimed that “there’s somewhere between a small signal and no signal that veganism is better with respect to all-cause mortality than omnivorism.” How is that a misleading way of summarizing the adjusted hazard ratio 0.85 (95% CI, 0.73–1.01), in either magnitude or direction? Should he have said that veganism is associated with higher mortality instead?
None of the comments you mentioned in that section claimed that veganism was associated with lower mortality in all subgroups (e.g. women). But even if they had, the hazard ratio for veganism among women was still in the “right” direction (below 1, though just slightly and not meaningfully). Other diets were (just slightly and not meaningfully) better among women, but none of the commenters claimed that veganism was better than all diets either.
Unrelatedly, I noticed that in this comment (and in other comments you’ve made regarding my points about confidence intervals) you don’t seem to argue that the sentence “[o]utcomes for veganism are [...] worse than everything except for omnivorism in women” is not misleading.
I think one issue here is that my phrasing was bad. I meant “support” as in “doesn’t support claims of superiority”, but reading it now it’s obvious that it could be read as “doesn’t support claims as fine”, which is not what I meant. I agree that veganism works for some people. I want to fix that, although am going to wait until I’m sure there aren’t other changes, because all changes need to be made in triplicate.
I appreciate you pointing out that potential reading so I could fix it. However I am extremely frustrated with this conversation overall. You are holding me to an exacting standard while making basic errors yourself, such as:
* Quoted me as saying “X is great”, when what I said was “If you believe Y, which you shouldn’t, X is great”
* implying that when I wrote this post I ignored a response from you, when the response was made a day after this was posted posted, after I pointed out your non-response in comments. (This was somewhat fixed after another commenter pointed it out)
* Your original comment on Change My Mind referred to this study as “missing context”, as if it was something I should have known and deliberately left out. That’s a loaded implication in general, but outright unfair when the post is titled “change my mind” and its entire point was asking for exactly that kind of information.
The second point here was not intended and I fixed it within 2 minutes of orthonormal pointing it out, so it doesn’t seem charitable to bring that up. (Though I just re-edited that comment to make this clearer).
The first point was already addressed here.
I’m not sure what to say regarding the third point other than that I didn’t mean to imply that you “should have known and deliberately left out” that study. I just thought it was (literally) useful context. Just edited that comment.
All of this also seems unrelated to this discussion. I’m not sure why me addressing your arguments is being construed as “holding [you] to an exacting standard.”