I think your response to SMTM is great. It points out all kinds of shortcomings in their representation of the underlying data. I don’t think lithium is anything like the comprehensive explanation they make it out to be. However, it’s also important to note that your analysis is (and claims to be) nothing more or less than a case for healthy skepticism. “It’s probably not the lithium” doesn’t mean “it’s not worth it to check.”
Note that Matthew Barnett, in his bet, claims to have credence of ~1% that the SMTM hypothesis is “true...”
… but didn’t even suggest what sort of evidence they could use as resolution criteria. Note: Matthew did later write a post (linked in his response to me below) doing exactly this. He just didn’t do it in the Twitter thread, partly because it’s a short format.
I doubt even SMTM thinks lithium explains literally 100% of the obesity epidemic. Matthew must not think they think that either. It’s very hard to interpret what these credences mean in the absence of a specific operationalization. I think it’s less than 1% likely that lithium has a perfect correlation with BMI (and thus that it completely explains obesity), but more than 1% likely that normal dietary doses of lithium are an important causal contributor to the obesity epidemic, via pathways that may be complex and not easy to parse from a single study or metric.
I’m skeptical that refusal to take a bet/call-out on Twitter amounts to a failure to stand by your beliefs. People are loss averse and have a variety of feelings about both bets and the kinds of people who offer them. This bet offer in particular seems to give off more heat than light. It doesn’t feel fun or collaborative when I read through the Twitter thread, and it should feel that way.
Edit: I am splitting off my thoughts on the hypothesis itself into a separate comment, because the social element and epistemic element feel like they distract from each other.
Note that Matthew Barnett, in his bet, claims to have credence of ~1% that the SMTM hypothesis is “true...” but didn’t even suggest what sort of evidence they could use as resolution criteria.
This feels unfair to me. I stated in the original tweet,
If [SMTM] takes me up on this offer, I will try to work with them to generate reasonable, unambiguous resolution criteria, and fair odds. I will risk up to $1000.
The reason why I didn’t suggest any evidence that might be relevant is because I wanted to see what they would come up with (that, and also Twitter encourages you to be brief). However, I later wrote a post engaging with cruxes that I think are important, and in particular, what I think would need to be false for the (alternative) palatability theory of obesity to be falsified.
Other people have also accused me of being unfair to SMTM here, mostly because of the perverse incentives involved with betting them, but I don’t really buy this accusation. Letting someone choose the criteria to a proposed bet maximizes the chances that they can come up with something fair. And nowhere did I continue to harass them about not betting me. I also don’t think their failure to bet me played a significant role in anyone’s argument for why SMTM’s theory is false, anywhere (either in my comments, tweets, or in Natalia’s).
I doubt even SMTM thinks lithium explains literally 100% of the obesity epidemic.
Maybe not, but SMTM did literally summarize their theory as, “These contaminants are the only cause of the obesity epidemic, and the worldwide increase in obesity rates since 1980 is entirely attributable to their effects.” (Note that my bet was not about lithium but was instead about their general theory, which just proposes some small set of contaminants).
However, I later wrote a post engaging with cruxes that I think are important, and in particular, what I think would need to be false for the (alternative) palatability theory of obesity to be falsified.
That does address the epistemic issue in the present moment. I hadn’t seen it. I will note that in my original comment.
Socially, I think it would have been preferable to first write your crux post, and then to approach them on Twitter. It doesn’t take persistent “harassment” (I didn’t see your post as harassment, and this seems like playful hyperbole to me) to create persistent social tension. A strident Twitter challenge to make a fairly large bet will do that all by itself. I have no idea whether the author(s?) at SMTM actually feel tense. It’s just how it resonated with me, on an emotional level.
I also agree that their failure to bet you is not relevant to most people’s credences for the lithium hypothesis, but their general approach to epistemics was a persistent theme in Natalia’s post, and this was the part that bugged me.
I doubt even SMTM thinks lithium explains literally 100% of the obesity epidemic.
Maybe not, but SMTM did literally summarize their theory as, “These contaminants are the only cause of the obesity epidemic, and the worldwide increase in obesity rates since 1980 is entirely attributable to their effects.” (Note that my bet was not about Lithium but was instead about their general theory, which just proposes some small set of contaminants).
Good catch, I wasn’t precise enough with my language. What I meant was, “I doubt even SMTM thinks lithium explains literally 100% of the prevalence of obesity,” and of course we should replace “lithium” with “environmental contaminants.”
I read them as giving the strongest possible version of their hypothesis. It would also be possible to give their statement about what they view as the most likely version of their hypothesis, in which environmental contaminants explain some range of percentages of the worldwide increase in obesity rates since 1980. These are two distinct and useful statements to make, but the strong statement is easier to make than the credence-based statement, and makes it easier for a new reader to integrate the subsequent argument.
Letting someone choose the criteria to a proposed bet maximizes the chances that they can come up with something fair.
True, but it’s unclear to me how this relates to the parent. Is your implicit argument that because of this, refusing to take the bet does amount to a failure to stand by their beliefs? Because giving someone the fairest possible framework for a bet doesn’t mean there can be no other objection to taking it, nor imply anything about their argument if they choose not to.
STMT’s reason for doing so was very reasonable. We don’t want them to want it to be true (any more than is inevitable). In fact, I would go so far as to argue that bets on predictions about the world should never involve any party actively investigating those claims, and you should be suspicious of any research from someone who is conflicted in this way.
And while you claim that their “failure” to bet you didn’t play much of a role in anyone’s argument, you sure do seem to be making a lot of fairly combative noise about it. Betting on predictions is can be a useful collaborative tool, but in this case it feels more like a weapon-and a deterrent to speculative investigation.
Is your implicit argument that because of this, refusing to take the bet does amount to a failure to stand by their beliefs?
No
And while you claim that their “failure” to bet you didn’t play much of a role in anyone’s argument, you sure do seem to be making a lot of fairly combative noise about it.
I feel like you are misrepresenting what happened here [ETA: reworded this statement]. I am not, nor have I anywhere, independently brought up the bet to make any claim about SMTM’s standards of rigor, so far as I can recall. So, whatever “noise” you think I’m making about this bet must merely be referring to the replies I’m making to people, who are saying I’m being unfair. This includes,
Replying on Twitter to SMTM several months ago explaining why I think the perverse incentives are not so bad.
Replying to people in this thread, and more recently on Twitter, who are accusing me of being unfair to SMTM, and claiming that this bet somehow plays a critical role in the discussion (despite being from January).
This is totally distinct from making loud noises, trying to draw attention to the bet (which was, in any case, mentioned only as a side-note in the original post).
Well this isn’t helpful! I was genuinely trying to understand what the point of the quoted statement is. In the context, it seemed like that was the most reasonable interpretation. If it isn’t, then it’d be more productive to explain what you did mean.
I’m sorry that you feel misrepresented. For me, continuing to argue (in response to criticism or otherwise) that there is something wrong with STMT not taking the bet, and that their stated reason is insufficient, and making what read to me like implicit accusations of dishonesty, seems a lot like ‘making combative noise’. It’s quite an imprecise charge, though, and perhaps unhelpful of me to make.
Anyway, I certainly don’t want to be making combative noise, and policing your tone isn’t really adding anything to the (important) object-level discussion, so I’ll beat a retreat.
For me, continuing to argue (in response to criticism or otherwise) that there is something wrong with STMT not taking the bet, and that their stated reason is insufficient, and making what read to me like implicit accusations of dishonesty, seems a lot like ‘making combative noise’.
For the record, I don’t think there’s anything wrong with what SMTM did in the thread.
[ETA: partially retracted] But you can see what you’re doing right? If you criticize me, and I reply to the criticism, and then you say, “Ahah! You are making combative noises by replying to criticism!” it’s not a damning statement about my intent. Of course it makes sense that I’m going to reply to criticism, if I think the criticism is not justified.
As for whether I made an implicit accusation of dishonestly, let me just clarify: I did not. I do not currently think SMTM is being actively dishonest about anything related to this discussion, as far as I can tell. And in general, I think few researchers practice explicit dishonesty about their primary research.
Strong disagreement is not the same thing as accusing someone of dishonesty. I don’t know what claim you’re referring to when you said you read me as implicitly making an accusation of dishonesty, but I don’t see it anywhere.
I certainly don’t “see what I’m doing”, because I wasn’t trying to do anything other than explain why your engagement with STMT seemed combative and unfairly accusatory. It did, and it does, reading it later. I hope/suspect that with the advantage of the same temporal remove, you will also see exactly why I and many others thought so.
I think I would have reworded some of my tweets to make the tone slightly better, if I had the benefit of hindsight (and had known how people would interpret what I said). But otherwise, I still don’t see why my offer to bet SMTM was “combative and unfairly accusatory”. I never accused them of dishonesty. I did suggest strong disagreement, but that seems only mildly combative. Mostly, I offered a friendly bet, and when they rejected it, I moved on.
ETA: In contrast to the twitter thread, I do think I was unfairly combative to you in particular. So, in the spirit of trying to be nicer, I’ve decided to partially retract some of my comments.
I think your response to SMTM is great. It points out all kinds of shortcomings in their representation of the underlying data. I don’t think lithium is anything like the comprehensive explanation they make it out to be. However, it’s also important to note that your analysis is (and claims to be) nothing more or less than a case for healthy skepticism. “It’s probably not the lithium” doesn’t mean “it’s not worth it to check.”
Note that Matthew Barnett, in his bet, claims to have credence of ~1% that the SMTM hypothesis is “true...”
… but didn’t even suggest what sort of evidence they could use as resolution criteria. Note: Matthew did later write a post (linked in his response to me below) doing exactly this. He just didn’t do it in the Twitter thread, partly because it’s a short format.
I doubt even SMTM thinks lithium explains literally 100% of the obesity epidemic. Matthew must not think they think that either. It’s very hard to interpret what these credences mean in the absence of a specific operationalization. I think it’s less than 1% likely that lithium has a perfect correlation with BMI (and thus that it completely explains obesity), but more than 1% likely that normal dietary doses of lithium are an important causal contributor to the obesity epidemic, via pathways that may be complex and not easy to parse from a single study or metric.
I’m skeptical that refusal to take a bet/call-out on Twitter amounts to a failure to stand by your beliefs. People are loss averse and have a variety of feelings about both bets and the kinds of people who offer them. This bet offer in particular seems to give off more heat than light. It doesn’t feel fun or collaborative when I read through the Twitter thread, and it should feel that way.
Edit: I am splitting off my thoughts on the hypothesis itself into a separate comment, because the social element and epistemic element feel like they distract from each other.
This feels unfair to me. I stated in the original tweet,
The reason why I didn’t suggest any evidence that might be relevant is because I wanted to see what they would come up with (that, and also Twitter encourages you to be brief). However, I later wrote a post engaging with cruxes that I think are important, and in particular, what I think would need to be false for the (alternative) palatability theory of obesity to be falsified.
Other people have also accused me of being unfair to SMTM here, mostly because of the perverse incentives involved with betting them, but I don’t really buy this accusation. Letting someone choose the criteria to a proposed bet maximizes the chances that they can come up with something fair. And nowhere did I continue to harass them about not betting me. I also don’t think their failure to bet me played a significant role in anyone’s argument for why SMTM’s theory is false, anywhere (either in my comments, tweets, or in Natalia’s).
Maybe not, but SMTM did literally summarize their theory as, “These contaminants are the only cause of the obesity epidemic, and the worldwide increase in obesity rates since 1980 is entirely attributable to their effects.” (Note that my bet was not about lithium but was instead about their general theory, which just proposes some small set of contaminants).
That does address the epistemic issue in the present moment. I hadn’t seen it. I will note that in my original comment.
Socially, I think it would have been preferable to first write your crux post, and then to approach them on Twitter. It doesn’t take persistent “harassment” (I didn’t see your post as harassment, and this seems like playful hyperbole to me) to create persistent social tension. A strident Twitter challenge to make a fairly large bet will do that all by itself. I have no idea whether the author(s?) at SMTM actually feel tense. It’s just how it resonated with me, on an emotional level.
I also agree that their failure to bet you is not relevant to most people’s credences for the lithium hypothesis, but their general approach to epistemics was a persistent theme in Natalia’s post, and this was the part that bugged me.
Good catch, I wasn’t precise enough with my language. What I meant was, “I doubt even SMTM thinks lithium explains literally 100% of the prevalence of obesity,” and of course we should replace “lithium” with “environmental contaminants.”
I read them as giving the strongest possible version of their hypothesis. It would also be possible to give their statement about what they view as the most likely version of their hypothesis, in which environmental contaminants explain some range of percentages of the worldwide increase in obesity rates since 1980. These are two distinct and useful statements to make, but the strong statement is easier to make than the credence-based statement, and makes it easier for a new reader to integrate the subsequent argument.
True, but it’s unclear to me how this relates to the parent. Is your implicit argument that because of this, refusing to take the bet does amount to a failure to stand by their beliefs? Because giving someone the fairest possible framework for a bet doesn’t mean there can be no other objection to taking it, nor imply anything about their argument if they choose not to.
STMT’s reason for doing so was very reasonable. We don’t want them to want it to be true (any more than is inevitable). In fact, I would go so far as to argue that bets on predictions about the world should never involve any party actively investigating those claims, and you should be suspicious of any research from someone who is conflicted in this way.
And while you claim that their “failure” to bet you didn’t play much of a role in anyone’s argument, you sure do seem to be making a lot of fairly combative noise about it. Betting on predictions is can be a useful collaborative tool, but in this case it feels more like a weapon-and a deterrent to speculative investigation.
No
I feel like you are misrepresenting what happened here [ETA: reworded this statement]. I am not, nor have I anywhere, independently brought up the bet to make any claim about SMTM’s standards of rigor, so far as I can recall. So, whatever “noise” you think I’m making about this bet must merely be referring to the replies I’m making to people, who are saying I’m being unfair. This includes,
Replying on Twitter to SMTM several months ago explaining why I think the perverse incentives are not so bad.
Replying to people in this thread, and more recently on Twitter, who are accusing me of being unfair to SMTM, and claiming that this bet somehow plays a critical role in the discussion (despite being from January).
This is totally distinct from making loud noises, trying to draw attention to the bet (which was, in any case, mentioned only as a side-note in the original post).
Well this isn’t helpful! I was genuinely trying to understand what the point of the quoted statement is. In the context, it seemed like that was the most reasonable interpretation. If it isn’t, then it’d be more productive to explain what you did mean.
I’m sorry that you feel misrepresented. For me, continuing to argue (in response to criticism or otherwise) that there is something wrong with STMT not taking the bet, and that their stated reason is insufficient, and making what read to me like implicit accusations of dishonesty, seems a lot like ‘making combative noise’. It’s quite an imprecise charge, though, and perhaps unhelpful of me to make.
Anyway, I certainly don’t want to be making combative noise, and policing your tone isn’t really adding anything to the (important) object-level discussion, so I’ll beat a retreat.
For the record, I don’t think there’s anything wrong with what SMTM did in the thread.
[ETA: partially retracted]
But you can see what you’re doing right? If you criticize me, and I reply to the criticism, and then you say, “Ahah! You are making combative noises by replying to criticism!” it’s not a damning statement about my intent. Of course it makes sense that I’m going to reply to criticism, if I think the criticism is not justified.As for whether I made an implicit accusation of dishonestly, let me just clarify: I did not. I do not currently think SMTM is being actively dishonest about anything related to this discussion, as far as I can tell. And in general, I think few researchers practice explicit dishonesty about their primary research.
Strong disagreement is not the same thing as accusing someone of dishonesty. I don’t know what claim you’re referring to when you said you read me as implicitly making an accusation of dishonesty, but I don’t see it anywhere.
I certainly don’t “see what I’m doing”, because I wasn’t trying to do anything other than explain why your engagement with STMT seemed combative and unfairly accusatory. It did, and it does, reading it later. I hope/suspect that with the advantage of the same temporal remove, you will also see exactly why I and many others thought so.
I think I would have reworded some of my tweets to make the tone slightly better, if I had the benefit of hindsight (and had known how people would interpret what I said). But otherwise, I still don’t see why my offer to bet SMTM was “combative and unfairly accusatory”. I never accused them of dishonesty. I did suggest strong disagreement, but that seems only mildly combative. Mostly, I offered a friendly bet, and when they rejected it, I moved on.
ETA: In contrast to the twitter thread, I do think I was unfairly combative to you in particular. So, in the spirit of trying to be nicer, I’ve decided to partially retract some of my comments.
Just to maximally clarify, I didn’t mean to suggest that the offer was itself inherently combative.
To clarify, was this double negative intended?
No, I corrected it.