It seems clear to me that it is a very bad example. I find that consistently the worst part of Eliezer’s non-fiction writing is that he fails to separate contentious claims from writings on unrelated subjects. Moreover, he usually discards the traditional view as ridiculous rather than admitting that its incorrectness is extremely non-obvious. He goes so far in this piece as to give the standard view a straw-man name and to state only the most laughable of its proponents’ justifications. This mars an otherwise excellent piece and I am unwilling to recommend this article to those who are not already reading LW.
in the virtue theory of traffic, drinking alcohol actually causes accidents due to divine punishment for the sin of intemperance
Of course not! The real reason drinkers cause more accidents is that low-conscientiousness people are both more likely to drink before driving and more likely to drive recklessly. (The impairment of reflexes due to alcohol does itself have an effect, but it’s not much larger than that due to e.g. sleep deprivation.) If a high-conscientiousness person was randomly assigned to the “drunk driving” condition, they would drive extra cautiously to compensate for their impairment. ;-)
(I’m exaggerating for comical effect, but I do believe a weaker version of this.)
“Extremely non-obvious”? Have you looked at how many calories one hour of exercise burns, and compared that to how many calories foodstuffs common in the First World contain?
I agree that focusing on input has far higher returns than focusing on output. Simple calorie comparison predicts it, and in my personal experience I’ve noted small appearance and weight changes after changes in exercise level and large appearance and weight changes after changes in intake level. That said, the traditional view- “eat less and exercise more”- has the direction of causation mostly right for both interventions and to represent it as just “exercise more” seems mistaken.
I was also distracted by the footnotes, since even though I found them quite funny, [3] at least is obviously wrong: “there’s no known intervention which can cause weight loss.” Sure there is—the effectiveness of bariatric surgery is quite well evidenced at this point (http://en.wikipedia.org/wiki/Bariatric_surgery#Weight_loss).
I generally share Eliezer’s assessment of the state of conventional wisdom in dietary science (abysmal), but careless formulations like this one are—well, distracting.
Also, even if he meant non-surgical interventions, it should be “which can reliably cause long-term weight loss”—there are people who lose weight by dieting, and a few of them don’t even gain it back.
I suspect this depends on the handling of the issue. Eliezer presenting his model of the world as “common sense,” straw manning the alternative, and then using fake data that backs up his preferences is, frankly, embarrassing.
This is especially troublesome because this is an introductory explanation to a technical topic- something Eliezer has done well before- and introductory explanations are great ways to introduce people to Less Wrong. But how can I send this to people I know who will notice the bias in the second paragraph and stop reading because that’s negative evidence about the quality of the article? How can I send this to people I know who will ask why he’s using two time-variant variables as single acyclic nodes, rather than a ladder (where exercise and weight at t-1 both cause exercise at t and weight at t)?
What would it look like to steel man the alternative? One of my physics professors called ‘calories in-calories out=change in fat’ the “physics diet,” since it was rooted in conservation of energy; that seems like a far better name. Like many things in physics, it’s a good first order approximation to the engineering reality, but there are meaningful second order terms to consider. “Calories in” is properly “calories absorbed” not “calories put into your mouth”- though we’ll note it’s difficult to absorb more calories than you put into your mouth. Similarly, calories out is non-trivial to measure- current weight and activity level can give you a broad guess, but it can be complicated by many things, like ambient temperature! Any attempt we make to control calories in and calories out will have to be passed through the psychology and physiology of the person in question, making them even more difficult to control in the field.
and about the method.
Compare the volume of discussion of the method and the overweight-exercise link in the comments.
How can I send this to people I know who will ask why he’s using two time-variant variables as single acyclic nodes, rather than a ladder (where exercise and weight at t-1 both cause exercise at t and weight at t)?
Why do you need to send this article to people who could ask that? If you’re saying “Oh, this should actually be modeled using causal markov chains...” then this is probably too basic for you.
Why do you need to send this article to people who could ask that?
Because I’m still a grad student, most of those people that I know are routinely engaged in teaching these sorts of concepts, and so will find articles like this useful for pedagogical reasons.
I’m torn. On the one hand, using the method to explain something the reader probably was not previously aware of is an awesome technique that I truly appreciate. Yet Vaniver’s point that controversial opinions should not be unnecessarily put into introductory sequence posts makes sense. It might turn off readers who would otherwise learn from the text, like nyan sandwich.
In my opinion, the best fix would be to steelman the argument as much as possible. Call it the physics diet, not the virtue-theory of metabolism. Add in an extra few sentences that really buff up the basics of the physics diet argument. And, at the end, include a note explaining why the physics diet doesn’t work (appetite increases as exercise increases).
The point Eliezer is addressing is the one that RichardKennaway brought up separately. Causal models can still function with feedback (in Causality, Pearl works through an economic model where price and quantity both cause each other, and have their own independent causes), but it’s a bit more bothersome.
A model where the three are one-time events- like, say, whether a person has a particular gene, whether or not they were breastfed, and their height as an adult- won’t have the problem of being cyclic, but will have the pedagogical problem that the causation is obvious from the timing of the events.
One could have, say, the weather witch’s prediction of whether or not there will be rain, whether or not you brought an umbrella with you, and whether or not it rained. Aside from learning, this will be an acyclic system that has a number of plausible underlying causal diagrams (with the presence of the witch making the example clearly fictional and muddying our causal intuitions, so we can only rely on the math).
In my opinion, the best fix would be to steelman the argument as much as possible.
The concept of inferential distance suggests to me that posts should try and make their pathways as short and straight as possible. Why write a double-length post that explains both causal models and metabolism, when you could write a single-length post that explains only causal models? (And, if metabolism takes longer to discuss than causal models, the post will mostly be about the illustrative detour, not the concept itself!)
The concept of inferential distance suggests to me that posts should try and make their pathways as short and straight as possible. Why write a double-length post that explains both causal models and metabolism, when you could write a single-length post that explains only causal models? (And, if metabolism takes longer to discuss than causal models, the post will mostly be about the illustrative detour, not the concept itself!)
You’ve convinced me. I now agree that EY should go back and edit the post to use a different more conventional example.
In my opinion, the best fix would be to steelman the argument as much as possible. Call it the physics diet, not the virtue-theory of metabolism.
“Physics diet” and “virtue-theory of metabolism” are not steelman and strawman of each other; they are quite different things. Proponents of the physics diet (e.g. John Walker) do not say that if you want to lose weight you should exercise more—they say you should eat less. EDIT: the strawman of this would be the theory that “excessive eating actually causes weight gain due to divine punishment for the sin of gluttony” (inspired by Yvain’s comment).
Seriously; that was intended to be an example. What’s it matter whether the nodes are labelled “exercise/overweight/internet” or “foo/bar/baz”? (But yeah, Footnote 1 doesn’t belong there, and Footnote 3 might mention eating.)
Taking a “contentious” point and resolving it in to a settled fact made the whole article vastly more engaging to me. It also struck me as an elegant demonstration of the value of the tool: It didn’t simply introduce the concept, it used it to accomplish something worthwhile.
Eliezer’s data is made up, but all the not-made-up research I’ve seen supports his actual conclusion. The net emotional result was the same for me as if he’d used the actual research, since my brain could substitute it in.
Perhaps I am weird in having this emotional link, or perhaps I am simply more familiar with the not-made-up research than you.
The net emotional result was the same for me as if he’d used the actual research, since my brain could substitute it in.
I understand. I think it’s important to watch out for these sorts of illusions of transparency, though, especially when dealing with pedagogical material. One of the heuristics I’ve been using is “who would I not recommend this to?”, because that will use social modules my brain is skilled at using to find holes and snags in the article. I don’t know how useful that heuristic will be to others, and welcome the suggestion of others.
perhaps I am simply more familiar with the not-made-up research than you.
I am not an expert in nutritional science, but it appears to me that there is controversy among good nutritionists. This post also aided my understanding of the issue. I also detail some more of my understanding in this comment down another branch.
EDIT: Also, doing some more poking around now, this seems relevant.
P.S. When quoting two people, it can be useful to attribute the quotes. I initially thought the second quote was your way of doing a snarky editorial comment on what I’d said, not quoting the article...
It’s not clear to me that the virtue theory of metabolism is a good example for this post, since it’s likely to be highly contentious.
It seems clear to me that it is a very bad example. I find that consistently the worst part of Eliezer’s non-fiction writing is that he fails to separate contentious claims from writings on unrelated subjects. Moreover, he usually discards the traditional view as ridiculous rather than admitting that its incorrectness is extremely non-obvious. He goes so far in this piece as to give the standard view a straw-man name and to state only the most laughable of its proponents’ justifications. This mars an otherwise excellent piece and I am unwilling to recommend this article to those who are not already reading LW.
Yeah, I didn’t even mind the topic, but I thought this particular sentence was pretty sketchy:
This sounds like a Fully General Mockery of any claim that humans can ever affect outcomes. For example:
And selectively applied Fully General Mockeries seem pretty Dark Artsy.
Of course not! The real reason drinkers cause more accidents is that low-conscientiousness people are both more likely to drink before driving and more likely to drive recklessly. (The impairment of reflexes due to alcohol does itself have an effect, but it’s not much larger than that due to e.g. sleep deprivation.) If a high-conscientiousness person was randomly assigned to the “drunk driving” condition, they would drive extra cautiously to compensate for their impairment. ;-)
(I’m exaggerating for comical effect, but I do believe a weaker version of this.)
“Extremely non-obvious”? Have you looked at how many calories one hour of exercise burns, and compared that to how many calories foodstuffs common in the First World contain?
I agree that focusing on input has far higher returns than focusing on output. Simple calorie comparison predicts it, and in my personal experience I’ve noted small appearance and weight changes after changes in exercise level and large appearance and weight changes after changes in intake level. That said, the traditional view- “eat less and exercise more”- has the direction of causation mostly right for both interventions and to represent it as just “exercise more” seems mistaken.
I was also distracted by the footnotes, since even though I found them quite funny, [3] at least is obviously wrong: “there’s no known intervention which can cause weight loss.” Sure there is—the effectiveness of bariatric surgery is quite well evidenced at this point (http://en.wikipedia.org/wiki/Bariatric_surgery#Weight_loss).
I generally share Eliezer’s assessment of the state of conventional wisdom in dietary science (abysmal), but careless formulations like this one are—well, distracting.
Also, even if he meant non-surgical interventions, it should be “which can reliably cause long-term weight loss”—there are people who lose weight by dieting, and a few of them don’t even gain it back.
I think that’s why it’s a good example. It induces genuine curiosity about the truth, and about the method.
I suspect this depends on the handling of the issue. Eliezer presenting his model of the world as “common sense,” straw manning the alternative, and then using fake data that backs up his preferences is, frankly, embarrassing.
This is especially troublesome because this is an introductory explanation to a technical topic- something Eliezer has done well before- and introductory explanations are great ways to introduce people to Less Wrong. But how can I send this to people I know who will notice the bias in the second paragraph and stop reading because that’s negative evidence about the quality of the article? How can I send this to people I know who will ask why he’s using two time-variant variables as single acyclic nodes, rather than a ladder (where exercise and weight at t-1 both cause exercise at t and weight at t)?
What would it look like to steel man the alternative? One of my physics professors called ‘calories in-calories out=change in fat’ the “physics diet,” since it was rooted in conservation of energy; that seems like a far better name. Like many things in physics, it’s a good first order approximation to the engineering reality, but there are meaningful second order terms to consider. “Calories in” is properly “calories absorbed” not “calories put into your mouth”- though we’ll note it’s difficult to absorb more calories than you put into your mouth. Similarly, calories out is non-trivial to measure- current weight and activity level can give you a broad guess, but it can be complicated by many things, like ambient temperature! Any attempt we make to control calories in and calories out will have to be passed through the psychology and physiology of the person in question, making them even more difficult to control in the field.
Compare the volume of discussion of the method and the overweight-exercise link in the comments.
Why do you need to send this article to people who could ask that? If you’re saying “Oh, this should actually be modeled using causal markov chains...” then this is probably too basic for you.
Because I’m still a grad student, most of those people that I know are routinely engaged in teaching these sorts of concepts, and so will find articles like this useful for pedagogical reasons.
This is not intended for readers who already know that much about causal models, btw, it’s a very very very basic intro.
I’m torn. On the one hand, using the method to explain something the reader probably was not previously aware of is an awesome technique that I truly appreciate. Yet Vaniver’s point that controversial opinions should not be unnecessarily put into introductory sequence posts makes sense. It might turn off readers who would otherwise learn from the text, like nyan sandwich.
In my opinion, the best fix would be to steelman the argument as much as possible. Call it the physics diet, not the virtue-theory of metabolism. Add in an extra few sentences that really buff up the basics of the physics diet argument. And, at the end, include a note explaining why the physics diet doesn’t work (appetite increases as exercise increases).
The point Eliezer is addressing is the one that RichardKennaway brought up separately. Causal models can still function with feedback (in Causality, Pearl works through an economic model where price and quantity both cause each other, and have their own independent causes), but it’s a bit more bothersome.
A model where the three are one-time events- like, say, whether a person has a particular gene, whether or not they were breastfed, and their height as an adult- won’t have the problem of being cyclic, but will have the pedagogical problem that the causation is obvious from the timing of the events.
One could have, say, the weather witch’s prediction of whether or not there will be rain, whether or not you brought an umbrella with you, and whether or not it rained. Aside from learning, this will be an acyclic system that has a number of plausible underlying causal diagrams (with the presence of the witch making the example clearly fictional and muddying our causal intuitions, so we can only rely on the math).
The concept of inferential distance suggests to me that posts should try and make their pathways as short and straight as possible. Why write a double-length post that explains both causal models and metabolism, when you could write a single-length post that explains only causal models? (And, if metabolism takes longer to discuss than causal models, the post will mostly be about the illustrative detour, not the concept itself!)
Should that be “cyclic”? I take it from Richard’s post that “acyclic” is what we want.
Yes, it should. Thanks for catching the typo!
You’ve convinced me. I now agree that EY should go back and edit the post to use a different more conventional example.
“Physics diet” and “virtue-theory of metabolism” are not steelman and strawman of each other; they are quite different things. Proponents of the physics diet (e.g. John Walker) do not say that if you want to lose weight you should exercise more—they say you should eat less. EDIT: the strawman of this would be the theory that “excessive eating actually causes weight gain due to divine punishment for the sin of gluttony” (inspired by Yvain’s comment).
Seriously; that was intended to be an example. What’s it matter whether the nodes are labelled “exercise/overweight/internet” or “foo/bar/baz”? (But yeah, Footnote 1 doesn’t belong there, and Footnote 3 might mention eating.)
Taking a “contentious” point and resolving it in to a settled fact made the whole article vastly more engaging to me. It also struck me as an elegant demonstration of the value of the tool: It didn’t simply introduce the concept, it used it to accomplish something worthwhile.
! From the article:
Eliezer’s data is made up, but all the not-made-up research I’ve seen supports his actual conclusion. The net emotional result was the same for me as if he’d used the actual research, since my brain could substitute it in.
Perhaps I am weird in having this emotional link, or perhaps I am simply more familiar with the not-made-up research than you.
I understand. I think it’s important to watch out for these sorts of illusions of transparency, though, especially when dealing with pedagogical material. One of the heuristics I’ve been using is “who would I not recommend this to?”, because that will use social modules my brain is skilled at using to find holes and snags in the article. I don’t know how useful that heuristic will be to others, and welcome the suggestion of others.
I am not an expert in nutritional science, but it appears to me that there is controversy among good nutritionists. This post also aided my understanding of the issue. I also detail some more of my understanding in this comment down another branch.
EDIT: Also, doing some more poking around now, this seems relevant.
Ahh, that heuristic makes sense! I wasn’t thinking in that context :)
P.S. When quoting two people, it can be useful to attribute the quotes. I initially thought the second quote was your way of doing a snarky editorial comment on what I’d said, not quoting the article...
Thanks for the suggestion, I’ve edited my comment to make it clearer.