I discovered lesswrong.com because someone left a printout of an article on the elliptical machine in my gym. I started reading it and have become hooked.
I’m a formally uneducated computer expert. The lack of formal education makes me a bit insecure, so I obsess over improving my thinking through literature on cognitive dissonance and biases, such as books from the library and also sites like this.
Nowadays I get paid to be a middle-manager at technology companies. Most of my career has been in Linux system administration as well as functional programming.
I’m a bit of a health nut. I adopted a whole-food plant-based diet (the “China Study” diet) because it seems most well supported in the literature, although a broad consensus on the topic has not emerged. I base this decision in part on my trust of experts with titles after their names, since I’m too out of my element to read and interpret most of the literature on my own. At the same time I have a personal anecdote that this works well, so those two are enough to convince me for now.
There are times when I find reading about rational thinking rather sobering. It’s clear that we were born with an irrational, “defective”, brain and that we would be so lucky if we could even make a small dent in improving our decision making. Improvements seem very hard to come by, I worry that all I’m really doing is learning to distrust my beliefs.
So that’s a nutshell full. How’s everyone else? :)
I discovered lesswrong.com because someone left a printout of an article on the elliptical machine in my gym. I started reading it and have become hooked.
Yes. I spent a lot of time reviewing critiques of The China Study (TCS), including Minger’s. At the end of it I came to the following conclusions.
Nutrition science is extraordinarily nonlinear
I’m definitely not qualified to deconstruct claims made about nutrition
TCS critics don’t seem very qualified either, especially when compared to the qualifications of the people advancing TCS
There’s no larger group of qualified people advancing a radically different approach
So, those are my reasons. I admit they’re not very satisfying. I’m spoiled by fields where, once you grok the formal proof you can be highly confident that the claim is correct.
No such luck with something as squishy as nutrition, it would seem.
When you know next to nothing about the topic at hand and the only choice is to trust authority or to rely on your own, almost certainly flawed judgment, I’d go with authority.
When you know next to nothing about the topic at hand and the only choice is to trust authority or to rely on your own, almost certainly flawed judgment, I’d go with authority.
When the topic is an important one, like health and nutrition, I’ll go learn about the topic.
I’m skeptical this is a great strategy for topics in general.
Nutrition, for example, doesn’t appear to be the kind of topic where you can just learn its axioms and build up an optimal human diet from first principles. It’s far too complicated.
Instead you need substantial education, training, experience and access, as well as a community that can help you support and refine your ideas. You need to gather evidence, you need to learn how to determine the quality of the evidence you’ve gathered, and you need to propose reasonable stories that fit the evidence.
Since I haven’t made health and nutrition my career most of these things will be hard or even impossible for me to come by. As such, my confidence in the quality of any amateur conclusions I come to must necessarily be low.
So, the most reasonable thing for me to do is trust authorities when it comes to nutrition.
I’m skeptical this is a great strategy for topics in general.
And rightly so :-) This is an approach that should be reserved for important topics.
Instead you need substantial education, training, experience and access, as well as a community
I think you’re setting the bar too high. What you describe will allow one to produce new research and that’s not the goal here. All you need to be able to do is to pass a judgement on conflicting claims—that’s much easier than gathering evidence and proposing stories.
In nutrition, for example, a lot of claims are contested and not by crackpots. Highly qualified people strongly disagree about basic issues, for example, the effects of dietary saturated fat. I am saying that you should read the arguments of both sides and form your opinion about them—not that you should apply to the NIH for a grant to do a definitive study.
Of course that means reading the actual papers, not dumbed down advice for hoi polloi.
By “learn”, I assume you mean read existing literature on the topic. In the case of health and nutrition (and most other medical topics), high-quality literature is rather sparse, both because of frequently bad statistical analyses and the fact that practically no one releases their raw data—only the results. (Seriously, what’s up with that?)
Given that the experts in the field are precisely those learning from and producing that same literature, the fact that the literature is generally low-quality doesn’t make me more inclined to trust them. (Though, as bad as academic nutrition science is, conventional wisdom and pop nutrition science seem to be worse.)
It does make it exceptionally hard to gain a good understanding of the field yourself, though. Unlike Lumifer, I’d say the correct move, unless you are yourself a nutritionist or a fitness nerd or otherwise inclined to spend a large portion of your life on this, is to reserve judgment.
Given that the experts in the field are precisely those learning from and producing that same literature, the fact that the literature is generally low-quality doesn’t make me more inclined to trust them.
In terms of statistics and data, yes, the papers they produce are fairly low-quality. In terms of domain-specific knowledge, however, I’d trust an expert over pretty much anyone else. That being said, I do agree with you here:
It does make it exceptionally hard to gain a good understanding of the field yourself, though. Unlike Lumifer, I’d say the correct move, unless you are yourself a nutritionist or a fitness nerd or otherwise inclined to spend a large portion of your life on this, is to reserve judgment.
Although I prefer trusting expert authority to making my own judgments on unfamiliar topics, gaining a good-enough understanding to figure out which experts to trust is still hard, especially with so many conflicting conclusions out there. This being the case, the strategy you propose—reserve judgment—is precisely what I do.
Ah, the old “choosing not to choose is itself a choice” move. Never was too convinced by that.
You can reserve judgment on the theory while taking some default stance on the practical issue. Depending on where you’re standing this might mean the standard diet for your culture (probably suboptimal, but arguably less suboptimal than whatever random permutations you might apply to it), or “common sense” (which I’m skeptical of in some ways, but it probably picks some low-hanging fruit), or imitating people or populations with empirically good results (the “Mediterranean diet” is a persistently popular target), or adopting a cautious stance toward dietary innovations from the last forty years or so (about when the obesity epidemic started taking off).
while taking some default stance on the practical issue
Your stance is a choice nevertheless and it necessary implies a particular theory of nutrition (even if that theory is not academically recognized and might be as simple as “eating whatever everyone else eats can’t be that bad”).
It’s an option—a point in a configuration space—but not a random option. The default is, almost tautologically, a stable equilibrium, while in a sufficiently complicated system almost all possible choices may move you away from that equilibrium in ways you don’t want.
Nutrition is a very complicated system. Of course, its fitness landscape might be friendlier than I’m giving it credit for here, but I don’t have any particular reason to assume that it is.
Well, of course. Where does the idea of a random choice even come from?
The default is, almost tautologically, a stable equilibrium
If by “default” you mean “whatever most people around me eat”, then no, not necessarily. Food changes. Examples would be the introduction of white rice (hence, beriberi) or mercury-polluted fish.
There is also the issue of the proper metric. If you want to optimize for health and longevity, there is no particular reason to consider the “default” to be close to optimal.
Where does the idea of a random choice even come from?
If you don’t have much good information about what the fitness landscape looks like—for example, if the literature is opaque and often contradictory—then there’s going to be a lot of randomness in the effects of any choices you make. It’s not random in the sense of a blind jump into the depths of the fitness landscape—the very concept of what counts as “food”, for example, screens off quite a bit—but even if the steps are short, you don’t know if you’re going to be climbing a hill or descending into a valley. And in complex optimization problems that have seen a lot of iteration, most choices are usually bad.
You can, of course, iterate on empirical differences, and most people do, but the cycle time’s long, the results are noisy, and a lot of people aren’t very good at that sort of reflection in the first place.
But it’s not that the choice is random—it’s that the consequences of choices are rather uncertain.
its fitness landscape might be friendlier
Well, first it’s well-bounded: there is both an upper bound on how much (in health and longevity) you can gain by manipulating your diet, and a clear lower bound (poisons tend to be obvious). Second, there is hope in untangling—eventually—all the underlying biochemistry so that we don’t have to treat the body as a mostly-black box.
Another thing is that there is a LOT of individual (or group) variation, something that most nutritional research tends to ignore, that is, treat it as unwanted noise.
A major problem is that it’s legally/politically/morally hard to experiment on humans, even with full consent.
By “learn”, I assume you mean read existing literature on the topic
Also around the topic, not to mention that learning necessarily involves a fair amount of one’s own thinking.
high-quality literature is rather sparse
I agree which makes relying of authority (and, usually, on mass media reinterpretations of authority) particularly suspect.
what’s up with that?
I think the usual explanation is privacy and medical ethics, but my cynical mind readily suggests that it’s much harder to critique a study if you can’t see the data...
No, I do not. I actually read the papers and see if they make sense. One of my long-standing complaints is that in medical research no one releases the data—it would be very useful to reanalyze it is a bit less brain-dead fashion.
Using the data set with the flawed inclusion of Tuoli, Campbell cites a strong association between animal protein and lipid intake as a reason to implicate animal foods with breast cancer. Yet using the revised data set, animal foods do not contribute significantly more fat to total lipid intake than do plant oils. As a result, any association between breast cancer and dietary fat could be linked to either animal or plant-sourced foods, and there is no justification for indicting only animal products.
Also, to continue quoting Minger,
..meat was not the dietary feature noted in my discussion of Tuoli: dairy was. Both the three-day diet survey and the frequency questionnaire reveal high intakes of dairy for Tuoli citizens, with the questionnaire indicating milk products are consumed an average of 330.3 days per year, and closer to 350 in one township.[98] In addition, despite Campbell’s comment that the Tuoli migrate seasonally and consume more vegetables and fruit for part of the year, the China Study frequency questionnaire indicates Tuoli’s vegetable intake is only twice per year and fruit intake is less than once per year on average.[99]
If Campbell believes both the three-day diet survey and frequency questionnaire were in error, I must question why Tuoli county was not excluded entirely from the data set—especially given its pronounced influence on virtually all associations involving meat, dairy, and animal protein, many of which Campbell cited as verification for his animal foods-disease hypothesis.
Why aren’t [the people who live in Tuoli] sick and diseased?
We have plenty of evidence showing hormone-pumped dairy, grain-fed meat, pasteurized and homogenized milk, processed lunch meats, and other monstrosities are bad for the human body. No debate there. But we do have a woeful lack of research on the effects of “clean” animal products—meat from wild or pastured animals fed good diets, milk that hasn’t been heat-zapped, antibiotic-free cheeses and yogurts, and so forth.
[...]
Is it possible the diseases we ascribe to animal products aren’t caused by animal products themselves, but by the chemicals, hormones, and treatment processes we expose them to? If the Tuoli are any indication, this may be the case. Hopefully future research will shed more light on the matter.
Or, you know, they’re an insular minority with peculiar nutritional requirements.
You’re quoting from the page which says right on top:
Important disclaimer: In light of new information, this post needs to be taken with a really whoppin’ huge grain of salt. It turns out Tuoli was “feasting” on the day the survey crew came for China Study I, so they were likely eating more calories, more wheat, more dairy, and so forth than they typically do the rest of the year. We can’t be completely sure what their normal diet did look at the time, but the questionnaire data (which is supposedly more reliable than the diet survey data) still suggests they were eating a lot of animal products and very little in the way of fruits or vegetables.
At any rate, I recommend not quoting this post or citing it as “evidence” for anything simply because of the uncertainty surrounding the Tuoli data in the China Study.
You seem to be more interested in creating gotchas than in finding out what’s actually happening in reality.
You’re quoting from the page which says right on top:
I was kind of waiting for you to point that out. Notice it’s a non-disclaimer anyway:
but the questionnaire data (which is supposedly more reliable than the diet survey data) still suggests they were eating a lot of animal products and very little in the way of fruits or vegetables.
In any case, I’m not using it as evidence for or against a particular diet. I’m using it as evidence of her research process. About a quarter of her criticism of TCS is based around Tuoli being an outlier, so it’s interesting that she also thought that their diet didn’t increase their rate of disease significantly, even before she found out the data was bad. It’s a clear sign of motivated cognition.
You seem to be more interested in creating gotchas than in finding out what’s actually happening in reality.
In general, you don’t seem very good at ascribing motives to me. Recall you were the one that asked for an example of what I found confusing.
I am sorry, did you miss that comment?
No, I didn’t.
But if you want to pretend Tuoli doesn’t exist, sure, you can pretend Tuoli doesn’t exist. What next?
That’s not even remotely close to what I said, and doesn’t really have anything to do with the point at hand.
I am going to call bullshit on that. You did a word search for “Tuoli” in a web page and that turned up 27 hits. That does not mean that there are 27 instances of using the Tuoli data to argue against TCS.
Section 1.2, for example, explicitly points out that taking Tuoli data out makes some Campbell claims to have much less support in the correlation numbers.
I think you’re being dishonest. This conversation is over.
I discovered lesswrong.com because someone left a printout of an article on the elliptical machine in my gym. I started reading it and have become hooked.
I’m a formally uneducated computer expert. The lack of formal education makes me a bit insecure, so I obsess over improving my thinking through literature on cognitive dissonance and biases, such as books from the library and also sites like this.
Nowadays I get paid to be a middle-manager at technology companies. Most of my career has been in Linux system administration as well as functional programming.
I’m a bit of a health nut. I adopted a whole-food plant-based diet (the “China Study” diet) because it seems most well supported in the literature, although a broad consensus on the topic has not emerged. I base this decision in part on my trust of experts with titles after their names, since I’m too out of my element to read and interpret most of the literature on my own. At the same time I have a personal anecdote that this works well, so those two are enough to convince me for now.
There are times when I find reading about rational thinking rather sobering. It’s clear that we were born with an irrational, “defective”, brain and that we would be so lucky if we could even make a small dent in improving our decision making. Improvements seem very hard to come by, I worry that all I’m really doing is learning to distrust my beliefs.
So that’s a nutshell full. How’s everyone else? :)
What article was that?
http://lesswrong.com/lw/xk/continuous_improvement/
Are you aware of Denise Minger’s dissection of the China Study?
Yes. I spent a lot of time reviewing critiques of The China Study (TCS), including Minger’s. At the end of it I came to the following conclusions.
Nutrition science is extraordinarily nonlinear
I’m definitely not qualified to deconstruct claims made about nutrition
TCS critics don’t seem very qualified either, especially when compared to the qualifications of the people advancing TCS
There’s no larger group of qualified people advancing a radically different approach
So, those are my reasons. I admit they’re not very satisfying. I’m spoiled by fields where, once you grok the formal proof you can be highly confident that the claim is correct.
No such luck with something as squishy as nutrition, it would seem.
General advice: learn causal inference. Getting strong causal claims empirically is not so simple...
I disagree with your approach (basically, trust authority), but that’s just me.
When you know next to nothing about the topic at hand and the only choice is to trust authority or to rely on your own, almost certainly flawed judgment, I’d go with authority.
When the topic is an important one, like health and nutrition, I’ll go learn about the topic.
I’m skeptical this is a great strategy for topics in general.
Nutrition, for example, doesn’t appear to be the kind of topic where you can just learn its axioms and build up an optimal human diet from first principles. It’s far too complicated.
Instead you need substantial education, training, experience and access, as well as a community that can help you support and refine your ideas. You need to gather evidence, you need to learn how to determine the quality of the evidence you’ve gathered, and you need to propose reasonable stories that fit the evidence.
Since I haven’t made health and nutrition my career most of these things will be hard or even impossible for me to come by. As such, my confidence in the quality of any amateur conclusions I come to must necessarily be low.
So, the most reasonable thing for me to do is trust authorities when it comes to nutrition.
And rightly so :-) This is an approach that should be reserved for important topics.
I think you’re setting the bar too high. What you describe will allow one to produce new research and that’s not the goal here. All you need to be able to do is to pass a judgement on conflicting claims—that’s much easier than gathering evidence and proposing stories.
In nutrition, for example, a lot of claims are contested and not by crackpots. Highly qualified people strongly disagree about basic issues, for example, the effects of dietary saturated fat. I am saying that you should read the arguments of both sides and form your opinion about them—not that you should apply to the NIH for a grant to do a definitive study.
Of course that means reading the actual papers, not dumbed down advice for hoi polloi.
By “learn”, I assume you mean read existing literature on the topic. In the case of health and nutrition (and most other medical topics), high-quality literature is rather sparse, both because of frequently bad statistical analyses and the fact that practically no one releases their raw data—only the results. (Seriously, what’s up with that?)
Given that the experts in the field are precisely those learning from and producing that same literature, the fact that the literature is generally low-quality doesn’t make me more inclined to trust them. (Though, as bad as academic nutrition science is, conventional wisdom and pop nutrition science seem to be worse.)
It does make it exceptionally hard to gain a good understanding of the field yourself, though. Unlike Lumifer, I’d say the correct move, unless you are yourself a nutritionist or a fitness nerd or otherwise inclined to spend a large portion of your life on this, is to reserve judgment.
In terms of statistics and data, yes, the papers they produce are fairly low-quality. In terms of domain-specific knowledge, however, I’d trust an expert over pretty much anyone else. That being said, I do agree with you here:
Although I prefer trusting expert authority to making my own judgments on unfamiliar topics, gaining a good-enough understanding to figure out which experts to trust is still hard, especially with so many conflicting conclusions out there. This being the case, the strategy you propose—reserve judgment—is precisely what I do.
That doesn’t help you when experts disagree.
You can’t—you’ve got to eat each day :-/
Ah, the old “choosing not to choose is itself a choice” move. Never was too convinced by that.
You can reserve judgment on the theory while taking some default stance on the practical issue. Depending on where you’re standing this might mean the standard diet for your culture (probably suboptimal, but arguably less suboptimal than whatever random permutations you might apply to it), or “common sense” (which I’m skeptical of in some ways, but it probably picks some low-hanging fruit), or imitating people or populations with empirically good results (the “Mediterranean diet” is a persistently popular target), or adopting a cautious stance toward dietary innovations from the last forty years or so (about when the obesity epidemic started taking off).
It looks obviously true to me.
Your stance is a choice nevertheless and it necessary implies a particular theory of nutrition (even if that theory is not academically recognized and might be as simple as “eating whatever everyone else eats can’t be that bad”).
It’s an option—a point in a configuration space—but not a random option. The default is, almost tautologically, a stable equilibrium, while in a sufficiently complicated system almost all possible choices may move you away from that equilibrium in ways you don’t want.
Nutrition is a very complicated system. Of course, its fitness landscape might be friendlier than I’m giving it credit for here, but I don’t have any particular reason to assume that it is.
Well, of course. Where does the idea of a random choice even come from?
If by “default” you mean “whatever most people around me eat”, then no, not necessarily. Food changes. Examples would be the introduction of white rice (hence, beriberi) or mercury-polluted fish.
There is also the issue of the proper metric. If you want to optimize for health and longevity, there is no particular reason to consider the “default” to be close to optimal.
I certainly agree.
If you don’t have much good information about what the fitness landscape looks like—for example, if the literature is opaque and often contradictory—then there’s going to be a lot of randomness in the effects of any choices you make. It’s not random in the sense of a blind jump into the depths of the fitness landscape—the very concept of what counts as “food”, for example, screens off quite a bit—but even if the steps are short, you don’t know if you’re going to be climbing a hill or descending into a valley. And in complex optimization problems that have seen a lot of iteration, most choices are usually bad.
You can, of course, iterate on empirical differences, and most people do, but the cycle time’s long, the results are noisy, and a lot of people aren’t very good at that sort of reflection in the first place.
But it’s not that the choice is random—it’s that the consequences of choices are rather uncertain.
Well, first it’s well-bounded: there is both an upper bound on how much (in health and longevity) you can gain by manipulating your diet, and a clear lower bound (poisons tend to be obvious). Second, there is hope in untangling—eventually—all the underlying biochemistry so that we don’t have to treat the body as a mostly-black box.
Another thing is that there is a LOT of individual (or group) variation, something that most nutritional research tends to ignore, that is, treat it as unwanted noise.
A major problem is that it’s legally/politically/morally hard to experiment on humans, even with full consent.
Also around the topic, not to mention that learning necessarily involves a fair amount of one’s own thinking.
I agree which makes relying of authority (and, usually, on mass media reinterpretations of authority) particularly suspect.
I think the usual explanation is privacy and medical ethics, but my cynical mind readily suggests that it’s much harder to critique a study if you can’t see the data...
Sounds to me that you’re trusting authority that just happens to be of a different sort.
No, I do not. I actually read the papers and see if they make sense. One of my long-standing complaints is that in medical research no one releases the data—it would be very useful to reanalyze it is a bit less brain-dead fashion.
Then why’d you recommend Minger’s criticism? Because as far as I can tell it doesn’t make sense.
Makes a lot of sense to me. What is it that doesn’t make sense to you?
Let’s start with the sturm and drang over Tuoli, I suppose. Why aren’t they an obvious outlier?
Um, it is.
To quote Minger
Also, to continue quoting Minger,
Yet elsewhere:
[...]
Or, you know, they’re an insular minority with peculiar nutritional requirements.
You said that Minger’s criticism of TCS “doesn’t make sense”. Did you actually have anything specific in mind?
I also don’t see much problems with the passages you quoted.
They contradict each other. Why isn’t Tuoli an outlier?
You’re quoting from the page which says right on top:
You seem to be more interested in creating gotchas than in finding out what’s actually happening in reality.
I am sorry, did you miss that comment?
But if you want to pretend Tuoli doesn’t exist, sure, you can pretend Tuoli doesn’t exist. What next?
I was kind of waiting for you to point that out. Notice it’s a non-disclaimer anyway:
In any case, I’m not using it as evidence for or against a particular diet. I’m using it as evidence of her research process. About a quarter of her criticism of TCS is based around Tuoli being an outlier, so it’s interesting that she also thought that their diet didn’t increase their rate of disease significantly, even before she found out the data was bad. It’s a clear sign of motivated cognition.
In general, you don’t seem very good at ascribing motives to me. Recall you were the one that asked for an example of what I found confusing.
No, I didn’t.
That’s not even remotely close to what I said, and doesn’t really have anything to do with the point at hand.
I don’t believe this is true—see this.
You still haven’t made any specific objections against Minger’s criticism of TCS.
You did mention motivated cognition, did you not?
27 instances. Section 1.2, 1.4, 1.8, 2.2, 3.1, and of course 3.3. “A quarter” is about correct, but let’s say “a fifth” if you’d like.
She depends too much on the Tuoli data—which she supposedly doesn’t trust anyway—to make her arguments.
I am going to call bullshit on that. You did a word search for “Tuoli” in a web page and that turned up 27 hits. That does not mean that there are 27 instances of using the Tuoli data to argue against TCS.
Section 1.2, for example, explicitly points out that taking Tuoli data out makes some Campbell claims to have much less support in the correlation numbers.
I think you’re being dishonest. This conversation is over.
Then I’d advise in the future you not offer to provide clarification when you’d prefer to quibble and assume bad faith where none exists.