This [people casually misinterpreting “A” as “A & non-B” when it’s contrasted with “A & B”] is covered in Tversky and Khaneman, 1983.
You mean the medical question? I’m not at all impressed with that one. This question and the subsequent one about its interpretation were worded in a way that takes a considerable mental effort to parse correctly, and is extremely unnatural for non-mathematicians, even highly educated ones like doctors. What I would guess happened was that the respondents skimmed the question without coming anywhere near the level of understanding that T&K assume in their interpretation of the results.
Again, when thinking about these experiments, one must imagine realistic people in a realistic setting, who are extremely unlikely to be willing to grapple with semantic subtleties that go beyond what they’ll gather with effortless casual skimming of the questions and offhand answers given without much thought.
This question and the subsequent one about its interpretation were worded in a way that takes a considerable mental effort to parse correctly, and is extremely unnatural for non-mathematicians, even highly educated ones like doctors.
A 55-year-old woman had pulmonary embolism documented angiographically 10 days after a cholecstectomy. Please rank order the following in terms of the probability that they will be among the conditions experienced by the patient (use 1 for the most likely and 6 for the least likely). Naturally, the patient could experience more than one of these conditions.
Dyspnea and hemiparesis
Calf pain
Pleuritic chest pain
Syncope and tachycardia
Hemiparesis
Hemoptysis
In what way is this question difficult for a doctor to parse? Give the subjects a little credit here.
How do you know that subjects did not interpret “Linda is a bank teller” to mean “Linda is a bank teller and is not active in the feminist movement”? For one thing, dear readers, I offer the observation that most bank tellers, even the ones who participated in anti-nuclear demonstrations in college, are probably not active in the feminist movement. So, even so, Teller should rank above Teller & Feminist.
Do you mean this one (from Conjunction Controversy): [...]
Yes, that’s the one.
In what way is this question difficult for a doctor to parse? Give the subjects a little credit here.
In my experience, outside of some very exceptional situations like e.g. in-depth discussions of complex technical subjects, the overwhelming majority of people, including highly educated people, simply don’t operate under the assumption that language should be understood in a logically precise and strict way. The standard human modus operandi is to skim the text in a very superficial manner and interpret it according to intuitive hunches and casual associations, guided by some strong preconceptions about what the writer is likely to be trying to say—and it’s up to you to structure your text so that it will be correctly understood with such an approach, or at least to give a clear and prominent warning that it should be read painstakingly in an unnaturally careful and literal way.
Some people are in the habit of always reading in a precise and literal way (stereotypically, mathematicians tend to be like that). I am also like that, and I’m sure many people here are too. But this is simply not the way the great majority of people function—including many people whose work includes complex and precise formal reasoning, but who don’t carry over this mode of thinking into the rest of their lives. In particular, from what I’ve seen, doctors typically don’t practice rigid formal reasoning much, and it’s definitely not reasonable to expect them to make such an effort in a situation where they lack any concrete incentive to do so.
I thought about this post for awhile—partially because I’ve just been too busy for LW the past few days—and I’m still pretty skeptical. I general, I think you’re right—people don’t closely read much of anything, or interpret much literally. I’ve seen enough economic experiments to know that subject rarely have even a basic grasp of the rules coming into the experiment, and only when the experiment begins and they start trying things out do they understand the environment we put them in.
However, in the heuristics and biases experiments what subjects are reading is only a couple of sentences long. In my exerpience, people tend to only skim when what they’re reading is long or complicated. So I find it fairly hard to believe that most people aren’t reading something like the Linda problem close enough to understand it—especially undergraduates at high end universities and trained doctors.
In my experience, people tend to only skim when what they’re reading is long or complicated. So I find it fairly hard to believe that most people aren’t reading something like the Linda problem close enough to understand it—especially undergraduates at high end universities and trained doctors.
My thinking about this topic is strongly influenced by my experiences from situations where I was in charge of organizing something, but without any formal authority over the people involved, with things based on an honor system and voluntary enthusiasm. In such situations, when I send off an email with instructions, I often find it a non-trivial problem to word things in a such a way that I’ll have peace of mind that it will be properly understood by all recipients.
In my experience, even very smart people with a technical or scientific background who normally display great intelligence and precision of thought in the course of their work will often skim and misunderstand questions and instructions worded in a precise but unnatural way, unless they have an incentive to make the effort to read the message with extra care and accuracy (e.g. if it’s coming from someone whose authority they fear). Maybe some bad experiences from the past have made me excessively cautious in this regard, but if I caught myself writing an email worded the same way as the doctors’ question by T&K and directed at people who won’t be inclined to treat it with special care—no matter how smart, except perhaps if they’re mathematicians—I would definitely rewrite it before sending.
see this comment.
also:
This is covered in Tversky and Khaneman, 1983. Also in conjunction controversy
Matt_Simpson:
You mean the medical question? I’m not at all impressed with that one. This question and the subsequent one about its interpretation were worded in a way that takes a considerable mental effort to parse correctly, and is extremely unnatural for non-mathematicians, even highly educated ones like doctors. What I would guess happened was that the respondents skimmed the question without coming anywhere near the level of understanding that T&K assume in their interpretation of the results.
Again, when thinking about these experiments, one must imagine realistic people in a realistic setting, who are extremely unlikely to be willing to grapple with semantic subtleties that go beyond what they’ll gather with effortless casual skimming of the questions and offhand answers given without much thought.
Do you mean this one (from Conjunction Controversy):
In what way is this question difficult for a doctor to parse? Give the subjects a little credit here.
Also note this about the Linda problem (also from Conjunction Controversy):
Matt_Simpson:
Yes, that’s the one.
In my experience, outside of some very exceptional situations like e.g. in-depth discussions of complex technical subjects, the overwhelming majority of people, including highly educated people, simply don’t operate under the assumption that language should be understood in a logically precise and strict way. The standard human modus operandi is to skim the text in a very superficial manner and interpret it according to intuitive hunches and casual associations, guided by some strong preconceptions about what the writer is likely to be trying to say—and it’s up to you to structure your text so that it will be correctly understood with such an approach, or at least to give a clear and prominent warning that it should be read painstakingly in an unnaturally careful and literal way.
Some people are in the habit of always reading in a precise and literal way (stereotypically, mathematicians tend to be like that). I am also like that, and I’m sure many people here are too. But this is simply not the way the great majority of people function—including many people whose work includes complex and precise formal reasoning, but who don’t carry over this mode of thinking into the rest of their lives. In particular, from what I’ve seen, doctors typically don’t practice rigid formal reasoning much, and it’s definitely not reasonable to expect them to make such an effort in a situation where they lack any concrete incentive to do so.
I thought about this post for awhile—partially because I’ve just been too busy for LW the past few days—and I’m still pretty skeptical. I general, I think you’re right—people don’t closely read much of anything, or interpret much literally. I’ve seen enough economic experiments to know that subject rarely have even a basic grasp of the rules coming into the experiment, and only when the experiment begins and they start trying things out do they understand the environment we put them in.
However, in the heuristics and biases experiments what subjects are reading is only a couple of sentences long. In my exerpience, people tend to only skim when what they’re reading is long or complicated. So I find it fairly hard to believe that most people aren’t reading something like the Linda problem close enough to understand it—especially undergraduates at high end universities and trained doctors.
OTOH, I’m open to any evidence you have
Matt_Simpson:
My thinking about this topic is strongly influenced by my experiences from situations where I was in charge of organizing something, but without any formal authority over the people involved, with things based on an honor system and voluntary enthusiasm. In such situations, when I send off an email with instructions, I often find it a non-trivial problem to word things in a such a way that I’ll have peace of mind that it will be properly understood by all recipients.
In my experience, even very smart people with a technical or scientific background who normally display great intelligence and precision of thought in the course of their work will often skim and misunderstand questions and instructions worded in a precise but unnatural way, unless they have an incentive to make the effort to read the message with extra care and accuracy (e.g. if it’s coming from someone whose authority they fear). Maybe some bad experiences from the past have made me excessively cautious in this regard, but if I caught myself writing an email worded the same way as the doctors’ question by T&K and directed at people who won’t be inclined to treat it with special care—no matter how smart, except perhaps if they’re mathematicians—I would definitely rewrite it before sending.