But I think it was vastly overstated and does not apply to “real life” situations in nearly the same way as in testing environments.
It’s certainly easier to demonstrate in testing environments. But I think the mistake of using ‘representativeness’ to judge probability does come up quite a bit in real life situations.
I’ll think they’re probably telling me about John, in part, because he has acheived something noteworthy—like playing NBA basketball.
But… it’s still a conjunction! You shouldn’t think John becomes more likely when another constraint is put on it. You might ask “did the first John never play in the NBA, or does that cover both cases?”
Typically, human minds are set up to deal with stories, not math, and using stories when math is appropriate is a way to leave yourself easily hackable. (Mentioning the NBA should not make you think there are more bank tellers, or that bank tellers are more athletic!)
Your reply is a good example—not to pick on you—of what I’m talking about.
Of course it’s “still a conjunction”. Of course the formal probability is lower in the case of the conjunction regardless of if John is 10 feet tall and can fly. But in the real world good instrumental rationality involves the audicity to come to the conclusion that John is a NBA basketball player despite the clues in the question. The answer might be the questioner is wrong about John, and that isn’t a valid option in the lab.
Your reply is a good example—not to pick on you—of what I’m talking about.
I’m pretty confident that I understand your position, and to me it looks like you’re falling exactly into the trap predicted by the fallacy. Would it be a good use of our time for me to explain why? (And, I suppose, areas where I think the fallacy is dangerous?)
You shouldn’t think John becomes more likely when another constraint is put on it. You might ask “did the first John never play in the NBA, or does that cover both cases?”
No. If you did reply with this to someone who approached you in a social situation, you’d be more likely to “lose” than if you were just polite and answered the question with your best guess.
It is socially awkward to do labwork in real world social environments. So, while your follow up questions might help you win in correctly identifying the highest probability for John’s career path, you’d lose in the social exchange because you would have acted like a weirdo.
It’s good to be aware of the conjunction fallacy. It’s good to be aware of lots of the stuff on LW. But when you go around using it to mercilessly pursue rationality with no regard for decorum, you end up doing poorly in real life.
The real heart of the conjunction fallacy is mistaking P(A|B) and P(B|A). Since those look very similar, let’s try to make them more distinct: P(description|attribute) and P(attribute|description), or representativeness and likeliness*.
When you hear “NBA player,” the representativeness for ‘tall and athletic’ skyrockets. If he was an NBA player, it’s almost certain that he’s tall and athletic. But the reverse inference- how much knowing that he’s tall and athletic increases the chance that he’s an NBA player- is much lower. And while the bank teller detail is strange, you probably aren’t likely to adjust the representativeness down much because of it, even though there are probably more former NBA players who are short or got fat after leaving the league than there are former NBA players that became bank tellers. (That is, you should pay as much attention to 1% probabilities as you should to 99% probabilities when doing Bayesian calculations, because both represent similar strengths of evidence.)
When details increase, the likeliness of a story has to not increase, assuming you’re logically omniscient, which is obviously a bad assumption. If I say that I’m wearing green, and then that I’m wearing blue, it’s more likely that I’m wearing just green than wearing green and blue, because any case in which I am wearing both I am wearing green. This is the core idea of burdensome details.
So lets talk examples. When an insurance salesman comes to your door, which question will he ask: “what’s the chance that you’ll die tomorrow and leave your loved ones without anyone to care for them?” or “what’s the chance that you’ll die tomorrow of a heart attack and leave your loved ones without anyone to care for them?” The second question tells a story- and if your estimate of dying is higher because they specified the cause of death (which necessarily leaves out other potential causes!), then by telling you a long list of potential causes, as well as many vivid details about the scenario, the salesman can get your perceived risk as high as he needs it to be to justify the insurance.
Now, you may make the omniscience counterargument from before- who is to say that your baseline is any good? Maybe you thought the risk was zero, but on second thought it’s actually nonzero. But I would argue that the way to fix a fault is by doing the right thing, not a different wrong thing. You say “Wow, that is scary. But what’s the actual risk, in numeric terms?”, because if you don’t trust yourself to estimate what your total risk of death is, then you probably shouldn’t trust yourself to estimate your partial risk of death.
*I use infrequently used terms to try to make it clear that I am referring to precisely defined mathematical entities.
But when you go around using it to mercilessly pursue rationality with no regard for decorum, you end up doing poorly in real life.
Agreed that it’s a good idea to be polite. Disagreed that the conjunction fallacy is just because people are polite. There are lots of experiments where people are just getting the formal math problem wrong or being primed into giving strange estimates.
But even if we suppose that the person is trying to ‘steelman the question,’ that is a dangerous thing to do in real life. “Did you get the tickets for Saturday?” She must mean Friday, because that’s when we’re going. “Yes, I got the tickets.” Friday: “I’m outside the theater, where are you?” “At work; we’re going tomorrow! …you got the tickets for tomorrow, right? Because now the show is sold out.”
you’d lose in the social exchange because you would have acted like a weirdo.
Yes, it’s a good social skill to judge the level of precision the other person wants in the conversation. Responding to an unimportant anecdote with a “well actually” is generally seen as a jerk move. But if you’re around people who see it as a jerk move to insist on precision when something meaningful actually depends on that precision, then you need to replace those people.
And if they were intentionally asking you a gotcha, and you skewer the gotcha, that’s a win for you and a loss for them.
But if you’re around people who see it as a jerk move to insist on precision when something meaningful actually depends on that precision, then you need to replace those people.
Huh? First, Linda’s occupation in the original example is trivial, since I don’t know Linda and could not care less about what she does for a living.
And “replacing” people is not how life works. To be successful, you’ll need navigate (without replacing) all types of folks.
And if they were intentionally asking you a gotcha, and you skewer the gotcha, that’s a win for you and a loss for them.
This sounds weird to me. Who does this?
Anyway… I get the conjunction fallacy. There are plenty of useful applications for it. I still think the core of how it is presented around here is goofy. Of course additional conjunctions = lower probability. And yep, that isn’t instantly intuitive so it’s good to know.
It’s certainly easier to demonstrate in testing environments. But I think the mistake of using ‘representativeness’ to judge probability does come up quite a bit in real life situations.
But… it’s still a conjunction! You shouldn’t think John becomes more likely when another constraint is put on it. You might ask “did the first John never play in the NBA, or does that cover both cases?”
Typically, human minds are set up to deal with stories, not math, and using stories when math is appropriate is a way to leave yourself easily hackable. (Mentioning the NBA should not make you think there are more bank tellers, or that bank tellers are more athletic!)
Your reply is a good example—not to pick on you—of what I’m talking about.
Of course it’s “still a conjunction”. Of course the formal probability is lower in the case of the conjunction regardless of if John is 10 feet tall and can fly. But in the real world good instrumental rationality involves the audicity to come to the conclusion that John is a NBA basketball player despite the clues in the question. The answer might be the questioner is wrong about John, and that isn’t a valid option in the lab.
I’m pretty confident that I understand your position, and to me it looks like you’re falling exactly into the trap predicted by the fallacy. Would it be a good use of our time for me to explain why? (And, I suppose, areas where I think the fallacy is dangerous?)
Sure.
No. If you did reply with this to someone who approached you in a social situation, you’d be more likely to “lose” than if you were just polite and answered the question with your best guess.
It is socially awkward to do labwork in real world social environments. So, while your follow up questions might help you win in correctly identifying the highest probability for John’s career path, you’d lose in the social exchange because you would have acted like a weirdo.
It’s good to be aware of the conjunction fallacy. It’s good to be aware of lots of the stuff on LW. But when you go around using it to mercilessly pursue rationality with no regard for decorum, you end up doing poorly in real life.
The real heart of the conjunction fallacy is mistaking P(A|B) and P(B|A). Since those look very similar, let’s try to make them more distinct: P(description|attribute) and P(attribute|description), or representativeness and likeliness*.
When you hear “NBA player,” the representativeness for ‘tall and athletic’ skyrockets. If he was an NBA player, it’s almost certain that he’s tall and athletic. But the reverse inference- how much knowing that he’s tall and athletic increases the chance that he’s an NBA player- is much lower. And while the bank teller detail is strange, you probably aren’t likely to adjust the representativeness down much because of it, even though there are probably more former NBA players who are short or got fat after leaving the league than there are former NBA players that became bank tellers. (That is, you should pay as much attention to 1% probabilities as you should to 99% probabilities when doing Bayesian calculations, because both represent similar strengths of evidence.)
When details increase, the likeliness of a story has to not increase, assuming you’re logically omniscient, which is obviously a bad assumption. If I say that I’m wearing green, and then that I’m wearing blue, it’s more likely that I’m wearing just green than wearing green and blue, because any case in which I am wearing both I am wearing green. This is the core idea of burdensome details.
So lets talk examples. When an insurance salesman comes to your door, which question will he ask: “what’s the chance that you’ll die tomorrow and leave your loved ones without anyone to care for them?” or “what’s the chance that you’ll die tomorrow of a heart attack and leave your loved ones without anyone to care for them?” The second question tells a story- and if your estimate of dying is higher because they specified the cause of death (which necessarily leaves out other potential causes!), then by telling you a long list of potential causes, as well as many vivid details about the scenario, the salesman can get your perceived risk as high as he needs it to be to justify the insurance.
Now, you may make the omniscience counterargument from before- who is to say that your baseline is any good? Maybe you thought the risk was zero, but on second thought it’s actually nonzero. But I would argue that the way to fix a fault is by doing the right thing, not a different wrong thing. You say “Wow, that is scary. But what’s the actual risk, in numeric terms?”, because if you don’t trust yourself to estimate what your total risk of death is, then you probably shouldn’t trust yourself to estimate your partial risk of death.
*I use infrequently used terms to try to make it clear that I am referring to precisely defined mathematical entities.
Agreed that it’s a good idea to be polite. Disagreed that the conjunction fallacy is just because people are polite. There are lots of experiments where people are just getting the formal math problem wrong or being primed into giving strange estimates.
But even if we suppose that the person is trying to ‘steelman the question,’ that is a dangerous thing to do in real life. “Did you get the tickets for Saturday?” She must mean Friday, because that’s when we’re going. “Yes, I got the tickets.” Friday: “I’m outside the theater, where are you?” “At work; we’re going tomorrow! …you got the tickets for tomorrow, right? Because now the show is sold out.”
Yes, it’s a good social skill to judge the level of precision the other person wants in the conversation. Responding to an unimportant anecdote with a “well actually” is generally seen as a jerk move. But if you’re around people who see it as a jerk move to insist on precision when something meaningful actually depends on that precision, then you need to replace those people.
And if they were intentionally asking you a gotcha, and you skewer the gotcha, that’s a win for you and a loss for them.
Huh? First, Linda’s occupation in the original example is trivial, since I don’t know Linda and could not care less about what she does for a living.
And “replacing” people is not how life works. To be successful, you’ll need navigate (without replacing) all types of folks.
This sounds weird to me. Who does this?
Anyway… I get the conjunction fallacy. There are plenty of useful applications for it. I still think the core of how it is presented around here is goofy. Of course additional conjunctions = lower probability. And yep, that isn’t instantly intuitive so it’s good to know.
Agreed. That’s why I gave a non-trivial example for the broader reference class of ‘steelmanning questions’ / ‘not noticing and pursuing confusion.’
Disagreed. Replacing people is costly, yes, but oftentimes the costs are worth paying.
It is one of many status games that people can play, and thus one that people sometimes do play.