Start by reading Gigerenzer’s critiques. E.g. I really like the study on how overconfidence goes away if you ask for frequencies rather than subjective probabilities—this actually gives you a rationality technique that you can apply in real life! (In my experience it works, but that’s an impression, not a statistical finding.) I also quite liked his point about how just telling subjects to assume random sampling is misleading. You can find a summary of two of his critiques in a LW post by Kaj Sotala, “Heuristics and Biases Biases?” or summat. Also fastandfrugal.com should have some links or links to links. Also worth noting is that Gigerenzer’s been cited many thousands of times and has written a few popular books. I especially like Gigerenzer because unlike many H&B folk he has a thorough knowledge of statistics, and he uses that knowledge to make very sharp critiques of Kahneman’s compare-to-allegedly-ideal-Bayesian-reasoner approach. (Of course it’s still possible to use a Bayesian approach, but the most convincing Bayesian papers I’ve seen were sophisticated (e.g. didn’t skimp on information theory) and applied only to very simple problems.)
I wouldn’t even say that the problem is overall in the H&B lit, it’s just that lots of H&B folk spin their results as if they somehow applied to real life situations. It’s intellectually dishonest, and leads to people like Eliezer having massive overconfidence in the relevance of H&B knowledge for personal rationality.
Tversky and Kahneman, hogwash? What? Can you explain? Or just mention something?
Start by reading Gigerenzer’s critiques. E.g. I really like the study on how overconfidence goes away if you ask for frequencies rather than subjective probabilities—this actually gives you a rationality technique that you can apply in real life! (In my experience it works, but that’s an impression, not a statistical finding.) I also quite liked his point about how just telling subjects to assume random sampling is misleading. You can find a summary of two of his critiques in a LW post by Kaj Sotala, “Heuristics and Biases Biases?” or summat. Also fastandfrugal.com should have some links or links to links. Also worth noting is that Gigerenzer’s been cited many thousands of times and has written a few popular books. I especially like Gigerenzer because unlike many H&B folk he has a thorough knowledge of statistics, and he uses that knowledge to make very sharp critiques of Kahneman’s compare-to-allegedly-ideal-Bayesian-reasoner approach. (Of course it’s still possible to use a Bayesian approach, but the most convincing Bayesian papers I’ve seen were sophisticated (e.g. didn’t skimp on information theory) and applied only to very simple problems.)
I wouldn’t even say that the problem is overall in the H&B lit, it’s just that lots of H&B folk spin their results as if they somehow applied to real life situations. It’s intellectually dishonest, and leads to people like Eliezer having massive overconfidence in the relevance of H&B knowledge for personal rationality.
Awesome, big thanks!