I… mostly just really like that this is one of the few posts in a while that seemed to make a credible advance at, you know, straightforwardly advancing the art of rationality. (Okay okay, Catching the Spark also counted. And… okay okay I can actually list a number of posts that engage directly with this. But, I feel like it’s not as common as I’d ideally like, or at least less direct. I liked that this post focused on it first-and-foremost, and established the context in which this was explicitly a rationality problem)
I particularly like laying out the further-research-directions at the end.
I think my main critique of this it doesn’t really lay out how often the Trapped Priors problem actually happens. Scott lists some examples of it happening (which I definitely have recognized in the wild). But I’m not sure if this is more of a rare edge case, a moderately common failure mode, or completely pervasive.
I too got the sense that this post was plausibly Important for the art of rationality. It seems to me to have the Sequences quality of quickly obsoleting itself (“oh, duh, this is obviously correct now“) and I now have a crisp handle for this kind of mental bug. i think that having that kind of handle is quite important.
You could certainly make an argument that every major ‘ism’ and religion is just a trapped prior. Racism, sexism, ageism—all of them, a person generates a conclusion, perhaps supplied by others, that negative traits about members of a particular class are true.
And it could be that these negative traits are more prevalent in the class. But they then generalize where they interpret during any meeting of a member of that class that the negative trait is in fact true. It’s also because the real world evidence is complicated. If you believe a member of a class is dumb as a rule, you can always pick some dumb behaviors or dismiss something smart they say as “too intellectual” or “is just reading the material he memorized”.
As an example, early in Obama’s term he gave a town hall where at least on video, the man on the fly appeared to answer each question with answers that were generally correctanswers for each subject. Yet conservative relatives of mine somehow saw this as evidence as to how incompetent/too academic whatever this politician was. If you believe that poor people are lazy it’s a similar thing. Or that homeless beggars are all faking it—oh look they have new shoes, must be lying.
Curated.
I… mostly just really like that this is one of the few posts in a while that seemed to make a credible advance at, you know, straightforwardly advancing the art of rationality. (Okay okay, Catching the Spark also counted. And… okay okay I can actually list a number of posts that engage directly with this. But, I feel like it’s not as common as I’d ideally like, or at least less direct. I liked that this post focused on it first-and-foremost, and established the context in which this was explicitly a rationality problem)
I particularly like laying out the further-research-directions at the end.
I think my main critique of this it doesn’t really lay out how often the Trapped Priors problem actually happens. Scott lists some examples of it happening (which I definitely have recognized in the wild). But I’m not sure if this is more of a rare edge case, a moderately common failure mode, or completely pervasive.
I too got the sense that this post was plausibly Important for the art of rationality. It seems to me to have the Sequences quality of quickly obsoleting itself (“oh, duh, this is obviously correct now“) and I now have a crisp handle for this kind of mental bug. i think that having that kind of handle is quite important.
You could certainly make an argument that every major ‘ism’ and religion is just a trapped prior. Racism, sexism, ageism—all of them, a person generates a conclusion, perhaps supplied by others, that negative traits about members of a particular class are true.
And it could be that these negative traits are more prevalent in the class. But they then generalize where they interpret during any meeting of a member of that class that the negative trait is in fact true. It’s also because the real world evidence is complicated. If you believe a member of a class is dumb as a rule, you can always pick some dumb behaviors or dismiss something smart they say as “too intellectual” or “is just reading the material he memorized”.
As an example, early in Obama’s term he gave a town hall where at least on video, the man on the fly appeared to answer each question with answers that were generally correct answers for each subject. Yet conservative relatives of mine somehow saw this as evidence as to how incompetent/too academic whatever this politician was. If you believe that poor people are lazy it’s a similar thing. Or that homeless beggars are all faking it—oh look they have new shoes, must be lying.