I’ve been drawing more connections the more I think about this.
Here’s one: belief-in-belief. Humans seem hard-wired to sometimes believe that they believe things different from their aliefs (i.e. their “real” beliefs as implied by their actions). Why? The usual answer is handwave politics handwave machiavellian intelligence hypothesis handwave. But what are the actual concrete mechanisms by which humans are able to gain by being good at “politics” in this particular way? What exactly are these “politics”, what is the form of the game, how do “politics” have such large evolutionary rewards?
One possible answer is that belief-in-belief evolved to solve coordination games, via exactly the sort of strategy in this post.
Example: a group is organizing a lion hunt, and the tribe’s shaman blesses them with lion-protection. They don’t want to actually believe that they’re magically protected against lions; that would imply terrible tactics like “casually stroll in and pick a one-on-one fight with a lion”. Nor do they want to pretend to believe to be magically protected; that would at-best trick other members of the tribe into thinking they’re magically protected against lions, which would just mean other tribe-members get slaughtered by bad tactics. (Yes, evolution is usually happy when competitors die, but in this case I’m assuming that humans in a larger tribe have higher reproductive fitness than in a smaller tribe or alone.) What the hunters really want is to pretend-to-pretend to be protected from lions (so everyone goes on the hunt), while also acting like they are not protected (e.g. for tactical purposes).
If the goal is to pretend-to-pretend, without actually believing or pretending to actually believe, then belief-in-belief is the ideal tool.
Seems like, step by step, every irrational human behavior will be explained as a higher form of rationality. At the end, we will realize that the only truly irrational people on this planet are the so-called rationalists.
It’s easy to say this if you’re surrounded by nerdy types who stubbornly refuse to leave simulacrum 1. But have you looked at other people these days??
Look at the Midwestern mom who just blew another $200 on the latest exercise fad that is definitely not going to give her the body she wants. Look at every business that went under because they failed to measure what really mattered. Look at anyone whose S3 sentiment has been so easily hijacked and commoditized by the social media outrage machines.
Nah, I’m thoroughly convinced that there is still an advantage in knowing what’s actually true, which means seeing the S levels for what they are.
I can’t tell if it’s related in a way relevant to the post, but the lion anecdote definitely reminds me of this description of CA’s vaccine rollout. CA wanted to make sure vaccines were distributed evenly or disproportionately towards disprivileged people. One effort towards this was zipcode restrictions, enforced with ID requirements. Unfortunately ID and proof of address are exactly the kind of things that privilege helps you access. Being poor, moving a lot, working weird hours, lacking legal right to be in the country… all make you less likely to have ID. The article alleges that the (unspoken) plan was to just not enforce those rules against people who looked disprivileged, but not every worker got the memo and lots of people who everyone wanted to have a vaccine were turned away because they didn’t have a bill with their address on it.
I’ve been drawing more connections the more I think about this.
Here’s one: belief-in-belief. Humans seem hard-wired to sometimes believe that they believe things different from their aliefs (i.e. their “real” beliefs as implied by their actions). Why? The usual answer is handwave politics handwave machiavellian intelligence hypothesis handwave. But what are the actual concrete mechanisms by which humans are able to gain by being good at “politics” in this particular way? What exactly are these “politics”, what is the form of the game, how do “politics” have such large evolutionary rewards?
One possible answer is that belief-in-belief evolved to solve coordination games, via exactly the sort of strategy in this post.
Example: a group is organizing a lion hunt, and the tribe’s shaman blesses them with lion-protection. They don’t want to actually believe that they’re magically protected against lions; that would imply terrible tactics like “casually stroll in and pick a one-on-one fight with a lion”. Nor do they want to pretend to believe to be magically protected; that would at-best trick other members of the tribe into thinking they’re magically protected against lions, which would just mean other tribe-members get slaughtered by bad tactics. (Yes, evolution is usually happy when competitors die, but in this case I’m assuming that humans in a larger tribe have higher reproductive fitness than in a smaller tribe or alone.) What the hunters really want is to pretend-to-pretend to be protected from lions (so everyone goes on the hunt), while also acting like they are not protected (e.g. for tactical purposes).
If the goal is to pretend-to-pretend, without actually believing or pretending to actually believe, then belief-in-belief is the ideal tool.
Seems like, step by step, every irrational human behavior will be explained as a higher form of rationality. At the end, we will realize that the only truly irrational people on this planet are the so-called rationalists.
(Not completely serious, but...)
It’s easy to say this if you’re surrounded by nerdy types who stubbornly refuse to leave simulacrum 1. But have you looked at other people these days??
Look at the Midwestern mom who just blew another $200 on the latest exercise fad that is definitely not going to give her the body she wants. Look at every business that went under because they failed to measure what really mattered. Look at anyone whose S3 sentiment has been so easily hijacked and commoditized by the social media outrage machines.
Nah, I’m thoroughly convinced that there is still an advantage in knowing what’s actually true, which means seeing the S levels for what they are.
This seems like a reasonable mechanism, but I thought we already had one: belief-in-belief makes it easier to lie without being caught.
I can’t tell if it’s related in a way relevant to the post, but the lion anecdote definitely reminds me of this description of CA’s vaccine rollout. CA wanted to make sure vaccines were distributed evenly or disproportionately towards disprivileged people. One effort towards this was zipcode restrictions, enforced with ID requirements. Unfortunately ID and proof of address are exactly the kind of things that privilege helps you access. Being poor, moving a lot, working weird hours, lacking legal right to be in the country… all make you less likely to have ID. The article alleges that the (unspoken) plan was to just not enforce those rules against people who looked disprivileged, but not every worker got the memo and lots of people who everyone wanted to have a vaccine were turned away because they didn’t have a bill with their address on it.
I forgot this wasn’t actually part of the original post. I think it should be.