I agree with a lot of claims in your comment, and I think it’s valuable to think through how status plays a role in many situations, including this.
There is an approach in your comments toward explaining someone’s behaviour that I disagree with, though it may just be a question of emphasis. A few examples:
My real model is that they took those ideas extra seriously because the people were nice and high status.
...a prerequisite for them changing their mind was that they were taken seriously and engaged with respectfully
These seem to me definitely true and simultaneously not that important*.
When I read that essay, at least half of it is heavily-laden with status concerns and psychological motivations. But, to reiterate: though pro-social community norms left this person open to having their mind changed by argument, probably the arguments still had to be made. (emphasis added)
The word ‘probably’ in that sentence feels false to me. I feel somewhat analogous to hearing someone argue that a successful tech startup is 100s of people working together in a company, and that basically running a tech startup is about status and incentives, though “probably code still had to be written” to make it successful. They’re both necessary.
More generally, there are two types of games going on. One we’re allowed to talk about, and one we’re not, or at least not very directly. And we have to coordinate on both levels to succeed. This generally warps how our words relate to reality, because we’re also using those words to do things we’re pretending to ourselves we’re not doing, to let everyone express their preferences and coordinate in the silent games. These silent games have real and crucial implications for how well we can coordinate and where resources must be spent. But once you realise the silent games are being played, it isn’t the right move to say that the silent games are the only games, or always the primary games.
I think the fact that the model doesn’t actually work is an important aspect of this. Many hackers would have done it as a cool project and released it without pomp, but this person put together a long essay, explicitly touting the importance of what they’d done and the impact it would have on history. Then, it turned out the model did not work, which must have been very embarrassing. It is fairly reasonable to suggest that the person then took the action that made them feel the best about their legacy and status: writing an essay about why they were not releasing the model for good rationalist approved reasons. It is not even necessarily the case that the person is aware that this is influencing the decision, this is a fully Elephant in the Brain situation.
Again, I agree that something in this reference class is likely happening. But, for example, the long essay was not onlyabout increasing the perceived importance of the action. It was also a strongly pro-social and cooperative move to the broader AI community to allow counterarguments to be presented, which is what successfully happened. There are multiple motives here, and (I think) it’s the case that the motive you point to was not the main one, even while it is a silent motive folks systematically avoid discussing.
--
*Actually I think that Connor in particular would’ve engaged with arguments even if they’d not been delivered respectfully, given that he responded substantively to many comments on Twitter/HackerNews/Medium, some of which were predominantly snark.
When Robin Hanson is interviewed about The Elephant in the Brain, he is often asked “Are you saying that status accounts for all of our behaviour?”. His reply is that he+KevinSimler aren’t arguing that the hidden motives are the only motive, but that they’re a far more common motive than we give credit for in our normal discourse. Here’s an example of him saying this kind of thing on the 80k podcast:
As we just said the example that, in education, your motive isn’t to learn the material, or when you go to the doctor, your motive isn’t to get well primarily, and the hidden motives are the actual motive. Now, how could I know what the hidden motives are, you might ask? The plan here, that’s where the book is … In each area, we identify the usual story, then we collect a set of puzzles that don’t make sense from the point of view of the usual story, strange empirical patterns, and then we offer an alternative motive that makes a lot more sense of those empirical patterns, and then we suggest that that is a stronger motive than the one we usually say.
Now, just to be clear, almost every area of human life is complicated, and there’s a lot of people with a lot of different details and so, of course, almost every possible motive shows up in almost every area of human life, so we can’t be talking about the only motive, and so the usual motive does actually apply sometimes. Actually, you could think of the analogy to the excuse that the dog ate my homework. It only works because sometimes dogs eat homework. We don’t say the dragon ate my homework. That wouldn’t fly, so the usual story is part of the story. It’s just a smaller part than we like to admit, and what we’re going to call the hidden motive, the real motive is a bigger part of the story, but it’s still not the only part.
I agree with a lot of claims in your comment, and I think it’s valuable to think through how status plays a role in many situations, including this.
There is an approach in your comments toward explaining someone’s behaviour that I disagree with, though it may just be a question of emphasis. A few examples:
These seem to me definitely true and simultaneously not that important*.
The word ‘probably’ in that sentence feels false to me. I feel somewhat analogous to hearing someone argue that a successful tech startup is 100s of people working together in a company, and that basically running a tech startup is about status and incentives, though “probably code still had to be written” to make it successful. They’re both necessary.
More generally, there are two types of games going on. One we’re allowed to talk about, and one we’re not, or at least not very directly. And we have to coordinate on both levels to succeed. This generally warps how our words relate to reality, because we’re also using those words to do things we’re pretending to ourselves we’re not doing, to let everyone express their preferences and coordinate in the silent games. These silent games have real and crucial implications for how well we can coordinate and where resources must be spent. But once you realise the silent games are being played, it isn’t the right move to say that the silent games are the only games, or always the primary games.
Again, I agree that something in this reference class is likely happening. But, for example, the long essay was not only about increasing the perceived importance of the action. It was also a strongly pro-social and cooperative move to the broader AI community to allow counterarguments to be presented, which is what successfully happened. There are multiple motives here, and (I think) it’s the case that the motive you point to was not the main one, even while it is a silent motive folks systematically avoid discussing.
--
*Actually I think that Connor in particular would’ve engaged with arguments even if they’d not been delivered respectfully, given that he responded substantively to many comments on Twitter/HackerNews/Medium, some of which were predominantly snark.
When Robin Hanson is interviewed about The Elephant in the Brain, he is often asked “Are you saying that status accounts for all of our behaviour?”. His reply is that he+KevinSimler aren’t arguing that the hidden motives are the only motive, but that they’re a far more common motive than we give credit for in our normal discourse. Here’s an example of him saying this kind of thing on the 80k podcast: