I don’t care what you think you’re saying—the primary operative takeaway for a large proportion of people, maybe everybody except recurring characters like Paul Christiano, is that even if their internal models say they have a solution, they should just shut up because they’re not you and can’t think correctly about these sorts of issues.
I think this is, unfortunately, true. One reason people might feel this way is because they view LessWrong posts through a social lens. Eliezer posts about how doomed alignment is and how stupid everyone else’s solution attempts are, that feels bad, you feel sheepish about disagreeing, etc.
But despite understandably having this reaction to the social dynamics, the important part of the situation is not the social dynamics. It is about finding technical solutions to prevent utter ruination. When I notice the status-calculators in my brain starting to crunch and chew on Eliezer’s posts, I tell them to be quiet, that’s not important, who cares whether he thinks I’m a fool. I enter a frame in which Eliezer is a generator of claims and statements, and often those claims and statements are interesting and even true, so I do pay attention to that generator’s outputs, but it’s still up to me to evaluate those claims and statements, to think for myself.
If Eliezer says everyone’s ideas are awful, that’s another claim to be evaluated. If Eliezer says we are doomed, that’s another claim to be evaluated. The point is not to argue Eliezer into agreement, or to earn his respect. The point is to win in reality, and I’m not going to do that by constantly worrying about whether I should shut up.
If I’m wrong on an object-level point, I’m wrong, and I’ll change my mind, and then keep working. The rest is distraction.
Sounds like same way we had a dumb questions post we need somewhere explicitly for posting dumb potential solutions that will totally never work, or something, maybe?
I think it’s unwise to internally label good-faith thinking as “dumb.” If I did that, I feel that I would not be taking my own reasoning seriously. If I say a quick take, or an uninformed take, I can flag it as such. But “dumb potential solutions that will totally never work”? Not to my taste.
That said, if a person is only comfortable posting under the “dumb thoughts incoming” disclaimer—then perhaps that’s the right move for them.
The point of that label is that for someone who already has the status-sense of “my ideas are probably dumb”, any intake point that doesn’t explicitly say “yeah, dumb stuff accepted here” will act as an emotional barrier. If you think what you’re carrying is trash, you’ll only throw it in the bin and not show it to anyone. If someone puts a brightly-colored bin right in front of you instead with “All Ideas Recycling! Two Cents Per Idea”, maybe you’ll toss it in there instead.
In the more general population, I believe the underlying sense to be a very common phenomenon, and easily triggered. Unless there is some other social context propping up a sense of equality, people will regularly feel dumb around you because you used a single long-and-classy-sounding word they didn’t know, or other similar grades of experience. Then they will stop telling you things. Including important things! If someone else who’s aligned can very overtly look less intimidating to step up and catch them, especially if they’re also volunteering some of the filtering effort that might otherwise make a broad net difficult to handle, that’s a huge win, especially because when people stop telling you things they often also stop listening and stop giving you the feedback you need to preserve alliances, much less try to convince them of anything “for real” rather than them walking away and feeling a sense of relief and throwing everything you said in the “that’s not for people like me” zone and never thinking about it again.
Notice what Aryeh Englander emphasized near the beginning of each of these secondary posts: “I noticed that while I had several points I wanted to ask about, I was reluctant to actually ask them”, “I don’t want to spam the group with half-thought-through posts, but I also want to post these ideas”. Beyond their truth value, these act as status-hedges (or anti-hedges, if you want to think of it in the sense of a hedge maze). They connect the idea of “I am feeling the same intimidation as you; I feel as dumb as you feel right now” with “I am acting like it’s okay to be open about this and giving you implicit permission to do the same”, thus helping puncture the bubble. (There is potentially some discussion to be had around the Sequences link I just edited in and what that implies for what can be expected socially, but I don’t want to dig too far unless people are interested and will only say that I don’t think relying on people putting that principle into practice most of the time is realistic in this context.)
Saying that people should not care about social dynamics and only about object level arguments is a failure at world modelling. People do care about social dynamics, if you want to win, you need to take that into account. If you think that people should act differently, well, you are right, but the people who counts are the real one, not those who live in your head.
Incentives matters. In today’s lesswrong, the threshold of quality for having your ideas heard (rather than everybody ganging up on you to explain how wrong you are) is much higher for people who disagree with Eliezer than for people who agree with him. Unsurprisingly, that means that people filter what they say at a higher rate if they disagree with Eliezer (or any other famous user honestly—including you.).
I wondered whether people would take away the message that “The social dynamics aren’t important.” I should have edited to clarify, so thanks for bringing this up.
Here was my intended message: The social dynamics are important, and it’s important to not let yourself be bullied around, and it’s important to make spaces where people aren’t pressured into conformity. But I find it productive to approach this situation with a mindset of “OK, whatever, this Eliezer guy made these claims, who cares what he thinks of me, are his claims actually correct?” This tactic doesn’t solve the social dynamics issues on LessWrong. This tactic just helps me think for myself.
So, to be clear, I agree that incentives matter, I agree that incentives are, in one way or another, bad around disagreeing with Eliezer (and, to lesser extents, with other prominent users). I infer that these bad incentives spring both from Eliezer’s condescension and rudeness, and also a range of other failures.
For example, if many people aren’t just doing their best to explain why they best-guess-of-the-facts agree with Eliezer—if those people are “ganging up” and rederiving the bottom line of “Eliezer has to be right”—then those people are failing at rationality,
or any other famous user honestly—including you.
For the record, I welcome any thoughtful commenter to disagree with me, for whatever small amount that reduces the anti-disagreement social pressure. I don’t negatively judge people who make good-faith efforts to disagree with me, even if I think their points are totally mistaken.
I think this is, unfortunately, true. One reason people might feel this way is because they view LessWrong posts through a social lens. Eliezer posts about how doomed alignment is and how stupid everyone else’s solution attempts are, that feels bad, you feel sheepish about disagreeing, etc.
But despite understandably having this reaction to the social dynamics, the important part of the situation is not the social dynamics. It is about finding technical solutions to prevent utter ruination. When I notice the status-calculators in my brain starting to crunch and chew on Eliezer’s posts, I tell them to be quiet, that’s not important, who cares whether he thinks I’m a fool. I enter a frame in which Eliezer is a generator of claims and statements, and often those claims and statements are interesting and even true, so I do pay attention to that generator’s outputs, but it’s still up to me to evaluate those claims and statements, to think for myself.
If Eliezer says everyone’s ideas are awful, that’s another claim to be evaluated. If Eliezer says we are doomed, that’s another claim to be evaluated. The point is not to argue Eliezer into agreement, or to earn his respect. The point is to win in reality, and I’m not going to do that by constantly worrying about whether I should shut up.
If I’m wrong on an object-level point, I’m wrong, and I’ll change my mind, and then keep working. The rest is distraction.
Sounds like same way we had a dumb questions post we need somewhere explicitly for posting dumb potential solutions that will totally never work, or something, maybe?
I have now posted a “Half-baked AI safety ideas thread” (LW version, EA Forum version) - let me know if that’s more or less what you had in mind.
I think it’s unwise to internally label good-faith thinking as “dumb.” If I did that, I feel that I would not be taking my own reasoning seriously. If I say a quick take, or an uninformed take, I can flag it as such. But “dumb potential solutions that will totally never work”? Not to my taste.
That said, if a person is only comfortable posting under the “dumb thoughts incoming” disclaimer—then perhaps that’s the right move for them.
The point of that label is that for someone who already has the status-sense of “my ideas are probably dumb”, any intake point that doesn’t explicitly say “yeah, dumb stuff accepted here” will act as an emotional barrier. If you think what you’re carrying is trash, you’ll only throw it in the bin and not show it to anyone. If someone puts a brightly-colored bin right in front of you instead with “All Ideas Recycling! Two Cents Per Idea”, maybe you’ll toss it in there instead.
In the more general population, I believe the underlying sense to be a very common phenomenon, and easily triggered. Unless there is some other social context propping up a sense of equality, people will regularly feel dumb around you because you used a single long-and-classy-sounding word they didn’t know, or other similar grades of experience. Then they will stop telling you things. Including important things! If someone else who’s aligned can very overtly look less intimidating to step up and catch them, especially if they’re also volunteering some of the filtering effort that might otherwise make a broad net difficult to handle, that’s a huge win, especially because when people stop telling you things they often also stop listening and stop giving you the feedback you need to preserve alliances, much less try to convince them of anything “for real” rather than them walking away and feeling a sense of relief and throwing everything you said in the “that’s not for people like me” zone and never thinking about it again.
Notice what Aryeh Englander emphasized near the beginning of each of these secondary posts: “I noticed that while I had several points I wanted to ask about, I was reluctant to actually ask them”, “I don’t want to spam the group with half-thought-through posts, but I also want to post these ideas”. Beyond their truth value, these act as status-hedges (or anti-hedges, if you want to think of it in the sense of a hedge maze). They connect the idea of “I am feeling the same intimidation as you; I feel as dumb as you feel right now” with “I am acting like it’s okay to be open about this and giving you implicit permission to do the same”, thus helping puncture the bubble. (There is potentially some discussion to be had around the Sequences link I just edited in and what that implies for what can be expected socially, but I don’t want to dig too far unless people are interested and will only say that I don’t think relying on people putting that principle into practice most of the time is realistic in this context.)
I for one really appreciate the ‘dumb-question’ area :)
Oh yes please. Maybe some tag that could be added to the comment. Maybe a comment in a different color.
Saying that people should not care about social dynamics and only about object level arguments is a failure at world modelling. People do care about social dynamics, if you want to win, you need to take that into account. If you think that people should act differently, well, you are right, but the people who counts are the real one, not those who live in your head.
Incentives matters. In today’s lesswrong, the threshold of quality for having your ideas heard (rather than everybody ganging up on you to explain how wrong you are) is much higher for people who disagree with Eliezer than for people who agree with him. Unsurprisingly, that means that people filter what they say at a higher rate if they disagree with Eliezer (or any other famous user honestly—including you.).
I wondered whether people would take away the message that “The social dynamics aren’t important.” I should have edited to clarify, so thanks for bringing this up.
Here was my intended message: The social dynamics are important, and it’s important to not let yourself be bullied around, and it’s important to make spaces where people aren’t pressured into conformity. But I find it productive to approach this situation with a mindset of “OK, whatever, this Eliezer guy made these claims, who cares what he thinks of me, are his claims actually correct?” This tactic doesn’t solve the social dynamics issues on LessWrong. This tactic just helps me think for myself.
So, to be clear, I agree that incentives matter, I agree that incentives are, in one way or another, bad around disagreeing with Eliezer (and, to lesser extents, with other prominent users). I infer that these bad incentives spring both from Eliezer’s condescension and rudeness, and also a range of other failures.
For example, if many people aren’t just doing their best to explain why they best-guess-of-the-facts agree with Eliezer—if those people are “ganging up” and rederiving the bottom line of “Eliezer has to be right”—then those people are failing at rationality,
For the record, I welcome any thoughtful commenter to disagree with me, for whatever small amount that reduces the anti-disagreement social pressure. I don’t negatively judge people who make good-faith efforts to disagree with me, even if I think their points are totally mistaken.