Excitement vs childishness
I’ve heard Robin Hanson and others make the argument that people are often biased towards the fast takeoff scenario because it’s exciting to think about (or “sexy”). On the other hand, there is a bias towards disbelieving the fast-takeoff scenario because it’s childish.
I think most of us can agree that both are indeed biases, i.e., should be assigned zero weight, because both are about attributes that don’t correlate with what’s true. So we have the excitement-bias and the childishness-bias. The question is, how do they compare?
It feels obvious to me that the childishness bias is far stronger. I see people signaling maturity all the time, and childishness seems to be extremely low status in the relevant circles. It’s so low status that it’s not uncommon to see people say things about dangers from AI that are accurately summarized as “even though I don’t know anything about this topic, I will evaluate its legitimacy based on how many childish sounding arguments I hear because obviously, they are not the real concern and people who defend them have zero credibility.” Even among those who provide other arguments, it seems more common than not that they also assume that the fast takeoff scenario is less likely a priori because it’s childish. Conversely, I’ve never heard anyone imply that fast takeoff must be true, or even likely, because it’s exciting. It seems that, to have a similar effect, the excitement bias would have to do some heavy lifting in a purely subconscious way.
Given all this, the fact that I’ve seen far more discussion about the excitement-bias than the childishness-bias seems wrong.
Disclaimer: I don’t think this is a strong argument that fast takeoff is likely, and I also don’t think that bias towards excitement is weak in general – just that it’s weak among the relevant class of people. Finally, I don’t think this bias is common on LessWrong, only just about everywhere else.
I’m glad you made that disclaimer. In our community—LW—the ratio of excitement/childishness bias is probably unusually high, perhaps even high enough that we need to be on higher guard against excitement. But in the wider community of “smart and/or important people thinking about AI” it seems pretty clear that childishness bias is much much stronger.
Counterevidence that the excitement bias afflicts people less than the childishness bias: Michael Wulfsohn’s essay The tyranny of the god scenario.
I’m not sure this is strong counterevidence, because it can be interpreted as a case of childishness bias overcoming excitement bias:
Indeed I’d go so far as to say if this is the best case we can think of of excitement bias being stronger, that’s pretty good evidence that in fact childishness bias is usually way stronger.
Like, I agree that Michael Wulfson is a person for whom the excitement bias was stronger, judging by his story. But even for him, the childishness bias was super strong too, and eventually won out. Most people are not like him; most people will find the childishness bias / excitement bias ratio much higher than he did.