I appreciate these views being stated clearly, and at once feel a positive feeling toward the author, and also am shaking my head No. As others have pointed out, the mistake theory here is confused.
I think it’s not exactly wrong. The way in which it’s right is this:
If people doing AGI research understood what we understand about the existential risk of AGI, most of them would stop, and AGI research would go much slower.
In other words, most people are amenable to reason on this point, in the sense that they’d respond to reasons to not do something that they’ve been convinced of. This is not without exception; some players, e.g. Larry Page (according to Elon Musk), want AGI to take the world from humanity.
The way in which the mistake theory is wrong is this:
Many people doing AGI research are not trying, and in many cases trying not, to understand what we understand about AGI risk.
So it’s not just a mistake. It’s a choice, that choice has motivations, and those motivations are in conflict with our motivations, insofar as they shelter themselves from reason.
So it’s not just a mistake. It’s a choice, that choice has motivations, and those motivations are in conflict with our motivations, insofar as they shelter themselves from reason.
This still seems, to me, like a special case of “mistake”.
I appreciate these views being stated clearly, and at once feel a positive feeling toward the author, and also am shaking my head No. As others have pointed out, the mistake theory here is confused.
I think it’s not exactly wrong. The way in which it’s right is this:
In other words, most people are amenable to reason on this point, in the sense that they’d respond to reasons to not do something that they’ve been convinced of. This is not without exception; some players, e.g. Larry Page (according to Elon Musk), want AGI to take the world from humanity.
The way in which the mistake theory is wrong is this:
So it’s not just a mistake. It’s a choice, that choice has motivations, and those motivations are in conflict with our motivations, insofar as they shelter themselves from reason.
(IIRC, Tegmark, who was present for the relevant event, has confirmed that Page had stated his position as described.)
This still seems, to me, like a special case of “mistake”.
It’s not just epistemic confusion that can be most easily corrected with good evidence and arguments. That’s what I think we’re talking about.
Well, I wrote about this here: https://www.lesswrong.com/posts/tMtMHvcwpsWqf9dgS/class-consciousness-for-those-against-the-class-system
But the internet loves to downvote without explaining why...