Agreed. I think a type of “stop AGI research” argument that’s under-deployed is that there’s no process or actor in the world that society would trust with unilateral godlike power. At large, people don’t trust their own governments, don’t trust foreign governments, don’t trust international organizations, and don’t trust corporations or their CEOs. Therefore, preventing anyone from building ASI anywhere is the only thing we can all agree on.
I expect this would be much more effective messaging with some demographics, compared to even very down-to-earth arguments about loss of control. For one, it doesn’t need to dismiss the very legitimate fear that the AGI would be aligned to values that a given person would consider monstrous. (Unlike “stop thinking about it, we can’t align it to any values!”.)
Agreed. I think a type of “stop AGI research” argument that’s under-deployed is that there’s no process or actor in the world that society would trust with unilateral godlike power. At large, people don’t trust their own governments, don’t trust foreign governments, don’t trust international organizations, and don’t trust corporations or their CEOs. Therefore, preventing anyone from building ASI anywhere is the only thing we can all agree on.
I expect this would be much more effective messaging with some demographics, compared to even very down-to-earth arguments about loss of control. For one, it doesn’t need to dismiss the very legitimate fear that the AGI would be aligned to values that a given person would consider monstrous. (Unlike “stop thinking about it, we can’t align it to any values!”.)
And it is, of course, true.