I expect AGI within 5 years. I give it a 95% chance that if an AGI is built, it will self-improve and wipe out humanity. In my view, the remaining 5% depends very little on who builds it. Someone who builds AGI while actively trying to end the world has almost exactly as much chance of doing so as someone who builds AGI for any other reason.
There is no “good guy with an AGI” or “marginally safer frontier lab.” There is only “oops, all entity smarter than us that we never figured out how to align or control.”
So what do you allocate the remaining 5% to? No matter who builds the AGI, there’s 5% chance that it doesn’t wipe out humanity because… what? (Or is it just model uncertainty?)
So what do you allocate the remaining 5% to? No matter who builds the AGI, there’s 5% chance that it doesn’t wipe out humanity because… what? (Or is it just model uncertainty?)
Yes, that’s my model uncertainty.