If you can prove that it’s you who melt all GPUs stealthy using AI-developed nanotech, it should be pretty obvious that the same AI without safety measures can kill everyone.
Scott Alexander once wrote that while it’s probably not wise to build AI organisation around pivotal act, if you find yourself in position where you can do it, you should do it, because, assuming you are not special genius decades ahead in AI development, if you can do pivotal act, someone else in AI can kill everyone.
I mean intelligence in wide sense, including wisdom, security mindset and self-control. And obviously, if I could build AI that can provide me such enhancement, I would enhance myself to solve full value-alignment problem, not give enhancement to random unchecked fools.
Yes, but that “I can’t let someone else handle this, I’ll do it myself behind their backs” generalized attitude is how actually we do get 100% all offed, no pivotal acts whatsoever. It’s delusion to think it leaves a measurable, non-infinitesimal window to actually succeeding—it does not. It simply leads to everyone racing and eventually someone who’s more reckless and thus faster “winning”. Or at best, it leads to a pivotal act by someone who then absolutely goes on to abuse their newfound power because no one can be inherently trusted with that level of control. That’s the best of the two worlds, but still bad.
Not quite.
If you live in the world where you can let others handle this, you can’t be in position to perform pivotal act, because others will successfully coordinate around not giving anyone (including you) unilateral capability to launch ASI.
And otherwise, if you find yourself in situation “there is a red button to melt all GPUs”, it means that others utterly failed to coordinate and you should pick the least bad world that remains possible.
If you can prove that it’s you who melt all GPUs stealthy using AI-developed nanotech, it should be pretty obvious that the same AI without safety measures can kill everyone.
Scott Alexander once wrote that while it’s probably not wise to build AI organisation around pivotal act, if you find yourself in position where you can do it, you should do it, because, assuming you are not special genius decades ahead in AI development, if you can do pivotal act, someone else in AI can kill everyone.
I mean intelligence in wide sense, including wisdom, security mindset and self-control. And obviously, if I could build AI that can provide me such enhancement, I would enhance myself to solve full value-alignment problem, not give enhancement to random unchecked fools.
Yes, but that “I can’t let someone else handle this, I’ll do it myself behind their backs” generalized attitude is how actually we do get 100% all offed, no pivotal acts whatsoever. It’s delusion to think it leaves a measurable, non-infinitesimal window to actually succeeding—it does not. It simply leads to everyone racing and eventually someone who’s more reckless and thus faster “winning”. Or at best, it leads to a pivotal act by someone who then absolutely goes on to abuse their newfound power because no one can be inherently trusted with that level of control. That’s the best of the two worlds, but still bad.
Not quite. If you live in the world where you can let others handle this, you can’t be in position to perform pivotal act, because others will successfully coordinate around not giving anyone (including you) unilateral capability to launch ASI. And otherwise, if you find yourself in situation “there is a red button to melt all GPUs”, it means that others utterly failed to coordinate and you should pick the least bad world that remains possible.