If we do as well with preventing AGI as we have with nuclear non-proliferation, we fail. And, nuclear non-proliferation has been more effective than some other regimes (chemical weapons, drugs, trade in endangered animals, carbon emissions, etc.). In addition, because of the need for relatively scarce elements, control over nuclear weapons is easier than control over AI.
And, as others have noted the incentives for develpong AI are far stronger than for developing nuclear weapons.
What makes you think we fail if it looks like nukes? If everyone agrees on alignment difficulty and we have few actors, it is not unreasonable for no one to push the button, just like they don’t with MAD.
There are currently nine countries who have deployed nuclear weapons. At least four of those nine are countries that the non-proliferation regime would have preferred to prevent having nuclear weapons.
An equivalent result in AGI would have four entities deploying AGI. (And in the AGI context, the problem is deployment not using the AGI in any particular way.)
Note that 8 of those countries have never used nukes, and all 9 of them if you start after the IAEA was founded.
Most people think if 500 entities had nukes, they would be used more. But with few, MAD can work. AGI doesn’t have MAD, but it has a similar dynamic if you convince everyone of the alignment problem.
If we do as well with preventing AGI as we have with nuclear non-proliferation, we fail. And, nuclear non-proliferation has been more effective than some other regimes (chemical weapons, drugs, trade in endangered animals, carbon emissions, etc.). In addition, because of the need for relatively scarce elements, control over nuclear weapons is easier than control over AI.
And, as others have noted the incentives for develpong AI are far stronger than for developing nuclear weapons.
What makes you think we fail if it looks like nukes? If everyone agrees on alignment difficulty and we have few actors, it is not unreasonable for no one to push the button, just like they don’t with MAD.
There are currently nine countries who have deployed nuclear weapons. At least four of those nine are countries that the non-proliferation regime would have preferred to prevent having nuclear weapons.
An equivalent result in AGI would have four entities deploying AGI. (And in the AGI context, the problem is deployment not using the AGI in any particular way.)
Note that 8 of those countries have never used nukes, and all 9 of them if you start after the IAEA was founded.
Most people think if 500 entities had nukes, they would be used more. But with few, MAD can work. AGI doesn’t have MAD, but it has a similar dynamic if you convince everyone of the alignment problem.