It still seems to me that international cooperation isn’t the right first step. If the US believes that AI is potentially world-ending, it should put its money where its mouth is, and first set up a national commission with the power to check AIs and AI training runs for safety, and ban them if needed. Then China will plausibly do the same as well, and from a cooperation of like-minded people in both countries’ safety commissions we can maybe get an international commission. But if you skip this first step, then China’s negotiators can reasonably say: why do you ask us for cooperation while you still continue AI development unchecked? This shows you don’t really believe it’s dangerous, and are just trying to gain an advantage.
It still seems to me that international cooperation isn’t the right first step. If the US believes that AI is potentially world-ending, it should put its money where its mouth is, and first set up a national commission with the power to check AIs and AI training runs for safety, and ban them if needed. Then China will plausibly do the same as well, and from a cooperation of like-minded people in both countries’ safety commissions we can maybe get an international commission. But if you skip this first step, then China’s negotiators can reasonably say: why do you ask us for cooperation while you still continue AI development unchecked? This shows you don’t really believe it’s dangerous, and are just trying to gain an advantage.