Thanks! Fwiw I agree with Zvi on “At a minimum, let’s not fire off a starting gun to a race that we might well not win, even if all of humanity wasn’t very likely to lose it, over a ‘missile gap’ style lie that we are somehow not currently in the lead.”
In some ways, this would be better if you can get universal buy-in, since there wouldn’t be a race for completion. There might be a race for alignment to particular subgroups? Which could be better or worse, depending.
Also, securing it against bringing insights and know-how back to a clandestine single-nation competitor seems like it would be very difficult. Like, if we had this kind of project being built, do I really believe there won’t be spies telling underground data centers and teams of researchers in Moscow and Washington everything it learns? And that governments will consistently put more effort into the shared project than the secret one?
I think the argument for combining separate US and Chinese projects into one global project is probably stronger than the argument for centralising US development. That’s because racing between US companies can potentially be handled by USG regulation, but racing between US and China can’t be similarly handled.
OTOH, the ‘info security’ benefits of centralisation mostly wouldn’t apply
I like a global project idea more, but I think it still has issues.
A global project would likely eliminate the racing concerns.
A global project would have fewer infosec issues. Hopefully, most state actors who could steal the weights are bought into the project and wouldn’t attack it.
Power concentration seems worse since more actors would have varying interests. Some countries would likely have ideological differences and might try to seize power over the project. Various checks and balances might be able to remedy this.
Why not just one global project?
My main take here is that it seems really unlikely that the US and China would agree to work together on this.
Zvi’s AI newsletter, latest installment https://www.lesswrong.com/posts/LBzRWoTQagRnbPWG4/ai-93-happy-tuesday, has a regular segment Pick Up the Phone arguing against this.
Thanks! Fwiw I agree with Zvi on “At a minimum, let’s not fire off a starting gun to a race that we might well not win, even if all of humanity wasn’t very likely to lose it, over a ‘missile gap’ style lie that we are somehow not currently in the lead.”
(You can find his ten mentions of that ~hashtag via the looking glass on thezvi.substack.com. huh, less regular than I thought.)
In some ways, this would be better if you can get universal buy-in, since there wouldn’t be a race for completion. There might be a race for alignment to particular subgroups? Which could be better or worse, depending.
Also, securing it against bringing insights and know-how back to a clandestine single-nation competitor seems like it would be very difficult. Like, if we had this kind of project being built, do I really believe there won’t be spies telling underground data centers and teams of researchers in Moscow and Washington everything it learns? And that governments will consistently put more effort into the shared project than the secret one?
Start small, once you have an attractive umbrella working for a few projects you can take in the rest of the US, the the world
I think the argument for combining separate US and Chinese projects into one global project is probably stronger than the argument for centralising US development. That’s because racing between US companies can potentially be handled by USG regulation, but racing between US and China can’t be similarly handled.
OTOH, the ‘info security’ benefits of centralisation mostly wouldn’t apply
That would seem to be better. As long a Putin and similar don’t get root access to an AGI as a result.
I like a global project idea more, but I think it still has issues.
A global project would likely eliminate the racing concerns.
A global project would have fewer infosec issues. Hopefully, most state actors who could steal the weights are bought into the project and wouldn’t attack it.
Power concentration seems worse since more actors would have varying interests. Some countries would likely have ideological differences and might try to seize power over the project. Various checks and balances might be able to remedy this.