I think this is the sort of conversation we should be having! [Side note: I think restricting compute is more effective than restricting research because you don’t need 100% buy in.
it’s easier to prevent people from manufacturing semiconductors than to keep people from learning ideas that fit on a napkin
It’s easier to prevent scientists in Eaccistan from having GPUs than to prevent scientists in Eaccistan from thinking.
The analogy to nuclear weapons is, I think, a good one. The science behind nuclear weapons is well known—what keeps them from being built is access to nuclear materials.
(Restricting compute also seriously restricts research. Research speed on neural nets is in large part bounded by how many experiments you run rather than ideas you have.)]
I think this is the sort of conversation we should be having! [Side note: I think restricting compute is more effective than restricting research because you don’t need 100% buy in.
it’s easier to prevent people from manufacturing semiconductors than to keep people from learning ideas that fit on a napkin
It’s easier to prevent scientists in Eaccistan from having GPUs than to prevent scientists in Eaccistan from thinking.
The analogy to nuclear weapons is, I think, a good one. The science behind nuclear weapons is well known—what keeps them from being built is access to nuclear materials.
(Restricting compute also seriously restricts research. Research speed on neural nets is in large part bounded by how many experiments you run rather than ideas you have.)]
This is not true for AGI.