I think this is the sort of conversation we should be having! [Side note: I think restricting compute is more effective than restricting research because you don’t need 100% buy in.
it’s easier to prevent people from manufacturing semiconductors than to keep people from learning ideas that fit on a napkin
It’s easier to prevent scientists in Eaccistan from having GPUs than to prevent scientists in Eaccistan from thinking.
The analogy to nuclear weapons is, I think, a good one. The science behind nuclear weapons is well known—what keeps them from being built is access to nuclear materials.
(Restricting compute also seriously restricts research. Research speed on neural nets is in large part bounded by how many experiments you run rather than ideas you have.)]
Well I asked this https://www.lesswrong.com/posts/X9Z9vdG7kEFTBkA6h/what-could-a-policy-banning-agi-look-like but roughly no one was interested—I had to learn about “born secret” https://en.wikipedia.org/wiki/Born_secret from Eric Weinstein in a youtube video.
FYI, while restricting compute manufacture is I would guess net helpful, it’s far from a solution. People can make plenty of conceptual progress given current levels of compute https://www.lesswrong.com/posts/sTDfraZab47KiRMmT/views-on-when-agi-comes-and-on-strategy-to-reduce . It’s not a way out, either. There are ways possibly-out. But approximately no one is interested in them.
Can you link?
https://www.youtube.com/watch?v=7x0W-9HPB2w&t=4200
(see timestamp)
I think this is the sort of conversation we should be having! [Side note: I think restricting compute is more effective than restricting research because you don’t need 100% buy in.
it’s easier to prevent people from manufacturing semiconductors than to keep people from learning ideas that fit on a napkin
It’s easier to prevent scientists in Eaccistan from having GPUs than to prevent scientists in Eaccistan from thinking.
The analogy to nuclear weapons is, I think, a good one. The science behind nuclear weapons is well known—what keeps them from being built is access to nuclear materials.
(Restricting compute also seriously restricts research. Research speed on neural nets is in large part bounded by how many experiments you run rather than ideas you have.)]
This is not true for AGI.