As others have said, I believe AI pauses by governments would absolutely be more serious and longer, preventing overhangs from building up too much.
The big worry I do have with pause proposals in practice is that I expect most realistic pauses to buy us several years at most, but not decades long because people will shift their incentives towards algorithmic progress, which isn’t very controllable by default, and I also expect there to be at most 1 OOM of compute left to build AGI which scales to superintelligence by the time we pause, meaning that it’s a very unstable policy as any algorithmic advances like AI search actually working in complicated domains would immediately blow up the pause, and there are likely strong incentives to break the pause once people realize what superintelligence means.
As others have said, I believe AI pauses by governments would absolutely be more serious and longer, preventing overhangs from building up too much.
The big worry I do have with pause proposals in practice is that I expect most realistic pauses to buy us several years at most, but not decades long because people will shift their incentives towards algorithmic progress, which isn’t very controllable by default, and I also expect there to be at most 1 OOM of compute left to build AGI which scales to superintelligence by the time we pause, meaning that it’s a very unstable policy as any algorithmic advances like AI search actually working in complicated domains would immediately blow up the pause, and there are likely strong incentives to break the pause once people realize what superintelligence means.
See here for one example:
https://yellow-apartment-148.notion.site/AI-Search-The-Bitter-er-Lesson-44c11acd27294f4495c3de778cd09c8d