Akash, your comment raises the good point that a short-timelines plan that doesn’t realize governments are a really important lever here is missing a lot of opportunities for safety. Another piece of the puzzle that comes out when you consider what governance measures we’d want to include in the short timelines plan is the “off-ramps problem” that’s sort of touched on in this post.
Basically, our short timelines plan needs to also include measures (mostly governance/policy, though also technical) that get us to a desirable off-ramp from geopolitical tensions brought about by the economic and military transformation resulting from AGI/ASI.
I don’t think there are good off-ramps that do not route through governments. This is one reason to include more government-focused outreach/measures in our plans.
MiloSal
Karma: 28
Impact in AI Safety Now Requires Specific Strategic Insight
I think it is much less clear that pluralism is good than you portray. I would not, for example, want other weapons of mass destruction to be pluralized.
I’m fairly confident that this would be better than the current situation, and primarily because of something that others haven’t touched on here.
The reason is that, regardless of who develops them, the first (militarily and economically) transformative AIs will cause extreme geopolitical tension and instability that is challenging to resolve safely. Resolving such a situation safely requires a well-planned off-ramp, which must route through extremely major national- or international-level decisions. Only governments are equipped to make decisions like these; private AGI companies certainly are not.
Therefore, unless development is at some point centralized in a USG project, there is no way to avoid the many paths to catastrophe that threaten the world during the period of extreme tension coinciding with AGI/ASI development.