The last paragraph stood out to me (emphasis mine).
Second, we believe it would be unintuitively risky and difficult to stop the creation of superintelligence. Because the upsides are so tremendous, the cost to build it decreases each year, the number of actors building it is rapidly increasing, and it’s inherently part of the technological path we are on, stopping it would require something like a global surveillance regime, and even that isn’t guaranteed to work. So we have to get it right.
There are efforts in AI governance that definitely don’t look like “global surveillance regime”! Taking the part above at face value, the authors seem to think that such efforts are not sufficient. But earlier on the post they talk about useful things that one could do in the AI governance field (lab coordination, independent IAEA-like authority), so I’m left confused about the authors’ models of what’s feasible and what’s not.
The passage also makes me worried that the authors are, despite their encouragement of coordination and audits, skeptical or even opposing to efforts to stop building dangerous AIs. (Perhaps this should have already been obvious from OpenAI pushing the capabilities frontier, but anyways.)
The last paragraph stood out to me (emphasis mine).
There are efforts in AI governance that definitely don’t look like “global surveillance regime”! Taking the part above at face value, the authors seem to think that such efforts are not sufficient. But earlier on the post they talk about useful things that one could do in the AI governance field (lab coordination, independent IAEA-like authority), so I’m left confused about the authors’ models of what’s feasible and what’s not.
The passage also makes me worried that the authors are, despite their encouragement of coordination and audits, skeptical or even opposing to efforts to stop building dangerous AIs. (Perhaps this should have already been obvious from OpenAI pushing the capabilities frontier, but anyways.)
When they say stopping I think they refer to stopping it forever, instead of slowing down, regulating and even pausing development.
Which I think is something pretty much everyone agrees on.