If the leading lab can’t stop critical models from leaking to actors that won’t use great deployment safety practices, approximately nothing else matters.
This seems somewhat overstated. You might hope that you can get the safety tax sufficiently low that you can just do full competition (e.g. even though there are rogue AIs, you just compete with this rogue AIs for power). This also requires offense-defense imbalance to not be too bad.
I overall agree that securing model weights in underrated and that it is plausibly the most important thing on current margins.
In principle, if reasonable actors start with a high fraction of resources (e.g. compute), then you might hope that they can keep that fraction of power (in expectation at least).
This seems somewhat overstated. You might hope that you can get the safety tax sufficiently low that you can just do full competition (e.g. even though there are rogue AIs, you just compete with this rogue AIs for power). This also requires offense-defense imbalance to not be too bad.
I overall agree that securing model weights in underrated and that it is plausibly the most important thing on current margins.
In principle, if reasonable actors start with a high fraction of resources (e.g. compute), then you might hope that they can keep that fraction of power (in expectation at least).
See also “The strategy-stealing assumption”. But also What does it take to defend the world against out-of-control AGIs?.