Yes, in reality, as people are approaching the “danger zone”, the safety work should gradually take more and more significant fraction.
I am impressed by OpenAI’s commitment of 20% of meaningful resources to AGI/ASI safety, and by the recent impressive diversification of their safety efforts (which is great, since we really need to cover all promising directions), but I expect that closer to the real danger zone, 50% of resources or even more allocated to safety would not be unreasonable (of course, separation between safety and capability is tricky, a lot of safety enhancers do boost capabilities or do have strong potential to boost capabilities, so those percentages of allocations might become more nuanced; while those allocations would need to be against “pure capability” roles, a lot of mixed roles are likely to be involved, people collaborating with advanced AI systems on how to make it safe for everyone is an example of a quintessentially mixed role).
So, what I really think is that if there is a robust ongoing safety effort in a lab which is already measured in dozens of percents of the overall force and resources, it should be relatively easy to gradually absorb the rest of the force and resources into that safety effort, if needed.
Whereas, if the safety effort is tiny and next to non-existent, then this is unlikely to work (and in such a case I would not expect a lab to be safety-conscious enough to pause anyway).
Yes, in reality, as people are approaching the “danger zone”, the safety work should gradually take more and more significant fraction.
I am impressed by OpenAI’s commitment of 20% of meaningful resources to AGI/ASI safety, and by the recent impressive diversification of their safety efforts (which is great, since we really need to cover all promising directions), but I expect that closer to the real danger zone, 50% of resources or even more allocated to safety would not be unreasonable (of course, separation between safety and capability is tricky, a lot of safety enhancers do boost capabilities or do have strong potential to boost capabilities, so those percentages of allocations might become more nuanced; while those allocations would need to be against “pure capability” roles, a lot of mixed roles are likely to be involved, people collaborating with advanced AI systems on how to make it safe for everyone is an example of a quintessentially mixed role).
So, what I really think is that if there is a robust ongoing safety effort in a lab which is already measured in dozens of percents of the overall force and resources, it should be relatively easy to gradually absorb the rest of the force and resources into that safety effort, if needed.
Whereas, if the safety effort is tiny and next to non-existent, then this is unlikely to work (and in such a case I would not expect a lab to be safety-conscious enough to pause anyway).