AI safety research is still solving easy problems. [...]
Capability development is getting AI safety research for free. [...]
AI safety research is speeding up capabilities. [...]
Even if (2) and (3) are true and (1) is mostly true (e.g. most safety research is worthless), I still think it can easily be worthwhile to indiscriminately increase the supply of safety research[1].
The core thing is a quantitative argument: there are far more people working on capabities than x-safety and if no one works on safety, nothing will happen at all.
There currently seems to be >10x as many people directly trying to build AGI/improve capabilities as trying to improve safety.
Suppose that the safety people have as good ideas and research ability as the capabilities people. (As a simplifying assumption.)
Then, if all the safety people switched to working full time on maximally advancing capabilities, this would only advance capabilites by less than 10%.
If, on the other hand, they stopped publically publishing safety work and this resulted in a 50% slow down, all safety work would slow down by 50%.
Naively, it seems very hard for publishing less to make sense if the number of safety researchers is much smaller than the number of capabilities researchers and safety researchers aren’t much better at capabilities than capabilities researchers.
Of course, there might be better things to do than indiscriminately increase the supply. E.g., maybe it is better to try to steer the direction of the field.
Even if (2) and (3) are true and (1) is mostly true (e.g. most safety research is worthless), I still think it can easily be worthwhile to indiscriminately increase the supply of safety research[1].
The core thing is a quantitative argument: there are far more people working on capabities than x-safety and if no one works on safety, nothing will happen at all.
Copying a version of this argument from a prior comment I made:
There currently seems to be >10x as many people directly trying to build AGI/improve capabilities as trying to improve safety.
Suppose that the safety people have as good ideas and research ability as the capabilities people. (As a simplifying assumption.)
Then, if all the safety people switched to working full time on maximally advancing capabilities, this would only advance capabilites by less than 10%.
If, on the other hand, they stopped publically publishing safety work and this resulted in a 50% slow down, all safety work would slow down by 50%.
Naively, it seems very hard for publishing less to make sense if the number of safety researchers is much smaller than the number of capabilities researchers and safety researchers aren’t much better at capabilities than capabilities researchers.
Of course, there might be better things to do than indiscriminately increase the supply. E.g., maybe it is better to try to steer the direction of the field.
If only...
(Oops, fixed.)