I can think of a plausible set of assumptions under which open AI is a good idea.
Suppose that in the crucial domain, safety work does not depend on capability work. AI safety researchers don’t need to know the latest capability breakthrough.
Obviously we want the work on capability to be behind the work on safety. If you think that making the safety part open source is going to improve the speed of its development, then open sourcing AI Safety is a good idea. And the safety researchers could still need a toy intelligence to test their work on. Nothing dangerous or even near the state of the art. Making that public could be sensible. Dangerous capabilities work should still be kept quiet until safety is more advanced.
This is a vision of the world where people cooperate on a global, open source AI safety project. So long as they finish before AI capability gets too high, anyone designing interesting new algorithms can just call “import human_values” to make their AI aligned. Who makes this AI is unimportant, so long as they use the library standard human_value instead of being totally stupid.
Of course, these are a set of not obviously false assumptions under which open AI makes sense, not necessarily the actual state of play, or even what open AI is thinking.
I can think of a plausible set of assumptions under which open AI is a good idea.
Suppose that in the crucial domain, safety work does not depend on capability work. AI safety researchers don’t need to know the latest capability breakthrough.
Obviously we want the work on capability to be behind the work on safety. If you think that making the safety part open source is going to improve the speed of its development, then open sourcing AI Safety is a good idea. And the safety researchers could still need a toy intelligence to test their work on. Nothing dangerous or even near the state of the art. Making that public could be sensible. Dangerous capabilities work should still be kept quiet until safety is more advanced.
This is a vision of the world where people cooperate on a global, open source AI safety project. So long as they finish before AI capability gets too high, anyone designing interesting new algorithms can just call “import human_values” to make their AI aligned. Who makes this AI is unimportant, so long as they use the library standard human_value instead of being totally stupid.
Of course, these are a set of not obviously false assumptions under which open AI makes sense, not necessarily the actual state of play, or even what open AI is thinking.