There’s an issue with AI Safety funding: many people agree it’s important for there to be an AI Safety field, and such a field should grow, evaluate the problem from different angles, build up a body of knowledge and cultivate people who think seriously about the problem full-time.
Non-safety AI Researchers should know that such a field exists, that it worth serious attention.
This idea is popular both because it is true, but (I think) also because it is a tempting way to pass the buck.
If you’re a funder who thinks AI Alignment is important, well, you’re not sure what actually needs doing, but you can at least agree to fund something generic that “builds the field.” A specific approach might be wrong, a generic approach can’t “be wrong”, and so your bullshit detectors don’t see any concrete red flags.
But… at some point, to actually make progress, you need to get into the weeds and make a plan that involves specific assumptions and try to implement that plan.
MIRI is (relatively) unique among AI Safety organizations in that it focuses on particular types of approaches. The researchers at MIRI aren’t a monolith, but they tend to share particular assumptions about the constraints of the problem, and whatsolving the problem would look like. And then trying to actually create that solution.
And they may be wrong in some of their assumptions. But someone has to have a plan that is specific enough that it might be wrong and try to implement that plan.
For this reason, I expect MIRI (and things like it) to be generally underfunded relative to more “generic seeming” organizations that aren’t making explicit claims.
If true, this claim should probably push you to be more willing to fund things that are attempting concrete solutions.
(I also happen to think they are not wrong in their assumptions, and one of the things that’s impressed me over the years is seeing their strategy shift over time as the landscape of AI changed, which I’ve found reassuring. I trust them to continue to update as new information comes out)
I should note that I’m not sure whether OpenAI is a point against this claim or not (I think not but for complicated reasons). My vague impression is that they do tend to have their own set of assumptions, and are working on reasonably concrete things (I think those assumptions are wrong but am not that confident).
I do lean towards OpenAI and MIRI should both be fully funded, OpenAI just seems to be getting a lot more funding due to Elon’s involvement and generally being more “traditionally prestigious”.
Insofar as I think OpenAI shouldn’t be funded, it’s because I think it might be actively harmful.
(epistemic status: I am not very informed about the current goings on at OpenAI, this is a random person rambling hearsay and making the best guesses they can without doing a thorough review of their blog, let alone talking to them)
The reasons it might be actively harmful is because it seems like a lot of their work is more like actuallydeveloping AI instead of AI Safety, and sharing AI developments with the world that might accelerate progress.
MIRI is the only organization I know of working directly on AI safety that I’ve heard talk extensively about differential-technological-development. i.e, do research that would only help build Aligned AGI and doesn’t accelerate generic AI that might feed into Unaligned AGI.
A related thought:
There’s an issue with AI Safety funding: many people agree it’s important for there to be an AI Safety field, and such a field should grow, evaluate the problem from different angles, build up a body of knowledge and cultivate people who think seriously about the problem full-time.
Non-safety AI Researchers should know that such a field exists, that it worth serious attention.
This idea is popular both because it is true, but (I think) also because it is a tempting way to pass the buck.
If you’re a funder who thinks AI Alignment is important, well, you’re not sure what actually needs doing, but you can at least agree to fund something generic that “builds the field.” A specific approach might be wrong, a generic approach can’t “be wrong”, and so your bullshit detectors don’t see any concrete red flags.
But… at some point, to actually make progress, you need to get into the weeds and make a plan that involves specific assumptions and try to implement that plan.
MIRI is (relatively) unique among AI Safety organizations in that it focuses on particular types of approaches. The researchers at MIRI aren’t a monolith, but they tend to share particular assumptions about the constraints of the problem, and what solving the problem would look like. And then trying to actually create that solution.
And they may be wrong in some of their assumptions. But someone has to have a plan that is specific enough that it might be wrong and try to implement that plan.
For this reason, I expect MIRI (and things like it) to be generally underfunded relative to more “generic seeming” organizations that aren’t making explicit claims.
If true, this claim should probably push you to be more willing to fund things that are attempting concrete solutions.
(I also happen to think they are not wrong in their assumptions, and one of the things that’s impressed me over the years is seeing their strategy shift over time as the landscape of AI changed, which I’ve found reassuring. I trust them to continue to update as new information comes out)
I should note that I’m not sure whether OpenAI is a point against this claim or not (I think not but for complicated reasons). My vague impression is that they do tend to have their own set of assumptions, and are working on reasonably concrete things (I think those assumptions are wrong but am not that confident).
I do lean towards OpenAI and MIRI should both be fully funded, OpenAI just seems to be getting a lot more funding due to Elon’s involvement and generally being more “traditionally prestigious”.
Further thoughts here:
Insofar as I think OpenAI shouldn’t be funded, it’s because I think it might be actively harmful.
(epistemic status: I am not very informed about the current goings on at OpenAI, this is a random person rambling hearsay and making the best guesses they can without doing a thorough review of their blog, let alone talking to them)
The reasons it might be actively harmful is because it seems like a lot of their work is more like actually developing AI instead of AI Safety, and sharing AI developments with the world that might accelerate progress.
MIRI is the only organization I know of working directly on AI safety that I’ve heard talk extensively about differential-technological-development. i.e, do research that would only help build Aligned AGI and doesn’t accelerate generic AI that might feed into Unaligned AGI.