I think one core issue here is that there are actually two debates going on. One is “how hard is the alignment problem?”; another is “how powerful are prosaic alignment techniques?” Broadly speaking, I’d characterise most of the disagreement as being on the first question. But you’re treating it like it’s mostly on the second question—like EY and everyone else are studying the same thing (cancer, in your metaphor) and just disagree about how to treat it.
That’s an interesting separation of the problem, because I really feel there is more disagreement on the second question than on the first.
My attempt to portray EY’s perspective is more like: he’s concerned with the problem of ageing, and a whole bunch of people have come along, said they agree with him, and started proposing ways to cure cancer using prosaic radiotherapy techniques. Now he’s trying to say: no, your work is not addressing the core problem of ageing, which is going to kill us unless we make a big theoretical breakthrough.
Funnily, aren’t the people currently working on ageing using quite prosaic techniques? I completely agree that one need to go for the big problems, especially ones that only appear in more powerful regimes (which is why I am adamant that there should be places for researchers to think about distinctly AGI problems and not have to rephrase everything in a way that is palatable to ML academia). But people like Paul and Evan and more are actually going for the core problems IMO, just anchoring a lot of their thinking in current ML technologies. So I have trouble understanding how prosaic alignment isn’t trying to solve the problem at all. Maybe it’s just a disagreement on how large the “prosaic alignment category” is?
Regardless of that, calling the debate “one sided” seems way too strong, especially given how many selection effects are involved. I mean, you could also call the debate about whether alignment is even a problem “one sided” − 95% of all ML researchers don’t think it’s a problem, or think it’s something we’ll solve easily. But for fairly similar meta-level reasons as why it’s good for them to listen to us in an open-minded way, it’s also good for prosaic alignment researchers to listen to EY in an open-minded way.
You definitely have a point, and I want to listen to EY in an open-minded way. It’s just harder when he writes things like everyone working on alignment is faking it and not giving much details. Also I feel that your comparison breaks a bit because compared to the debate with ML researchers (where most people against alignment haven’t even thought about the basics and make obvious mistakes), the other parties in this debate have thought long and hard about alignment. Maybe not as much as EY, but clearly much more than the ML researchers in the whole “is alignment even a problem” debate.
(As a side note, I’d be curious what credence you place on EY’s worldview being more true than the prosaic alignment worldview.)
At the moment I feel like I don’t have a good enough model of EY’s worldview, plus I’m annoyed by his statements, so any credence I give now would be biased against his worldview.
Now, your complaint might be that MIRI has not made their case enough over the last few years. If that’s the main issue, then stay tuned; as Rob said, this is just the preface to a bunch of relevant material.
I really feel there is more disagreement on the second question than on the first
What is this feeling based on? One way we could measure this is by asking people about how much AI xrisk there is conditional on there being no more research explicitly aimed at aligning AGIs. I expect that different people would give very different predictions.
People like Paul and Evan and more are actually going for the core problems IMO, just anchoring a lot of their thinking in current ML technologies.
Everyone agrees that Paul is trying to solve foundational problems. And it seems strange to criticise Eliezer’s position by citing the work of MIRI employees.
It’s just harder when he writes things like everyone working on alignment is faking it and not giving much details.
Thanks for the detailed comment!
That’s an interesting separation of the problem, because I really feel there is more disagreement on the second question than on the first.
Funnily, aren’t the people currently working on ageing using quite prosaic techniques? I completely agree that one need to go for the big problems, especially ones that only appear in more powerful regimes (which is why I am adamant that there should be places for researchers to think about distinctly AGI problems and not have to rephrase everything in a way that is palatable to ML academia). But people like Paul and Evan and more are actually going for the core problems IMO, just anchoring a lot of their thinking in current ML technologies. So I have trouble understanding how prosaic alignment isn’t trying to solve the problem at all. Maybe it’s just a disagreement on how large the “prosaic alignment category” is?
You definitely have a point, and I want to listen to EY in an open-minded way. It’s just harder when he writes things like everyone working on alignment is faking it and not giving much details. Also I feel that your comparison breaks a bit because compared to the debate with ML researchers (where most people against alignment haven’t even thought about the basics and make obvious mistakes), the other parties in this debate have thought long and hard about alignment. Maybe not as much as EY, but clearly much more than the ML researchers in the whole “is alignment even a problem” debate.
At the moment I feel like I don’t have a good enough model of EY’s worldview, plus I’m annoyed by his statements, so any credence I give now would be biased against his worldview.
Yeah, excited about that!
What is this feeling based on? One way we could measure this is by asking people about how much AI xrisk there is conditional on there being no more research explicitly aimed at aligning AGIs. I expect that different people would give very different predictions.
Everyone agrees that Paul is trying to solve foundational problems. And it seems strange to criticise Eliezer’s position by citing the work of MIRI employees.
As Rob pointed out above, this straightforwardly mischaracterises what Eliezer said.