Sorry but my rough impression from the post is you seem to be at least as confused about where the difficulties are as average of alignment researchers you think are not on the ball—and the style of somewhat strawmanning everyone & strong words is a bit irritating.
Maybe I’m getting it wrong, but it seems the model you have for why everyone is not on the ball is something like “people are approaching it too much from a theory perspective, and promising approach is very close to how empirical ML capabilities research works” & “this is a type of problem where you can just throw money at it and attract better ML talent”.
I don’t think these two insights are promising.
Also, again, maybe I’m getting it wrong, but I’m confused how similar you are imagining the current systems to be to the dangerous systems. It seems either the superhuman-level problems (eg not lying in a way no human can recognize) are somewhat continuous with current problems (eg not lying), and in that case it is possible to study them empirically. Or they are not. But different parts of the post seem to point in different directions. (Personally I think the problem is somewhat continuous, but many of the human-in-the-loop solutions are not, and just break down.)
Also, with what you find promising I’m confused what do you think the ‘real science’ to aim for is - on one hand it seems you think the closer the thing is to how ML is done in practice the more real science it is. On the other hand, in your view all deep learning progress has been empirical, often via dumb hacks and intuitions (this isn’t true imo).
I’m just really, really skeptical that a bunch of abstract work on decision theory and similar [from MIRI and similar independent researchers] will get us there. My expectation is that alignment is an ML problem, and you can’t solve alignment utterly disconnected from actual ML systems.
I think earlier forms of this research focused on developing new, alignable algorithms, rather than aligning existing deep learning algorithms. However, a reader of the first quote might think “wow, those people actually thought galaxy-brained decision theory stuff was going to work on deep learning systems!”
So for example, I might have a view like: we could either build AI by having systems which perform inference and models that we understand that have like interpretable beliefs about the world and then act on those beliefs, or I could build systems by having opaque black boxes and doing optimization over those black boxes. I might believe that the first kind of AI is easier to align, so one way that I could make the alignment tax smaller is just by advancing that kind of AI, which I expect to be easier to align.
This is not a super uncommon view amongst academics. It also may be familiar here because I would say it describes MIRI’s view; they sort of take the outlook that some kinds of AI just look hard to align. We want to build an understanding such that we can build the kind of AI that is easier to align.
Sorry but my rough impression from the post is you seem to be at least as confused about where the difficulties are as average of alignment researchers you think are not on the ball—and the style of somewhat strawmanning everyone & strong words is a bit irritating.
Maybe I’m getting it wrong, but it seems the model you have for why everyone is not on the ball is something like “people are approaching it too much from a theory perspective, and promising approach is very close to how empirical ML capabilities research works” & “this is a type of problem where you can just throw money at it and attract better ML talent”.
I don’t think these two insights are promising.
Also, again, maybe I’m getting it wrong, but I’m confused how similar you are imagining the current systems to be to the dangerous systems. It seems either the superhuman-level problems (eg not lying in a way no human can recognize) are somewhat continuous with current problems (eg not lying), and in that case it is possible to study them empirically. Or they are not. But different parts of the post seem to point in different directions. (Personally I think the problem is somewhat continuous, but many of the human-in-the-loop solutions are not, and just break down.)
Also, with what you find promising I’m confused what do you think the ‘real science’ to aim for is - on one hand it seems you think the closer the thing is to how ML is done in practice the more real science it is. On the other hand, in your view all deep learning progress has been empirical, often via dumb hacks and intuitions (this isn’t true imo).
I am interested in examples of non-empirical (theoretically based) deep learning progress.
Can you elaborate on why you think this is false? I’m curious.
On a related note, this part might be misleading:
I think earlier forms of this research focused on developing new, alignable algorithms, rather than aligning existing deep learning algorithms. However, a reader of the first quote might think “wow, those people actually thought galaxy-brained decision theory stuff was going to work on deep learning systems!”
For more details, see Paul Christiano’s 2019 talk on “Current work in AI alignment”: