Humanity doesn’t build AGI in the first place, via governance, monitoring, etc.
Humanity doesn’t build AGI in the first place, via somebody doing a “pivotal act” that does not require AGI (e.g. release nanobots that spread across the earth melting all computer chips)
Somebody figures out how to build safe AGI
(Your #4 is my #1, your #1 is my #1 or #3, your #2 is my #3, your #3 is one aspect of my #3 (i.e., it’s getting at an approach to ease outer alignment, but we would still also need to solve inner alignment).)
Yeah, I suspect that EY thinks #1 is even less likely than #3, and that he doesn’t think about #1 at all except to poo-poo people who are even trying. My impression is that EY has done work brainstorming on #2 in the past, but that he has given up.
For my part, I’m more open-minded and happy for people to be trying different things in parallel—see for example what I wrote under “objection #6” here (expressing an open mind that someone might solve #1).
This list is a lot better in terms of comprehensiveness since most things are probably in one of the three buckets, but one of my main points was that there are a lot of approaches with a chance of working that have little overlap, and lumping them together glosses over that. There are probably several ideas that aren’t on my list and also in your #1.
Or differently put, the opinion that #1 is hopeless only makes sense if you have an argument that applies to all things in #1, but I question whether you can justifiably have such an argument. (I know you don’t have it, I’m just saying that the category you’ve created may be doing some unjustified work here.)
Edit: and also (this is something I almost included in the post but then didn’t), I’m also skeptical about lumping #3 together since e.g. agent foundations and debate don’t seem to have a lot of overlap, but they’re both ideas that aim to solve the entire problem. And EY seems more specialized in agent foundations specifically than all of alignment.
I would break down the possibility space as:
Humanity doesn’t build AGI in the first place, via governance, monitoring, etc.
Humanity doesn’t build AGI in the first place, via somebody doing a “pivotal act” that does not require AGI (e.g. release nanobots that spread across the earth melting all computer chips)
Somebody figures out how to build safe AGI
(Your #4 is my #1, your #1 is my #1 or #3, your #2 is my #3, your #3 is one aspect of my #3 (i.e., it’s getting at an approach to ease outer alignment, but we would still also need to solve inner alignment).)
Yeah, I suspect that EY thinks #1 is even less likely than #3, and that he doesn’t think about #1 at all except to poo-poo people who are even trying. My impression is that EY has done work brainstorming on #2 in the past, but that he has given up.
For my part, I’m more open-minded and happy for people to be trying different things in parallel—see for example what I wrote under “objection #6” here (expressing an open mind that someone might solve #1).
This list is a lot better in terms of comprehensiveness since most things are probably in one of the three buckets, but one of my main points was that there are a lot of approaches with a chance of working that have little overlap, and lumping them together glosses over that. There are probably several ideas that aren’t on my list and also in your #1.
Or differently put, the opinion that #1 is hopeless only makes sense if you have an argument that applies to all things in #1, but I question whether you can justifiably have such an argument. (I know you don’t have it, I’m just saying that the category you’ve created may be doing some unjustified work here.)
Edit: and also (this is something I almost included in the post but then didn’t), I’m also skeptical about lumping #3 together since e.g. agent foundations and debate don’t seem to have a lot of overlap, but they’re both ideas that aim to solve the entire problem. And EY seems more specialized in agent foundations specifically than all of alignment.
(Good correction on the inner alignment thing.)