I’ve copied over and lightly edited some comments I left on a draft. Note I haven’t reread the post in detail; sorry if these were addressed somewhere.
Writing down quick thoughts after reading the intro and before reading the rest:
I have two major reasons to be skeptical of actively slowing down AI (setting aside feasibility):
1. It makes it easier for a future misaligned AI to take over by increasing overhangs, both via compute progress and algorithmic efficiency progress. (This is basically the same sort of argument as “Every 18 months, the minimum IQ necessary to destroy the world drops by one point.”)
2. Such strategies are likely to disproportionately penalize safety-conscious actors.
(As a concrete example of (2), if you build public support, maybe the public calls for compute restrictions on AGI companies and this ends up binding the companies with AGI safety teams but not the various AI companies that are skeptical of “AGI” and “AI x-risk” and say they are just building powerful AI tools without calling it AGI.)
For me personally there’s a third reason, which is that (to first approximation) I have a limited amount of resources and it seems better to spend that on the “use good alignment techniques” plan rather than the “try to not build AGI” plan. But that’s specific to me.
After reading the doc I think I agree with most of the specific object-level points, and disagree with the overall thrust.
There seems to be some equivocation between two different viewpoints:
1. We can make a big difference by just not building AGI for a long time (say decades). Maybe this is hard / morally wrong, maybe not, we haven’t actually tried, and we aren’t even considering it. (To this view my objection would be the two points above; I’m confused why neither one appeared in the doc.)
2. There are simple straightforward things that seem clearly good, so it’s wrong for people to say it’s too hard / morally dubious. (To this view my objection is that people don’t say that, people aren’t opposed to it, and people are doing the simple straightforward things. Though perhaps we know different people.)
I feel like as I read this doc I was being pushed towards ambiguating between these two views, so that I come out with a belief like “Wow, there’s all this straightforward obvious stuff, but there’s a huge taboo against it and no one even considers it”. Nowhere do you actually say that but it’s very much the vibe I get.
All that being said, I strongly agree that there are simple straightforward things that seem clearly good and we should do them! I just think that people do consider them already. For example, on your list of 11 actions in the “Restraint is not terrorism, usually”, I think I have done or am doing (1), (2), (4), (5), (6), (7), (9), and (11).
(Written now, not previously) Overall my take is that most of the specific object-level arguments make sense and are valuable as a response to some ill-considered memes that get thrown around in this community, but it isn’t a great take on the overall question of whether and how to slow down AI because it doesn’t consider the best counterarguments. (Which I’ll note that I wrote down before reading the full essay.)
FYI I think your first skepticism was mentioned in the safety from speed section; she concludes that section:
These [objections] all seem plausible. But also plausibly wrong. I don’t know of a decisive analysis of any of these considerations, and am not going to do one here. My impression is that they could basically all go either way.
She mentions your second skepticism near the top, but I don’t see anywhere she directly addresses it.
I’ve copied over and lightly edited some comments I left on a draft. Note I haven’t reread the post in detail; sorry if these were addressed somewhere.
Writing down quick thoughts after reading the intro and before reading the rest:
I have two major reasons to be skeptical of actively slowing down AI (setting aside feasibility):
1. It makes it easier for a future misaligned AI to take over by increasing overhangs, both via compute progress and algorithmic efficiency progress. (This is basically the same sort of argument as “Every 18 months, the minimum IQ necessary to destroy the world drops by one point.”)
2. Such strategies are likely to disproportionately penalize safety-conscious actors.
(As a concrete example of (2), if you build public support, maybe the public calls for compute restrictions on AGI companies and this ends up binding the companies with AGI safety teams but not the various AI companies that are skeptical of “AGI” and “AI x-risk” and say they are just building powerful AI tools without calling it AGI.)
For me personally there’s a third reason, which is that (to first approximation) I have a limited amount of resources and it seems better to spend that on the “use good alignment techniques” plan rather than the “try to not build AGI” plan. But that’s specific to me.
After reading the doc I think I agree with most of the specific object-level points, and disagree with the overall thrust.
There seems to be some equivocation between two different viewpoints:
1. We can make a big difference by just not building AGI for a long time (say decades). Maybe this is hard / morally wrong, maybe not, we haven’t actually tried, and we aren’t even considering it. (To this view my objection would be the two points above; I’m confused why neither one appeared in the doc.)
2. There are simple straightforward things that seem clearly good, so it’s wrong for people to say it’s too hard / morally dubious. (To this view my objection is that people don’t say that, people aren’t opposed to it, and people are doing the simple straightforward things. Though perhaps we know different people.)
I feel like as I read this doc I was being pushed towards ambiguating between these two views, so that I come out with a belief like “Wow, there’s all this straightforward obvious stuff, but there’s a huge taboo against it and no one even considers it”. Nowhere do you actually say that but it’s very much the vibe I get.
All that being said, I strongly agree that there are simple straightforward things that seem clearly good and we should do them! I just think that people do consider them already. For example, on your list of 11 actions in the “Restraint is not terrorism, usually”, I think I have done or am doing (1), (2), (4), (5), (6), (7), (9), and (11).
(Written now, not previously) Overall my take is that most of the specific object-level arguments make sense and are valuable as a response to some ill-considered memes that get thrown around in this community, but it isn’t a great take on the overall question of whether and how to slow down AI because it doesn’t consider the best counterarguments. (Which I’ll note that I wrote down before reading the full essay.)
FYI I think your first skepticism was mentioned in the safety from speed section; she concludes that section:
She mentions your second skepticism near the top, but I don’t see anywhere she directly addresses it.
Thanks, that’s good to know.