I’m a bit disappointed by this article. From the title, I fought it would be something like “A list of strategies AI might use to kill all humanity”, not “A list of reasons AIs are incredibly dangerous, and people who disagree are wrong”. Arguably, it’s not very good at being the second.
But “ways AI could be lethal on an extinction level” is a pretty interesting subject, and (from what I’ve read on LW) somewhat under-explored. Like… what’s our threat model?
For instance, the basic Terminator scenario of “the AI triggers a nuclear war” seems unlikely to me. A nuclear war would produce a lot of EMPs, shut down a lot of power plants and blow up a lot of data centers. Even if the AI is backed up in individual laptops or in Starlink satellites, it would lose any way of interacting with the outside world. Boston dynamics robots would shut down because there are no more miners producing coal for the coal plant that produced the electricity these robots need to run. (and, you know, all the other million parts of the supply chain being lost).
In fact, even if an unfriendly AI escaped its sandbox, it might not want to kill us immediately. It would want to wait until we’ve developed some technologies in the right directions: more automation in data-centers and power plants, higher numbers of drones and versatile androids, better nanotechnology, etc.
That’s not meant to be reassuring. The AI would still kill us eventually, and it wouldn’t sit tight in the meantime. It would influence political and economic processes to make sure no other AI can concurrence it. This could take many forms, from the covert (eg manipulating elections and flooding social networks with targeted disinformation) to the overt (eg assassinating AI researchers or bombing OpenAI datacenters). The point is that its interventions would look “soft” at first compared to the “flood the planet with nanbots and kill everyone at the same time” scenario, because it would be putting its pieces in place for that scenario to happen.
Again, that doesn’t mean the AI would lose. If you’re Afghanistan and you’re fighting against the US, you’re not going to win just because the US is unwilling to immediately jump to nukes. In fact, if the US is determined to win at all costs and will prefer using nukes over losing, you’re still fucked. But the war will look like you have a fighting chance during the initial phases, because the enemy will be going easy on you in preparation for the final phase.
All that is just uninformed speculating, of course. Again, my main point is that I haven’t really seen discussions of these scenarios and what the probable limits of an unfriendly AI would be. The question probably deserves to be explored more.
I’m a bit disappointed by this article. From the title, I fought it would be something like “A list of strategies AI might use to kill all humanity”, not “A list of reasons AIs are incredibly dangerous, and people who disagree are wrong”. Arguably, it’s not very good at being the second.
But “ways AI could be lethal on an extinction level” is a pretty interesting subject, and (from what I’ve read on LW) somewhat under-explored. Like… what’s our threat model?
For instance, the basic Terminator scenario of “the AI triggers a nuclear war” seems unlikely to me. A nuclear war would produce a lot of EMPs, shut down a lot of power plants and blow up a lot of data centers. Even if the AI is backed up in individual laptops or in Starlink satellites, it would lose any way of interacting with the outside world. Boston dynamics robots would shut down because there are no more miners producing coal for the coal plant that produced the electricity these robots need to run. (and, you know, all the other million parts of the supply chain being lost).
In fact, even if an unfriendly AI escaped its sandbox, it might not want to kill us immediately. It would want to wait until we’ve developed some technologies in the right directions: more automation in data-centers and power plants, higher numbers of drones and versatile androids, better nanotechnology, etc.
That’s not meant to be reassuring. The AI would still kill us eventually, and it wouldn’t sit tight in the meantime. It would influence political and economic processes to make sure no other AI can concurrence it. This could take many forms, from the covert (eg manipulating elections and flooding social networks with targeted disinformation) to the overt (eg assassinating AI researchers or bombing OpenAI datacenters). The point is that its interventions would look “soft” at first compared to the “flood the planet with nanbots and kill everyone at the same time” scenario, because it would be putting its pieces in place for that scenario to happen.
Again, that doesn’t mean the AI would lose. If you’re Afghanistan and you’re fighting against the US, you’re not going to win just because the US is unwilling to immediately jump to nukes. In fact, if the US is determined to win at all costs and will prefer using nukes over losing, you’re still fucked. But the war will look like you have a fighting chance during the initial phases, because the enemy will be going easy on you in preparation for the final phase.
All that is just uninformed speculating, of course. Again, my main point is that I haven’t really seen discussions of these scenarios and what the probable limits of an unfriendly AI would be. The question probably deserves to be explored more.