It definitely is the case that a pivotal act that isn’t “disruptive” isn’t a pivotal act. But I think not all disruptive acts have a significant cost in human lives.
To continue with the ‘burn all GPUs’ example, note that while some industries are heavily dependent on GPUs, most industries are instead heavily dependent on CPUs. The hospital’s power will still be on if all GPUs melt, and probably their monitors will still work (if the nanobots can somehow distinguish between standalone GPUs and ones embedded into motherboards). Transportation networks will probably still function, and so on. Cryptocurrencies, entertainment industries, and lots of AI applications will be significantly impacted, but this seems recoverable.
But I do think Eliezer’s main claim is: some people will lash out in desperation when cornered (“Well, maybe starting WWIII will help with AI risk!”), and Eliezer is not one of those people. So if he makes a call of the form “disruption that causes 10M deaths”, it’s because the other option looked actually worse, and so this is ‘safer’. [If you’re one of the people tied up on the trolley tracks, you want the person at the lever to switch it!]
AI can run on CPUs (with a certain inefficiency factor), so only burning all GPUs doesn’t seem like it would be sufficient. As for disruptive acts that are less deadly, it would be nice to have some examples but Eliezer says they’re too far out of the Overton Window to mention.
If what you’re saying about Eliezer’s claim is accurate, it does seem disingenuous to frame “The only worlds where humanity survives are ones where people like me do something extreme and unethical” as “I won’t do anything extreme and unethical [because humanity is doomed anyway]”. It makes Eliezer dangerous to be around if he’s mistaken, and if you’re significantly less pessimistic than he is (if you assign >10^-6 probability to humanity surviving), he’s mistaken in most of the worlds where humanity survives. Which are the worlds that matter the most.
And yeah, it’s nice that Eliezer claims that Eliezer can violate ethical injunctions because he’s smart enough, after repeatedly stating that people who violate ethical injunctions because they think they’re smart enough are almost always wrong. I don’t doubt he’ll pick the option that looks actually better to him. It’s just that he’s only human—he’s running on corrupted hardware like the rest of us.
It definitely is the case that a pivotal act that isn’t “disruptive” isn’t a pivotal act. But I think not all disruptive acts have a significant cost in human lives.
To continue with the ‘burn all GPUs’ example, note that while some industries are heavily dependent on GPUs, most industries are instead heavily dependent on CPUs. The hospital’s power will still be on if all GPUs melt, and probably their monitors will still work (if the nanobots can somehow distinguish between standalone GPUs and ones embedded into motherboards). Transportation networks will probably still function, and so on. Cryptocurrencies, entertainment industries, and lots of AI applications will be significantly impacted, but this seems recoverable.
But I do think Eliezer’s main claim is: some people will lash out in desperation when cornered (“Well, maybe starting WWIII will help with AI risk!”), and Eliezer is not one of those people. So if he makes a call of the form “disruption that causes 10M deaths”, it’s because the other option looked actually worse, and so this is ‘safer’. [If you’re one of the people tied up on the trolley tracks, you want the person at the lever to switch it!]
AI can run on CPUs (with a certain inefficiency factor), so only burning all GPUs doesn’t seem like it would be sufficient. As for disruptive acts that are less deadly, it would be nice to have some examples but Eliezer says they’re too far out of the Overton Window to mention.
If what you’re saying about Eliezer’s claim is accurate, it does seem disingenuous to frame “The only worlds where humanity survives are ones where people like me do something extreme and unethical” as “I won’t do anything extreme and unethical [because humanity is doomed anyway]”. It makes Eliezer dangerous to be around if he’s mistaken, and if you’re significantly less pessimistic than he is (if you assign >10^-6 probability to humanity surviving), he’s mistaken in most of the worlds where humanity survives. Which are the worlds that matter the most.
And yeah, it’s nice that Eliezer claims that Eliezer can violate ethical injunctions because he’s smart enough, after repeatedly stating that people who violate ethical injunctions because they think they’re smart enough are almost always wrong. I don’t doubt he’ll pick the option that looks actually better to him. It’s just that he’s only human—he’s running on corrupted hardware like the rest of us.