I’ll assume you’re familiar with the trolley problem and the fat man variant.
There’s a subtlety to it: if you pushed the fat man and diverted the trolley, you can explain what you did when the police comes, and might get away with something like negligent manslaughter—depending on your jurisdiction, after a trial you might even go unpunished.
Do you push the fat man if you know for a fact you’ll be tried and convicted of murder and nobody will thank you for saving the lives of those who were on the track? I think that’s a much higher bar.
But that’s not the ultimate trolley problem. The ultimate trolley problem is this.
The trolley is AGI. It is on track to crash into the world and kill all humans. Strong regulation and powerful public pressure of sufficient intensity to possibly avert this will arrive in time (=the trolley gets on the other track) iff there’s a very public accident with a runaway AI that kills lots of people and is barely contained (=pushing the fat man). And you’ll be investigated thoroughly -nothing in your digital footprints can indicate your true intent (such as your association with places like Less Wrong) because in that case people who shout about AI safety will be associated with terrorism and their recommendations will be ignored extra hard.
What are the ethics of that? Are you THAT consequentialist?
AI safety: the ultimate trolley problem
I’ll assume you’re familiar with the trolley problem and the fat man variant.
There’s a subtlety to it: if you pushed the fat man and diverted the trolley, you can explain what you did when the police comes, and might get away with something like negligent manslaughter—depending on your jurisdiction, after a trial you might even go unpunished.
Do you push the fat man if you know for a fact you’ll be tried and convicted of murder and nobody will thank you for saving the lives of those who were on the track? I think that’s a much higher bar.
But that’s not the ultimate trolley problem. The ultimate trolley problem is this.
The trolley is AGI. It is on track to crash into the world and kill all humans. Strong regulation and powerful public pressure of sufficient intensity to possibly avert this will arrive in time (=the trolley gets on the other track) iff there’s a very public accident with a runaway AI that kills lots of people and is barely contained (=pushing the fat man). And you’ll be investigated thoroughly -nothing in your digital footprints can indicate your true intent (such as your association with places like Less Wrong) because in that case people who shout about AI safety will be associated with terrorism and their recommendations will be ignored extra hard.
What are the ethics of that? Are you THAT consequentialist?