Because there’s no consensus, your version of CEV would not interfere, and the 80% would be free to kill the 20%.
There may be a distinction between “the AI will not prevent the 80% from killing the 20%” and “nothing will prevent the 80% from killing the 20%” that is getting lost in your phrasing. I am not convinced that the math doesn’t make them equivalent, in the long run—but I’m definitely not convinced otherwise.
I’m assuming the 80% are capable of killing the 20% unless the AI interferes. That’s part of the thought experiment. It’s not unreasonable, since they are 4 times as numerous. But if you find this problematic, suppose it’s 99% killing 1% at a time. It doesn’t really matter.
My point is that we currently have methods of preventing this that don’t require an AI, and which do pretty well. Why do we need the AI to do it? Or more specifically, why should we reject an AI that won’t, but may do other useful things?
There have been, and are, many mass killings of minority groups and of enemy populations and conscripted soldiers at war. If we cure death and diseases, this will become the biggest cause of death and suffering in the world. It’s important and we’ll have to deal with it eventually.
The AI under discussion not just won’t solve the problem, it would (I contend) become a singleton and prevent me from building another AI that does solve the problem. (If it chooses not to become a singleton, it will quickly be supplanted by an AI that does try to become one.)
There may be a distinction between “the AI will not prevent the 80% from killing the 20%” and “nothing will prevent the 80% from killing the 20%” that is getting lost in your phrasing. I am not convinced that the math doesn’t make them equivalent, in the long run—but I’m definitely not convinced otherwise.
I’m assuming the 80% are capable of killing the 20% unless the AI interferes. That’s part of the thought experiment. It’s not unreasonable, since they are 4 times as numerous. But if you find this problematic, suppose it’s 99% killing 1% at a time. It doesn’t really matter.
My point is that we currently have methods of preventing this that don’t require an AI, and which do pretty well. Why do we need the AI to do it? Or more specifically, why should we reject an AI that won’t, but may do other useful things?
There have been, and are, many mass killings of minority groups and of enemy populations and conscripted soldiers at war. If we cure death and diseases, this will become the biggest cause of death and suffering in the world. It’s important and we’ll have to deal with it eventually.
The AI under discussion not just won’t solve the problem, it would (I contend) become a singleton and prevent me from building another AI that does solve the problem. (If it chooses not to become a singleton, it will quickly be supplanted by an AI that does try to become one.)