Because ‘CEV’ must be instantiated on a group of agents (usually humans). Some humans are assholes. So for some value of aGroup, CEV does assholish things. Hopefully the group of all humans doesn’t create a CEV that makes FAI> an outright uFAI from our perspective but we certainly shouldn’t count on it.
Some humans are assholes. So for some value of aGroup, CEV does assholish things.
That’s not necessarily true. CEV isn’t precisely defined but it’s intended to represent the idealized version of our desires and meta-desires. So even if we take a group of assholes, they don’t necessarily want to be assholes, or want to want to be assholes, or maybe they wouldn’t want to if they knew more and were smarter.
I refer, of course, to people whose preferences really are different to our own. Coherent Extrapolated Assholes. I don’t refer to people who would really have preferences that I would consider acceptable if they just knew a bit more.
You asked for an explanation of how a correctly implemented ‘CEV’ could want something abhorrent. That’s how.
There is an unfortunate tendency to glorify the extrapolation process and pretend that it makes any given individual or group have acceptable values. It need not.
Upvoted for the phrase “Coherent Extrapolated Assholes”. Best. Insult. Ever.
Seriously, though, I don’t think there are many CEAs around, anyway. (This doesn’t mean there are none, either. (I was going to link to this as an example of one, but I’m not sure Hitler would have done what he did had he known about late-20th-century results about heterosis, Ashkenazi Jew intelligence, etc.)) This mean that I think it’s very, very unlikely for CEV to be evil (and even less likely to be evil>), unless the membership criteria to aGroup are gerrymandered to make it so.
There is an unfortunate tendency to glorify the extrapolation process and pretend that it makes any given individual or group have acceptable values. It need not.
It seemed odd to me that so few people were bothered by the claims that CEV shouldn’t care much about the inputs. If you expect it to give similar results if you put in a chimpanzee and a murderer and Archimedes, then why put in anything at all instead of just printing out the only results it gives?
Because ‘CEV’ must be instantiated on a group of agents (usually humans). Some humans are assholes. So for some value of aGroup, CEV does assholish things. Hopefully the group of all humans doesn’t create a CEV that makes FAI> an outright uFAI from our perspective but we certainly shouldn’t count on it.
That’s not necessarily true. CEV isn’t precisely defined but it’s intended to represent the idealized version of our desires and meta-desires. So even if we take a group of assholes, they don’t necessarily want to be assholes, or want to want to be assholes, or maybe they wouldn’t want to if they knew more and were smarter.
I refer, of course, to people whose preferences really are different to our own. Coherent Extrapolated Assholes. I don’t refer to people who would really have preferences that I would consider acceptable if they just knew a bit more.
You asked for an explanation of how a correctly implemented ‘CEV’ could want something abhorrent. That’s how.
There is an unfortunate tendency to glorify the extrapolation process and pretend that it makes any given individual or group have acceptable values. It need not.
Upvoted for the phrase “Coherent Extrapolated Assholes”. Best. Insult. Ever.
Seriously, though, I don’t think there are many CEAs around, anyway. (This doesn’t mean there are none, either. (I was going to link to this as an example of one, but I’m not sure Hitler would have done what he did had he known about late-20th-century results about heterosis, Ashkenazi Jew intelligence, etc.)) This mean that I think it’s very, very unlikely for CEV to be evil (and even less likely to be evil>), unless the membership criteria to aGroup are gerrymandered to make it so.
It seemed odd to me that so few people were bothered by the claims that CEV shouldn’t care much about the inputs. If you expect it to give similar results if you put in a chimpanzee and a murderer and Archimedes, then why put in anything at all instead of just printing out the only results it gives?