Because ‘CEV’ must be instantiated on a group of agents (usually humans). Some humans are assholes. So for some value of aGroup, CEV does assholish things. Hopefully the group of all humans doesn’t create a CEV that makes FAI> an outright uFAI from our perspective but we certainly shouldn’t count on it.
Some humans are assholes. So for some value of aGroup, CEV does assholish things.
That’s not necessarily true. CEV isn’t precisely defined but it’s intended to represent the idealized version of our desires and meta-desires. So even if we take a group of assholes, they don’t necessarily want to be assholes, or want to want to be assholes, or maybe they wouldn’t want to if they knew more and were smarter.
I refer, of course, to people whose preferences really are different to our own. Coherent Extrapolated Assholes. I don’t refer to people who would really have preferences that I would consider acceptable if they just knew a bit more.
You asked for an explanation of how a correctly implemented ‘CEV’ could want something abhorrent. That’s how.
There is an unfortunate tendency to glorify the extrapolation process and pretend that it makes any given individual or group have acceptable values. It need not.
Upvoted for the phrase “Coherent Extrapolated Assholes”. Best. Insult. Ever.
Seriously, though, I don’t think there are many CEAs around, anyway. (This doesn’t mean there are none, either. (I was going to link to this as an example of one, but I’m not sure Hitler would have done what he did had he known about late-20th-century results about heterosis, Ashkenazi Jew intelligence, etc.)) This mean that I think it’s very, very unlikely for CEV to be evil (and even less likely to be evil>), unless the membership criteria to aGroup are gerrymandered to make it so.
There is an unfortunate tendency to glorify the extrapolation process and pretend that it makes any given individual or group have acceptable values. It need not.
It seemed odd to me that so few people were bothered by the claims that CEV shouldn’t care much about the inputs. If you expect it to give similar results if you put in a chimpanzee and a murderer and Archimedes, then why put in anything at all instead of just printing out the only results it gives?
If you believe in moral progress (and CEV seems to rely on that position), then there’s every reason to think that future-society would want to make changes to how we live, if future-society had the capacity to make that type of intervention.
In short, wouldn’t you change the past to prevent the occurrence of chattel slavery if you could? (If you don’t like that example, substitute preventing the October revolution or whatever example fits your preferences).
Punishment from the future is spooky enough. Imagine what an anti-Guns of the South would be like for the temporal locals. Not pleasant, that’s for sure.
Doesn’t CEV implicitly assert that there exists a set of moral assertions M that is more reliably moral than anything humans assert today, and that it’s possible for a sufficiently intelligent system to derive M?
That sure sounds like a belief in moral progress to me.
Granted, it doesn’t imply that humans left to their own devices will achieve moral progress. But the same is true of technological progress.
Doesn’t CEV implicitly assert that there exists a set of moral assertions M that is more reliably moral than anything humans assert today, and that it’s possible for a sufficiently intelligent system to derive M?
The implicit assertion is “Greater or Equal”, not “Greater”.
Run on a True Conservative it will return the morals that the conservative currently has.
Mm. I’ll certainly agree that anyone for whom that’s true deserves the title “True Conservative.”
I don’t think I’ve ever met anyone who meets that description, though I’ve met people who would probably describe themselves that way.
Presumably, someone who believes this is true of themselves would consider the whole notion of extrapolating the target definition for a superhumanly powerful optimization process to be silly, though, and consider the label CEV to be technically accurate, in the same sense that I’m currently extrapolating the presence of my laptop, but to imply falsehoods.
Really, please explain (or PM me if it would require breaking the gag rule on Roko’s scenario). Why would CEV want that?
Because ‘CEV’ must be instantiated on a group of agents (usually humans). Some humans are assholes. So for some value of aGroup, CEV does assholish things. Hopefully the group of all humans doesn’t create a CEV that makes FAI> an outright uFAI from our perspective but we certainly shouldn’t count on it.
That’s not necessarily true. CEV isn’t precisely defined but it’s intended to represent the idealized version of our desires and meta-desires. So even if we take a group of assholes, they don’t necessarily want to be assholes, or want to want to be assholes, or maybe they wouldn’t want to if they knew more and were smarter.
I refer, of course, to people whose preferences really are different to our own. Coherent Extrapolated Assholes. I don’t refer to people who would really have preferences that I would consider acceptable if they just knew a bit more.
You asked for an explanation of how a correctly implemented ‘CEV’ could want something abhorrent. That’s how.
There is an unfortunate tendency to glorify the extrapolation process and pretend that it makes any given individual or group have acceptable values. It need not.
Upvoted for the phrase “Coherent Extrapolated Assholes”. Best. Insult. Ever.
Seriously, though, I don’t think there are many CEAs around, anyway. (This doesn’t mean there are none, either. (I was going to link to this as an example of one, but I’m not sure Hitler would have done what he did had he known about late-20th-century results about heterosis, Ashkenazi Jew intelligence, etc.)) This mean that I think it’s very, very unlikely for CEV to be evil (and even less likely to be evil>), unless the membership criteria to aGroup are gerrymandered to make it so.
It seemed odd to me that so few people were bothered by the claims that CEV shouldn’t care much about the inputs. If you expect it to give similar results if you put in a chimpanzee and a murderer and Archimedes, then why put in anything at all instead of just printing out the only results it gives?
If you believe in moral progress (and CEV seems to rely on that position), then there’s every reason to think that future-society would want to make changes to how we live, if future-society had the capacity to make that type of intervention.
In short, wouldn’t you change the past to prevent the occurrence of chattel slavery if you could? (If you don’t like that example, substitute preventing the October revolution or whatever example fits your preferences).
It’s more agnostic on the issue. It works just as well for the ultimate conservative.
I wouldn’t torture innocent people to prevent it, no.
Punishment from the future is spooky enough. Imagine what an anti-Guns of the South would be like for the temporal locals. Not pleasant, that’s for sure.
It’s more agnostic on the issue. It works just as well for the ultimate conservative.
Doesn’t CEV implicitly assert that there exists a set of moral assertions M that is more reliably moral than anything humans assert today, and that it’s possible for a sufficiently intelligent system to derive M?
That sure sounds like a belief in moral progress to me.
Granted, it doesn’t imply that humans left to their own devices will achieve moral progress. But the same is true of technological progress.
The implicit assertion is “Greater or Equal”, not “Greater”.
Run on a True Conservative it will return the morals that the conservative currently has.
Mm.
I’ll certainly agree that anyone for whom that’s true deserves the title “True Conservative.”
I don’t think I’ve ever met anyone who meets that description, though I’ve met people who would probably describe themselves that way.
Presumably, someone who believes this is true of themselves would consider the whole notion of extrapolating the target definition for a superhumanly powerful optimization process to be silly, though, and consider the label CEV to be technically accurate, in the same sense that I’m currently extrapolating the presence of my laptop, but to imply falsehoods.