Sure, you can negotiate with an agent with conflicting values, but I don’t think its beside the point.
You can get a sociopath to cooperate with non-sociopaths by making them trade off for things they do care about, or using coercive power. But Clippy doesn’t have any concerns other than paperclips to trade off against its concern for paperclips, and we’re not in a position to coerce Clippy, because Clippy is powerful enough to treat us as an obstacle to be destroyed. The fact that the non-sociopath majority can more or less keep the sociopath minority under control doesn’t mean that we could persuade agents whose values deviate far from our own to accommodate us if we didn’t have coercive power over them.
Prawnoffate’s point to begin with was that humans could and would change their fundamental values on new information about what is moral. I suggested sociopaths as an example of people who wouldn’t change their values to conform to those of other people on the basis of argument or evidence, nor would ordinary humans change their fundamental values to a sociopath’s.
If we’ve progressed to a discussion of whether it’s possible to coerce less powerful agents into behaving in accordance with our values, I think we’ve departed from the context in which sociopaths were relevant in the first place.
That’s beside the point, though. Just because two agents have incompatible values doesn’t mean they can’t be persuaded otherwise.
ETA: in other words, persuading a sociopath to act like they’re ethical or vice versa is possible. It just doesn’t rewire their terminal values.
Sure, you can negotiate with an agent with conflicting values, but I don’t think its beside the point.
You can get a sociopath to cooperate with non-sociopaths by making them trade off for things they do care about, or using coercive power. But Clippy doesn’t have any concerns other than paperclips to trade off against its concern for paperclips, and we’re not in a position to coerce Clippy, because Clippy is powerful enough to treat us as an obstacle to be destroyed. The fact that the non-sociopath majority can more or less keep the sociopath minority under control doesn’t mean that we could persuade agents whose values deviate far from our own to accommodate us if we didn’t have coercive power over them.
Clippy is a superintelligence. Humans, neurotypical or no, are not.
I’m not saying it’s necessarily rational for sociopaths to act moral or vice versa. I’m saying people can be (and have been) persuaded of this.
Prawnoffate’s point to begin with was that humans could and would change their fundamental values on new information about what is moral. I suggested sociopaths as an example of people who wouldn’t change their values to conform to those of other people on the basis of argument or evidence, nor would ordinary humans change their fundamental values to a sociopath’s.
If we’ve progressed to a discussion of whether it’s possible to coerce less powerful agents into behaving in accordance with our values, I think we’ve departed from the context in which sociopaths were relevant in the first place.
Oh, sorry, I wasn’t disagreeing with you about that, just nitpicking your example. Should have made that clearer ;)