The trouble here is that deep disagreements aren’t often symmetrically held with the same intensity. Consider the following situation:
Say we have Protag and Villain. Villain goes around torturing people and happens upon Protag’s brother. Protag’s brother is subsequently tortured and killed. Protag is unable to forgive Villain but Villain has nothing personal against Protag. Which of the following is the outcome?
Protag says “Villain must not go to Eudaemonia” so neither Protag nor Villain go to Eudaemonia
Protag says “Villain must not go to Eudaemonia” so Protag cannot go to Eudaemonia. Villain says “I don’t care what happens to Protag; he can go if he wants” so Villain gets to go to Eudaemonia
Protag says “Villain must not go to Eudaemonia” but it doesn’t matter because next month they talk to someone else they disagree with and both go to Eudaemonia anyway
The first case is sad but understandable here—but also allows extremist purple-tribe members to veto non-extremist green-tribe members (where purple and green ideologies pertain to something silly like “how to play pool correctly”). The second case is perverse. The third case is just “violate people’s preferences for retribution, but with extra steps.”
I’m confused—please forgive me if this is a dumb comment, this is my first contribution.
What was the purpose of the post if the idea was, on its own, not durable enough to stand? I’m genuinely confused on how this would avoid harming the ‘good’ people in the short term.
Yep. Good thing a real AI would come up with a much better idea! :)
How does this post expand our thoughts of AI if it “would come up with a better idea”? I’m not trying to criticize you (hence why I didn’t downvote this post). I just want to better understand its intention so that I can understand LW better.
What was the purpose of the post if the idea was, on its own, not durable enough to stand?
I think there’s plenty of value in showing problems with unworkable-but-attractive ideas. Fiction is a fine medium for this, as it’s entertaining in it’s own right.
I guess I’ll say a few words in defense of doing something like this… Supposing we’re taking an ethically consequentialist stance. In that case, the only purpose of punishment, basically, is to serve as a deterrent. But in our glorious posthuman future, nanobots will step in before anyone is allowed to get hurt, and crimes will be impossible to commit. So deterrence is no longer necessary and the only reason to punish people is due to spite. But if people are feeling spiteful towards one another on Eudaimonia that would kill the vibe. Being able to forgive one person you disagree with seems like a pretty low bar where being non-spiteful is concerned. (Other moral views might consider punishment to be a moral imperative even if it isn’t achieving anything from a consequentialist point of view. But consequentialism is easily the most popular moral view on LW according to this survey.)
A more realistic scheme might involve multiple continents for people with value systems that are strongly incompatible, perhaps allowing people to engage in duels on a voluntary basis if they’re really sure that is what they want to do.
In any case, the name of the site is “Less Wrong” not “Always Right”, so I feel pretty comfortable posting something which I suspect may be flawed and letting commenters find flaws (and in fact that was part of why I made this post, to see what complaints people would have, beyond the utility of sharing a fun whimsical story. But overall the post was more optimized for whimsy.)
The trouble here is that deep disagreements aren’t often symmetrically held with the same intensity. Consider the following situation:
Say we have Protag and Villain. Villain goes around torturing people and happens upon Protag’s brother. Protag’s brother is subsequently tortured and killed. Protag is unable to forgive Villain but Villain has nothing personal against Protag. Which of the following is the outcome?
Protag says “Villain must not go to Eudaemonia” so neither Protag nor Villain go to Eudaemonia
Protag says “Villain must not go to Eudaemonia” so Protag cannot go to Eudaemonia. Villain says “I don’t care what happens to Protag; he can go if he wants” so Villain gets to go to Eudaemonia
Protag says “Villain must not go to Eudaemonia” but it doesn’t matter because next month they talk to someone else they disagree with and both go to Eudaemonia anyway
The first case is sad but understandable here—but also allows extremist purple-tribe members to veto non-extremist green-tribe members (where purple and green ideologies pertain to something silly like “how to play pool correctly”). The second case is perverse. The third case is just “violate people’s preferences for retribution, but with extra steps.”
Yep. Good thing a real AI would come up with a much better idea! :)
I’m confused—please forgive me if this is a dumb comment, this is my first contribution.
What was the purpose of the post if the idea was, on its own, not durable enough to stand? I’m genuinely confused on how this would avoid harming the ‘good’ people in the short term.
How does this post expand our thoughts of AI if it “would come up with a better idea”? I’m not trying to criticize you (hence why I didn’t downvote this post). I just want to better understand its intention so that I can understand LW better.
Thanks
I think there’s plenty of value in showing problems with unworkable-but-attractive ideas. Fiction is a fine medium for this, as it’s entertaining in it’s own right.
It’s fiction ¯\_(ツ)_/¯
I guess I’ll say a few words in defense of doing something like this… Supposing we’re taking an ethically consequentialist stance. In that case, the only purpose of punishment, basically, is to serve as a deterrent. But in our glorious posthuman future, nanobots will step in before anyone is allowed to get hurt, and crimes will be impossible to commit. So deterrence is no longer necessary and the only reason to punish people is due to spite. But if people are feeling spiteful towards one another on Eudaimonia that would kill the vibe. Being able to forgive one person you disagree with seems like a pretty low bar where being non-spiteful is concerned. (Other moral views might consider punishment to be a moral imperative even if it isn’t achieving anything from a consequentialist point of view. But consequentialism is easily the most popular moral view on LW according to this survey.)
A more realistic scheme might involve multiple continents for people with value systems that are strongly incompatible, perhaps allowing people to engage in duels on a voluntary basis if they’re really sure that is what they want to do.
In any case, the name of the site is “Less Wrong” not “Always Right”, so I feel pretty comfortable posting something which I suspect may be flawed and letting commenters find flaws (and in fact that was part of why I made this post, to see what complaints people would have, beyond the utility of sharing a fun whimsical story. But overall the post was more optimized for whimsy.)