Assume B choose without coercion, but assume A always knows what B will choose and can set up various facts in the world to determine B’s choice. Is B free?
So, just checking before I answer: you’re claiming that no direct, gun-to-the-head coercion is employed, but Omega can always predict your actions and responses, and sets things up to ensure you will choose a specific thing.
Are you free, or are you in some sense “serving” Omega? I answer: The latter, very, very, very definitely.
If we take it out of abstract language, real people manipulate each-other all the time, and we always condemn it as a violation of the ethical principle of free choice. Yes, sometimes there are principles higher than free choice, as with a parent who might say, “Do your homework or you get no dessert” (treat that sentence as a metasyntactic variable for whatever you think is appropriate parenting), but we still prefer, all else equal, that our choices and those of others be less manipulated rather than more.
Just because fraud and direct coercion are the usual standards for proving a violation of free choice in a court of law, for instance in order to invalidate a legal contract, does not mean that these are the all-and-all of the underlying ethics of free choice.
Are you free, or are you in some sense “serving” Omega? I answer: The latter, very, very, very definitely.
Then if Omega is superintelligent, it has a problem: every single decision it makes increases or decreases the probability of someone answering something or other, possibly by a large amount. It seems Omega cannot avoid being coercive, just because it’s so knowledgeable.
We don’t quite know that, and there’s also the matter of whether Omega is deliberately optimizing those people or they’re just reacting to Omega’s optimizing the inanimate world (which I would judge to be acceptable and, yes, unavoidable).
So, just checking before I answer: you’re claiming that no direct, gun-to-the-head coercion is employed, but Omega can always predict your actions and responses, and sets things up to ensure you will choose a specific thing.
Are you free, or are you in some sense “serving” Omega? I answer: The latter, very, very, very definitely.
If we take it out of abstract language, real people manipulate each-other all the time, and we always condemn it as a violation of the ethical principle of free choice. Yes, sometimes there are principles higher than free choice, as with a parent who might say, “Do your homework or you get no dessert” (treat that sentence as a metasyntactic variable for whatever you think is appropriate parenting), but we still prefer, all else equal, that our choices and those of others be less manipulated rather than more.
Just because fraud and direct coercion are the usual standards for proving a violation of free choice in a court of law, for instance in order to invalidate a legal contract, does not mean that these are the all-and-all of the underlying ethics of free choice.
Then if Omega is superintelligent, it has a problem: every single decision it makes increases or decreases the probability of someone answering something or other, possibly by a large amount. It seems Omega cannot avoid being coercive, just because it’s so knowledgeable.
We don’t quite know that, and there’s also the matter of whether Omega is deliberately optimizing those people or they’re just reacting to Omega’s optimizing the inanimate world (which I would judge to be acceptable and, yes, unavoidable).
It is an interesting question, though, and illustrates the challenges with “liberty” as a concept in these circumstances.
Well yes. It’s also why many people have argued in favor of Really Powerful Optimizers just… doing absolutely nothing.
That I don’t see.