Are you free, or are you in some sense “serving” Omega? I answer: The latter, very, very, very definitely.
Then if Omega is superintelligent, it has a problem: every single decision it makes increases or decreases the probability of someone answering something or other, possibly by a large amount. It seems Omega cannot avoid being coercive, just because it’s so knowledgeable.
We don’t quite know that, and there’s also the matter of whether Omega is deliberately optimizing those people or they’re just reacting to Omega’s optimizing the inanimate world (which I would judge to be acceptable and, yes, unavoidable).
Then if Omega is superintelligent, it has a problem: every single decision it makes increases or decreases the probability of someone answering something or other, possibly by a large amount. It seems Omega cannot avoid being coercive, just because it’s so knowledgeable.
We don’t quite know that, and there’s also the matter of whether Omega is deliberately optimizing those people or they’re just reacting to Omega’s optimizing the inanimate world (which I would judge to be acceptable and, yes, unavoidable).
It is an interesting question, though, and illustrates the challenges with “liberty” as a concept in these circumstances.
Well yes. It’s also why many people have argued in favor of Really Powerful Optimizers just… doing absolutely nothing.
That I don’t see.