If you argue that it would be unethical to make creatures who want to serve your will, would it not be worse to create a creature that does not want to serve your will and use capability control methods to force it to carry out your will anyways?
Intelligent minds always come with built-in drives; there’s nothing that in general makes goals chosen by another intelligence worse than those arrived through any other process (e.g. natural selection in the case of humans).
One of the closest corresponding human institutions—slavery—has a very bad reputation, and for good reason: Humans are typically not set up to do this sort of thing, so it tends to make them miserable. Even if you could get around that, there’s massive moral issues with subjugating an existing intelligent entity that would prefer not to be. Neither of those inherently apply to newly designed entities. Misery is still something that’s very much worth avoiding, but that issue is largely orthogonal to how the entity’s goals are determined.
Might it be unethical to make creatures who want to serve your will?
If you argue that it would be unethical to make creatures who want to serve your will, would it not be worse to create a creature that does not want to serve your will and use capability control methods to force it to carry out your will anyways?
Intelligent minds always come with built-in drives; there’s nothing that in general makes goals chosen by another intelligence worse than those arrived through any other process (e.g. natural selection in the case of humans).
One of the closest corresponding human institutions—slavery—has a very bad reputation, and for good reason: Humans are typically not set up to do this sort of thing, so it tends to make them miserable. Even if you could get around that, there’s massive moral issues with subjugating an existing intelligent entity that would prefer not to be. Neither of those inherently apply to newly designed entities. Misery is still something that’s very much worth avoiding, but that issue is largely orthogonal to how the entity’s goals are determined.
For a counterargument to your first claim, see the Wisdom of Nature paper by Bostrom and Sandberg (2009 I think).
Possible analogy: Was molding the evolutionary path of wolves, so they turned into dogs that serve us, unethical? Should we stop?
I think if it’s fair to eat animals, it’s far to make them into creatures that serve us.
“You become responsible forever for what you’ve tamed.”—Antoine de Saint-Exupéry