I am assuming [the AI] acts, and therefore makes choices, and therefore has preferences, and therefore can have preferences which conflict with the preferences of other minds (including human minds).
An AI can indeed have preferences that conflict with human preferences, but if it doesn’t start out with such preferences, it’s unclear how it comes to have them later.
On the other hand, if it starts out with dubious preferences, we’re in trouble from the outset.
I am assuming [the AI] acts, and therefore makes choices, and therefore has preferences, and therefore can have preferences which conflict with the preferences of other minds (including human minds).
An AI can indeed have preferences that conflict with human preferences, but if it doesn’t start out with such preferences, it’s unclear how it comes to have them later.
On the other hand, if it starts out with dubious preferences, we’re in trouble from the outset.