Yes. It might be doing exactly what it was designed to do, but its designer was clearly stupid or cruel and had different goals than I’d prefer the AI to have.
Where are you getting your goals from? What are you, except your design? You are what Azathoth build. There is no ideal you that you should’ve become, but which Azathoth failed to make.
Azathoth designed me with conflicting goals. Subconsciously, I value status, but if I were to take a pill that made me care entirely about making the world better and nothing else, I would. Just because “evolution” built that into me doesn’t make it bad, but it definitely did not give me a coherent volition. I have determined for my self which parts of humanity’s design are counterproductive, based on the thousand shards of desire.
Where are you getting your goals from? What are you, except your design? You are what Azathoth build. There is no ideal you that you should’ve become, but which Azathoth failed to make.
Azathoth designed me with conflicting goals. Subconsciously, I value status, but if I were to take a pill that made me care entirely about making the world better and nothing else, I would. Just because “evolution” built that into me doesn’t make it bad, but it definitely did not give me a coherent volition. I have determined for my self which parts of humanity’s design are counterproductive, based on the thousand shards of desire.
Would you sign up to be tortured so that others don’t suffer dust specks?
(“If we are here to make others happy, what are the others here for?”)
Yes.
A better analogy would be asking about a pill that caused pain asymbolia.