Sam Clarke comments on Less Realistic Tales of Doom

[Sam Clarke]

Will MacAskill calls this the “actual alignment problem”

Wei Dai has written a lot about related concerns in posts like The Argument from Philosophical Difficulty