Stuart_Armstrong comments on Plausibly, almost every powerful algorithm would be manipulative

[Stuart_Armstrong]

cannot be steered away from maximizing their goal by ad-hoc variations in the training protocol.

That, and the fact these ad-hoc variations can introduce new goals that the programmers are not aware of.