wizzwizz4 comments on Plausibly, almost every powerful algorithm would be manipulative

[wizzwizz4]

If producing (or committing to produce) an error state results in a change in utility /​ fitness, then it may end up optimised for /​ against.

[Stuart_Armstrong]

Yep!