Stuart_Armstrong comments on Siren worlds and the perils of over-optimised search

[Stuart_Armstrong]

If I cannot trust the Oracle AI, dear God, why am I having it explore potential eutopian future worlds for me?

Because I haven’t read Less Wrong? ^_^

This is another argument against using constrained but non-friendly AI to do stuff for us...