Stuart_Armstrong comments on Self-confirming predictions can be arbitrarily bad

[Stuart_Armstrong]

And such a predictor cannot unlock new corners of strategy space, or generate self-reinforcing predictions, because the past sequence on which it’s trained won’t have those features.

See my last paragraph above; I don’t think we can rely on predictors not unlocking new corners of strategy space, because it may be able to learn gradually how to do so.