I think I’m still confused—a naive sequence predictor is _OF COURSE_ broken by perverse or adversarial unmodelled (because of the naivety of the predictor) behaviors. And such a predictor cannot unlock new corners of strategy space, or generate self-reinforcing predictions, because the past sequence on which it’s trained won’t have those features.
And such a predictor cannot unlock new corners of strategy space, or generate self-reinforcing predictions, because the past sequence on which it’s trained won’t have those features.
See my last paragraph above; I don’t think we can rely on predictors not unlocking new corners of strategy space, because it may be able to learn gradually how to do so.
I think I’m still confused—a naive sequence predictor is _OF COURSE_ broken by perverse or adversarial unmodelled (because of the naivety of the predictor) behaviors. And such a predictor cannot unlock new corners of strategy space, or generate self-reinforcing predictions, because the past sequence on which it’s trained won’t have those features.
See my last paragraph above; I don’t think we can rely on predictors not unlocking new corners of strategy space, because it may be able to learn gradually how to do so.