But while you still would not have come close to what a Solomonoff approach might do, you have still learned a great deal about your model’s reliability in a way that I can’t see as having any connection with your time and KL-related approach.
I think there is a connection. Namely, the methods you mentioned are possible mechanisms of a learning process but
]) is a quantification of the expected impact of this learning process.
Yes, I see what you mean—the mean/expectation of how big the divergence between our current probability distribution and the future probability distribution—but this seems like a post hoc or purely descriptive approach: how do we estimate how much divergence there may be?
Having gotten estimates of future divergence, quantifying the divergence may then be useful, but it seems like putting the horse before the cart to start with your measure.
I think there is a connection. Namely, the methods you mentioned are possible mechanisms of a learning process but
Yes, I see what you mean—the mean/expectation of how big the divergence between our current probability distribution and the future probability distribution—but this seems like a post hoc or purely descriptive approach: how do we estimate how much divergence there may be?
Having gotten estimates of future divergence, quantifying the divergence may then be useful, but it seems like putting the horse before the cart to start with your measure.