Surely something like the expected variance of log(p/(1−p)) would be a much simpler way of formalising this, no? The probability over time is just a stochastic process, and OP is expecting the variance of this process to be very high in the near future.
The variance over time depends on how you gather information in the future, making it less general. For example, I may literally never learn enough about meteorology to update my credence about the winds from 0.5. Nevertheless, there’s still an important sense in which this credence is more fragile than my credence about coins, because I could update it.
I guess you could define it as something like “the variance if you investigated it further”. But defining what it means to investigate further seems about as complicated as defining the reference class of people you’re trading against. Also variance doesn’t give you the same directional information—e.g. OP would bet on doom at 2% or bet against it at 16%.
Overall though, as I said above, I don’t know a great way to formalize this, and would be very interested in attempts to do so.
Wait, why doesn’t the entropy of your posterior distribution capture this effect? In the basic example where we get to see samples from a bernoulli process, the posterior is a beta distribution that gets ever sharper around the truth. If you compute the entropy of the posterior, you might say something like “I’m unlikely to change my mind about this, my posterior only has 0.2 bits to go until zero entropy”. That’s already a quantity which estimates how much future evidence will influence your beliefs.
The thing that distinguishes the coin case from the wind case is how hard it is to gather additional information, not how much more information could be gathered in principle. In theory you could run all sorts of simulations that would give you informative data about an individual flip of the coin, it’s just that it would be really hard to do so/very few people are able to do so. I don’t think the entropy of the posterior captures this dynamic.
Surely something like the expected variance of log(p/(1−p)) would be a much simpler way of formalising this, no? The probability over time is just a stochastic process, and OP is expecting the variance of this process to be very high in the near future.
The variance over time depends on how you gather information in the future, making it less general. For example, I may literally never learn enough about meteorology to update my credence about the winds from 0.5. Nevertheless, there’s still an important sense in which this credence is more fragile than my credence about coins, because I could update it.
I guess you could define it as something like “the variance if you investigated it further”. But defining what it means to investigate further seems about as complicated as defining the reference class of people you’re trading against. Also variance doesn’t give you the same directional information—e.g. OP would bet on doom at 2% or bet against it at 16%.
Overall though, as I said above, I don’t know a great way to formalize this, and would be very interested in attempts to do so.
Wait, why doesn’t the entropy of your posterior distribution capture this effect? In the basic example where we get to see samples from a bernoulli process, the posterior is a beta distribution that gets ever sharper around the truth. If you compute the entropy of the posterior, you might say something like “I’m unlikely to change my mind about this, my posterior only has 0.2 bits to go until zero entropy”. That’s already a quantity which estimates how much future evidence will influence your beliefs.
The thing that distinguishes the coin case from the wind case is how hard it is to gather additional information, not how much more information could be gathered in principle. In theory you could run all sorts of simulations that would give you informative data about an individual flip of the coin, it’s just that it would be really hard to do so/very few people are able to do so. I don’t think the entropy of the posterior captures this dynamic.