Talking about increments of 5% runs counter to my intuitions regarding good thinking about probability estimates. For most purposes, the difference between 90% and 95% is significantly larger than the difference between 50% and 55%. Think in logs.
Yes, near the extremes it makes a difference—but we’re using a Brier scoring rule, averaged over all days a forecast is open. That makes thinking in logs less important − 99% isn’t much worse than 100% on errors. I’ll discuss that in pt.2 under ‘loss function’.
It depends on whether you’re using probabilities epistemically or instrumentally. Changing the probability of A from 90% to 95% doesn’t affect your expected utility any more than changing it from 50% to 55%.
The change in expected utility given constant decisions is the same for any 5% change in probability regardless of where the baseline is for the change. However, that “given constant decisions” criterion may be less likely to hold for a change from 90-95% than it is for a change from 50-55%. If you have to choose whether to risk a negative consequence of not-A in exchange for some benefit, for example, then it matters whether the expected negative utility of not-A just fell by a tenth or by half.
Talking about increments of 5% runs counter to my intuitions regarding good thinking about probability estimates. For most purposes, the difference between 90% and 95% is significantly larger than the difference between 50% and 55%. Think in logs.
Yes, near the extremes it makes a difference—but we’re using a Brier scoring rule, averaged over all days a forecast is open. That makes thinking in logs less important − 99% isn’t much worse than 100% on errors. I’ll discuss that in pt.2 under ‘loss function’.
Hooray!
It depends on whether you’re using probabilities epistemically or instrumentally. Changing the probability of A from 90% to 95% doesn’t affect your expected utility any more than changing it from 50% to 55%.
The change in expected utility given constant decisions is the same for any 5% change in probability regardless of where the baseline is for the change. However, that “given constant decisions” criterion may be less likely to hold for a change from 90-95% than it is for a change from 50-55%. If you have to choose whether to risk a negative consequence of not-A in exchange for some benefit, for example, then it matters whether the expected negative utility of not-A just fell by a tenth or by half.
Yeah, that’s why I said “For most purposes”.