These sorts of observations sound promising for someone’s potential as a forecaster. But by themselves, they are massively easier to cherry-pick, fudge, omit, or re-define things, versus proper forecasts.
When you see other people make non-specific “predictions”, how do you score them? How do you know the scoring that you’re doing is coherent, and isn’t rationalizing? How do you avoid the various pitfalls that Tetlock wrote about? How do you *ducks stern glance* score yourself on any of that, in a way that you’ll know isn’t rationalizing?
For emphasis, in this comment you reinforce that you consider it a successful advance prediction. This gives very little information about your forecasting accuracy. We don’t even know what your actual distribution is, and it’s a long time before this resolves, we only know it went in your direction. I claim that to critique other people’s proper-scored forecasts, you should be transparent and give your own.
EDIT: Pasted from another comment I wrote:
Instead of that actual [future resolution] reality, and because of how abruptly the community ended up shifting, Eliezer seems to be interpreting that to mean that his position about that reality is not extreme enough. Those 2 things are somewhat related but pretty weakly, so it seems like rationalizing for him to frame it as showing his forecast isn’t extreme enough.
These sorts of observations sound promising for someone’s potential as a forecaster. But by themselves, they are massively easier to cherry-pick, fudge, omit, or re-define things, versus proper forecasts.
When you see other people make non-specific “predictions”, how do you score them? How do you know the scoring that you’re doing is coherent, and isn’t rationalizing? How do you avoid the various pitfalls that Tetlock wrote about? How do you *ducks stern glance* score yourself on any of that, in a way that you’ll know isn’t rationalizing?
For emphasis, in this comment you reinforce that you consider it a successful advance prediction. This gives very little information about your forecasting accuracy. We don’t even know what your actual distribution is, and it’s a long time before this resolves, we only know it went in your direction. I claim that to critique other people’s proper-scored forecasts, you should be transparent and give your own.
EDIT: Pasted from another comment I wrote: