I’ve been thinking similar things about predictability recently. Different variables have different levels of predictability, it seems very clear. I’m also under the impression that the examples in the Superforecasting study were quite specific. It seems likely that similar problems to what they studied have essentially low predictability 5-10 years out (and that is interesting information!), but this has limited relevance on other possible interesting questions.
the only the way to arrive at some unbiased global sense of whether the future is predictable is to come up with some way of enumerating and weighing all possible facts about the future universe… which is an impossible problem. So we’re left with the unsatisfying truth that the future is neither predictable or unpredictable—it depends on which features of the future you are considering.
While I agree with the specifics, I don’t think that the answer to a question like, “What is the average predictability of all possible statements” would be all that interesting. We generally care about a very small subset of “all possible statements.” It seems pretty reasonable to me that we could learn about the predictability of the kinds of things we’re interested. That said, i feel like we can get most of the benefits of this by just having calibrated forecasters try predicting all of these things, and seeing what their resolution numbers are. So I don’t think we need to do a huge amount of work running tests for the sole purpose of better understanding long-term predictability.
I’ve been thinking similar things about predictability recently. Different variables have different levels of predictability, it seems very clear. I’m also under the impression that the examples in the Superforecasting study were quite specific. It seems likely that similar problems to what they studied have essentially low predictability 5-10 years out (and that is interesting information!), but this has limited relevance on other possible interesting questions.
While I agree with the specifics, I don’t think that the answer to a question like, “What is the average predictability of all possible statements” would be all that interesting. We generally care about a very small subset of “all possible statements.” It seems pretty reasonable to me that we could learn about the predictability of the kinds of things we’re interested. That said, i feel like we can get most of the benefits of this by just having calibrated forecasters try predicting all of these things, and seeing what their resolution numbers are. So I don’t think we need to do a huge amount of work running tests for the sole purpose of better understanding long-term predictability.
I left some longer comments in the EA Forum Post discussion.