My impression (based on using Metaculus a lot) is that, while questions like this may give you a reasonable ballpark estimate and it’s great that they exist, they’re nowhere close to being efficient enough for it to mean much when they fail to move. As a proxy for the amount of mental effort that goes into it, there’s only been three comments on the linked question in the last month. I’ve been complaining about people calling Metaculus a “prediction market” because if people think it’s a prediction market then they’ll assume there’s a point to be made like “if you can tell that the prediction is inefficient, then why aren’t you rich, at least in play money?” But the estimate you’re seeing is just a recency-weighted median of the predictions of everyone who presses the button, not weighted by past predictive record, and not weighted by willingness-to-bet, because there’s no buying or selling and everyone makes only one prediction. It’s basically a poll of people who are trying to get good results (in terms of Brier/log score and Metaculus points) on their answers.
My impression (based on using Metaculus a lot) is that, while questions like this may give you a reasonable ballpark estimate and it’s great that they exist, they’re nowhere close to being efficient enough for it to mean much when they fail to move. As a proxy for the amount of mental effort that goes into it, there’s only been three comments on the linked question in the last month. I’ve been complaining about people calling Metaculus a “prediction market” because if people think it’s a prediction market then they’ll assume there’s a point to be made like “if you can tell that the prediction is inefficient, then why aren’t you rich, at least in play money?” But the estimate you’re seeing is just a recency-weighted median of the predictions of everyone who presses the button, not weighted by past predictive record, and not weighted by willingness-to-bet, because there’s no buying or selling and everyone makes only one prediction. It’s basically a poll of people who are trying to get good results (in terms of Brier/log score and Metaculus points) on their answers.