If you think the estimates are made using the same or better information than you have, and are representative (unbiased in selection or reporting) of the true beliefs of the estimators. If these do not hold, the aggregate estimate MAY still be better than yours, or your independent estimate may be better.
If median is significantly different from the mean of a group of estimates, beware. Depending on the reasons you see for the variance, you may prefer to throw out outliers and then take the median/mean (which will be closer together).
Generally, use it as evidence in calculating a posterior from your prior, rather than adjusting your prior. The trick is in not double-counting evidence that you’re using directly which the public estimates are also depending on.
For Metaculus as evidence, there’s not much mechanism for correction—nobody is making money by moving the prediction toward truth. Which implies that it’s not very good for any more than a trigger to look deeper if you’re surprised by a result. You’ll have to figure out the reason for the surprising prediction, and use those reasons as evidence (if you agree with them), not just the resulting predictions.
If you think the estimates are made using the same or better information than you have, and are representative (unbiased in selection or reporting) of the true beliefs of the estimators. If these do not hold, the aggregate estimate MAY still be better than yours, or your independent estimate may be better.
If median is significantly different from the mean of a group of estimates, beware. Depending on the reasons you see for the variance, you may prefer to throw out outliers and then take the median/mean (which will be closer together).
Generally, use it as evidence in calculating a posterior from your prior, rather than adjusting your prior. The trick is in not double-counting evidence that you’re using directly which the public estimates are also depending on.
For Metaculus as evidence, there’s not much mechanism for correction—nobody is making money by moving the prediction toward truth. Which implies that it’s not very good for any more than a trigger to look deeper if you’re surprised by a result. You’ll have to figure out the reason for the surprising prediction, and use those reasons as evidence (if you agree with them), not just the resulting predictions.