the population estimate for the population estimate of “yes”, will still be low.
Thus “yes”, the correct answer, will be surprisingly popular.
I missed the ‘population estimate of the population estimate’ part … took me a while to understand why you said surprisingly popular :)
Also, this aeon article explains the above concept pretty well.
My takeaways from the aeon article:
Experts are more likely to recognise that other people will disagree with them. Novices betray themselves by being unable to fathom any position other than their own
This ‘meta-knowledge’, ability to know your beliefs and the causes for them, is one of the best predictors for expertise. Studies that incorporate such data, discard predictions that are very wrong on the meta-knowledge scale, perform generally better (except when everybody believes something that will happen).
The reason for this is that often the crowd is sourcing its predictions from the same resources. Thus, the predictions ‘double-count’ these sources. But experts or people who are more aware of both sides of the story, will be able to give more rational views.
So essentially when doing a poll/survey if you ask people what they believe and then ask them the number of other people who might believe the same thing, you will be able to differentiate between novices (who are often unaware of alternative views/beliefs) and experts (who will generally know that other views exist and also better predict how many people believe what). Discard novice opinions and use expert opinions to find the answer.
Those are real-world explanations for why the method might work, which helps reinforce the method in practice. But the mathematics work out in situations where everyone is a rational Bayesian expert, with access to different types of private information.
Nice article!
I missed the ‘population estimate of the population estimate’ part … took me a while to understand why you said surprisingly popular :)
Also, this aeon article explains the above concept pretty well.
My takeaways from the aeon article:
Experts are more likely to recognise that other people will disagree with them. Novices betray themselves by being unable to fathom any position other than their own
This ‘meta-knowledge’, ability to know your beliefs and the causes for them, is one of the best predictors for expertise. Studies that incorporate such data, discard predictions that are very wrong on the meta-knowledge scale, perform generally better (except when everybody believes something that will happen).
The reason for this is that often the crowd is sourcing its predictions from the same resources. Thus, the predictions ‘double-count’ these sources. But experts or people who are more aware of both sides of the story, will be able to give more rational views.
So essentially when doing a poll/survey if you ask people what they believe and then ask them the number of other people who might believe the same thing, you will be able to differentiate between novices (who are often unaware of alternative views/beliefs) and experts (who will generally know that other views exist and also better predict how many people believe what). Discard novice opinions and use expert opinions to find the answer.
Those are real-world explanations for why the method might work, which helps reinforce the method in practice. But the mathematics work out in situations where everyone is a rational Bayesian expert, with access to different types of private information.