This seems like a really hard problem: if a market like this “wins,” so that having a lot of points makes you high-status, people will try to game it, and if gaming it is easy, this will kill respect for the market.
This is not really a problem at Metaculus. Metaculus has metrics for player prediction, community prediction and Metaculus prediction on the questions someone answers.
The community prediction could be changed with sockpuppets but the Metaculus prediction can’t. You can judge people on how near they are to the Metaculus prediction in predictive accuracy or whether they even outperform it.
Metaculus however decides to keep that metric mostly private and makes it nonpublic. Metaculus has the problem of not wanting to embarrass users who make a lot of predictions when those predictions are on average bad
The Metaculus equivalent of buying Likes or Amazon reviews. On priors, if Amazon can’t squash this problem, it probably can’t be squashed.
I don’t think Amazon makes a serious effort at battling review fraud just as Youtube doesn’t make a serious effort at comment quality when it easily could do something about it.
Amazon also has a harder problem because as it’s ground truth is less clear.
This is not really a problem at Metaculus. Metaculus has metrics for player prediction, community prediction and Metaculus prediction on the questions someone answers.
The community prediction could be changed with sockpuppets but the Metaculus prediction can’t. You can judge people on how near they are to the Metaculus prediction in predictive accuracy or whether they even outperform it.
Metaculus however decides to keep that metric mostly private and makes it nonpublic. Metaculus has the problem of not wanting to embarrass users who make a lot of predictions when those predictions are on average bad
I don’t think Amazon makes a serious effort at battling review fraud just as Youtube doesn’t make a serious effort at comment quality when it easily could do something about it.
Amazon also has a harder problem because as it’s ground truth is less clear.