Let’s say we’re having a prediction contest where we want people to pre-register their probability estimates for the outcomes of a set of discrete events (e.g. the candidates for various races in an upcoming election). We have a fixed sum of, say, $100 that we must award to the winner(s) in some way. We are not allowed to award any more or less than that. How can we determine the award in such a way as to incentivize people to honestly report their estimates?
However, I’m pretty sure this doesn’t work. A proper scoring rule incentivizes honest estimates if the goal is to maximize your total score; but in this case, maximizing your score is not the goal. Under this system, I’m incentivized to make estimates that may have a suboptimal expected score as long as the expectation is high-variance, because that will increase the probability that I get the top spot.
Can you figure out a system that avoids this problem?
(Distributing the prize randomly or in partial shares is allowed. However, destroying money or creating extra money is not.)
[Question] Is there a good way to award a fixed prize in a prediction contest?
Let’s say we’re having a prediction contest where we want people to pre-register their probability estimates for the outcomes of a set of discrete events (e.g. the candidates for various races in an upcoming election). We have a fixed sum of, say, $100 that we must award to the winner(s) in some way. We are not allowed to award any more or less than that. How can we determine the award in such a way as to incentivize people to honestly report their estimates?
The naïve approach would be to do the following:
Score everyone’s predictions according to some proper scoring rule
Award the $100 to whoever has the highest score.
However, I’m pretty sure this doesn’t work. A proper scoring rule incentivizes honest estimates if the goal is to maximize your total score; but in this case, maximizing your score is not the goal. Under this system, I’m incentivized to make estimates that may have a suboptimal expected score as long as the expectation is high-variance, because that will increase the probability that I get the top spot.
Can you figure out a system that avoids this problem?
(Distributing the prize randomly or in partial shares is allowed. However, destroying money or creating extra money is not.)