The estimation game you describe sounds a lot like the party game Wits and Wagers, though with the added challenge of predicting what the other players may predict as well.
I like the idea behind your game though. One way you may be able to help it teach to calibrate your own confidence intervals is to have everyone also guess an X% confidence range for their guess (whatever you decide is a good range). Then, each time the answer falls within that range, award the player P points, and whenever it is outside the range, penalize them P/(1-X) points (e.g. confidence of 90%, give 1 point for correct range and −10 for incorrect). To keep the ranges tight, offer another bonus to whoever had the tightest correct bounds.
I tried it out and it is much easier to play than my game and thus is somewhat more fun. But it also has less insights.
My experience is as follows:
The trivia questions are not difficult enough. It is very seldom that values lie outside a times 2-range. And ‘surprises’ are rare.
The questions have an american cultural bias (no wonder)
The ‘going over’ rule is simple but totally skews the betting and guessing.
The simple payout-rules cause gaming for higher payouts thus mixing confidence and probability in non-trivial ways.
The two-phase setup where you can look where the ‘experts’ bet is interesting but doesn’t help with confidence calibration.
It really is optimized for playability. I think it does some calibration of (over)confidence and it builds intuition for probability and risk-trade-offs.
But—and that is my main point—it doesn’t have clear concepts. The concepts are all mingled up, skewed, hidden. You may gain intuition but it will not help you toward overcoming e.g. overconfidence bias or egocentric bias.
I still think for that the concept must be sufficiently present to be able to reflect and consciously use it.
Indeed. I wasn’t aware of http://en.wikipedia.org/wiki/Wits_and_Wagers. There is no German match for it (yet).
I actually considered using a confidence rating but discarded it because it was too easy to game the rules for it—but that was during the first test games that were about estimating your own performance via multiple estimates (called ‘strict variant’ in the post). When I switched to estimating the range of the other players guesses I didn’t reconsider confidence ranges. Could be an interesting variant.
The current gameplay is really simple even though it includes a min-max range. There was a earlier and more complex version that had a 90% range (or more precisely an 1-1/N range) which earned N points if it excluded exactly 1 guess, 1 if it excluded none and 0 if more than 1. That was OK but didn’t gain much. The most surprises were still registered by the min-max range.
The estimation game you describe sounds a lot like the party game Wits and Wagers, though with the added challenge of predicting what the other players may predict as well.
I like the idea behind your game though. One way you may be able to help it teach to calibrate your own confidence intervals is to have everyone also guess an X% confidence range for their guess (whatever you decide is a good range). Then, each time the answer falls within that range, award the player P points, and whenever it is outside the range, penalize them P/(1-X) points (e.g. confidence of 90%, give 1 point for correct range and −10 for incorrect). To keep the ranges tight, offer another bonus to whoever had the tightest correct bounds.
I discovered that Wits and Wagers was actually discussed here: http://lesswrong.com/lw/14u/wits_and_wagers/
I tried it out and it is much easier to play than my game and thus is somewhat more fun. But it also has less insights.
My experience is as follows:
The trivia questions are not difficult enough. It is very seldom that values lie outside a times 2-range. And ‘surprises’ are rare.
The questions have an american cultural bias (no wonder)
The ‘going over’ rule is simple but totally skews the betting and guessing.
The simple payout-rules cause gaming for higher payouts thus mixing confidence and probability in non-trivial ways.
The two-phase setup where you can look where the ‘experts’ bet is interesting but doesn’t help with confidence calibration.
It really is optimized for playability. I think it does some calibration of (over)confidence and it builds intuition for probability and risk-trade-offs.
But—and that is my main point—it doesn’t have clear concepts. The concepts are all mingled up, skewed, hidden. You may gain intuition but it will not help you toward overcoming e.g. overconfidence bias or egocentric bias.
I still think for that the concept must be sufficiently present to be able to reflect and consciously use it.
Indeed. I wasn’t aware of http://en.wikipedia.org/wiki/Wits_and_Wagers. There is no German match for it (yet). I actually considered using a confidence rating but discarded it because it was too easy to game the rules for it—but that was during the first test games that were about estimating your own performance via multiple estimates (called ‘strict variant’ in the post). When I switched to estimating the range of the other players guesses I didn’t reconsider confidence ranges. Could be an interesting variant.
The current gameplay is really simple even though it includes a min-max range. There was a earlier and more complex version that had a 90% range (or more precisely an 1-1/N range) which earned N points if it excluded exactly 1 guess, 1 if it excluded none and 0 if more than 1. That was OK but didn’t gain much. The most surprises were still registered by the min-max range.