Say I were convinced that there is valuable wisdom of the crowds when guessing about the number of jelly beans in a jar.
I would be especially surprised by the average of humans’ guesses being accurate because I would have thought more relevant the humans’ average estimation error by percent of difference between the jar and the guess, whether it was an overestimation or an underestimation.
A guess of 425 beans is off by 425 beans and 50%, a guess of 1275 is off by 425 beans and 33.33%, and a guess of 1700 is off by 850 beans and 50%.
For a young child glancing at a map, which is a worse guess for how many states are in the U.S., 1 or 100? 10 or 100?
Say I were convinced that there is valuable wisdom of the crowds when guessing about the number of jelly beans in a jar.
I would be especially surprised by the average of humans’ guesses being accurate because I would have thought more relevant the humans’ average estimation error by percent of difference between the jar and the guess, whether it was an overestimation or an underestimation.
A guess of 425 beans is off by 425 beans and 50%, a guess of 1275 is off by 425 beans and 33.33%, and a guess of 1700 is off by 850 beans and 50%.
For a young child glancing at a map, which is a worse guess for how many states are in the U.S., 1 or 100? 10 or 100?
I think 1 and 10 are worse.
This can be solved by taking the error on a log scale rather than a linear scale.