Down-voted for not acknowledging or not realizing that you were also lucky.
As people make their guesses, the mean of their guesses will fluctuate. This doesn’t mean the number of jellybeans is fluctuating, of course. You were lucky that your well-educated guess happened to be closer than any other guess.
Suppose people’s guesses were normally distributed about the true number of jellybeans with a specific standard deviation. Even with these ideal assumptions, the average of N guesses doesn’t converge that quickly to the mean, and I don’t suppose it converges faster than the probability that any one guess is closer to the true number than your mean.
Actually, with the assumptions you gave, as far as I can tell from memory and a quick look at wikipedia, the standard deviation of the average of N guesses is proportional to sqrt(N) so, whatever the average error of one guess is, the average error of the average of 16 guesses is 4 times less, making it a significantly better guess. This quantity is known as the standard error of the mean.
Great, I feel like we’re making good progress. (wisdom of the crowd..)
the standard deviation of the average of N guesses is proportional to 1/sqrt(N)
Yes, for example from here, if the standard deviation of the individual guesses is s, the standard deviation of the average of N guesses will be s / sqrt(N).
… And this represents the typical error of seven_and_sixes strategy, within one standard deviation.
Now—to see if the strategy typically wins—we just need the number for: given N guesses, what is the expected minimum error of the N guesses? (That is, the average minimum of the set of differences between each guess and the mean?)
I would guess that this is proportional 1/N, whereas the average method gives 1/sqrt(N), so the probability that you win is O(1/sqrt(N)). In reality it would be worse, since, if you were not the last to go, you would only have the average of M guesses, with M < N. However, the average person has a probability of winning of 1/(N+1) so your probability of winning is sqrt(N) times better than the average person’s, unless you can do even better with some other skill that you have. This analysis is complicated by exceptionally bad guessers and other average-takers, but not significantly. I accept your downvote because sixes was still lucky to win and did not acknowledge this.
Oh wait. I think we’ve got some errors to fix. I won’t have time immediately, but I’ll edit my comment to reflect any changes you make. I knew what you meant, or you knew what I meant, but now it’s confusing..
I’m not sure what you mean. Looking back I did make some errors in my analysis, but I’m not particularly motivated to correct them, since I doubt my conclusion will qualitatively change, though if I got the right probability of winning, it would be a coincidence. Maybe I’ll feel more curious about the right answer another time.
I fixed that the standard deviation of the average of N guesses is proportional to 1/sqrt(N)
in this comment.
I don’t think you should ‘guess’ that the minimum error is proportional 1/N, since its relationship with N is exactly what we need to know. Let’s wait to see if someone knows.
… But do you realize that 1/N decreases faster than 1/sqrt(N), so that your guess would indicate that the average strategy would rarely win?
I don’t think you should ‘guess’ that the minimum error is proportional 1/N, since its relationship with N is exactly what we need to know.
I mistakenly thought I had a qualitative argument that was too simple to bother spelling out. Unfortunately, it was also too simple to be correct. :-) However, see constant’s comment; my intuition appeared to be probably right after all.
… But do you realize that 1/N decreases faster than 1/sqrt(N), so that your guess would indicate that the average strategy would rarely win?
Yes. If my analysis is correct, the method would be very unlikely to win, but it would be much better than using the same method as the other competitors, precisely because 1/N decreases faster than 1/sqrt(N).
As the minimum error approaches zero, the probability that the next guess will reduce the minimum error becomes proportional to the minimum error itself—that’s because it’s like trying to hit a target, and the probability of hitting the target is proportional to the size of the target. This only applies to the error close to zero, because that allows us to treat the probability distribution as essentially flat in that neighborhood, so we don’t have to worry about the shape of the curve.
If the next guess does reduce the minimum error, then, on average, it will reduce the minimum error by half. As above, we’re treating the probability distribution as essentially flat.
So, we expect that after some number n guesses, the minimum error is reduced by half. We expect that after 2n more guesses, the minimum error is reduced again by half. Assuming this is what happens, then we expect that after 4n more guesses, the minimum error is reduced by half again.
The reduction in error that we’re seeing in this imagined playing out is approximately inversely proportional to the number of guesses. The total number of guesses goes from n to n+2n=3n, to n+2n+4n=7n, etc. If we keep going, the total number of guesses becomes 15n, 31n, etc. This approaches a doubling of total guesses. And the error after each approximate doubling is half what it was before.
This is far from a proof. This is crude, fallible reasoning. It’s my best estimate, that’s all.
Now, I’m not sure that my comment was obvious or not. I thought that it would be.
Do people generally agree that this strategy should not be expected to win most of the time? (I would guess not even half the time, but this is just above where my estimate would be for large N.)
The guesses in most contests won’t be normally distributed, period, so they won’t be normally distributed around the true number. They’d be closer to log-normal or some such distribution, I would guess.
Long ago, I looked at some data on people’s guesses in probability calibration exercises. Exercises like: how many eggs do Americans consume annually—write down a number such that you judge it X% probable the true number is less than or equal, and (100-X)% probable it is greater. I vaguely recall some such thing about the distribution of guesses. A very large number of guesses were many orders of magnitude too low, as well.
Are you aware of the wisdom of crowds phenomenon (Surowiecki) and correcting a lucky misapplication / hasty application of it, or are you simply unaware of it? I don’t get a sense either way.
Also, I wonder whether the correct mean for this sort of wisdom of crowds averaging is arithmetic or geometric. I would be inclined to think that the geometric mean is the appropriate mean.
I’m aware of the wisdom of crowds phenomenon—I believe sixes_and_sevens used a good strategy. (Yes.) I haven’t read about the corrections, but I wouldn’t be surprised if there are some ways to optimize the process. (Why do you ask this second question?)
I wonder whether the correct mean for this sort of wisdom of crowds averaging is arithmetic or geometric.
I think this is an interesting question too. Perhaps someone knows the math for this; alternatively, it would take a few minutes to test it with some made-up guess distributions. (Any takers?)
Down-voted for not acknowledging or not realizing that you were also lucky.
As people make their guesses, the mean of their guesses will fluctuate. This doesn’t mean the number of jellybeans is fluctuating, of course. You were lucky that your well-educated guess happened to be closer than any other guess.
Suppose people’s guesses were normally distributed about the true number of jellybeans with a specific standard deviation. Even with these ideal assumptions, the average of N guesses doesn’t converge that quickly to the mean, and I don’t suppose it converges faster than the probability that any one guess is closer to the true number than your mean.
Actually, with the assumptions you gave, as far as I can tell from memory and a quick look at wikipedia, the standard deviation of the average of N guesses is proportional to sqrt(N) so, whatever the average error of one guess is, the average error of the average of 16 guesses is 4 times less, making it a significantly better guess. This quantity is known as the standard error of the mean.
Great, I feel like we’re making good progress. (wisdom of the crowd..)
Yes, for example from here, if the standard deviation of the individual guesses is s, the standard deviation of the average of N guesses will be s / sqrt(N).
… And this represents the typical error of seven_and_sixes strategy, within one standard deviation.
Now—to see if the strategy typically wins—we just need the number for: given N guesses, what is the expected minimum error of the N guesses? (That is, the average minimum of the set of differences between each guess and the mean?)
I would guess that this is proportional 1/N, whereas the average method gives 1/sqrt(N), so the probability that you win is O(1/sqrt(N)). In reality it would be worse, since, if you were not the last to go, you would only have the average of M guesses, with M < N. However, the average person has a probability of winning of 1/(N+1) so your probability of winning is sqrt(N) times better than the average person’s, unless you can do even better with some other skill that you have. This analysis is complicated by exceptionally bad guessers and other average-takers, but not significantly. I accept your downvote because sixes was still lucky to win and did not acknowledge this.
Oh wait. I think we’ve got some errors to fix. I won’t have time immediately, but I’ll edit my comment to reflect any changes you make. I knew what you meant, or you knew what I meant, but now it’s confusing..
I’m not sure what you mean. Looking back I did make some errors in my analysis, but I’m not particularly motivated to correct them, since I doubt my conclusion will qualitatively change, though if I got the right probability of winning, it would be a coincidence. Maybe I’ll feel more curious about the right answer another time.
I fixed that the standard deviation of the average of N guesses is proportional to 1/sqrt(N) in this comment.
I don’t think you should ‘guess’ that the minimum error is proportional 1/N, since its relationship with N is exactly what we need to know. Let’s wait to see if someone knows.
… But do you realize that 1/N decreases faster than 1/sqrt(N), so that your guess would indicate that the average strategy would rarely win?
I mistakenly thought I had a qualitative argument that was too simple to bother spelling out. Unfortunately, it was also too simple to be correct. :-) However, see constant’s comment; my intuition appeared to be probably right after all.
Yes. If my analysis is correct, the method would be very unlikely to win, but it would be much better than using the same method as the other competitors, precisely because 1/N decreases faster than 1/sqrt(N).
As the minimum error approaches zero, the probability that the next guess will reduce the minimum error becomes proportional to the minimum error itself—that’s because it’s like trying to hit a target, and the probability of hitting the target is proportional to the size of the target. This only applies to the error close to zero, because that allows us to treat the probability distribution as essentially flat in that neighborhood, so we don’t have to worry about the shape of the curve.
If the next guess does reduce the minimum error, then, on average, it will reduce the minimum error by half. As above, we’re treating the probability distribution as essentially flat.
So, we expect that after some number n guesses, the minimum error is reduced by half. We expect that after 2n more guesses, the minimum error is reduced again by half. Assuming this is what happens, then we expect that after 4n more guesses, the minimum error is reduced by half again.
The reduction in error that we’re seeing in this imagined playing out is approximately inversely proportional to the number of guesses. The total number of guesses goes from n to n+2n=3n, to n+2n+4n=7n, etc. If we keep going, the total number of guesses becomes 15n, 31n, etc. This approaches a doubling of total guesses. And the error after each approximate doubling is half what it was before.
This is far from a proof. This is crude, fallible reasoning. It’s my best estimate, that’s all.
Granted. I wasn’t claiming it was a guaranteed winning strategy, and a fair element of luck was obviously involved.
Now, I’m not sure that my comment was obvious or not. I thought that it would be.
Do people generally agree that this strategy should not be expected to win most of the time? (I would guess not even half the time, but this is just above where my estimate would be for large N.)
It seems appropriate to ‘ask the crowd’..
Strategy shouldn’t win contest most of the time.
Strategy should win contest most of the time.
Not obvious either way.
Karma balance.
per JoshuaZ’a request:
Strategy will win more frequently than trying to estimate but won’t win most of the time
Strategy will win more frequently than trying to estimate but won’t win most of the time.
Can we have as an option “strategy will win more frequently than trying to estimate but won’t win most of the time”?
Sure..
Strategy should win contest most of the time.
Strategy shouldn’t win contest most of the time.
Not obvious either way.
Karma balance.
The guesses in most contests won’t be normally distributed, period, so they won’t be normally distributed around the true number. They’d be closer to log-normal or some such distribution, I would guess.
Sounds reasonable. How do you know this / what do you base that on?
Long ago, I looked at some data on people’s guesses in probability calibration exercises. Exercises like: how many eggs do Americans consume annually—write down a number such that you judge it X% probable the true number is less than or equal, and (100-X)% probable it is greater. I vaguely recall some such thing about the distribution of guesses. A very large number of guesses were many orders of magnitude too low, as well.
Are you aware of the wisdom of crowds phenomenon (Surowiecki) and correcting a lucky misapplication / hasty application of it, or are you simply unaware of it? I don’t get a sense either way.
Also, I wonder whether the correct mean for this sort of wisdom of crowds averaging is arithmetic or geometric. I would be inclined to think that the geometric mean is the appropriate mean.
I’m aware of the wisdom of crowds phenomenon—I believe sixes_and_sevens used a good strategy. (Yes.) I haven’t read about the corrections, but I wouldn’t be surprised if there are some ways to optimize the process. (Why do you ask this second question?)
I think this is an interesting question too. Perhaps someone knows the math for this; alternatively, it would take a few minutes to test it with some made-up guess distributions. (Any takers?)