I just realised my own voting (and I suspect that of most people) was inefficient.
Once I decided on all of my votes I should have decreased all of the votes by 1, including putting a −1 vote on any that I had previously been neutral on (ignoring the off-by-one error thing for the moment).
This wouldn’t have changed the net effect of my vote but would have given me extra points to spend (the small cost of paying for negative results would have been more than offset by the large benefit from decreasing the positive votes).
I think most other people made the same mistake (well, it’s a mistake if voting effect size was a high priority rather than, say, speed) due to the large number of neutral votes (~71%) and the ratio between positive vs negative votes (4.8 : 1) although both of these might have been effected somewhat by the off-by-one correction.
Even without the off-by-one bug, if you were able to vote on all the nominees, then the most efficient use of points would have been to make your average vote as close to 0 as possible. For instance, imagine there were 50 nominees, and one voter rated 5 of them at 10 to use their 500 points. If instead, they rated each of those 5 at 9, and the other 45 at −1, that would be 405+45=450 points, with exactly the same overall impact on relative standings. (In algebraic terms, this is a simple quadratic decomposition, akin to the fact that mean squared error equals bias squared plus variance).
It’s clear from the vote results that the average voter did not ensure mean-0 and thus probably left some voting power on the table. The average of all votes is substantially positive.
This is even more “irrational” for those voters who were also authors of one or more of the works. Such voters were not allowed to vote on their own works, but could have helped their works win by voting all other works down. In other words, the incentive would be to have their average vote be negative, not just zer0.
I myself subtracted 2 from all my default-calculated votes for the above two reasons. Frankly, this 2-point difference (reduced to 1 in many cases by the off-by-one bug correction) was less than I considered selfishly rational, but I didn’t want to take too much advantage of such “underhanded” strategy. Looking at the voting results, I don’t think it’s likely that anybody else but me did this.
I wonder whether there is a way to take someone’s vote and infer a more optimal allocation of the votes, and then scale that up to use the full available points, so that we could potentially estimate the size of the impact of this.
There were also something like 10 who didn’t spend their full vote-ballot, so my guess is that optimality concerns aren’t a super big deal for many users, though I generally think that we should align the natural interaction with the system with the one that also spends your points most effectively, since anything else just weirdly biases the results towards people who either just vote differently naturally, or are thinking more about meta-level voting strategies, neither of which seems like a particularly good bias.
Note that I considered this, but saw that the final decision of what would be included was subjective anyway, and it would be a stronger signal to the judges to spend a lot of my points on a few posts I thought really deserved it and would otherwise be underrated.
Thanks, I was trying to work out a simple way to calculate how many times it would be worthwhile subtracting extra 1s—making the average close to 0 is a helpful simple rule.
Note that one of the desiderata I’d prefer out of this is “positive votes actually mean ‘I liked this post’ and negative votes actually mean ‘I disliked this post’ as opposed to ‘I strategically voted this number to have a cumulative effect across my overall votes’”.
(This isn’t crucial but it does make the swarmplots more intuitive to read)
There’s frequently a tradeoff between “less strategic incentives” and “more-intelligible under honesty”. I don’t think that you should pick the former every time, but it is certainly better to err a little bit on the side of the former and get good-but-slightly-more-confusing results, than to err on the side of the latter and get results that are neither good nor intelligible (because strategic voting has ruined that, too).
I agree, just looking for Pareto improvements if they exist (since we didn’t try much at all this year for them, it seemed plausible such things existed)
Put a cap on the variance (standard deviation squared) of votes, rather than the sum-squared of votes? That would be equivalent to subtracting the mean from each vote …
(Maybe you have to count the zero votes in the calculation)
Still seems fairly confusing to me. Like, the act of futzing around with your vote totals shouldn’t give you the (temporary) impression that you can get more points in a way that can’t actually get you more points.
I just realised my own voting (and I suspect that of most people) was inefficient.
Once I decided on all of my votes I should have decreased all of the votes by 1, including putting a −1 vote on any that I had previously been neutral on (ignoring the off-by-one error thing for the moment).
This wouldn’t have changed the net effect of my vote but would have given me extra points to spend (the small cost of paying for negative results would have been more than offset by the large benefit from decreasing the positive votes).
I think most other people made the same mistake (well, it’s a mistake if voting effect size was a high priority rather than, say, speed) due to the large number of neutral votes (~71%) and the ratio between positive vs negative votes (4.8 : 1) although both of these might have been effected somewhat by the off-by-one correction.
It’s far worse than that.
Even without the off-by-one bug, if you were able to vote on all the nominees, then the most efficient use of points would have been to make your average vote as close to 0 as possible. For instance, imagine there were 50 nominees, and one voter rated 5 of them at 10 to use their 500 points. If instead, they rated each of those 5 at 9, and the other 45 at −1, that would be 405+45=450 points, with exactly the same overall impact on relative standings. (In algebraic terms, this is a simple quadratic decomposition, akin to the fact that mean squared error equals bias squared plus variance).
It’s clear from the vote results that the average voter did not ensure mean-0 and thus probably left some voting power on the table. The average of all votes is substantially positive.
This is even more “irrational” for those voters who were also authors of one or more of the works. Such voters were not allowed to vote on their own works, but could have helped their works win by voting all other works down. In other words, the incentive would be to have their average vote be negative, not just zer0.
I myself subtracted 2 from all my default-calculated votes for the above two reasons. Frankly, this 2-point difference (reduced to 1 in many cases by the off-by-one bug correction) was less than I considered selfishly rational, but I didn’t want to take too much advantage of such “underhanded” strategy. Looking at the voting results, I don’t think it’s likely that anybody else but me did this.
I wonder whether there is a way to take someone’s vote and infer a more optimal allocation of the votes, and then scale that up to use the full available points, so that we could potentially estimate the size of the impact of this.
There were also something like 10 who didn’t spend their full vote-ballot, so my guess is that optimality concerns aren’t a super big deal for many users, though I generally think that we should align the natural interaction with the system with the one that also spends your points most effectively, since anything else just weirdly biases the results towards people who either just vote differently naturally, or are thinking more about meta-level voting strategies, neither of which seems like a particularly good bias.
Note that I considered this, but saw that the final decision of what would be included was subjective anyway, and it would be a stronger signal to the judges to spend a lot of my points on a few posts I thought really deserved it and would otherwise be underrated.
Thanks, I was trying to work out a simple way to calculate how many times it would be worthwhile subtracting extra 1s—making the average close to 0 is a helpful simple rule.
Do you have suggestions for what to do instead, that’d roughly preserve the advantages of the current system, while being more intuitive?
Just renormalize votes to be mean-0 before scaling.
Note that one of the desiderata I’d prefer out of this is “positive votes actually mean ‘I liked this post’ and negative votes actually mean ‘I disliked this post’ as opposed to ‘I strategically voted this number to have a cumulative effect across my overall votes’”.
(This isn’t crucial but it does make the swarmplots more intuitive to read)
There’s frequently a tradeoff between “less strategic incentives” and “more-intelligible under honesty”. I don’t think that you should pick the former every time, but it is certainly better to err a little bit on the side of the former and get good-but-slightly-more-confusing results, than to err on the side of the latter and get results that are neither good nor intelligible (because strategic voting has ruined that, too).
I agree, just looking for Pareto improvements if they exist (since we didn’t try much at all this year for them, it seemed plausible such things existed)
Put a cap on the variance (standard deviation squared) of votes, rather than the sum-squared of votes? That would be equivalent to subtracting the mean from each vote …
(Maybe you have to count the zero votes in the calculation)
Will that be obvious while the person is voting? (I guess this may be more of a UI question than a voting-theory question)
I can imagine, similar to how we have a button for ‘re-order the posts’, we could have a button for ‘normalise my votes’.
Still seems fairly confusing to me. Like, the act of futzing around with your vote totals shouldn’t give you the (temporary) impression that you can get more points in a way that can’t actually get you more points.
The whole point is that it does give you more points, I think.
Yeah, I actually found this to be a pretty annoying artifact of the voting system once I realized it.
Yeah! I also noticed this when looking over the results; there was a paragraph on it in the OP that I cut.