I’m quite curious how this ordering correlated with the original LessWrong Karma of each post, if that analysis hasn’t been done yet. Perhaps I’d be more curious to better understand what a great ordering would be. I feel like there are multiple factors taken into account when voting, and it’s also quite possible that the userbase represents multiple clusters that would have distinct preferences.
The “Click Here If You Would Like A More Comprehensive Vote Data Spreadsheet” link includes both vote totals and karma, making it easy to calculate the correlation using Google Sheet’s CORRELATE function. Pearson correlation between karma and vote count is 0.355, or if we throw away the outlier of Affordance Widths that was heavily downvoted due to its author, 0.425.
Interesting. From the data, it looks like there’s a decent linear correlation up to around 150 Karma or so, and then after that the correlation looks more nebulous.
That seems like weak evidence of karma info-cascades: posts with more karma get more upvotes *simply because* they have more karma, in a way which ultimately doesn’t correlate with their “true value” (as measured by the review process).
Potential mediating causes include users being anchored by karma, or more karma causing a larger share of the attention of the userbase (due to various sorting algorithms).
It was 1st June 2018 that we built strong/weak upvotes—before then you had to always vote your max strength. I could imagine that being responsible for the apparent info-cascades in very popular post.
If voters are at all consistent, you’d expect at lease some positive correlation because the same factors that made them upvote for karma also made upvote for the Review.
Beyond that, I’m guessing people voted for the posts they’d read, and people would have read higher karma posts more often since they get more exposure, e.g. sticking around the Latest Posts list for longer.
I’m quite curious how this ordering correlated with the original LessWrong Karma of each post, if that analysis hasn’t been done yet. Perhaps I’d be more curious to better understand what a great ordering would be. I feel like there are multiple factors taken into account when voting, and it’s also quite possible that the userbase represents multiple clusters that would have distinct preferences.
The “Click Here If You Would Like A More Comprehensive Vote Data Spreadsheet” link includes both vote totals and karma, making it easy to calculate the correlation using Google Sheet’s CORRELATE function. Pearson correlation between karma and vote count is 0.355, or if we throw away the outlier of Affordance Widths that was heavily downvoted due to its author, 0.425.
Scatterplots with “Affordance Widths” removed:
And it’s even larger (r = −0.46) between amount of karma and ranking in the vote.
That’s an interesting measure, let’s plot that too. (Ranks reversed so that rank 1 is represented as 74, rank 2 as 73, and so on.)
Interesting. From the data, it looks like there’s a decent linear correlation up to around 150 Karma or so, and then after that the correlation looks more nebulous.
That seems like weak evidence of karma info-cascades: posts with more karma get more upvotes *simply because* they have more karma, in a way which ultimately doesn’t correlate with their “true value” (as measured by the review process).
Potential mediating causes include users being anchored by karma, or more karma causing a larger share of the attention of the userbase (due to various sorting algorithms).
It was 1st June 2018 that we built strong/weak upvotes—before then you had to always vote your max strength. I could imagine that being responsible for the apparent info-cascades in very popular post.
If voters are at all consistent, you’d expect at lease some positive correlation because the same factors that made them upvote for karma also made upvote for the Review.
Beyond that, I’m guessing people voted for the posts they’d read, and people would have read higher karma posts more often since they get more exposure, e.g. sticking around the Latest Posts list for longer.
Correlation looks good all the way except for the three massive outliers at the top of the rankings.