Attention is a limited resource. I don’t have the time or interest to read every comment on LessWrong. So what is karma even for? I use the karma score for one simple yes-no question: “Is reading this worth my time?”.
Is displaying the number of upvotes minus downvotes really the best way to answer the question? Karma should not be about mere popularity, or path dependence based on whoever voted first. The weighting system is an improvement, but I think we can do better.
Display the estimated probability (as a percentage) that the post is worth my time.
Users will start with a reasonable personal karma score (say less that 50%), while established users with high karma on the old system will start with somewhat higher personal scores. Then the initial (prior) score of a post is the score of its author.
Users can then upvote or downvote posts, and this will be taken as Bayesian evidence about the quality of that post, shifting its score from its prior in the direction of the vote.
The probability that any given vote is accurate will be basted on the voters’ karma percentage. Those with high personal karma will be assumed to have better judgement (because they write high karma posts), so their vote will have more weight.
And finally, the user’s personal karma percentage will be adjusted from their prior by using the karma percentage of their posts as Bayesian evidence. This means that the personal karma percentage of a user is the estimated probability that the user’s next post is worth reading.
This is an interesting idea, although I’m not sure if the degree-of-(in)accuracy of the percentage would end up being more useful than the current weighted karma. (i.e, converting upvotes into a percentage-of-worth-it-ness seems like it’d have to go through several iterations before reaching something that was accurate enough to be better)
I’m not exactly sure what you mean by “iterations” here. Is it about getting enough votes? Or about what conversion function to use when grandfathering established users?
I think it would be possible to experiment with the current data. You have a record of the dates of all posts and votes so far. Rather than grandfathering in established users with some human-estimated prior, give everyone the same starting score, and try computing their current karma% from scratch. See if it gives you reasonable answers. See if it finds hidden gems. Try a different prior (with enough votes, any reasonable choice should get similar results). This won’t answer questions about incentives, but it will give you a good comparison to the current unbounded karma system.
Bayesian Karma
Attention is a limited resource. I don’t have the time or interest to read every comment on LessWrong. So what is karma even for? I use the karma score for one simple yes-no question: “Is reading this worth my time?”.
Is displaying the number of upvotes minus downvotes really the best way to answer the question? Karma should not be about mere popularity, or path dependence based on whoever voted first. The weighting system is an improvement, but I think we can do better.
Display the estimated probability (as a percentage) that the post is worth my time.
Users will start with a reasonable personal karma score (say less that 50%), while established users with high karma on the old system will start with somewhat higher personal scores. Then the initial (prior) score of a post is the score of its author.
Users can then upvote or downvote posts, and this will be taken as Bayesian evidence about the quality of that post, shifting its score from its prior in the direction of the vote.
The probability that any given vote is accurate will be basted on the voters’ karma percentage. Those with high personal karma will be assumed to have better judgement (because they write high karma posts), so their vote will have more weight.
And finally, the user’s personal karma percentage will be adjusted from their prior by using the karma percentage of their posts as Bayesian evidence. This means that the personal karma percentage of a user is the estimated probability that the user’s next post is worth reading.
This is an interesting idea, although I’m not sure if the degree-of-(in)accuracy of the percentage would end up being more useful than the current weighted karma. (i.e, converting upvotes into a percentage-of-worth-it-ness seems like it’d have to go through several iterations before reaching something that was accurate enough to be better)
I’m not exactly sure what you mean by “iterations” here. Is it about getting enough votes? Or about what conversion function to use when grandfathering established users?
I think it would be possible to experiment with the current data. You have a record of the dates of all posts and votes so far. Rather than grandfathering in established users with some human-estimated prior, give everyone the same starting score, and try computing their current karma% from scratch. See if it gives you reasonable answers. See if it finds hidden gems. Try a different prior (with enough votes, any reasonable choice should get similar results). This won’t answer questions about incentives, but it will give you a good comparison to the current unbounded karma system.