LessWrong has a karma system, mostly based off of Reddit’s karma system, with some improvements and tweaks to it. I’ve thought a lot about more improvements to it, but one roadblock that I always run into when trying to improve the karma system, is that it actually serves a lot of different uses, and changing it in one way often means completely destroying its ability to function in a different way. Let me try to summarize what I think the different purposes of the karma system are:
Helping users filter content
The most obvious purpose of the karma system is to determine how long a post is displayed on the frontpage, and how much visibility it should get.
Being a social reward for good content
This aspect of the karma system comes out more when thinking about Facebook “likes”. Often when I upvote a post, it is more of a public signal that I value something, with the goal that the author will feel rewarded for putting their effort into writing the relevant content.
Creating common-knowledge about what is good and bad
This aspect of the karma system comes out the most when dealing with debates, though it’s present in basically any karma-related interaction. The fact that the karma of a post is visible to everyone, helps people establish common knowledge of what the community considers to be broadly good or broadly bad. Seeing a an insult downvoted, does more than just filter it out of people’s feeds, it also makes it so that anyone who stumbles accross it learns something about the norms of the community.
Being a low-effort way of engaging with the site
On lesswrong, Reddit and Facebook, karma is often the simplest action you can take on the site. This means its usually key for a karma system like that to be extremely simple, and not require complicated decisions, since that would break the basic engagement loop with the site.
Problems with alternative karma systems
Here are some of the most common alternatives to our current karma system, and how they perform on the above dimensions:
Eigenkarma as weighted by a set of core users
The basic idea here is that you try to signal-boost a small set of trusted users, by giving people voting power that is downstream from the initially defined set of users.
There are some problems with this. The first one is whether to assign any voting power to new users. If you don’t you remove a large part of the value of having a low-effort way of engaging with your site.
It also forces you to separate the points that you get on your content, from your total karma score, from your “karma-trust score” which introduces some complexity into the system. It also makes it so that increases in the points of your content, no longer neatly correspond to voting events, because the underlying reputation graph is constantly shifting and changing, making the social reward signal a lot weaker.
In exchange for this, you likely get a system that is better at filtering content, and probably has better judgement about what should be made common-knowledge or not.
Prediction-based system
I was talking with Scott Garrabrant today, who was excited about a prediction-based karma system. The basic idea is to just have a system that tries to do its best to predict what rating you are likely to give to a post, based on your voting record, the post, and other people’s votes.
In some sense this is what Youtube and Facebook are doing in their systems, though he was unhappy with the transparency of what they were doing.
The biggest sacrifice I see in creating this system, is the loss in the ability to create common knowledge, since now all votes are ultimately private, and the ability for karma to establish social norms, or just common knowledge about foundational facts that the community is built around, is greatly diminished.
I also think it diminishes the degree to which votes can serve as a social reward signal, since there is no obvious thing to inform the user of when their content got votes on. No number that went up or down, just a few thousand weights in some distant predictive matrix, or neural net.
Augmenting experts
A similar formulation to the eigenkarma system is the idea of trying to augment experts, by rewarding users in proportion to how successful they are at predicting how a trusted expert would vote, and then using that predicted expert’s vote as the reward signal. Periodically, you do query the trusted expert, and use that to calibrate and train the users who are trying to predict the expert.
This still allows you to build common-knowledge, and allows you to have effective reward signals (“simulated Eliezer upvoted your comment”), but does run into problems when it comes to being a low-effort way of engaging with the site. The operation of “what would person X think about this comment” is a much more difficult one than “did I like this comment?”, and as such might deter a large number of users from using your site.
What is the purpose of karma?
LessWrong has a karma system, mostly based off of Reddit’s karma system, with some improvements and tweaks to it. I’ve thought a lot about more improvements to it, but one roadblock that I always run into when trying to improve the karma system, is that it actually serves a lot of different uses, and changing it in one way often means completely destroying its ability to function in a different way. Let me try to summarize what I think the different purposes of the karma system are:
Helping users filter content
The most obvious purpose of the karma system is to determine how long a post is displayed on the frontpage, and how much visibility it should get.
Being a social reward for good content
This aspect of the karma system comes out more when thinking about Facebook “likes”. Often when I upvote a post, it is more of a public signal that I value something, with the goal that the author will feel rewarded for putting their effort into writing the relevant content.
Creating common-knowledge about what is good and bad
This aspect of the karma system comes out the most when dealing with debates, though it’s present in basically any karma-related interaction. The fact that the karma of a post is visible to everyone, helps people establish common knowledge of what the community considers to be broadly good or broadly bad. Seeing a an insult downvoted, does more than just filter it out of people’s feeds, it also makes it so that anyone who stumbles accross it learns something about the norms of the community.
Being a low-effort way of engaging with the site
On lesswrong, Reddit and Facebook, karma is often the simplest action you can take on the site. This means its usually key for a karma system like that to be extremely simple, and not require complicated decisions, since that would break the basic engagement loop with the site.
Problems with alternative karma systems
Here are some of the most common alternatives to our current karma system, and how they perform on the above dimensions:
Eigenkarma as weighted by a set of core users
The basic idea here is that you try to signal-boost a small set of trusted users, by giving people voting power that is downstream from the initially defined set of users.
There are some problems with this. The first one is whether to assign any voting power to new users. If you don’t you remove a large part of the value of having a low-effort way of engaging with your site.
It also forces you to separate the points that you get on your content, from your total karma score, from your “karma-trust score” which introduces some complexity into the system. It also makes it so that increases in the points of your content, no longer neatly correspond to voting events, because the underlying reputation graph is constantly shifting and changing, making the social reward signal a lot weaker.
In exchange for this, you likely get a system that is better at filtering content, and probably has better judgement about what should be made common-knowledge or not.
Prediction-based system
I was talking with Scott Garrabrant today, who was excited about a prediction-based karma system. The basic idea is to just have a system that tries to do its best to predict what rating you are likely to give to a post, based on your voting record, the post, and other people’s votes.
In some sense this is what Youtube and Facebook are doing in their systems, though he was unhappy with the transparency of what they were doing.
The biggest sacrifice I see in creating this system, is the loss in the ability to create common knowledge, since now all votes are ultimately private, and the ability for karma to establish social norms, or just common knowledge about foundational facts that the community is built around, is greatly diminished.
I also think it diminishes the degree to which votes can serve as a social reward signal, since there is no obvious thing to inform the user of when their content got votes on. No number that went up or down, just a few thousand weights in some distant predictive matrix, or neural net.
Augmenting experts
A similar formulation to the eigenkarma system is the idea of trying to augment experts, by rewarding users in proportion to how successful they are at predicting how a trusted expert would vote, and then using that predicted expert’s vote as the reward signal. Periodically, you do query the trusted expert, and use that to calibrate and train the users who are trying to predict the expert.
This still allows you to build common-knowledge, and allows you to have effective reward signals (“simulated Eliezer upvoted your comment”), but does run into problems when it comes to being a low-effort way of engaging with the site. The operation of “what would person X think about this comment” is a much more difficult one than “did I like this comment?”, and as such might deter a large number of users from using your site.
This is really good and I missed it until now. I vote for you making this a full-on post. I think it’s fine as is for that.