I like the idea of using the Open Thread for testing new karma systems.
Adding multidimensionalilty to it certainly seems like a good idea. In my experience, karma scores on comments seem to be correlated not just to quality of content but also to how well it aligns with the community narrative, to entertainment value, to the prior status of the commenter, and even to the timing of the comment relative to that of the post. Disentangling these would be helpful.
But then, what is it we really want karma to represent? If community members are not vigilant in how we rate things, the karma system is ripe for Goodharting. It’s easy to feel compelled to try whatever it takes to get that karma counter to increment.
In my opinion, karma ought to represent how much a comment or post should be seen by other members of the community, in terms of how useful it is for promoting good ideas specifically or rational thinking/behavior generally. Upvotes/downvotes are only (somewhat weak) Bayesian evidence for or against this.
I like the idea of using the Open Thread for testing new karma systems.
Adding multidimensionalilty to it certainly seems like a good idea. In my experience, karma scores on comments seem to be correlated not just to quality of content but also to how well it aligns with the community narrative, to entertainment value, to the prior status of the commenter, and even to the timing of the comment relative to that of the post. Disentangling these would be helpful.
But then, what is it we really want karma to represent? If community members are not vigilant in how we rate things, the karma system is ripe for Goodharting. It’s easy to feel compelled to try whatever it takes to get that karma counter to increment.
In my opinion, karma ought to represent how much a comment or post should be seen by other members of the community, in terms of how useful it is for promoting good ideas specifically or rational thinking/behavior generally. Upvotes/downvotes are only (somewhat weak) Bayesian evidence for or against this.