Stop displaying user’s Karma total, so that there is no numbers-go-up reward for posting lots of mediocre stuff, instead count the number of comments/posts in some upper quantile by Karma (which should cash out as something like 15+ Karma for comments). Use that number where currently Karma is used, like vote weights. (Also, display the number of comments below some negative threshold like −3 in the last few months.)
In some ways this sounds better than either my proposal or Raemon’s. But there is still the spurious upvoting issue, so a metric should be able to not get too excited about a few highly upvoted things.
FYI a similar I’ve thinking about is “you see the total karma of the user’s top ~20 comments/posts”, which you can initially improve by writing somewhat-good-comments but you’ll quickly max that out. That metric emphasizes “what was their best content like?”, your metric is something like “how much ‘at least pretty solid’ content do they have?” and I’m not sure which is better.
There are some viral posts (including on community drama) where (half of) everything gets unusually highly upvoted, compared to normal. So the inclination I get is to recalibrate thresholds according to post’s popularity and the current year, to count fewer such comments (that’s too messy, isn’t worth it, but other things should be robust to this effect). This is why I specifically proposed number of 15+ Karma comments, not their total Karma. Also, the total number still counts as some sort of “total contribution” as opposed to the less savory “user quality”.
I think avoiding the negative selection failure modes is an important point. I’m mulling over how to think about it.
Do you have a thing you’re imagining with “positive selection?” that you expect to work?
Stop displaying user’s Karma total, so that there is no numbers-go-up reward for posting lots of mediocre stuff, instead count the number of comments/posts in some upper quantile by Karma (which should cash out as something like 15+ Karma for comments). Use that number where currently Karma is used, like vote weights. (Also, display the number of comments below some negative threshold like −3 in the last few months.)
Something like an h-index might be better than a total.
In some ways this sounds better than either my proposal or Raemon’s. But there is still the spurious upvoting issue, so a metric should be able to not get too excited about a few highly upvoted things.
Mmm. Yeah something in that space makes sense.
FYI a similar I’ve thinking about is “you see the total karma of the user’s top ~20 comments/posts”, which you can initially improve by writing somewhat-good-comments but you’ll quickly max that out. That metric emphasizes “what was their best content like?”, your metric is something like “how much ‘at least pretty solid’ content do they have?” and I’m not sure which is better.
There are some viral posts (including on community drama) where (half of) everything gets unusually highly upvoted, compared to normal. So the inclination I get is to recalibrate thresholds according to post’s popularity and the current year, to count fewer such comments (that’s too messy, isn’t worth it, but other things should be robust to this effect). This is why I specifically proposed number of 15+ Karma comments, not their total Karma. Also, the total number still counts as some sort of “total contribution” as opposed to the less savory “user quality”.