I think this might be another way the karma system continues to be a feature of a map that doesn’t necessarily represent the territory.
High karma seems to me to be about being on board with LW orthodoxy, articulating that clearly in posts and comments and supporting the aims of the group.
I guess there is nothing wrong with that...though that is exactly how “karma points” are handed out in the church.
This is basically saying, “People will listen to your opinion X amount more inside LW for doing this volunteer work.” It is the exact same way in the church for those who evangelize a lot or display some other evidence of their commitment to the group.
The problem, I think, is that proofreading e-books or evangelizing the group’s message is not the slightest bit related to having a view that is in line with reality which is, as I understood it, one of the main aims of rationality.
High karma seems to me to be about being on board with LW orthodoxy, articulating that clearly in posts and comments and supporting the aims of the group.
The criticism you proffer has been made independently several times. The usual answer is that cogent, intelligent criticism that shows familiarity with the premises and conclusions of LW “orthodoxy” (and clearly locates its criticism as a problem with either the plausibility of a premise or the validity of an argument, or both) tends to get upvoted. You just have to show you’ve done your homework.
The problem is that exactly what qualifies as “homework” is determined by the in-group. And, as I said, this is exactly how it works in the church.
Nevermind that though. My point was really that karma isn’t tied to who is right, it is tied to who we like or who furthers our preferential ends. This karma-for-work deal is another example of that.
If karma is a popularity system, then fine. But there seems to be a lingering sentiment it is more about rationality, and how a given comment or commenter is in line with it. That’s not the case when you are giving people points to do tasks.
If we feel that we want to keep track of instrumentally useful contributions to the community but also want karma to remain a more-or-less pure representation of the reception of a user’s comments, then the obvious way to reflect that seems to be to create a metric other than karma to represent the former. This might also be a useful way of reflecting certain actions that have traditionally been rewarded by means of upvotes on comments bragging about them, like posting survey responses or donating to CFAR or MIRI. Granted, the development resources for this aren’t likely to appear in the near term.
Elsewhere in these comments I’ve mentioned the XP metric that the Everything2 community created to fill a karma-like niche. It was after my time, but I’m told they ended up creating a “GP” metric (RPG metaphor, yes) that worked similarly to this.
My point was really that karma isn’t tied to who is right, it is tied to who we like or who furthers our preferential ends.
Barring an objective method for telling what arguments are right, this is the way any human-run evaluation system (including e.g. formal peer review or university grades) has to work. You can try to eliminate the “who we like” part by trying to blind the identities of the people in question, but since one cannot assess degree-of-correctness directly, one has to rely on some other criteria, e.g. the extent to which the comment seems to take previous work into account. And those other criteria and their parameters, like what counts as “previous work”, are ultimately set by an in-group consensus. (I felt that James Paul Gee had a particularly good elaboration of this.)
But I do agree that awarding karma for work distances karma from correctness even further than would be necessary. (Not sure whether it’s a bad thing, though.)
At the very low end, a certain amount of karma must be earned for certain functionality to become available to a user account. But past a certain point—somewhere in the 200-500 range—more karma just doesn’t matter very much when assessing a user account. After that point, the meaning of a karma score attaches much more strongly to specific comments than to users.
The corollary is that the karma reward will incent newcomers a lot more than old hands. I don’t have a problem with that.
My point was really that karma isn’t tied to who is right, it is tied to who we like or who furthers our preferential ends.
The “who we like” part seems to be how some people operate, especially vis-a-vis mass downvoting. I view that as counterproductive. What the phrase “furthers our preferential ends” denotes is not clear to me; if it means something like an upvote is meant to signal “I want to see more like this comment” and a downvote the opposite, then my understanding of LW consensus is, yes, that’s the idea.
But past a certain point—somewhere in the 200-500 range—more karma just doesn’t matter very much when assessing a user account.
Huh? If karma isn’t very meaningful past a certain count, why keep track of it at all? Why not just call everybody who reached 500 karma points “vetted” and leave it at that? (I suspect the answer is that karma does matter to some significant portion of the people here, but I’m open to hearing why you think otherwise.)
The “who we like” part seems to be how some people operate, especially vis-a-vis mass downvoting. I view that as counterproductive. What the phrase “furthers our preferential ends” denotes is not clear to me
“Who we like” could include mass downvoting. More than it, it involves applause lights hanging above certain members who espouse popular views or who have done something, apart from making a rational comment, to garner favor in the group. This could be proofreading a LW-approved text, organizing a meetup, etc., etc.
The key characteristic of this earned karma is that is has zero to do making a direct contribution to a more accurate map.
It’s is just interesting to me because it’s a lot like the church from whence I came. Signaling devotion to the cause becomes more important than being right about the merits of the cause.
LW is a community specifically committed to the mission of “refining rationality” and, therefore, (you’d think) making sure things like karma systems work to incentivize members toward that end. I don’t see that happening.
LW is the best blog/forum I’ve seen—virtually troll free, consistent flow of interesting articles, thoughtful & well-written comments, lots of people waaaay smarter than me, etc. That is part of why it is so interesting that the karma system seems so...weak.
Huh? If karma isn’t very meaningful past a certain count, why keep track of it at all? Why not just call everybody who reached 500 karma points “vetted” and leave it at that? (I suspect the answer is that karma does matter to some significant portion of the people here, but I’m open to hearing why you think otherwise.)
I didn’t say it was strictly meaningless or negligibly meaningful.
The reasons for not having an explicit “vetted” status in lieu of accumulating karma are to a certain extent historical—it wasn’t thought of when the LW karma system was implemented, since that was adapted whole-hog from the Reddit codebase. I think the reason why such a change hasn’t been made in the meantime is three-fold: (i) it would obviate the “Top Contributor, 30 Days” status incentive (the most feasible way to top that list is to write highly upvoted front page articles), (ii) it would obviate any loss-aversion-motivated engagement induced by each account’s “karma in the last 30 days” score; and (iii) on general “if it ain’t broke don’t fix it” grounds.
It’s is just interesting to me because it’s a lot like the church from whence I came. Signaling devotion to the cause becomes more important than being right about the merits of the cause.
Yes, churches win at creating socially cohesive communities. If your complaint is that that karma system induces undue (in your view) social cohesiveness, my response is, “feature, not bug”. (Here’s some LW canon on the usefulness and pitfalls of social cohesiveness.) If you suspect that the harmfulness of the system outweighs the usefulness, set some standards for harmfulness and usefulness and then collect some evidence for and against that hypothesis. Keep in mind that by design, the biggest rewards go to high-quality front page posts (like this one criticizing time spent kibitzing on LW).
I didn’t say it was strictly meaningless or negligibly meaningful.
Nor did I say you said that. You said this:
But past a certain point—somewhere in the 200-500 range—more karma just doesn’t matter very much when assessing a user account.
And I replied with this:
If karma isn’t very meaningful past a certain count, why keep track of it at all?
My apologies if you feel I rephrased you inaccurately, or missed your meaning.
The reasons for not having an explicit “vetted” status in lieu of accumulating karma are to a certain extent historical—it wasn’t thought of when the LW karma system was implemented, since that was adapted whole-hog from the Reddit codebase. I think the reason why such a change hasn’t been made in the meantime is three-fold: (i) it would obviate the “Top Contributor, 30 Days” status incentive (the most feasible way to top that list is to write highly upvoted front page articles), (ii) it would obviate any loss-aversion-motivated engagement induced by each account’s “karma in the last 30 days” score; and (iii) on general “if it ain’t broke don’t fix it” grounds.
To the system being historical: “That’s the way we’ve always done it” isn’t a very good reason for any policy or behavior to continue.
To (i) and (ii): Keep 30-day karma and ditch cumulative karma. No need to obviate anything.
To (iii): Begging the question.
Yes, churches win at creating socially cohesive communities. If your complaint is that that karma system induces undue (in your view) social cohesiveness, my response is, “feature, not bug”.
Social cohesive is fine, of course. I agree it is a feature. And it is great!
It (social cohesiveness) shouldn’t take priority over adherence to reality. When it does, it’s buggy. And it is what happens in the church. In my experience, they value “unity” over rationality. At LW, that is a clear no-no. (I agree it is a no-no. Make sure you are correct first; only then be unified. Lest dogma tends to ensue.)
If you suspect that the harmfulness of the system outweighs the usefulness, set some standards for harmfulness and usefulness and then collect some evidence for and against that hypothesis.
This is way outside my level of interest in, or commitment to, LW. I’ve given what I believe to be a reasonable criticism of the karma system (one that you mention has been independently noted many times) and made what I believe to be an accurate and helpful analogy (i.e. how “karma” works in the church).
It’s on the record for anyone who is interested to do with it what they’d like, or ignore it altogether.
Keep in mind that by design, the biggest rewards go to high-quality front page posts (like this one criticizing time spent kibitzing on LW).
One of my favorite posts. Though I would say reading and interacting on LW is really good for rationality novices—like me.
At some point, it does become a bit of an anti-rational engagment. For me, in this thread, that time is now.
What happens if old karma is only displayed as a percentage, rather than as a number? That way you know generally what the community thinks of their post quality, without more-prolific posters overwhelming less frequent posters?
LW is the best blog/forum I’ve seen—virtually troll free, consistent flow of interesting articles, thoughtful & well-written comments, lots of people waaaay smarter than me, etc. That is part of why it is so interesting that the karma system seems so…weak.
Are you sure that the other features you mention aren’t in part because of the “weak” karma system? Trolls get actively downvoted, for instance.
I’d imagine the existing system works pretty well towards several ends… troll deterence and the writing quality of posts/comments being a couple examples.
I think this might be another way the karma system continues to be a feature of a map that doesn’t necessarily represent the territory.
High karma seems to me to be about being on board with LW orthodoxy, articulating that clearly in posts and comments and supporting the aims of the group.
I guess there is nothing wrong with that...though that is exactly how “karma points” are handed out in the church.
This is basically saying, “People will listen to your opinion X amount more inside LW for doing this volunteer work.” It is the exact same way in the church for those who evangelize a lot or display some other evidence of their commitment to the group.
The problem, I think, is that proofreading e-books or evangelizing the group’s message is not the slightest bit related to having a view that is in line with reality which is, as I understood it, one of the main aims of rationality.
The criticism you proffer has been made independently several times. The usual answer is that cogent, intelligent criticism that shows familiarity with the premises and conclusions of LW “orthodoxy” (and clearly locates its criticism as a problem with either the plausibility of a premise or the validity of an argument, or both) tends to get upvoted. You just have to show you’ve done your homework.
The problem is that exactly what qualifies as “homework” is determined by the in-group. And, as I said, this is exactly how it works in the church.
Nevermind that though. My point was really that karma isn’t tied to who is right, it is tied to who we like or who furthers our preferential ends. This karma-for-work deal is another example of that.
If karma is a popularity system, then fine. But there seems to be a lingering sentiment it is more about rationality, and how a given comment or commenter is in line with it. That’s not the case when you are giving people points to do tasks.
If we feel that we want to keep track of instrumentally useful contributions to the community but also want karma to remain a more-or-less pure representation of the reception of a user’s comments, then the obvious way to reflect that seems to be to create a metric other than karma to represent the former. This might also be a useful way of reflecting certain actions that have traditionally been rewarded by means of upvotes on comments bragging about them, like posting survey responses or donating to CFAR or MIRI. Granted, the development resources for this aren’t likely to appear in the near term.
Elsewhere in these comments I’ve mentioned the XP metric that the Everything2 community created to fill a karma-like niche. It was after my time, but I’m told they ended up creating a “GP” metric (RPG metaphor, yes) that worked similarly to this.
badges
Barring an objective method for telling what arguments are right, this is the way any human-run evaluation system (including e.g. formal peer review or university grades) has to work. You can try to eliminate the “who we like” part by trying to blind the identities of the people in question, but since one cannot assess degree-of-correctness directly, one has to rely on some other criteria, e.g. the extent to which the comment seems to take previous work into account. And those other criteria and their parameters, like what counts as “previous work”, are ultimately set by an in-group consensus. (I felt that James Paul Gee had a particularly good elaboration of this.)
But I do agree that awarding karma for work distances karma from correctness even further than would be necessary. (Not sure whether it’s a bad thing, though.)
At the very low end, a certain amount of karma must be earned for certain functionality to become available to a user account. But past a certain point—somewhere in the 200-500 range—more karma just doesn’t matter very much when assessing a user account. After that point, the meaning of a karma score attaches much more strongly to specific comments than to users.
The corollary is that the karma reward will incent newcomers a lot more than old hands. I don’t have a problem with that.
The “who we like” part seems to be how some people operate, especially vis-a-vis mass downvoting. I view that as counterproductive. What the phrase “furthers our preferential ends” denotes is not clear to me; if it means something like an upvote is meant to signal “I want to see more like this comment” and a downvote the opposite, then my understanding of LW consensus is, yes, that’s the idea.
Huh? If karma isn’t very meaningful past a certain count, why keep track of it at all? Why not just call everybody who reached 500 karma points “vetted” and leave it at that? (I suspect the answer is that karma does matter to some significant portion of the people here, but I’m open to hearing why you think otherwise.)
“Who we like” could include mass downvoting. More than it, it involves applause lights hanging above certain members who espouse popular views or who have done something, apart from making a rational comment, to garner favor in the group. This could be proofreading a LW-approved text, organizing a meetup, etc., etc.
The key characteristic of this earned karma is that is has zero to do making a direct contribution to a more accurate map.
It’s is just interesting to me because it’s a lot like the church from whence I came. Signaling devotion to the cause becomes more important than being right about the merits of the cause.
LW is a community specifically committed to the mission of “refining rationality” and, therefore, (you’d think) making sure things like karma systems work to incentivize members toward that end. I don’t see that happening.
LW is the best blog/forum I’ve seen—virtually troll free, consistent flow of interesting articles, thoughtful & well-written comments, lots of people waaaay smarter than me, etc. That is part of why it is so interesting that the karma system seems so...weak.
I didn’t say it was strictly meaningless or negligibly meaningful.
The reasons for not having an explicit “vetted” status in lieu of accumulating karma are to a certain extent historical—it wasn’t thought of when the LW karma system was implemented, since that was adapted whole-hog from the Reddit codebase. I think the reason why such a change hasn’t been made in the meantime is three-fold: (i) it would obviate the “Top Contributor, 30 Days” status incentive (the most feasible way to top that list is to write highly upvoted front page articles), (ii) it would obviate any loss-aversion-motivated engagement induced by each account’s “karma in the last 30 days” score; and (iii) on general “if it ain’t broke don’t fix it” grounds.
Yes, churches win at creating socially cohesive communities. If your complaint is that that karma system induces undue (in your view) social cohesiveness, my response is, “feature, not bug”. (Here’s some LW canon on the usefulness and pitfalls of social cohesiveness.) If you suspect that the harmfulness of the system outweighs the usefulness, set some standards for harmfulness and usefulness and then collect some evidence for and against that hypothesis. Keep in mind that by design, the biggest rewards go to high-quality front page posts (like this one criticizing time spent kibitzing on LW).
Nor did I say you said that. You said this:
And I replied with this:
My apologies if you feel I rephrased you inaccurately, or missed your meaning.
To the system being historical: “That’s the way we’ve always done it” isn’t a very good reason for any policy or behavior to continue.
To (i) and (ii): Keep 30-day karma and ditch cumulative karma. No need to obviate anything.
To (iii): Begging the question.
Social cohesive is fine, of course. I agree it is a feature. And it is great!
It (social cohesiveness) shouldn’t take priority over adherence to reality. When it does, it’s buggy. And it is what happens in the church. In my experience, they value “unity” over rationality. At LW, that is a clear no-no. (I agree it is a no-no. Make sure you are correct first; only then be unified. Lest dogma tends to ensue.)
This is way outside my level of interest in, or commitment to, LW. I’ve given what I believe to be a reasonable criticism of the karma system (one that you mention has been independently noted many times) and made what I believe to be an accurate and helpful analogy (i.e. how “karma” works in the church).
It’s on the record for anyone who is interested to do with it what they’d like, or ignore it altogether.
One of my favorite posts. Though I would say reading and interacting on LW is really good for rationality novices—like me.
At some point, it does become a bit of an anti-rational engagment. For me, in this thread, that time is now.
Tap.
So people who don’t post on LW for a month or more become indistinguishable from newbies?
What happens if old karma is only displayed as a percentage, rather than as a number? That way you know generally what the community thinks of their post quality, without more-prolific posters overwhelming less frequent posters?
I am still not quite sure what is the problem we are trying to solve here.
What exactly do you hope to gain by screwing around with the karma system?
I like this suggestion.
Are you sure that the other features you mention aren’t in part because of the “weak” karma system? Trolls get actively downvoted, for instance.
I’d imagine the existing system works pretty well towards several ends… troll deterence and the writing quality of posts/comments being a couple examples.