Well, to start with I wanted to see if others agree that a problem exists here. If most people are satisfied with how karma is working in these cases, then there is not much point in me spending a lot of time writing out long explanations and justifications, and trying to find solutions. So at this stage, I’m basically saying “This looks wrong to me. What do you think?” I think I did give some explanations and justifications, but I accept that more are needed if an eventual change to the karma system is to be made.
Ok, as one data point, I don’t see a particular problem here. The higher rated posts in your examples deserved higher ratings in my opinion. Karma mostly functions as I expect it to function.
Thanks, but can you explain why you think people who post wrong arguments deserve to get more karma than those who correct the wrong arguments? Suppose I thought of uninverted’s argument, but then realized that it’s wrong, so I don’t post my original argument, and instead correct him when he posts his. I end up with less karma than if I hadn’t spent time thinking things through and realizing the flaw in my reasoning. Why do we want to discourage “less wrong” thinking in this way?
It seems to me that the way karma works now encourages people to think up arguments that support the majority view and then post them as soon as they can without thinking things through. Why is this good, or “expected”?
First, I think you’re missing a karma pattern that I’ve noticed which is that the first post in a thread often gets more extreme votes (scores of greater absolute magnitude) than subsequent posts. I imagine this is because more people read the earlier posts in a thread and interest/readership drops off the deeper the nesting gets. I don’t see any simple way to ‘fix’ that—it has the potential to be gamed but I don’t think gaming the system in that respect is a major problem here.
Second I don’t think karma strictly reflects ‘correctness’ of arguments, nor do I think it necessarily should. People award karma for attributes other than correctness. For example I imagine some of the upvotes on uninverted’s “But I don’t want to be a really big integer!” comment were drive-by upvotes for an amusing remark. Some of those upvoters won’t have stayed for the followup discussion, others might have awarded more karma for pithy and amusing than accurate but dry. I think points-for-humour is as likely an explanation here as points-for-majority-opinion. Maybe you don’t think karma should be awarded for attributes other than correctness. If so, go ahead and bring it up and see what the rest of the community thinks.
As a side note, I think you probably shouldn’t have chosen a thread where you were a participant as an example. It gives the slight impression that your real complaint is that uninverted got more brownie points than you even though you were right and it’s just not fair. If I didn’t recognize your username as a regular and generally high-value contributor I might not have given you the benefit of the doubt on that.
As a side note, I think you probably shouldn’t have chosen a thread where you were a participant as an example. It gives the slight impression that your real complaint is that uninverted got more brownie points than you even though you were right and it’s just not fair.
I was given that impression somewhat but then on reflection I realized a more likely prompt for the Wei’s frustration was the Nelson/Komponisto/Knox affair. Not wanting to bring that issue up yet again, he chose some other similar examples that didn’t come with as much baggage. That one of them was his own was just unfortunate.
That would presumably require a fairly different rating system, under the current system median karma would mean posts could only ever score −1, 0 or 1. That doesn’t seem like an improvement to me.
the first post in a thread often gets more extreme votes (scores of greater absolute magnitude) than subsequent posts.
In that the main effect of karma on the reader is to sort posts, comparing scores at different levels of nesting is irrelevant. It is a very biased heuristic to read only comments at, say, karma > 5. I don’t know if anyone uses this heuristic. A lot of people only read comments at karma > −4, which probably has a similar bias, but I wouldn’t worry about it.
because more people read the earlier posts in a thread and interest/readership drops off the deeper the nesting gets.
People who post replies are serving a smaller audience than people who post higher level comments. It is possibly good that they are proportionately rewarded with karma.
I don’t see any simple way to ‘fix’ that—it has the potential to be gamed but I don’t think gaming the system in that respect is a major problem here
It’s not so much a potential to be gamed, as encouraging people to post without thinking things through, as well as misleading readers as to which arguments are correct. I don’t know if there is a simple fix or not, but if we can agree that it’s a problem, then we can at least start thinking about possible solutions.
Maybe you don’t think karma should be awarded for attributes other than correctness. If so, go ahead and bring it up and see what the rest of the community thinks.
In a case where a comment is both funny and incorrect, I think we should prioritize the correctness. After all, this is “Less Wrong”, not “Less Bored”.
If I didn’t recognize your username as a regular and generally high-value contributor I might not have given you the benefit of the doubt on that.
I was too lazy to find another example, and counting on the benefit of the doubt. :)
ETA: Also, I think being upvoted for supporting the majority opinion is clearly a strong reason for what happened, especially in Exhibit B, where the comment is deep in the middle of a thread, and has no humor value.
It’s not so much a potential to be gamed, as encouraging people to post without thinking things through, as well as misleading readers as to which arguments are correct.
I’m not sure that’s true. As I originally said, the first comment in a thread often gets karma of greater absolute magnitude. Bad posts get voted down more harshly as well as good posts getting voted up more. I think the higher readership for top level comments explains this. It means that from a karma-gaming perspective posting a top level comment is only a good move if you are confident it will be received positively.
In a case where a comment is both funny and incorrect, I think we should prioritize the correctness.
How about other attributes not directly related to correctness? How should niceness be judged relative to correctness for example?
I still think there’s a conflict between you wanting people to give more upvotes to things that are correct but fewer to things that ‘agree with the majority opinion’. I don’t think people upvote because a comment ‘agrees with the majority opinion’, they upvote because a comment agrees with their opinion. That tends to produce greater upvotes for the majority view. In your second example I think the greater upvotes for James Andrix reflect the fact that he is more correct. Your real complaint seems to be that the majority opinion is wrong on this issue. The best way to fix that is to make a better argument for the other view, not to complain that people are failing to recognize correct arguments and upvote them.
Ok, as one data point, I don’t see a particular problem here. The higher rated posts in your examples deserved higher ratings in my opinion. Karma mostly functions as I expect it to function.
Thanks, but can you explain why you think people who post wrong arguments deserve to get more karma than those who correct the wrong arguments?
Mattnewport did not claim or otherwise imply that he thought that.
I agree with you, but I think it has to do with the way people vote (mainly voting in favor of things they agree with and against things they disagree with), and with which comments are read by whom. In other words, changing the karma system probably is not a way to address it: people have to change their behavior.
It seems a little inconsistent to expect people to vote up things for being correct but not to vote up things simply because they agree with them. I tend to agree with things I think are correct and disagree with things I think are incorrect.
I think Wei Dai was saying that people should vote up strong arguments, even if they disagree with the conclusion. I do this sometimes, and I think it’s a good thing to do.
Well, to start with I wanted to see if others agree that a problem exists here. If most people are satisfied with how karma is working in these cases, then there is not much point in me spending a lot of time writing out long explanations and justifications, and trying to find solutions. So at this stage, I’m basically saying “This looks wrong to me. What do you think?” I think I did give some explanations and justifications, but I accept that more are needed if an eventual change to the karma system is to be made.
Ok, as one data point, I don’t see a particular problem here. The higher rated posts in your examples deserved higher ratings in my opinion. Karma mostly functions as I expect it to function.
Thanks, but can you explain why you think people who post wrong arguments deserve to get more karma than those who correct the wrong arguments? Suppose I thought of uninverted’s argument, but then realized that it’s wrong, so I don’t post my original argument, and instead correct him when he posts his. I end up with less karma than if I hadn’t spent time thinking things through and realizing the flaw in my reasoning. Why do we want to discourage “less wrong” thinking in this way?
It seems to me that the way karma works now encourages people to think up arguments that support the majority view and then post them as soon as they can without thinking things through. Why is this good, or “expected”?
First, I think you’re missing a karma pattern that I’ve noticed which is that the first post in a thread often gets more extreme votes (scores of greater absolute magnitude) than subsequent posts. I imagine this is because more people read the earlier posts in a thread and interest/readership drops off the deeper the nesting gets. I don’t see any simple way to ‘fix’ that—it has the potential to be gamed but I don’t think gaming the system in that respect is a major problem here.
Second I don’t think karma strictly reflects ‘correctness’ of arguments, nor do I think it necessarily should. People award karma for attributes other than correctness. For example I imagine some of the upvotes on uninverted’s “But I don’t want to be a really big integer!” comment were drive-by upvotes for an amusing remark. Some of those upvoters won’t have stayed for the followup discussion, others might have awarded more karma for pithy and amusing than accurate but dry. I think points-for-humour is as likely an explanation here as points-for-majority-opinion. Maybe you don’t think karma should be awarded for attributes other than correctness. If so, go ahead and bring it up and see what the rest of the community thinks.
As a side note, I think you probably shouldn’t have chosen a thread where you were a participant as an example. It gives the slight impression that your real complaint is that uninverted got more brownie points than you even though you were right and it’s just not fair. If I didn’t recognize your username as a regular and generally high-value contributor I might not have given you the benefit of the doubt on that.
I was given that impression somewhat but then on reflection I realized a more likely prompt for the Wei’s frustration was the Nelson/Komponisto/Knox affair. Not wanting to bring that issue up yet again, he chose some other similar examples that didn’t come with as much baggage. That one of them was his own was just unfortunate.
Upvoted for the correct inference. This is definitely one of those rare times when laziness failed to pay off. :)
Median karma would de-emphasize number of voters and put greater emphasis on the score they assigned.
That would presumably require a fairly different rating system, under the current system median karma would mean posts could only ever score −1, 0 or 1. That doesn’t seem like an improvement to me.
Yes; I imagine a range from 0⁄5 to 5⁄5 as per eg Amazon book rating sites. One problem with this however is that people don’t use the whole range.
In that the main effect of karma on the reader is to sort posts, comparing scores at different levels of nesting is irrelevant. It is a very biased heuristic to read only comments at, say, karma > 5. I don’t know if anyone uses this heuristic. A lot of people only read comments at karma > −4, which probably has a similar bias, but I wouldn’t worry about it.
People who post replies are serving a smaller audience than people who post higher level comments. It is possibly good that they are proportionately rewarded with karma.
It’s not so much a potential to be gamed, as encouraging people to post without thinking things through, as well as misleading readers as to which arguments are correct. I don’t know if there is a simple fix or not, but if we can agree that it’s a problem, then we can at least start thinking about possible solutions.
In a case where a comment is both funny and incorrect, I think we should prioritize the correctness. After all, this is “Less Wrong”, not “Less Bored”.
I was too lazy to find another example, and counting on the benefit of the doubt. :)
ETA: Also, I think being upvoted for supporting the majority opinion is clearly a strong reason for what happened, especially in Exhibit B, where the comment is deep in the middle of a thread, and has no humor value.
I’m not sure that’s true. As I originally said, the first comment in a thread often gets karma of greater absolute magnitude. Bad posts get voted down more harshly as well as good posts getting voted up more. I think the higher readership for top level comments explains this. It means that from a karma-gaming perspective posting a top level comment is only a good move if you are confident it will be received positively.
How about other attributes not directly related to correctness? How should niceness be judged relative to correctness for example?
I still think there’s a conflict between you wanting people to give more upvotes to things that are correct but fewer to things that ‘agree with the majority opinion’. I don’t think people upvote because a comment ‘agrees with the majority opinion’, they upvote because a comment agrees with their opinion. That tends to produce greater upvotes for the majority view. In your second example I think the greater upvotes for James Andrix reflect the fact that he is more correct. Your real complaint seems to be that the majority opinion is wrong on this issue. The best way to fix that is to make a better argument for the other view, not to complain that people are failing to recognize correct arguments and upvote them.
Mattnewport did not claim or otherwise imply that he thought that.
I agree with you, but I think it has to do with the way people vote (mainly voting in favor of things they agree with and against things they disagree with), and with which comments are read by whom. In other words, changing the karma system probably is not a way to address it: people have to change their behavior.
Yes, I agree, and by “karma system” I meant to include how people think they should vote.
It seems a little inconsistent to expect people to vote up things for being correct but not to vote up things simply because they agree with them. I tend to agree with things I think are correct and disagree with things I think are incorrect.
I think Wei Dai was saying that people should vote up strong arguments, even if they disagree with the conclusion. I do this sometimes, and I think it’s a good thing to do.