On what basis will free people vote on an idea they disagree with but that is explained well?
A hilarious but unrelated pun?
A brilliant comment on a post that has nothing to do with it?
A valid point by a well known troll?
I can immediately answer that the valid point by the troll should be voted up, and it seems that the disagreed-with idea that is explained well should at least not be voted down.
It seems like the only criterion for the rating of comment/post be the degree to which it contributes to healthy discussion (well-explained, on-topic, not completely stupid). However, there is an strong tendency for people to vote comments based on whether they disagree with them or not, which is very bad for healthy discussion. It discourages new ideas and drives away visitors with differing opinions when they see a page full of highly rated comments for a particular viewpoint (cf. reddit).
The feature I would recommend most for this website is a dual voting feature: one vote up/down for the quality of the post/comment, and one for whether you agree or disagree with it. This would allow quality, disagreeable comments to float to the top while allowing everyone to satisfy their urge to express their opinion. It also would force people to make a cognitive distinction between the two categories.
Even people like me who try to base their ratings independent of their agreement with the comment are biased in their assessment of the quality. It would be very healthy to read a comment you agree with and would normally upvote (because your quality standards have been biased downward) only to see that a large fraction of the community finds the argument poor.
Incidentally, you might allow voting for humor or on-topic-ness so that people can (say) still be funny every once in a while without directly contributing to the current discussion per se.
(Sorry that was so long. It was something I had been thinking about for awhile.)
I disagree, because I see these factors as necessarily closely connected, in any person’s mind. I rate not quality of prose, but quality of communicated idea, as it comes through. If I think that the idea is silly, I rate it down. If the argument moves me, communicating a piece of knowledge that I at least give a chance of changing my understanding of something, then the message was valuable. It doesn’t matter whether the context was to imply a conclusion I agree or disagree with, it only matters whether the idea contributes something to my understanding.
If agreement votes aren’t going to be used, why not do away with them altogether and just use the current system to vote based on quality only? True comments are higher quality than false comments so agreement should factor into quality judgments anyway.
If agreement votes aren’t going to be used, why not do away with them altogether and just use the current system to vote based on quality only?
I like Jess’s proposal because I think it has a better chance of working in practice. Most of us, I think, do want to express agreement / disagreement, and I think separating it out into a separate vote would work better with real humans’ cognitive systems than relying on people following an explicit instruction to ignore one of their motivations. [Yes, I would like to see a study testing this assumption somehow, but in the meantime, that’s the prediction my subjective probability is going into...]
Besides, I would find the agree/disagree info interesting. And I think it probably reduces “me too” posts. And the info presumably could be used for the “most controversial” page.
(edited: s/separate out their motivations/ignore one of their motivations/)
I think other people have said it, but Slashdot has one of the best commenting structures around.
Different values for different categories (funny, insightful etc....), anonymous posting, reputation, very clear thread structure. All sorts of fun stuff
Because quality and truth are separate judgments in practice, and forcing them to be conflated into a single scale is losing information. To the extent that truth is positively correlated with quality this will fall out automatically: highly truthy posts will tend to have high quality. Low quality and high truth are not opposites.
I agree it’s losing information, but that’s something you have to weigh against the inconvenience of multiple dimensions. To the extent that truth is positively correlated with quality you’re just making people click twice, and I suspect clicks are a limited resource.
As I see it the voting system is there to put comments in a convenient order and remove the really bad ones from sight, not to provide opinion poll information.
That’s exactly the point: voting is supposed to put comments in order according to quality, so that you can read the worthwhile comments in a reasonable time. My claim is that the current voting system will not do this well at all and that a dual voting system will be better. (That second bit is just a guess). The opinion poll information is just a nice side effect.
OK, so according to you and Benja the point is to have the agree/disagree buttons there mainly as a lightning rod to prevent agreement from affecting quality votes. That’s a good point, but I wonder if it’s worth it and if there are better ways to accomplish the same thing.
I also wonder if there should be a button labeled “malevolent cantaloupe” so the unserious people will click on that instead of voting.
I’m not sure this is obviously right. I would probably insist upon some usability study to determine how people actually use such features. Of course, if the cost is low such a study could just be implementing them and seeing how it works.
I imagine there’s a name for this cognitive bias, but I’ve noticed well-informed folks tend to think agreeable opinions are better-argued, and less agreeable ones are worse-argued (probably a species of confirmation bias).
For example, someone posting against physicalism might get downvoted quickly by people who say “but they didn’t even consider Dennett’s response to this premise”. But they might not have the same objections on-hand to an unsound argument in favor of physicalism.
I’d prefer a clear explanation of intended semantics of voting, linked to on “About” page, and posted one of these days on the front page to get anyone’s attention and users’ suggestions.
It might also be good to stick a reminder of what up-voting is intended to mean right next to the up-vote and down-vote buttons. Or to change the names: instead of “vote up” and “vote down”, perhaps something like “high-quality discussion” and “low-quality discussion”.
How about buttons “High quality”, “Low quality”, “Accurate”, “Inaccurate”. We’re increasing options here, but there’s probably a nice way to design the interface to reduce the cognitive load.
Using the word “vote” seems broken here more generally—we aren’t implementing some democratic process, we’re aggregating judgments (read: collecting evidence) across a population.
“High quality” / “Low quality” has good brevity, but for myself I’m still tempted to blend in agreement/disagreement with my ratings when I picture those words—to regard comments I disagree with as “low quality”. If we could have the question “Does this add to or subtract from the conversation?” surrounded by up/down arrows (or by “adds” / “subtracts”), I imagine myself voting better.
For example, I just up-voted James Andrix’s and Kurige’s comments about their religious beliefs.
I up-voted the comments because they’re good data, I’m glad the commenters shared it, and it looks like stuff more eyes should look at within the thread. But I hesitated, because “up-voting” gives the appearance of agreement. Rating Kurige’s comment “high quality” feels a bit similar, like calling it “high quality reasoning”. But clicking up-arrow next to the question “Does this add to the conversation?” would feel obvious, to me in this case.
A second question about the semantics of voting: should I be up-voting all good posts regardless of score, and down-voting all bad posts regardless of score, or should I be voting to correct points-numbers that are misaligned with post quality?
Upon reflection, I’d say we should be voting to correct points-numbers that are misaligned with post quality. Otherwise, if people continue to up-vote more posts than they down-vote, comments will accumulate more and more points the older they get, and a setting like “show me all posts above 3” won’t be meaningful across threads or comment-ages.
Added: The “odd social dynamics” point is good. I’ll follow Eliezer and thomblake here.
I would actually say that voting to correct post-quality would lead to some odd social dynamics. I’ll vote up if something is negative for no reason, but otherwise I’ll vote my opinion, not corrections—unless something seems really out of line.
I’d rather have the sum of people’s individual components then see everyone trying to correct everyone else’s voting.
On what basis will free people vote on an idea they disagree with but that is explained well? A hilarious but unrelated pun? A brilliant comment on a post that has nothing to do with it? A valid point by a well known troll?
I can immediately answer that the valid point by the troll should be voted up, and it seems that the disagreed-with idea that is explained well should at least not be voted down.
It seems like the only criterion for the rating of comment/post be the degree to which it contributes to healthy discussion (well-explained, on-topic, not completely stupid). However, there is an strong tendency for people to vote comments based on whether they disagree with them or not, which is very bad for healthy discussion. It discourages new ideas and drives away visitors with differing opinions when they see a page full of highly rated comments for a particular viewpoint (cf. reddit).
The feature I would recommend most for this website is a dual voting feature: one vote up/down for the quality of the post/comment, and one for whether you agree or disagree with it. This would allow quality, disagreeable comments to float to the top while allowing everyone to satisfy their urge to express their opinion. It also would force people to make a cognitive distinction between the two categories.
Even people like me who try to base their ratings independent of their agreement with the comment are biased in their assessment of the quality. It would be very healthy to read a comment you agree with and would normally upvote (because your quality standards have been biased downward) only to see that a large fraction of the community finds the argument poor.
Incidentally, you might allow voting for humor or on-topic-ness so that people can (say) still be funny every once in a while without directly contributing to the current discussion per se.
(Sorry that was so long. It was something I had been thinking about for awhile.)
I disagree, because I see these factors as necessarily closely connected, in any person’s mind. I rate not quality of prose, but quality of communicated idea, as it comes through. If I think that the idea is silly, I rate it down. If the argument moves me, communicating a piece of knowledge that I at least give a chance of changing my understanding of something, then the message was valuable. It doesn’t matter whether the context was to imply a conclusion I agree or disagree with, it only matters whether the idea contributes something to my understanding.
This makes… quite a lot of sense, actually. And of course the posts would be sorted by quality votes, not agreement votes.
If agreement votes aren’t going to be used, why not do away with them altogether and just use the current system to vote based on quality only? True comments are higher quality than false comments so agreement should factor into quality judgments anyway.
I like Jess’s proposal because I think it has a better chance of working in practice. Most of us, I think, do want to express agreement / disagreement, and I think separating it out into a separate vote would work better with real humans’ cognitive systems than relying on people following an explicit instruction to ignore one of their motivations. [Yes, I would like to see a study testing this assumption somehow, but in the meantime, that’s the prediction my subjective probability is going into...]
Besides, I would find the agree/disagree info interesting. And I think it probably reduces “me too” posts. And the info presumably could be used for the “most controversial” page.
(edited: s/separate out their motivations/ignore one of their motivations/)
I think other people have said it, but Slashdot has one of the best commenting structures around.
Different values for different categories (funny, insightful etc....), anonymous posting, reputation, very clear thread structure. All sorts of fun stuff
Because quality and truth are separate judgments in practice, and forcing them to be conflated into a single scale is losing information. To the extent that truth is positively correlated with quality this will fall out automatically: highly truthy posts will tend to have high quality. Low quality and high truth are not opposites.
I agree it’s losing information, but that’s something you have to weigh against the inconvenience of multiple dimensions. To the extent that truth is positively correlated with quality you’re just making people click twice, and I suspect clicks are a limited resource.
As I see it the voting system is there to put comments in a convenient order and remove the really bad ones from sight, not to provide opinion poll information.
That’s exactly the point: voting is supposed to put comments in order according to quality, so that you can read the worthwhile comments in a reasonable time. My claim is that the current voting system will not do this well at all and that a dual voting system will be better. (That second bit is just a guess). The opinion poll information is just a nice side effect.
OK, so according to you and Benja the point is to have the agree/disagree buttons there mainly as a lightning rod to prevent agreement from affecting quality votes. That’s a good point, but I wonder if it’s worth it and if there are better ways to accomplish the same thing.
I also wonder if there should be a button labeled “malevolent cantaloupe” so the unserious people will click on that instead of voting.
I’m not sure this is obviously right. I would probably insist upon some usability study to determine how people actually use such features. Of course, if the cost is low such a study could just be implementing them and seeing how it works.
I imagine there’s a name for this cognitive bias, but I’ve noticed well-informed folks tend to think agreeable opinions are better-argued, and less agreeable ones are worse-argued (probably a species of confirmation bias).
For example, someone posting against physicalism might get downvoted quickly by people who say “but they didn’t even consider Dennett’s response to this premise”. But they might not have the same objections on-hand to an unsound argument in favor of physicalism.
I’d prefer a clear explanation of intended semantics of voting, linked to on “About” page, and posted one of these days on the front page to get anyone’s attention and users’ suggestions.
It might also be good to stick a reminder of what up-voting is intended to mean right next to the up-vote and down-vote buttons. Or to change the names: instead of “vote up” and “vote down”, perhaps something like “high-quality discussion” and “low-quality discussion”.
Not sure about that—those labels at least would look ugly. Maybe a title attribute on the “vote up” and “vote down” would be sufficient.
How about buttons “High quality”, “Low quality”, “Accurate”, “Inaccurate”. We’re increasing options here, but there’s probably a nice way to design the interface to reduce the cognitive load.
Using the word “vote” seems broken here more generally—we aren’t implementing some democratic process, we’re aggregating judgments (read: collecting evidence) across a population.
I completely agree about the word “vote”.
“High quality” / “Low quality” has good brevity, but for myself I’m still tempted to blend in agreement/disagreement with my ratings when I picture those words—to regard comments I disagree with as “low quality”. If we could have the question “Does this add to or subtract from the conversation?” surrounded by up/down arrows (or by “adds” / “subtracts”), I imagine myself voting better.
For example, I just up-voted James Andrix’s and Kurige’s comments about their religious beliefs.
I up-voted the comments because they’re good data, I’m glad the commenters shared it, and it looks like stuff more eyes should look at within the thread. But I hesitated, because “up-voting” gives the appearance of agreement. Rating Kurige’s comment “high quality” feels a bit similar, like calling it “high quality reasoning”. But clicking up-arrow next to the question “Does this add to the conversation?” would feel obvious, to me in this case.
Yep, what I wrote is just based on my best guess. A usability study would be great.
Also, I am going with the crowd and changing to a user name with an underscore
A second question about the semantics of voting: should I be up-voting all good posts regardless of score, and down-voting all bad posts regardless of score, or should I be voting to correct points-numbers that are misaligned with post quality?
Upon reflection, I’d say we should be voting to correct points-numbers that are misaligned with post quality. Otherwise, if people continue to up-vote more posts than they down-vote, comments will accumulate more and more points the older they get, and a setting like “show me all posts above 3” won’t be meaningful across threads or comment-ages.
Added: The “odd social dynamics” point is good. I’ll follow Eliezer and thomblake here.
I would actually say that voting to correct post-quality would lead to some odd social dynamics. I’ll vote up if something is negative for no reason, but otherwise I’ll vote my opinion, not corrections—unless something seems really out of line.
I’d rather have the sum of people’s individual components then see everyone trying to correct everyone else’s voting.
Agreed—I’ve been using voting the same way:
if (score < 0 && myPreferredScore >=0)
vote up
else
vote my conscience
Edit: had to camelCase the name above due to odd behavior of underscores. I should learn MarkDown.