Meta: What do you think of a karma vote checklist?
I’m imagining an optional checklist offered if you downvote, with the list possibly including troll, poor spelling/grammar, false, redundant....
For that matter, if an optional checklist for downvotes makes sense, then perhaps there should also be one for upvotes: sensible, informative, funny, caused an update....
I’m imagining a little chart appearing if your cursor is on a karma number, the way the proportions of stars do for amazon reviews.
I don’t know how much trouble this would be to program—I’m just floating the idea.
I’m new here and might not understand the present karma system completely or correctly. I like it, in certain ways, but I also know, from long internet history, that systems like this can be abused.
A well-known and acknowledged internet troll just openly threatened (on RationalWiki, where I’ve retired) to come here and harass me. I know what he does. I’m not concerned about argument from him, the karma system will handle that. However, he will also do these things, it can be predicted:
*He will look at all my past contributions and will down-vote them as much as allowed.
*He will register new accounts as needed. He’s highly skilled at this.
*He will look for any method of gaming the system, he will probe for vulnerabilities.
*He doesn’t care about the site purpose. He cares about winning a game.
The present karma system looks vulnerable to activity like this. I don’t see any clear sign that he’s been active so far. I reached a nadir of about −40, which is not surprising, I had raised certain issues that might be unpopular here. I modified my behavior, that’s a positive effect of the karma system. I’m at −2 right now.
His threat might be empty. However, these are the problems with the karma system that I see:
The voting is anonymous and there is no accountability. There is a suggestion that’s been made that downvoting should have a cost. It’s possible that all voting should have a cost. Otherwise we get voting (an action with consequences) with no personal responsibility (leading to a weighting toward people who really don’t care, but just respond, knee-jerk, possibly irrationally).
Voting systems ideally represent what happens in the brain. We have affective and aversive responses, and we do make decisions based on the overall weight. However, rational process, internally, can look at each response and value or devalue it, and the same happens in social processes with responsible actors. In the karma system, there is no way to look at what is producing up-votes and down-votes, and most votes are not accompanied by any comment at all.
Voting is presently three-valued, like Range 3 voting: i.e., values of −1, 0, +1. While this can be a great voting system (substantially better than binary), the total votes in each category, in real systems, can make a big difference in subsequent process. I.e., a net of −1 based on a single downvote, is a very different creature than the same net with 50 ups and 51 downs. The latter is probably of high interest! It would indicate a true divided community, as distinct from one that doesn’t care, it could indicate an area that needs more discussion. If it’s +50 and −53, it would indicate the same thing, the difference is in the noise, but now the karma system would inhibit the very discussion needed.
This leads to some immediate suggestions:
Report the votes in each category, not just the total.
Increase the resolution, i.e., say, allow double voting in each category of vote, and categorize these separately. (The system then becomes Range 5.)
Consider systems that make users more accountable. Perhaps report for each user how many upvotes they cast and how many downvotes. Or even make voting not anonymous. In real deliberative organizations, all opinion is public, and secret ballot is never used for issues, only for certain kinds of elections. An Objection to Consideration of the Question, for example, is subject to immediate, public vote. And if someone still wants to raise the issue, they know to whom to talk, individually. That is a device that increases social intelligence (distributed conversations).
Use a percentage rule for consequences of vote totals, rather than a fixed difference.
It is a general situation that internet process is at a primitive stage. Moving toward simulation of intelligent decision-making process could greatly improve the effectiveness of any society.
The karma system is a great step toward this, but appears vulnerable in certain ways.
Could you give a link?
I’m not sure it will do much good, but here is the post, and this is a permanent link to the discussion as it stands now. This was a goodbye post, to AD, one of the seemingly saner members of the RationalWiki community, an elected moderator. There is a link in my goodbye post back to AD’s comment in a discussion that included history, but that’s a lot more than I expect people here to be interested in. Suffice it to say that the user has a history of being exactly what he says he is, a highly effective troll. He says “professional.”
(To understand some of the discussion, “promote” on RatWiki means “remove sysop privileges” or sometimes “block.”)
Again, an idea with prior art—Slashdot and SensibleErection both used this for an order of magnitude longer than LW has been around; SE even adds reasons, sometimes, for special occasions. However, newer forums like reddit and Hacker News (even with its quality obsession) have abandoned those methods in favor of a simple scalar vote.
I am going to turn from a lurker to do the manual version of this.
I am not sure I am subcultured enough to align my votes with the consensus. Doing it this way can elict more explicit reasoning of why posts get voted the way they do. Of course doing this will automatically make the comment meta-discussion. I plan to do very short, word or phrase like motivations for both why the post was worthwile and why it wasn’t.
I won’t read all threads and I am not doing a public service (so don’t count on me doing this). I am thinking of leaving a comment on every thread start post that I read and always voting either up or down. For discussion in comments I will think I will have a heuristic of judging long and critical (at sole my intuitive criterion) posts. Long posts drain more attention so they better use it productively. Non-throwaway posts are more in need of feedback and dialog.
I think that reasons for up and downvote are quite distinct few per person voting. Having that information plainly in view would take a lot of implicit misunderstanding away. Off-course there are risks in more subtle and harder to pronounce feedback being trampled. Not everyone wants to be crockered.
Anyone have more reservations that I could be aware of beforehand? Know that I am open to discussion regarding this behaviour.
I have ceased to do this. It felt like being a judge and atleast a couple of discussion posts I probably discouraged the poster more with a reasoned downvote than with a silent downvote. It brought to my attention that I didn’t really have consistent, good or defensible way of voting. Not sure whether this expliciation would cause this to other users. My conception on what karma is and how it should be used has been mangled.
I am surprised that anyone from the internet can come and highjack the voting system to use as they see fit. I am putting the hammer down realising I didn’t know what I was building.
Thanks for writing the followup.
I think you should try it and see how it works out.
I also can’t see any downsides except that it may take up more time than you want, but that may simply mean that you read fewer articles and write more comments.
It’s possible that if everyone posted explanations for their votes, LW would get cluttered with those comments, but there’s no reason to think everyone is going to start posting explanations. Or perhaps the solution would be that an upvote for an explanation is the only thing which doesn’t require an explanation—the upvote is equivalent to “Me too”.
I suspect new posters frequently don’t know precisely why their reply has been downvoted. Giving them more information will help them improve the quality of their future replies.
Not a bad idea. Popup on thumbs up/ thumbs down. Particularly useful for down votes, where an unexplained karma down vote often just seems impolite to me—it’s just not very constructive to say “you stink” if it’s unclear what you’re complaining about.
But I maintain that the main problem here isn’t that we have too few ways to vote, but too few ways to use votes and other information to filter posts in a user customized fashion. Well, I shouldn’t say “we”, because some people have a further problem that they just don’t want some posts to exist, or be seen by anyone.
Slashdot has something like this.
Does the Slashdot system work well?
No.
EDIT: Okay, still no, but Slashdot has a very different culture than over here. It’s all news postings, there’s a huge rush, most posts are shit and spam, the fact that some good stuff manages to rise to the top is a minor miracle. The truth is, we have a culture that encourages more thought per posts, a smaller number of active posters, and if you do write something crappy or unpopular, most people’ll just reveal it anyway. On Slashdot, doing through all the comments on one post would take hours for a typical post, so you write something one mod downvoted cause he had some free ones to spare or he doesn’t like you very much, no one sees it, and your Karma drops to below zero, so no one will ever see your post ever again.
Or you can require downvoters to leave a comment explaining what they found objectionable.
The Stack Exchange Q&A forums encourage (though not require) explaining downvotes. They also charge 1 reputation point for each downvote. The quality of the discussion seems to be very high.
Explanations might be useful for new users (like me—if I got downvoted, it would be good to know what I’d gotten wrong) but if people had to explain every time they downvote a troll it might be a bit of a waste of time. If trollfeeding is already a problem on here, that would surely make it worse...
The proper way of dealing with trolls is to ignore them altogether. Even downvoting them gives them the attention they don’t deserve.
I disagree that downvoting is not worth it. Downvoting is a clear signal to others that this is a comment not worth paying attention to, and prevents them from responding to a troll comment.
Can’t they judge for themselves?
Takes time.
Sure, but it seems better than relying on the majority opinion.
The majority opinion isn’t always accurate, but it is generally useful, especially in a narrow community of your choosing.
One way or another, you have to face the trade off between the expected value of the time you will lose reading the comment, and the expected value you will extract from the comment. In other words, how can you decide if a comment is worth your time or not? There are several data points that can help you make your decision:
Overall quality of comments, that you know from prior experience with the site.
Whether it belongs to a thread you participated in. (That’s not objective, but what counts is the value the comment has to you)
Length of the comment (the longer ones tend to be more valuable, but also takes more time to read)
Upvotes and downvotes (highly depends on how much you trust the community there)
The content itself doesn’t count, because you need to read the comment to judge it.
Now assume your time is valuable, and you need to sort the weed from the crop. Without a voting system, you’re left with the first three criteria, maybe only the first two. In the end, you are likely to read either few comments, or almost all of them. Voting systems can help you do better.
The problem is, you also have to judge the voting patterns. To what extent do you agree with the votes you see? The more you do, the more you can let the community judge for you. Enjoy your time.
In order to do that, you have to read low scoring comments too.
Only for a time.
Voting patterns can change with time and with respect to different issues.
Another thing to keep in mind in this regard, is that this forum has a filter that blocks posts with enough of a negative score. Downvoting a comment posted by a troll can swiftly block it from public consumption, which means only those who want to read it will.
I’ve been thinking that there needs to be specific feedback with the votes, myself. I’m going to release a “prevent endless September” plan for you all to critique soon which I am willing to volunteer to help accomplish (I’m a web pro) which recommends requiring users to give feedback—I feel that the anonymous, feedback-less, thumb downs with no accountability are a bane to innovative types, as it allows these commonly misunderstood folks to be censored without providing the feedback required for them to figure out how to communicate effectively. What I would like to see rather than a drop down is a box where you have to type at least one word as a reason. That way, you can always put something appropriate, yet it’s still not hard to type “trolling” and it’s not cumbersome for more advanced people to leave something specific like “appeal to popularity” or “optimism bias” as constructive criticism for the user.
Edit: Actually it would really be best to have a drop down for categories of up/down votes AND a box where you type at least one word. Except for trolling. Where you just hit “Trolling”. Or spam.
By the way, setting up a “reason” option, is an excellent idea, properly implemented. It could be a checklist, with one option being to enter a specific explanation. This then becomes metacomment, only in-the-face of those concerned to look at it. Layering.
Your checklist omits the main purpose actually served by upvoting and downvoting; determining the LW consensus. The main factor (but not the only factor) determining a vote is whether the rater agrees with the poster. Perhaps that’s as it should be, since without it, LW could never evolve a party line (so to speak).
You could add agree/disagree to the list, but that would undercut your purpose, focusing on quality rather than agreement. But it isn’t abstract quality, if there is such a thing, that LW is after: it wants to advance its brand of rationalism, which requires deciding what that brand is. This is the main function of voting and karma, but it is subject to much denial.
In theory, upvotes are supposed to indicate what you want more of, and downvotes are what you’d like to see less of.
I don’t know how many LWers use them that way, rather than for agree/disagree.
Let me substantiate. Here’s a post where I recently articulated and applied the LW “line.”
On the other hand, a post in this thread, where I advance an idea that should be important to to people who build a rationalist community:
Not to be immodest, but this insight is the product of years of watching and experimenting with LW, and I only reached it recently. If it’s true, it’s important because LW is obstructed in constructing a rational community when it ignores the primary function of its “institutions” and substitutes idealistic thinking (‘an upvote means you want more of the same’) for a functional analysis.
Once you see the actual role of karma, you might realize that it couldn’t be otherwise. A a massive intellectual community must find a way to evolve a dynamic consensus. It requires objective incentives to coordinate on a singe outlook (or on a narrow spectrum of outlooks).
In point of fact, this how leading LWers sometimes speak. For example, LukeProg argued that the community isn’t a cult around E.Y. because early on, his posts were upvoted in some cases when E.Y.’s were downvoted. LukeProg’s comment implies LW took an ideological or practical direction through the karma mechanism.
In this comment I’m not evaluating the karma mechanism but pointing out that it is L.W.’s soul.
I experiment with these things. Based on a period of years, during which I occupied different personae, it’s clear to me that I can get upvoted reliably if I clearly articulate and intelligently apply the LW-consensus analyses. I will get downvoted severely if I—stupidly or intelligently—articulate an original position that contradicts the LW consensus. As I say, I don’t see how it can be otherwise, if LW is to function as a community with consensus views on eclectic matters.
I wonder what you mean by “in theory.” If that means “according to the standard LW rhetoric,” I agree.
Also, if down votes were not anonymous, people would have to be accountable for them. I don’t think its a good idea for people to have the ability to censor anonymously. That gives a single person the power to deter others from responding to them, and it gives a group of three the ability to get them ignored. I’m totally willing to be accountable for defending any down votes I give. How do you guys feel about having to be capable of defending your down votes because they’re public?
Anonymous downvoting allows karma assassinations etc.
Non-anonymous downvoting means that every time you downvote something you expose yourself to a conflict. When thinking about a downvote, you will also instinctively think about the chance that the person will get angry and punish you—by expressing hostility towards you, or by downvoting some of your comments. This could make people less likely to downvote some problematic kinds of behavior. It could become: “yeah, someone should downvote this, but why me?”
It seems like we should be willing to defend our downvotes, but in some situations this could be equal to feeding the trolls, or discouraging people from downvoting applause lights.
We need separate buttons and votes need various changes to be effective at encouraging improvement. Negative karma cannot provide new users with useful feedback, deter trolls and inform the popularity sorting system all at the same time. Here is why:
Unexplained downvotes don’t tell users what to improve, so they’re as good as ignored
If we feel a need to have a way to mark the low quality comments, and provide feedback to discourage annoyance, we should definitely ask that person to give a few words of feedback. As a new person who occasionally gets vote downs, I have absolutely no clue whether there is any pattern to them. They’re not encouraging me to acculturate at all. It just looks random. I’ve theorized that maybe somebody doesn’t like me, or that maybe I made a spelling error I didn’t catch, or maybe a troll is harassing me. I have no clue. This speculation isn’t getting me anywhere. I need specific feedback.
What would help me (and other new users) improve
If the vote downs provided at least a couple of words of feedback, and didn’t censor me, I’d be happy with that. Getting “Not new information” or “joke was not funny” would be much preferred to random anonymous discouragement. A category would be better than nothing, but a couple specific words would be best.
Reserving feedback for older members will reduce endless September risk and increase feedback value
The way this would reduce endless September risk is by highlighting the interactions with older users, which will amplify their ability to maintain culture if the ratio of new to old people is thrown off. Also, as a new user, I want to know that the feedback I’m getting is from somebody helpful, not a troll, or a newbie. This doesn’t guarantee that old member’s feedback is useful but it would prevent me from brushing them off thinking I might be getting harassed by trolls. If there’s a possibility trolls are giving the feedback, I really can’t take it seriously.
Attaching a punishment to a downvote sacrifices honest feedback in favor of not punishing the user
Consider this: You dislike somebody’s comment, but you don’t see them as a troll or a bad person. They’ve got two downvotes. Do you downvote them further and hide their comment? Probably not. That’s what a lot of people are thinking, because most of the time when I see downvotes, it’s just one or two. You know what I think when one or two people didn’t like my comment? “Can’t make all the people happy all the time. Oh well.”
Now if TEN people vote me down, I know something’s going on. THEN I can be completely sure it’s not a personal vendetta or someone having a bad day. We cannot take vote downs as a serious source of feedback if we only get one or two, and people won’t give you more than one or two if you’re punished at three.
Merging upvotes and downvotes into one figure promotes black and white thinking about popularity
Also, seeing the number of up votes versus down votes would be useful. If I say something that gets 40 up votes and 43 down votes, all I see in the current system is down votes. I may think most people are ignoring me or don’t like what I said. It could simply be that I said something controversial—and that can be a good thing. To avoid discouraging controversy, or to put it another way, to prevent conformity, showing both sides of the story is important. It may be the practice of showing only one side of the story that promotes conformity. Showing both might be a good way to prevent conformity.
How the separate ban button could work for trolls:
Giving a large portion of users the power to ban trolls would expedite the process of removing them. This can’t be a small number of people, or it would invite politically motivated “assassinations”. This can’t be new users because if new users flood the forum they may ban old users due to cultural misunderstandings who were contributing to the culture. If, say 5% of older active users press the ban button, it would disable the troll’s account. Currently, that would mean around 25 to 50 people if Vladimir is correct. They can’t be people with too little activity—this would encourage trolls and malicious users to create 50 dummy accounts, wait a few months, and then ban people they don’t like. The ban button can be totally anonymous and invisible, preventing trolls from getting any feedback at all, because it is completely separate from the voting system.
You can, in fact, do this already.
Love it when I totally miss something like that. fixes my comment
Unexplained upvotes are ambiguous, too. I frequently make several points in a comment. If it gets upvoted, I don’t know what people like.
I’m dubious about public votes. Anonymous and unaccountable has problems, but I don’t think actual karma counts turn out to be wildly unreasonable.
Public karma votes would probably lead to long quarrels about which votes are reasonable and to (more?) karma coalitions.
I’m not sure whether it’s obvious here, but I’m rather conflict-averse, which means that a public vote system would make me less likely to downvote the more aggressive comments and posts.
Does anyone have experience with a public vote system? How did it work out?
A “public vote system” has been used for centuries in standard deliberative process. You go to a Town Meeting and think that a question should not be considered, and you so move, and that is subject to immediate and very public vote. Private voting systems have been used and often have an abusive effect. Such systems, in standard process, when allowed, generally require a supermajority. Elections are an exception, where secret ballots are standard.
Much comment here seems to assume yes/no on “private.” It’s possible to collect data on “impressions” that is private, and it is not necessarily abusive. It can become abusive when this is used in a fixed decision-making system.
The karma system is quite popular, and the way it works should not, ideally, be damaged by “improvements.” Improvements may address the ways that it does not work, and there are a number.. There are many good ideas in this thread. Some of them, implemented raw, could do harm. Hence the need for discussion and the development of informed consensus, which can be very different from raw, knee-jerk consensus. Such raw consensus can be used to develop starting points, and is worthy of respect, but not worship.
Otherwise a community is vulnerable to cascades and to confirmation bias.
Standard deliberative process uses committee systems for topics not ready for full consideration and vote. The conversations take place in small groups, where brainstorming may be more open and less harmful, and, ideally, all significant points of view are represented in those groups. Distributed communication is essential for sound and efficient social process.
This sounds great when added to the end of fortune cookies.
I’m imagining two men with handguns forcing a blindfolded girl into a car so they can get a cut of the ransom money.
I don’t know how much trouble this would be to program—I’m just floating the idea.
Replies to downvoted comments now cost 5 karma apparently. Thank God I have over 6,000 karma!
Hey look it seems like replying to a comment that is a reply to a comment that was negative doesn’t cost anything. So as long as one person takes the 5 karma hit, the disincentive no longer applies. Good to know!
Doesn’t work for me. I had to pay 5 karma to write this comment.
Right, you have to be quick about it. But that just means I have to sacrifice 5 more karma. Obviously someone should make a bot that does this automatically. Agentic intelligence beats stupidity on the part of mechanism designers. This is a warning to all mechanism designers. I can defeat your system. Don’t be idiots, you idiots. Yay artificial intelligence. (Obviously if they create a bot that can ban bots then I can create a bot that creates more bots than they can efficiently ban. P versus NP. This would be completely and immediately obvious to you, mechanism designers, if you’d studied algorithmic mechanism design. But you’re idiots, so you didn’t, you idiots. I hope the AI fixes the problems with your brains because clearly there are many.)
http://xkcd.com/810/
Of course, this is a straightforward problem to fix in the mechanism design: Just make responses to downvoted comments start at −5 karma, instead of having a direct penalty, as suggested elsewhere. I think that suggestion was for unrelated reasons, but it also fixes this little loophole.
This would discourage me much more than the current mechanism: I care very little about my total karma score. (I’m not saying it would be a good thing.
You’re dumb sometimes, army1987, but other times your replies are spot-on and concise, so I like you. :) #perhapsthesamecouldbesaidofallreligions
Hey guys, don’t worry. I’ll take that 5 karma hit for you. Just send me a PM and I’ll try to get back to you. Just ’cuz I love you guys so much. <3
Mmmmm paying that 5 karma fee is kinda addictive. Mmmmmmmmmmmm. Your own… personal… Jesus...
Someone to read others’ stupid comments, someone to reply to them...
I’m sorry WIll it seems we won’t scapegoat you no matter how hard you try. Maybe you should try being king of something or start a start up?
Pope, actually, after the apocalypse. Thinking about taking the name Hilarius II. But we’re not yet ready for the apocalypse.
Oh I just can’t wait to be Pope!
What about your girlfriend?
If you read the history of papery, you’ll find that’s not an issue.
On the other hand, Will strikes me as type to actually care about following cannon law.
If Will was pope, he would have papal infallibility, and thus could speak ex cathedra on all matters of what is and what is not a cannon.
Trivia: Although the Society of Friends is generally a good example of theological liberalism, Quaker Cannon does exist. However, most scholars of Catholic cannon law hold it to be illusory.
I think an artillery specialist would have more authority on all matters of what is and what is not a cannon.