Have any other online forums tried something similar? If so, knowing what results they had seems decently valuable. I say decently instead of a stronger word because what works for one community doesn’t necessarily work for another, especially one as unique as LessWrong.
Hmm didn’t really find anything similar, but here are some examples of rating systems I found that looked interesting (though not necessarily relevant):
2-factor rating systems
SaidIt: (1) Insightful & (2) Fun
SaidIt is a Reddit alternative which seeks to “create an environment that encourages thought-provoking discussion”. SaidIt has two types of upvotes to choose from: 1) insightful, and 2) fun.[1]
Goodfilms: (1) quality & (2) rewatchability
Goodfilms is a movie site for users to rate, review, share films and find movies to watch. Users rate movies on two dimensions: quality and rewatchability. The ratings are displayed as a scatterplot, giving users a better sense of the type of movie (e.g. most people agree it is highly rewatchable, but there is disagreement on its quality ⇒ may not be very good, but is fun to watch).[2]
Scatterplot for Starship Troppers ratings
Suggestion by Majestic121: (1) Agree/Disagree & (2) Productive/Unproductive
A Hacker News comment by Majestic121 suggests a 2-factor voting system:
Up/Down : Agree/Disagree Left/Right : Makes the discussion go backward/forward
This way you could express disagreement while acknowledging that the point is interesting, or like a joke without having it drown a conversation
Suggestion by captainmuon: (1) Promote/Bury & (2) Reward
Hacker News comment by captainmuon: Promote/Bury and Reward buttons
Up/downvotes always have multiple conflicting dimensions.
The post is factually right / wrong
Confirms to the site rules / breaks the rules
I agree / disagree
I want to promote / bury this post
Reward poster with XP / punish
I am usually very pragmatic and upvote a post when I want other people to read it (because I want to see the discussion, or because I want to spread the idea). I also upvote to reward the poster.
I don’t tend to downvote factually wrong posts when they are still interesting, because that limits the chance they get good discussion, and because I don’t want to punish somebody for Being Wrong On The Internet. I do downvote positions that I find bad in order to reduce their reach.
It would be probably possible to have a site that implements two dimensions as a cross (maybe only for users with a certain XP) although the UX might not be to great. Maybe it is a good idea to have “promote/bury” and “reward” buttons?
Others
Pol.is: displays a cluster graph of participants based on their voting patterns on statements
Pol.is is a platform where participants submit statements for others to vote on (agree/disagree/pass), and participants are then clustered based on their votes. People can see this map of voters and are incentivized to craft statements that also appeal to members of other groups to gain more support, thus converging on a consensus.[3]
In this graph, statements are positioned more closely to statements which were voted on similarly. Participants, in turn, are positioned more closely to statements on which they agreed, and further from statements on which they disagreed. This means participants who voted similarly are closer together.
Example of cluster graph from Taiwan’s Uber case study
People in Taiwan were invited to discuss the regulation of Uber. At the beginning, there were two groups: pro-Uber and anti-Uber. As people tried to submit statements that would gain more supporters, they converged on a set of seven comments that majority agreed on, such as “It should be permissible for a for-hire driver to join multiple fleets and platforms.” These suggestions shaped the regulations that were eventually adopted by the government. [3]
Tweakers: users can assign scores from −1 to +3, with detailed guidelines on how to vote
Tweakers is a Dutch technology website. Users can assign a score of +3 (insightful), +2 (informative), +1 (on-topic), 0 (off-topic/irrelevant), −1 (unwanted) to comments. The median score is displayed, but users can click to view the breakdown of votes.[4] 0 scores with 0 votes are displayed in a different color from 0 votes with >= 1 votes.
There are detailed guidelines on what each score means, as well as how to handle common scenarios, such as:
Comments about spelling, typos etc.: −1, because they should be reported in the Dear Editorial Forum instead
Inaccurate comments: no less than 0 if it is well-intentioned, −1 is for socially inappropriate behavior
Opinions you disagree with: comments should not receive lower ratings simply because they conflict with your opinion. To express dissatisfaction, reply the comment with your refutation.
Strategic voting e.g. upvoting a comment because you think its score is too low, rather than because you think it deserves the high score: do not do this
The guidelines also vary based on the context, with the guidelines explaining how moderation practices should differ for downloads, product reviews, and other pages.
(Disclaimer: site was in Dutch so I used Google Translate)
Slashdot: voting by assigning different types of labels (e.g. insightful, redundant)
Moderators can assign different labels to comments, which will add or deduct a point from the comment’s score. There are descriptions of what each label means in the FAQ. The labels are as follows:
They should have dummy agree/disagree buttons that disappear once selected and has no function other than to satisfy the malcontents that the machine has recorded their opinion.
Have any other online forums tried something similar? If so, knowing what results they had seems decently valuable. I say decently instead of a stronger word because what works for one community doesn’t necessarily work for another, especially one as unique as LessWrong.
Hmm didn’t really find anything similar, but here are some examples of rating systems I found that looked interesting (though not necessarily relevant):
2-factor rating systems
SaidIt: (1) Insightful & (2) Fun
SaidIt is a Reddit alternative which seeks to “create an environment that encourages thought-provoking discussion”. SaidIt has two types of upvotes to choose from: 1) insightful, and 2) fun.[1]
Goodfilms: (1) quality & (2) rewatchability
Goodfilms is a movie site for users to rate, review, share films and find movies to watch. Users rate movies on two dimensions: quality and rewatchability. The ratings are displayed as a scatterplot, giving users a better sense of the type of movie (e.g. most people agree it is highly rewatchable, but there is disagreement on its quality ⇒ may not be very good, but is fun to watch).[2]
Suggestion by Majestic121: (1) Agree/Disagree & (2) Productive/Unproductive
A Hacker News comment by Majestic121 suggests a 2-factor voting system:
Suggestion by captainmuon: (1) Promote/Bury & (2) Reward
Hacker News comment by captainmuon: Promote/Bury and Reward buttons
Others
Pol.is: displays a cluster graph of participants based on their voting patterns on statements
Pol.is is a platform where participants submit statements for others to vote on (agree/disagree/pass), and participants are then clustered based on their votes. People can see this map of voters and are incentivized to craft statements that also appeal to members of other groups to gain more support, thus converging on a consensus.[3]
Description of graph extracted from Pol.is report:
Example case study: vTaiwan Uber
People in Taiwan were invited to discuss the regulation of Uber. At the beginning, there were two groups: pro-Uber and anti-Uber. As people tried to submit statements that would gain more supporters, they converged on a set of seven comments that majority agreed on, such as “It should be permissible for a for-hire driver to join multiple fleets and platforms.” These suggestions shaped the regulations that were eventually adopted by the government. [3]
Other Pol.is case studies and reports: https://compdemocracy.org/Case-studies
Tweakers: users can assign scores from −1 to +3, with detailed guidelines on how to vote
Tweakers is a Dutch technology website. Users can assign a score of +3 (insightful), +2 (informative), +1 (on-topic), 0 (off-topic/irrelevant), −1 (unwanted) to comments. The median score is displayed, but users can click to view the breakdown of votes.[4] 0 scores with 0 votes are displayed in a different color from 0 votes with >= 1 votes.
There are detailed guidelines on what each score means, as well as how to handle common scenarios, such as:
Comments about spelling, typos etc.: −1, because they should be reported in the Dear Editorial Forum instead
Inaccurate comments: no less than 0 if it is well-intentioned, −1 is for socially inappropriate behavior
Opinions you disagree with: comments should not receive lower ratings simply because they conflict with your opinion. To express dissatisfaction, reply the comment with your refutation.
Strategic voting e.g. upvoting a comment because you think its score is too low, rather than because you think it deserves the high score: do not do this
The guidelines also vary based on the context, with the guidelines explaining how moderation practices should differ for downloads, product reviews, and other pages.
(Disclaimer: site was in Dutch so I used Google Translate)
Slashdot: voting by assigning different types of labels (e.g. insightful, redundant)
Moderators can assign different labels to comments, which will add or deduct a point from the comment’s score. There are descriptions of what each label means in the FAQ. The labels are as follows:
Normal (default setting)
Offtopic
Flamebait
Troll
Redundant
Insightful
Interesting
Informative
Funny
Overrated
Underrated
Placebo button :p
Hacker News comment by kevin_thibedeau:
https://saidit.net/s/SaidIt/comments/37r/welcome_to_saiditnet/
https://web.archive.org/web/20200812094557/http://goodfil.ms/blog/posts/2012/08/22/why-ratings-systems-dont-work/
https://www.technologyreview.com/2018/08/21/240284/the-simple-but-ingenious-system-taiwan-uses-to-crowdsource-its-laws/
https://tweakers.net/info/faq/karma/#tab:1-2
https://pol.is/report/r32beaksmhwesyum6kaur