I really like this experimentation. Some thoughts:
Regarding finding the ideal set of axes: I wonder if it would make sense to give quite a few of them (that seem plausibly good), and then collect data for a month or so, and then select a subset based on usage and orthogonality. Rather than tentatively trying new axes in a more one-by-one fashion, that is. You’d explicitly tell users that the axes are being experimented with, and to vote on the axes which seem most appropriate. This might also be a way to collect axis ideas—if the user can’t find the axis that they want, they can click a button to suggest one. Relying on the in-the-moment intuitions of users could be a great way to quickly search the “axis space”.
I really like the “seeks truth/conflict” axis. A comment has an inherent “gravity” to it which makes it inappropriate/costly for pointing out “small” things. If a comment is very slightly hostile, then there’s a kind of social cost to pointing it out, since it really isn’t worth a whole comment. This results in a threshold under which incivility/conflict-seeking can simmer—being essentially immune to criticism.
One weird experiment that probably wouldn’t work, but which I’d love to see is for the reactions to be more like a tag system, where there are potentially hundreds of different tags. They’re essentially “quick comments”, and could be quite “subtle” in their meaning. It would be a bit like platforms that allow you to react with any emoji, except that you can be much more precise with your reactions—e.g. “Unnecessary incivility” or “Interesting direction” or “Good steelman” or “Please expand” or “Well-written” or “Hand-wavy” or “Goodhart’s Law” (perhaps implying that the concept is relevant in a way that’s unacknowledged by the author). There could also be some emergent use-cases with tags. For example, tags could be used as a way for a commenter to poll the people reading the comment by asking them to tag a digit between 1 and 5, for example.
There are lots of ways this idea could end up being a net negative—in particular it may be that any level of subtlety beyond a few basic voting axes really would benefit from a comment in almost all cases, and then that comment essentially becomes the “tag” that people can vote on. Still, I’d love to see an experiment.
This isn’t about this experiment specifically, but: One problem with showing an absolute vote count is that it relies on people explicitly not voting on something if they think it has reached an appropriate level of upvotes/downvotes. E.g. if a comment that you think is kinda bad has a score of +10, you might downvote it, but if it already has a score of −3, you might leave it because to downvote further would be “too harsh”. This obviously isn’t ideal, because a couple of hours later that −3 comment could have climbed to +5 and so it turns out you should have actually downvoted it. There are a few ways to solve this—e.g. use more of a star rating system, or cap the upside and downside (but keep the real votes so that e.g. if a comment gets to −10, only −5 is displayed and reflected in the user’s karma, but it would require 6 upvotes to get to −4), display as a ratio plus total number of votes, etc. - they all have their trade-offs though, so I’m not sure there’s a clear solution here. This is another place where tags are interesting, because if everyone things a comment is just slightly conflict-seeking, then they can use the “Slightly conflict-seeking” tag, and they can all vote on that without giving the comment author the impression that everyone thinks their comment is extremely conflict seeking.
Like I said, I love this experimentation—please keep at it! I think this topic is completely underappreciated by basically every social platform.
I really like this experimentation. Some thoughts:
Regarding finding the ideal set of axes: I wonder if it would make sense to give quite a few of them (that seem plausibly good), and then collect data for a month or so, and then select a subset based on usage and orthogonality. Rather than tentatively trying new axes in a more one-by-one fashion, that is. You’d explicitly tell users that the axes are being experimented with, and to vote on the axes which seem most appropriate. This might also be a way to collect axis ideas—if the user can’t find the axis that they want, they can click a button to suggest one. Relying on the in-the-moment intuitions of users could be a great way to quickly search the “axis space”.
I really like the “seeks truth/conflict” axis. A comment has an inherent “gravity” to it which makes it inappropriate/costly for pointing out “small” things. If a comment is very slightly hostile, then there’s a kind of social cost to pointing it out, since it really isn’t worth a whole comment. This results in a threshold under which incivility/conflict-seeking can simmer—being essentially immune to criticism.
One weird experiment that probably wouldn’t work, but which I’d love to see is for the reactions to be more like a tag system, where there are potentially hundreds of different tags. They’re essentially “quick comments”, and could be quite “subtle” in their meaning. It would be a bit like platforms that allow you to react with any emoji, except that you can be much more precise with your reactions—e.g. “Unnecessary incivility” or “Interesting direction” or “Good steelman” or “Please expand” or “Well-written” or “Hand-wavy” or “Goodhart’s Law” (perhaps implying that the concept is relevant in a way that’s unacknowledged by the author). There could also be some emergent use-cases with tags. For example, tags could be used as a way for a commenter to poll the people reading the comment by asking them to tag a digit between 1 and 5, for example.
There are lots of ways this idea could end up being a net negative—in particular it may be that any level of subtlety beyond a few basic voting axes really would benefit from a comment in almost all cases, and then that comment essentially becomes the “tag” that people can vote on. Still, I’d love to see an experiment.
This isn’t about this experiment specifically, but: One problem with showing an absolute vote count is that it relies on people explicitly not voting on something if they think it has reached an appropriate level of upvotes/downvotes. E.g. if a comment that you think is kinda bad has a score of +10, you might downvote it, but if it already has a score of −3, you might leave it because to downvote further would be “too harsh”. This obviously isn’t ideal, because a couple of hours later that −3 comment could have climbed to +5 and so it turns out you should have actually downvoted it. There are a few ways to solve this—e.g. use more of a star rating system, or cap the upside and downside (but keep the real votes so that e.g. if a comment gets to −10, only −5 is displayed and reflected in the user’s karma, but it would require 6 upvotes to get to −4), display as a ratio plus total number of votes, etc. - they all have their trade-offs though, so I’m not sure there’s a clear solution here. This is another place where tags are interesting, because if everyone things a comment is just slightly conflict-seeking, then they can use the “Slightly conflict-seeking” tag, and they can all vote on that without giving the comment author the impression that everyone thinks their comment is extremely conflict seeking.
Like I said, I love this experimentation—please keep at it! I think this topic is completely underappreciated by basically every social platform.
Thanks!