A lot of my recent thoughts on moderation have included an emphasis on “rate limiting as a primary mod tool.” This is very much in the “Ray has a hypothesis and it might not work out” phase, but I think it has promise.
The general principle is “moderators should be able to employ the least-amount-of-constraint/force possible to enforce a decision or give that decision teeth.” A full ban is a pretty extreme option, and being limited to that option makes me much more hesitant to take any kind of action against a person.
A rate limit is a pretty flexible tool. You can rate limit comments, posts, or (possibly) wordcount. You can limit by hour, day, week or more. It can be limited to a specific post, or across posts in general (i.e. N comments per post per unit-of-time), or sitewide (N comments total per unit of time)
But at least some people have reported experiencing limits as very costly, and have described it as “being banned.” Ruby has argued it might basically just feel like moderators are trying to ban people without really acknowledging the magnitude of what they’re doing.
I’m curious how people feel about it. I want to outline a bit how I’m thinking about it.
There’s roughly 3 ways I think I want to use rate-limits
1. As a soft, temporary tool to give moderator-warnings a little more teeth.
If someone(s) is/are spiralling into a demon thread, we could give them a temporary “1 comment per ~hour” rate limit on that particular post, to force them to slow down and chill out. If they’ve done something that seems particularly norm-violating, we can give them a temporary significant rate limit. The alternative here would be to do nothing, or give them a temporary 1-week ban. It seems at least in some cases better to do the rate-limit, although I could imagine in some cases “taking an actual break for a week” would be better.
2. As a response to serious problem users who are either new, or established but have been given previous warnings.
The alternatives are “permaban,” “doing nothing”, and “moderators consistently have to be-on-top-of-things giving ‘temp-bans’ or other local-reactions to serious problem users each time they do something bad.”
Doing nothing means leaving the garden untended, falling into entropy. “Moderators have to keep paying attention” is just… way too much work. From this perspective, a significant rate-limit is harsh, but the alternative seems probably worse. Rate limits give the user an option for proving they can change their commenting habits.
A thing that might change my mind here is if in practice, either most users just prefer exile to second-class-citizenship, or if longterm rate limits just in practice turn out to be a trap for people who really should move on.
3. As a default-state for new users.
New users sometimes show up and start commenting prolifically, before they really understand the site norms or background knowledge.
I think it’s a pretty reasonable position for “Users start out with one comment per day. If they get even slightly upvoted, they get a bit more commenting permissions, and if they get, say, 5 or 10 karma they become unfettered.” I predict that, once the site fully shifted over to this, it’d feel fairly reasonable and gamified in a positive way.
There’s an awkwardness right now of “there’s a lot of users I would have wanted to start out more rate-limited in this way, but who showed up in the past year or 2, who’ve been making lots of downvoted comments and not really learning from it.” So there is an awkward/painful readjustment period for those users.
The main alternative to this (which is what we’re actually doing right now, as of a couple weeks ago), is manually approving each individual comment from new users, and mostly not letting them post unfettered until they’ve posted at least one reasonable-looking one.
The other alternative is, of course, to just let people start posting/commenting willy nilly, but I think that’s mostly bad for both existing users (who get a much lower signal/noise ratio) and new users (who get downvoted and criticized a lot).
Problem 1: You can’t see which people are rate limited, and that makes conversations confusing.
Potential solution: Add a small icon on comments that conveys “this user is under a rate limit and can’t respond right away”. (potential problem with solution: feels too much like a Scarlet Letter? But I think it can be designed in a fairly subdued way that seems mostly utilitarian.
Problem 2: Maybe even with the icon, it’s just a lot of cognitive overhead for users keeping track of how rate limiting affects conversations?
I don’t know how big a deal this would be, and presumably it depends on how often it’s used. If it turned out to be a big deal in practice, it might be a major update for me.
Problem 3: Some authors might actively like users that are under a rate limit.
Potential solution: give authors a whitelist tool that lets them allow particular friends/colleagues to comment unfettered on their posts (or, on specific posts)
Problem 4: The new-user-setting might be too strict. Vladimir Nesov was worried about this, saying “There should be fast tracks that present no practical limits to the new users. First few comments should be available immediately upon registration, possibly regenerating quickly. This should only degrade if there is downvoting or no upvoting, and the limits should go away completely according to an algorithm that passes backtesting on first comments made by users in good standing who started commenting within the last 3-4 years.”
We have done some backtesting like this (looking at the early comments from good users), so we have a rough idea of what to expect here. There were some good-users-by-post-people’s-lights who got downvoted in their first few comments.
I’m… not sure I buy this is that big a problem? Obviously if this caused good users to give up on the site and leave, I’d consider that a major loss. But that’s not my first guess of what would happen – I think potentially-good-users would mostly see “oh, this place actually has standards. Lemme try and meet the standards and get in”, and maybe that takes them an extra day or so? That seems fine to me.
But if I were persuaded this was more prone-to-people-giving-up I’d update a lot.
(Currently we’re doing manual approval of new user comments which is I think mostly an even stricter regime than this implies. This user found it explicitly good to get rejected rather than subjected to downvotes and annoyed criticism. I’m not sure how common an experience that’ll be, and am unsure how to integrate this all into a new cohesive vision for new-user moderation)
...
...
Anyways, those are some thoughts about rate limiting. I am curious how people feel about this compared to other options.
“Rate limiting” as a mod tool
A lot of my recent thoughts on moderation have included an emphasis on “rate limiting as a primary mod tool.” This is very much in the “Ray has a hypothesis and it might not work out” phase, but I think it has promise.
The general principle is “moderators should be able to employ the least-amount-of-constraint/force possible to enforce a decision or give that decision teeth.” A full ban is a pretty extreme option, and being limited to that option makes me much more hesitant to take any kind of action against a person.
A rate limit is a pretty flexible tool. You can rate limit comments, posts, or (possibly) wordcount. You can limit by hour, day, week or more. It can be limited to a specific post, or across posts in general (i.e. N comments per post per unit-of-time), or sitewide (N comments total per unit of time)
But at least some people have reported experiencing limits as very costly, and have described it as “being banned.” Ruby has argued it might basically just feel like moderators are trying to ban people without really acknowledging the magnitude of what they’re doing.
I’m curious how people feel about it. I want to outline a bit how I’m thinking about it.
There’s roughly 3 ways I think I want to use rate-limits
1. As a soft, temporary tool to give moderator-warnings a little more teeth.
If someone(s) is/are spiralling into a demon thread, we could give them a temporary “1 comment per ~hour” rate limit on that particular post, to force them to slow down and chill out. If they’ve done something that seems particularly norm-violating, we can give them a temporary significant rate limit. The alternative here would be to do nothing, or give them a temporary 1-week ban. It seems at least in some cases better to do the rate-limit, although I could imagine in some cases “taking an actual break for a week” would be better.
2. As a response to serious problem users who are either new, or established but have been given previous warnings.
The alternatives are “permaban,” “doing nothing”, and “moderators consistently have to be-on-top-of-things giving ‘temp-bans’ or other local-reactions to serious problem users each time they do something bad.”
Doing nothing means leaving the garden untended, falling into entropy. “Moderators have to keep paying attention” is just… way too much work. From this perspective, a significant rate-limit is harsh, but the alternative seems probably worse. Rate limits give the user an option for proving they can change their commenting habits.
A thing that might change my mind here is if in practice, either most users just prefer exile to second-class-citizenship, or if longterm rate limits just in practice turn out to be a trap for people who really should move on.
3. As a default-state for new users.
New users sometimes show up and start commenting prolifically, before they really understand the site norms or background knowledge.
I think it’s a pretty reasonable position for “Users start out with one comment per day. If they get even slightly upvoted, they get a bit more commenting permissions, and if they get, say, 5 or 10 karma they become unfettered.” I predict that, once the site fully shifted over to this, it’d feel fairly reasonable and gamified in a positive way.
There’s an awkwardness right now of “there’s a lot of users I would have wanted to start out more rate-limited in this way, but who showed up in the past year or 2, who’ve been making lots of downvoted comments and not really learning from it.” So there is an awkward/painful readjustment period for those users.
The main alternative to this (which is what we’re actually doing right now, as of a couple weeks ago), is manually approving each individual comment from new users, and mostly not letting them post unfettered until they’ve posted at least one reasonable-looking one.
The other alternative is, of course, to just let people start posting/commenting willy nilly, but I think that’s mostly bad for both existing users (who get a much lower signal/noise ratio) and new users (who get downvoted and criticized a lot).
(note: I previously wrote about this here)
Potential risks/costs/problems
Problem 1: You can’t see which people are rate limited, and that makes conversations confusing.
Potential solution: Add a small icon on comments that conveys “this user is under a rate limit and can’t respond right away”. (potential problem with solution: feels too much like a Scarlet Letter? But I think it can be designed in a fairly subdued way that seems mostly utilitarian.
Problem 2: Maybe even with the icon, it’s just a lot of cognitive overhead for users keeping track of how rate limiting affects conversations?
I don’t know how big a deal this would be, and presumably it depends on how often it’s used. If it turned out to be a big deal in practice, it might be a major update for me.
Problem 3: Some authors might actively like users that are under a rate limit.
Potential solution: give authors a whitelist tool that lets them allow particular friends/colleagues to comment unfettered on their posts (or, on specific posts)
Problem 4: The new-user-setting might be too strict. Vladimir Nesov was worried about this, saying “There should be fast tracks that present no practical limits to the new users. First few comments should be available immediately upon registration, possibly regenerating quickly. This should only degrade if there is downvoting or no upvoting, and the limits should go away completely according to an algorithm that passes backtesting on first comments made by users in good standing who started commenting within the last 3-4 years.”
We have done some backtesting like this (looking at the early comments from good users), so we have a rough idea of what to expect here. There were some good-users-by-post-people’s-lights who got downvoted in their first few comments.
I’m… not sure I buy this is that big a problem? Obviously if this caused good users to give up on the site and leave, I’d consider that a major loss. But that’s not my first guess of what would happen – I think potentially-good-users would mostly see “oh, this place actually has standards. Lemme try and meet the standards and get in”, and maybe that takes them an extra day or so? That seems fine to me.
But if I were persuaded this was more prone-to-people-giving-up I’d update a lot.
(Currently we’re doing manual approval of new user comments which is I think mostly an even stricter regime than this implies. This user found it explicitly good to get rejected rather than subjected to downvotes and annoyed criticism. I’m not sure how common an experience that’ll be, and am unsure how to integrate this all into a new cohesive vision for new-user moderation)
...
...
Anyways, those are some thoughts about rate limiting. I am curious how people feel about this compared to other options.