Another thing I’ve noticed is that almost all the users are trying.
I haven’t thought about whether these rate-limits are justified (I currently think at least 1⁄4 of them are unjustified and 1⁄2 are okay), but I want to point out that post/comment quality is real. That is, some users have higher quality comments than others (due to reasoning in the comment, combativeness, how often this leads to good discussion, etc.) often for illegible reasons, this substantially affects the value readers get, and this is predictive of their future content. It follows that if moderators want to reduce the incidence of low-quality content beyond what is caught by simple rules, then they cannot defend themselves perfectly against accusations of arbitrariness. The signal-to-noise ratio of LW is very important, and IMO this justifies mods making judgment calls.
Take MiguelDev, who posts extremely long posts consisting mostly of LLM output. My guess is that the experiments are mediocre due to lack of rigor, with a small possibility that they are good. They are not egregiously bad. But as evidenced by the low karma, few people get value from reading these extremely long posts. I would like to see much less of this content on the frontpage because it decreases the SNR; maybe three posts per year is okay. Therefore I’m fine with this user being rate-limited by moderator fiat to something like one post per month. If moderators started rate-limiting Nora Belrose or someone else whose work I thought was particularly good, they would lose my confidence, but this hasn’t happened yet.
I agree about providing explanations for bans or ratelimits that are functionally bans though.
[note: I am technically a mod but in practice that means I give the team my opinions and occasionally curate something. The following is my opinion. I think it has some overlap with team members’ opinion but I don’t know how much, or how much of that opinion has made it into policy]
A counterintuitive effect is that bad comments are often worse for SNR than bad posts. Bad posts seem like they’d be worse, because posts are more important than comments and are placed higher. But karma works better (on average) on posts: bad or mediocre posts get enough downvotes, or at least not enough upvotes to compete, and gently disappear. But comments’ views scale with the viewership of the original post, so a mediocre comment on a popular post will get lots of attention regardless of its karma. If a post gets enough comments that low karma comments can’t get much attention, they still compete with new high-quality comments, and cut into the attention for the latter.
And even if no one else sees a bad comment, they are still likely to be read by the author and annoy them. If this gets bad enough, authors may stop reading their comment sections or stop posting altogether.
If a post gets enough comments that low karma comments can’t get much attention, they still compete with new high-quality comments, and cut into the attention for the latter.
Seems like this could be addressed by changing the comment sorting algorithm to favor recent comments more?
I disagree. Negative comments often provide feedback to the author he wouldn’t get elsewhere. And if you are annoyed by it you can filter them out (settings → hide low votes).
No. There can be many means in between or different altogether.
But back to my original comment: It was about the not made explicit action of what to do with bad comments. I agree that the dynamic for posts and comments is different. But I disagree with what I saw was the push that negative comments should be stronger discouraged because they have higher weight.
But when rereading, I see that you don’t say what to do about these comments. You only point out negative effects. What is your proposal?
Note: I’m in favor of tending the garden and discouraging orcs and banning trolls. But I’m also in favor of critical and negative remarks. Reduce their visibility maybe, but don’t completely prevent them.
AFAIK there was a wave of rate limits, not bans. I think it’s a huge error to conflate those. Most importantly, you can complain on-site about being rate limited in a way you can’t complain about being banned.
I have complaints about implementation but the theory seems sound. I’d like the team to put more work into implementation or treat false positives as more costly, but that’s easy for me to say since I’m not the one that has to do it.
Complaints:
the combination of imperfect filtering and no communication seems bad to me. How are people supposed to know their ban was a mistake and asking will help, instead of annoying mods further.
“retroactive to a year ago” sounds pretty bad to me. But I don’t think that’s the right frame. I think the team meant to intervene and not rate limit people who’d had an issue 11 months ago but have been great since. habryka described at least one ban as a mistake in comments on this post, so sounds like this was inconsistent. But conceptually I think it was supposed to be “we have a new tool for detecting people who have been below standards this entire time” not “we raised the bar”.
I’m not seeing any active rate limits. Do you know when you observed it? It’s certainly the case that an automatic rate limit could have kicked in and then, as voting changed, been removed.
Yeah, I am also not seeing anything. Maybe it was something temporary, but I thought we had set it up to leave a trace if any automatic rate limits got applied in the past.
Curious what symptom Nora observed (GreaterWrong has been having some problems with rate-limit warnings that I’ve been confused by, so I can imagine that looking like a rate-limit from our side).
I still agree with myself above and think this is a bad moderation decision. Although I don’t know the full story and don’t see you on the moderation log.
Any published examples of work product by the moderators or any moderators of a similar site? Even if the reasoning is black box the output should be inspectable.
“This user I will give a 5 for quality, and this a 3, and see here this user made an attack here and see how 10 comments later everyone is arguing in in this series of exchanges”.
I wasn’t aware anything like this existed, reddit I thought was SOTA, and that’s automated karma visibility, stateless, and mods usually only ban for bright line rules or abuse their power and ban any dissent.
I was accused of degrading whole comment sections, I just want to see it. Show how you know this happens. Or if not me, any examples of this.
All I see is there’s maybe a back and forth with 1 user. Unremarkable on reddit, what’s the harm?
Also I wanted you to note the karma system is a hair from auto banning Nora. She seems to be afloat only a few points recently.
I haven’t thought about whether these rate-limits are justified (I currently think at least 1⁄4 of them are unjustified and 1⁄2 are okay), but I want to point out that post/comment quality is real. That is, some users have higher quality comments than others (due to reasoning in the comment, combativeness, how often this leads to good discussion, etc.) often for illegible reasons, this substantially affects the value readers get, and this is predictive of their future content. It follows that if moderators want to reduce the incidence of low-quality content beyond what is caught by simple rules, then they cannot defend themselves perfectly against accusations of arbitrariness. The signal-to-noise ratio of LW is very important, and IMO this justifies mods making judgment calls.
Take MiguelDev, who posts extremely long posts consisting mostly of LLM output. My guess is that the experiments are mediocre due to lack of rigor, with a small possibility that they are good. They are not egregiously bad. But as evidenced by the low karma, few people get value from reading these extremely long posts. I would like to see much less of this content on the frontpage because it decreases the SNR; maybe three posts per year is okay. Therefore I’m fine with this user being rate-limited by moderator fiat to something like one post per month. If moderators started rate-limiting Nora Belrose or someone else whose work I thought was particularly good, they would lose my confidence, but this hasn’t happened yet.
I agree about providing explanations for bans or ratelimits that are functionally bans though.
[note: I am technically a mod but in practice that means I give the team my opinions and occasionally curate something. The following is my opinion. I think it has some overlap with team members’ opinion but I don’t know how much, or how much of that opinion has made it into policy]
A counterintuitive effect is that bad comments are often worse for SNR than bad posts. Bad posts seem like they’d be worse, because posts are more important than comments and are placed higher. But karma works better (on average) on posts: bad or mediocre posts get enough downvotes, or at least not enough upvotes to compete, and gently disappear. But comments’ views scale with the viewership of the original post, so a mediocre comment on a popular post will get lots of attention regardless of its karma. If a post gets enough comments that low karma comments can’t get much attention, they still compete with new high-quality comments, and cut into the attention for the latter.
And even if no one else sees a bad comment, they are still likely to be read by the author and annoy them. If this gets bad enough, authors may stop reading their comment sections or stop posting altogether.
Seems like this could be addressed by changing the comment sorting algorithm to favor recent comments more?
I disagree. Negative comments often provide feedback to the author he wouldn’t get elsewhere. And if you are annoyed by it you can filter them out (settings → hide low votes).
Shouldn’t one make a distinction between negative/critical and bad?
Sure, but literally bad ones will quickly get downvoted and the poster banned. This is about the less clearcut cases, right?
It sounds like you don’t think there should be any user-focused mod response between “nothing” and “banned”. Is that correct?
No. There can be many means in between or different altogether.
But back to my original comment: It was about the not made explicit action of what to do with bad comments. I agree that the dynamic for posts and comments is different. But I disagree with what I saw was the push that negative comments should be stronger discouraged because they have higher weight.
But when rereading, I see that you don’t say what to do about these comments. You only point out negative effects. What is your proposal?
Note: I’m in favor of tending the garden and discouraging orcs and banning trolls. But I’m also in favor of critical and negative remarks. Reduce their visibility maybe, but don’t completely prevent them.
Rate limiting. If I was pope I’d make a few tweaks, but I think the concept is fundamentally sound and the implementation good enough.
Then we agree about the general moderation of LW.
Did your comment also apply to the latest automated bans?
AFAIK there was a wave of rate limits, not bans. I think it’s a huge error to conflate those. Most importantly, you can complain on-site about being rate limited in a way you can’t complain about being banned.
I have complaints about implementation but the theory seems sound. I’d like the team to put more work into implementation or treat false positives as more costly, but that’s easy for me to say since I’m not the one that has to do it.
Complaints:
the combination of imperfect filtering and no communication seems bad to me. How are people supposed to know their ban was a mistake and asking will help, instead of annoying mods further.
“retroactive to a year ago” sounds pretty bad to me. But I don’t think that’s the right frame. I think the team meant to intervene and not rate limit people who’d had an issue 11 months ago but have been great since. habryka described at least one ban as a mistake in comments on this post, so sounds like this was inconsistent. But conceptually I think it was supposed to be “we have a new tool for detecting people who have been below standards this entire time” not “we raised the bar”.
I actually did get rate-limited today, unfortunately.
I’m not seeing any active rate limits. Do you know when you observed it? It’s certainly the case that an automatic rate limit could have kicked in and then, as voting changed, been removed.
Yeah, I am also not seeing anything. Maybe it was something temporary, but I thought we had set it up to leave a trace if any automatic rate limits got applied in the past.
Curious what symptom Nora observed (GreaterWrong has been having some problems with rate-limit warnings that I’ve been confused by, so I can imagine that looking like a rate-limit from our side).
I still agree with myself above and think this is a bad moderation decision. Although I don’t know the full story and don’t see you on the moderation log.
I don’t know what caused it exactly, and it seems like I’m not rate limited anymore.
Any published examples of work product by the moderators or any moderators of a similar site? Even if the reasoning is black box the output should be inspectable.
“This user I will give a 5 for quality, and this a 3, and see here this user made an attack here and see how 10 comments later everyone is arguing in in this series of exchanges”.
I wasn’t aware anything like this existed, reddit I thought was SOTA, and that’s automated karma visibility, stateless, and mods usually only ban for bright line rules or abuse their power and ban any dissent.
I was accused of degrading whole comment sections, I just want to see it. Show how you know this happens. Or if not me, any examples of this.
All I see is there’s maybe a back and forth with 1 user. Unremarkable on reddit, what’s the harm?
Also I wanted you to note the karma system is a hair from auto banning Nora. She seems to be afloat only a few points recently.