I’ve moderated a few forums before, and with that experience in mind I’d have to agree that there’s a huge, and generally hugely negative, selection bias at play in online response to moderator decisions. It’d be foolish to take those responses as representative of the entire userbase, and I’ve seen more than one forum suffer as a result of such a misconception.
That being said, though, I think it’s risky to write off online user feedback in favor of physical. The people you encounter privately are just as much a filtered set as those who post feedback here, though the filters point in different directions: you’re selecting people involved in the LW interpersonal community, for one thing, which filters out new and casual users right off the bat, and since they’re probably more likely to be personally friendly to you we can also expect affect heuristics to come into play. Skepticism toward certain LW norms may also be selected against, which could lead people to favor new policies reinforcing those norms. Moreover, I’ve noticed a trend in the Bay Area group—not necessarily an irrational one, but a noticeable one—toward treating the online community as low-quality relative to local groups, which we might expect to translate into antipathy towards its status quo.
I don’t know what the weightings should be, but if you’re looking for a representative measure of user preferences I think it’d be wise to take both groups into account to some extent.
I’ve moderated a few forums before, and with that experience in mind I’d have to agree that there’s a huge, and generally hugely negative, selection bias at play in online response to moderator decisions. It’d be foolish to take those responses as representative of the entire userbase, and I’ve seen more than one forum suffer as a result of such a misconception.
That being said, though, I think it’s risky to write off online user feedback in favor of physical. The people you encounter privately are just as much a filtered set as those who post feedback here, though the filters point in different directions: you’re selecting people involved in the LW interpersonal community, for one thing, which filters out new and casual users right off the bat, and since they’re probably more likely to be personally friendly to you we can also expect affect heuristics to come into play. Skepticism toward certain LW norms may also be selected against, which could lead people to favor new policies reinforcing those norms. Moreover, I’ve noticed a trend in the Bay Area group—not necessarily an irrational one, but a noticeable one—toward treating the online community as low-quality relative to local groups, which we might expect to translate into antipathy towards its status quo.
I don’t know what the weightings should be, but if you’re looking for a representative measure of user preferences I think it’d be wise to take both groups into account to some extent.