Empirical tests are important. In fact, I would probably alter the publicly-visible workings of the site to get better data that can help us discriminate between the competing hypotheses about how LW continues to have the high s/n ratio it does have and about how to protect it from various stresses (like discussion of mind-killers). The online rationality quiz hosted by darius at http://wry.me:7002/ is an example of the kind of thing I would add to the site.
As to my hypothesis that growing the membership too quickly risks LW becoming just another online community, that can be tested without altering the public face of LW: in particular, it can be tested by collecting statistics on the rate of the appearance of new commentators, posters and voters and by comparing that to the rate at which established voters stop voting.
I will admit that I stopped voting about 6 weeks ago in response to the increase in bad comments probably caused by influx of new participants from the Harry Potter fanfic and the (simultaneous with the fanfic) efforts by one of the SIAI visiting fellows to apply SEO techniques to the site. I am by disposition easily annoyed, and when I exercise my “critical / judgmental” mental state enough, I tend to get caught up in that state and cannot get myself out of it, which bad for my health and for my personal relationships, and it was just too annoying and too internally costly to pay enough attention to comments by new writers and the usual old writers of poor-quality comments to vote on them. So, you see, I already have one piece of empirical evidence (namely my observation of my own behavior) that LW is more stressed than it was 6 months ago, and consequently now is not the time to add a new stressor.
The only way I know of for LW to continue to have a positive effect on the world is for it to keep the quality of its conversation high. (Of course, the benefits it already has had on the world, e.g., in the form of increasing the rationality of readers, can compound themselves, e.g., by the more rational readers and former readers doing good in the world outside LW. But that would happen even if LW disappeared today.)
jimrandomh has identified a very potent way to improve the world: try to apply whatever processes make LW as good as it is to a greater fraction of the public discourse.
The strongest argument I know of against jimrandomh’s proposal is that it does not talk about why he thinks LW came to have the high s/r ratio it does have and how and under what conditions that quality is likely to be lost.
As to my hypothesis that growing the membership too quickly risks LW becoming just another online community, that can be tested without altering the public face of LW: in particular, it can be tested by collecting statistics on the rate of the appearance of new commentators, posters and voters and by comparing that to the rate at which established voters stop voting.
The most damning argument I know of against jimrandomh’s proposal is that it does not talk about why he thinks LW came to have the high s/r ratio it does have and how and under what conditions that quality is likely to be lost.
Up voted for this. We need to take a good hard look at our current user base and figure out why Lesswrong seems to be (mostly) working.
Quickly growing membership can indeed endanger the quality of the discourse and lower the s/n ratio; an important question is how quickly is too quickly. I suppose there are lots of rational people out there who are unaware of LW from whose membership the community may profit.
If the high quality is a mere result of high average rationality level of current members, it would be sufficient to filter newcomers according to their rationality. That can be done by karma system and the growth can be still pretty quick (I assume somebody repeatedly voted to negative total would lose interest in matter of days.)
If, on the other hand, the main reason for the high s/n ratio lies in some surplus quality which isn’t simple result of members’ individual rationality, but rather set of customs, unwritten laws, atmosphere, or hard to describe “spirit” of the community, then the acceptable growth rates would be much slower, as new members have to be acclimatised before they become a majority.
Then we need data of course. Are there any accessible membership statistics showing a systematically increasing rate of new registrations? Is there a real danger, or is the influx of new members only a temporary phenomenon? And why do you think jimrandomh’s proposal to broaden the scope of LW implies larger membership increase? (I also think so, but I would like to hear your reasons.)
Anyway, you seem to have some ideas about how to preserve the LW’s quality; perhaps you could write some summary thereof either as a comment here or on the top level.
Is there a real danger, or is the influx of new members only a temporary phenomenon?
IMHO a large temporary influx of new members like LW has had over the last 6 months is a real danger to the quality of the conversation on LW because once enough of the “pillars of the community” become discouraged and leave, nothing short of the establishment of a new web site or some kind of “hard reset” of LW would have a significant probability of bringing the quality back again.
We seem to be holding for now, but my impression is that we could easily go off course in a mere matter of months if we don’t tread carefully.
We need to analyze what the top contributors think and feel about the community and what motivates them to participate. We also need to figure out what the masses of registered posters are like, when did they arrive and for what periods they where active.
prase is not sure whether “the high quality [of LW] is a mere result of high average rationality level of current members” or whether “the main reason for the high s/n ratio lies in some surplus quality which isn’t simple result of members’ individual rationality, but rather set of customs, unwritten laws, atmosphere, or hard to describe “spirit” of the community”.
I am not sure either, but if it were easy for highly rational people to combine their rationality in organizations, associations or public discourses, I would expect organizations, associations and public discourses to have been more effective and less pernicious than they have. Consider organizations like the CIA, FBI, Harvard University and the White House that can pick their employees from a large pool of extremely committed, intelligent and well-educated applicants. Consider the public discourse conducted by elite journalists and editors during heyday of the New York Times, Washington Post, etc. Those elite journalists for example contributed greatly to the irrational witch hunt around sexual abuse at day-care centers and “recovered memory” sex-abuse trials in the 1980s.
You are probably right, and I tend now to favour slower membership growth.
But another issue comes to mind: we should have some more objective methods to measure the s/n ratio whether or not membership increases, because any community are in danger of falling prey to mutual reassurances how great and exceptional they are.
There is the danger that LW will become a mutual-admiration society, but if it does, the worst effect will probably be that people like you and I will have to find other places for discussion.
If SIAI becomes a mutual-admiration society, that is more serious, but LWers who are not SIAI insiders will have little control over whether that happens. (And the insiders I have gotten to know certainly seem able enough to prevent the possibility.)
So the question becomes, Is the risk that LW will become a mutual-admiration society higher than the risk that “confronting wrongness wherever it appears” (jimrandomh’s proposal in jimrandomh’s words) will change LW in such a way that the voters and commentators who have made it what it is will stop voting or commentating?
I haven’t meant it as a dilemma “mutual admiration society” vs. “indiscriminate battle against wrongness”, that would hardly make sense. I am even not really afraid of becoming mutual admiration society or cult or something like.
I only intended to ask a question (more or less unrelated to the original discussion): how reliably do we know that the s/n ratio is really high? There is a lot of room for bias here, since “this is an exceptionally rational community” is what we like to hear, while people with different opinion aren’t heard: why would they participate in a rationalist community, if they thought it weren’t so much rational after all? Put in another way, any community which values rationality—independently on how do they define it and whether they really meet their needs—is likely to produce such self-assuring statements.
So when I hear about how LW is great, I am a little bit worried that my (and everybody else’s) agreement may be biased. As always, a good thing would be to have either an independent judge, or a set of objective criteria and tests. That could also help to determine whether the LW standards are improving or deteriorating in time.
Empirical tests are important. In fact, I would probably alter the publicly-visible workings of the site to get better data that can help us discriminate between the competing hypotheses about how LW continues to have the high s/n ratio it does have and about how to protect it from various stresses (like discussion of mind-killers). The online rationality quiz hosted by darius at http://wry.me:7002/ is an example of the kind of thing I would add to the site.
As to my hypothesis that growing the membership too quickly risks LW becoming just another online community, that can be tested without altering the public face of LW: in particular, it can be tested by collecting statistics on the rate of the appearance of new commentators, posters and voters and by comparing that to the rate at which established voters stop voting.
I will admit that I stopped voting about 6 weeks ago in response to the increase in bad comments probably caused by influx of new participants from the Harry Potter fanfic and the (simultaneous with the fanfic) efforts by one of the SIAI visiting fellows to apply SEO techniques to the site. I am by disposition easily annoyed, and when I exercise my “critical / judgmental” mental state enough, I tend to get caught up in that state and cannot get myself out of it, which bad for my health and for my personal relationships, and it was just too annoying and too internally costly to pay enough attention to comments by new writers and the usual old writers of poor-quality comments to vote on them. So, you see, I already have one piece of empirical evidence (namely my observation of my own behavior) that LW is more stressed than it was 6 months ago, and consequently now is not the time to add a new stressor.
The only way I know of for LW to continue to have a positive effect on the world is for it to keep the quality of its conversation high. (Of course, the benefits it already has had on the world, e.g., in the form of increasing the rationality of readers, can compound themselves, e.g., by the more rational readers and former readers doing good in the world outside LW. But that would happen even if LW disappeared today.)
jimrandomh has identified a very potent way to improve the world: try to apply whatever processes make LW as good as it is to a greater fraction of the public discourse.
The strongest argument I know of against jimrandomh’s proposal is that it does not talk about why he thinks LW came to have the high s/r ratio it does have and how and under what conditions that quality is likely to be lost.
Up voted for this. We need to take a good hard look at our current user base and figure out why Lesswrong seems to be (mostly) working.
Quickly growing membership can indeed endanger the quality of the discourse and lower the s/n ratio; an important question is how quickly is too quickly. I suppose there are lots of rational people out there who are unaware of LW from whose membership the community may profit.
If the high quality is a mere result of high average rationality level of current members, it would be sufficient to filter newcomers according to their rationality. That can be done by karma system and the growth can be still pretty quick (I assume somebody repeatedly voted to negative total would lose interest in matter of days.)
If, on the other hand, the main reason for the high s/n ratio lies in some surplus quality which isn’t simple result of members’ individual rationality, but rather set of customs, unwritten laws, atmosphere, or hard to describe “spirit” of the community, then the acceptable growth rates would be much slower, as new members have to be acclimatised before they become a majority.
Then we need data of course. Are there any accessible membership statistics showing a systematically increasing rate of new registrations? Is there a real danger, or is the influx of new members only a temporary phenomenon? And why do you think jimrandomh’s proposal to broaden the scope of LW implies larger membership increase? (I also think so, but I would like to hear your reasons.)
Anyway, you seem to have some ideas about how to preserve the LW’s quality; perhaps you could write some summary thereof either as a comment here or on the top level.
IMHO a large temporary influx of new members like LW has had over the last 6 months is a real danger to the quality of the conversation on LW because once enough of the “pillars of the community” become discouraged and leave, nothing short of the establishment of a new web site or some kind of “hard reset” of LW would have a significant probability of bringing the quality back again.
We seem to be holding for now, but my impression is that we could easily go off course in a mere matter of months if we don’t tread carefully.
We need to analyze what the top contributors think and feel about the community and what motivates them to participate. We also need to figure out what the masses of registered posters are like, when did they arrive and for what periods they where active.
prase is not sure whether “the high quality [of LW] is a mere result of high average rationality level of current members” or whether “the main reason for the high s/n ratio lies in some surplus quality which isn’t simple result of members’ individual rationality, but rather set of customs, unwritten laws, atmosphere, or hard to describe “spirit” of the community”.
I am not sure either, but if it were easy for highly rational people to combine their rationality in organizations, associations or public discourses, I would expect organizations, associations and public discourses to have been more effective and less pernicious than they have. Consider organizations like the CIA, FBI, Harvard University and the White House that can pick their employees from a large pool of extremely committed, intelligent and well-educated applicants. Consider the public discourse conducted by elite journalists and editors during heyday of the New York Times, Washington Post, etc. Those elite journalists for example contributed greatly to the irrational witch hunt around sexual abuse at day-care centers and “recovered memory” sex-abuse trials in the 1980s.
You are probably right, and I tend now to favour slower membership growth.
But another issue comes to mind: we should have some more objective methods to measure the s/n ratio whether or not membership increases, because any community are in danger of falling prey to mutual reassurances how great and exceptional they are.
There is the danger that LW will become a mutual-admiration society, but if it does, the worst effect will probably be that people like you and I will have to find other places for discussion.
If SIAI becomes a mutual-admiration society, that is more serious, but LWers who are not SIAI insiders will have little control over whether that happens. (And the insiders I have gotten to know certainly seem able enough to prevent the possibility.)
So the question becomes, Is the risk that LW will become a mutual-admiration society higher than the risk that “confronting wrongness wherever it appears” (jimrandomh’s proposal in jimrandomh’s words) will change LW in such a way that the voters and commentators who have made it what it is will stop voting or commentating?
I haven’t meant it as a dilemma “mutual admiration society” vs. “indiscriminate battle against wrongness”, that would hardly make sense. I am even not really afraid of becoming mutual admiration society or cult or something like.
I only intended to ask a question (more or less unrelated to the original discussion): how reliably do we know that the s/n ratio is really high? There is a lot of room for bias here, since “this is an exceptionally rational community” is what we like to hear, while people with different opinion aren’t heard: why would they participate in a rationalist community, if they thought it weren’t so much rational after all? Put in another way, any community which values rationality—independently on how do they define it and whether they really meet their needs—is likely to produce such self-assuring statements.
So when I hear about how LW is great, I am a little bit worried that my (and everybody else’s) agreement may be biased. As always, a good thing would be to have either an independent judge, or a set of objective criteria and tests. That could also help to determine whether the LW standards are improving or deteriorating in time.