Even then, the response bias might be bad enough to preclude any conclusions about LessWrong.
Yes, this is a good point.
Note that it’s still evidence, in the sense that changing the results of the survey ought to change your opinions about the AS-spectrum average over the cluster of Less Wrong readers.
Of course there is a potentially huge response bias—according to sitemeter there were 3000 visitors today, meaning that the response bias could in theory be up to 60 x . (which means that with maximum response bias, i.e. all nonresponders are scoring an average of 16 on the AQ test and don’t have a diagnosis, LW could have a normal AS rate.)
various factors will affect who responds, but until you have a hypothesis about which way that bias points, all that does is add noise. I would predict that random sampling of people who ever look at the LW homepage would yield lower AS-spectrum results, but that random sampling of people who read regularly and/or comment probably wouldn’t.
Yes, this is a good point.
Note that it’s still evidence, in the sense that changing the results of the survey ought to change your opinions about the AS-spectrum average over the cluster of Less Wrong readers.
Of course there is a potentially huge response bias—according to sitemeter there were 3000 visitors today, meaning that the response bias could in theory be up to 60 x . (which means that with maximum response bias, i.e. all nonresponders are scoring an average of 16 on the AQ test and don’t have a diagnosis, LW could have a normal AS rate.)
various factors will affect who responds, but until you have a hypothesis about which way that bias points, all that does is add noise. I would predict that random sampling of people who ever look at the LW homepage would yield lower AS-spectrum results, but that random sampling of people who read regularly and/or comment probably wouldn’t.
Wouldn’t the maximum response bias be if all the non-responders scored 0 on the scale?
This is highly implausible since we didn’t get a single response below 10, but yes, technically that would be an even bigger bias.