Based mostly on Unnamed’s comment regarding a poll on homosexuality, I expect it would be lower than what you found for self-selected interested respondents, but I don’t have much confidence in this, and I’m not familiar enough with AQ variance to say how much lower.
I have some experience with trying to do survey research right (i.e. in a manner that results in true conclusions about the population), and I have seen so many ways that convenience samples can make me wrong (non-response bias is just a start), that I generally don’t bother much with trying to correct for one particular bias. Instead, I stick to random samples or use estimators that can handle convenience samples. But I am genuinely intrigued by your idea (from the very end of your post) to use a prior on non-response bias and a prior on AQ distribution to estimate the true AQ distribution of LW users. This wouldn’t handle e.g. problems with question wording or lies in responses, but I would expect those problems to be small in comparison to the non-response bias. Could you elaborate, or point me to any survey methods papers that explain this approach in detail? Where would you get the priors?
Based mostly on Unnamed’s comment regarding a poll on homosexuality, I expect it would be lower than what you found for self-selected interested respondents, but I don’t have much confidence in this, and I’m not familiar enough with AQ variance to say how much lower.
I have some experience with trying to do survey research right (i.e. in a manner that results in true conclusions about the population), and I have seen so many ways that convenience samples can make me wrong (non-response bias is just a start), that I generally don’t bother much with trying to correct for one particular bias. Instead, I stick to random samples or use estimators that can handle convenience samples. But I am genuinely intrigued by your idea (from the very end of your post) to use a prior on non-response bias and a prior on AQ distribution to estimate the true AQ distribution of LW users. This wouldn’t handle e.g. problems with question wording or lies in responses, but I would expect those problems to be small in comparison to the non-response bias. Could you elaborate, or point me to any survey methods papers that explain this approach in detail? Where would you get the priors?