I generally like surveys! Here’s a silly little survey I did that 100 people filled out, here’s a survey John Wentworth did on people’s technical background that 250 people that I know informs his writing. I think small surveys that directly answer key questions are very cheap and worthwhile.
It’s important to do a good job on a survey that you try to make the schelling annual survey for ~10k people on the site to complete. One user made a mess of it in 2017 and the survey died (link, same link with different comments), and another user also didn’t succeed in reviving in 2020 (link).
I think it’d be a nice-to-have to get an annual survey going, especially if it was run by someone who was trying to test particular hypotheses. For instance, if it were me, a bunch of questions on how users use the site that will help the LW team inform new feature development.
So I think it’s fine for you to do a basic demographics-and-beliefs survey, though I think it’s a bit much to demand/expect everyone to take your survey. Calling it “General Census” is a demand that people should actually fill it out that you have to develop buy-in for. Maybe you get lucky and everyone actually fills it out, but if it doesn’t then those who filled it out will be unhappy with you for making them spend effort on a stag hunt where they didn’t get the stag, and also people will trust you less-than-baseline in the future for such stag hunts.
I don’t want to block people from trying things (which is why I didn’t try to in our brief PMs and shared you on the q’s I’d gathered), nor am I freely endorsing any user who wants to run a survey that tries to take up 100s of hours of LW users.
I want to do a good job on this one. The decision to mostly re-use previous questions was a deliberate attempt to return to the Scott Alexander era, and my main changes were in trying to avoid what I see as the flaws in 2017 and 2020. 2017 used special software that wound up with software issues, so I went back to Google Forms. 2020 didn’t get seen by very many people, so I made an effort to get this one more visibility. So far nobody has mentioned a software issue and we already have more responses than 2020, so I’m feeling relatively good about how it’s going.
The goal of having a clear return to form overrode most of the new questions I wanted to explore. The remnants of that are the “Do You Organize Less Wrong Meetups?” question (I’m curious what percentage of people who read LW go to meetups, and what percentage of people who go to meetups run them!) and the “Most Important Lesson” question that I plan to feed directly into Meetups-In-A-Box activities to emphasize those lessons. In the end I decided I wanted the census to look familiar and reliable, so the new questions in Section 10 are mostly goofy and everything else is pretty standard.
“Demand” seems a bit stronger than I wanted to communicate but we might have a difference in connotations. It’s now called the Unofficial General Census, which might lower that somewhat. I was going for “encourage” rather than “demand.” Still, if this attempt flops and people subtract some of my Calling For Stag Hunt points, I would find that reasonable and fair. I’m also happy with the amount of endorsement; this census isn’t official, I’m not associated with the LessWrong team, most surveys shouldn’t wind up in front of the whole site, but this census isn’t blocked or dis-endorsed either.
I generally like surveys! Here’s a silly little survey I did that 100 people filled out, here’s a survey John Wentworth did on people’s technical background that 250 people that I know informs his writing. I think small surveys that directly answer key questions are very cheap and worthwhile.
It’s important to do a good job on a survey that you try to make the schelling annual survey for ~10k people on the site to complete. One user made a mess of it in 2017 and the survey died (link, same link with different comments), and another user also didn’t succeed in reviving in 2020 (link).
I think it’d be a nice-to-have to get an annual survey going, especially if it was run by someone who was trying to test particular hypotheses. For instance, if it were me, a bunch of questions on how users use the site that will help the LW team inform new feature development.
So I think it’s fine for you to do a basic demographics-and-beliefs survey, though I think it’s a bit much to demand/expect everyone to take your survey. Calling it “General Census” is a demand that people should actually fill it out that you have to develop buy-in for. Maybe you get lucky and everyone actually fills it out, but if it doesn’t then those who filled it out will be unhappy with you for making them spend effort on a stag hunt where they didn’t get the stag, and also people will trust you less-than-baseline in the future for such stag hunts.
I don’t want to block people from trying things (which is why I didn’t try to in our brief PMs and shared you on the q’s I’d gathered), nor am I freely endorsing any user who wants to run a survey that tries to take up 100s of hours of LW users.
I want to do a good job on this one. The decision to mostly re-use previous questions was a deliberate attempt to return to the Scott Alexander era, and my main changes were in trying to avoid what I see as the flaws in 2017 and 2020. 2017 used special software that wound up with software issues, so I went back to Google Forms. 2020 didn’t get seen by very many people, so I made an effort to get this one more visibility. So far nobody has mentioned a software issue and we already have more responses than 2020, so I’m feeling relatively good about how it’s going.
The goal of having a clear return to form overrode most of the new questions I wanted to explore. The remnants of that are the “Do You Organize Less Wrong Meetups?” question (I’m curious what percentage of people who read LW go to meetups, and what percentage of people who go to meetups run them!) and the “Most Important Lesson” question that I plan to feed directly into Meetups-In-A-Box activities to emphasize those lessons. In the end I decided I wanted the census to look familiar and reliable, so the new questions in Section 10 are mostly goofy and everything else is pretty standard.
“Demand” seems a bit stronger than I wanted to communicate but we might have a difference in connotations. It’s now called the Unofficial General Census, which might lower that somewhat. I was going for “encourage” rather than “demand.” Still, if this attempt flops and people subtract some of my Calling For Stag Hunt points, I would find that reasonable and fair. I’m also happy with the amount of endorsement; this census isn’t official, I’m not associated with the LessWrong team, most surveys shouldn’t wind up in front of the whole site, but this census isn’t blocked or dis-endorsed either.
Thanks. All seems good here to me.