I actually mostly agree with you. I hesitated a long time before posting this because I didn’t think I had enough/the-right-kind of work done to justify sharing. But ultimately, the reason I posted it is the same reason I still think it’s a good idea: Action is better than inaction, and a big problem I think people in our demographic face is overthinking and underdoing. Michaelos’ recent post in another thread strikes me as very true. (It may not, in fact, be true, but it definitely matches up with other things I know). If I’m taking actions to solve a problem, I can learn from my mistakes, get feedback and try new approaches. (Thank you for your feedback, by the way.)
There are already half-baked efforts to “expand the rationality movement” underway. A half-baked attempt to figure out if that’s even the right goal is not ideal, but I think it’s better than nothing.
I didn’t spend otherwise important, productive time doing this. I was converting useless time in an elevator into:
1) Some new information about what people think about rationality
2) Some new information about how to ask people questions and get productive answers
3) Practice at talking to random people in general
4) Practice talking about rationality without evangelizing (yes, I realize I didn’t do a great job at it, but it’s something that I can only improve at with practice)
(I didn’t see the definition as important so I could start deliberately evangelizing, but so that if the conversation went in a particular direction we’d have something ready to say)
I DID spend “potential productive” time writing up this report and setting up the google doc, but that was time that taught me how to write up a Less Wrong post and your feedback has given me things to think about to improve for next time, so thank you for that.
We talked about hiring real researchers at our meetups. We didn’t end up doing it, mostly because from everything we knew, the official channels to do so were expensive and we had no idea what nonofficial channels we might successfully work with. If you do have recommendations on how to actually go about this, that’d be great.
But regardless, I think this is was a useful exercise for me and I think it would be a useful exercise for many people here. The current data set is near useless, but the experience acquiring it was not, and I think as I/we got better at acquiring data it could become less useless. Even if we got a more scientifically useful polling company to answer the specific question “What do people think about the word rationality?” I think it would still be useful for us to practice talking to people about it.
Writing everything down on a google doc might not actually be useful for the purpose of evaluating the information accurately, but it gets us into the practice of recording and checking over data.
Action can be way worse than inaction, if what you end up doing is misleading yourself or doing harm to your cause.
I don’t think what you’ve done is necessarily misleading or harmful, as long as you don’t consider it anything more than incomplete, qualitative research into the range of responses the word “rationality” gets from random people.
But you really, really need to decide what the point of this exercise is. Are you trying to gather useful data, or make people feel more positive about rationality, or just get comfortable talking to random people? It kind of seems like at the moment, you mainly want to find post-hoc reasons why the exercise was “useful”.
Here’s my suggestion: if you’re trying to do a survey, decide on your demographic(s) of interest. Get everyone on Less Wrong to ask around until they find a sympathiser who works in a branding/marketing survey organisation, and can slip in an extra question in a survey, asking how people respond to the term “rationality”.
Failing that, collaboratively draw up a proper survey protocol and get Less Wrongers to administer it to a random sample of a people. Think it through before you do it: e.g. stopping people outside on the street would be more representative than limiting it to a certain building. You could signal that you’re an official survey person by carrying a clipboard (not by wielding a recording device). You could improve participation by stating initially that you only have one question which will take 15 seconds, then not trying to start a discussion. You could improve participation among younger women by making it clear that you’re doing a survey, so they’re not concerned you’re trying to start an abstract philosophical conversation as a pretext to get them into bed.
I think this could have great potential, especially if you comparatively test alternative terms to “rationality”. Richard Dawkins tried to popularise the term “Brights” for people who don’t believe in the supernatural. If he’d done even the amount of field testing you have already done, he would have realised it sounds unsufferably smug. So I think your impulse to do market research is a good one.
It kind of seems like at the moment, you mainly want to find post-hoc reasons why the exercise was “useful”.
I did use all of those reasons to justify why I thought I should do it beforehand. But I have noticed myself repeating those reasons to make myself feel more justified. (Also possible that my primary motivation in doing so in the first place was the social-skill development one)
In any case, I think your recommendations for how to proceed are good ones.
Another idea—if you can’t find someone skilled in market research to do this for you at a discount or free, read a textbook about how to assess potential new brands to help with designing the survey.
I actually mostly agree with you. I hesitated a long time before posting this because I didn’t think I had enough/the-right-kind of work done to justify sharing. But ultimately, the reason I posted it is the same reason I still think it’s a good idea: Action is better than inaction, and a big problem I think people in our demographic face is overthinking and underdoing. Michaelos’ recent post in another thread strikes me as very true. (It may not, in fact, be true, but it definitely matches up with other things I know). If I’m taking actions to solve a problem, I can learn from my mistakes, get feedback and try new approaches. (Thank you for your feedback, by the way.)
There are already half-baked efforts to “expand the rationality movement” underway. A half-baked attempt to figure out if that’s even the right goal is not ideal, but I think it’s better than nothing.
I didn’t spend otherwise important, productive time doing this. I was converting useless time in an elevator into:
1) Some new information about what people think about rationality 2) Some new information about how to ask people questions and get productive answers 3) Practice at talking to random people in general 4) Practice talking about rationality without evangelizing (yes, I realize I didn’t do a great job at it, but it’s something that I can only improve at with practice)
(I didn’t see the definition as important so I could start deliberately evangelizing, but so that if the conversation went in a particular direction we’d have something ready to say)
I DID spend “potential productive” time writing up this report and setting up the google doc, but that was time that taught me how to write up a Less Wrong post and your feedback has given me things to think about to improve for next time, so thank you for that.
We talked about hiring real researchers at our meetups. We didn’t end up doing it, mostly because from everything we knew, the official channels to do so were expensive and we had no idea what nonofficial channels we might successfully work with. If you do have recommendations on how to actually go about this, that’d be great.
But regardless, I think this is was a useful exercise for me and I think it would be a useful exercise for many people here. The current data set is near useless, but the experience acquiring it was not, and I think as I/we got better at acquiring data it could become less useless. Even if we got a more scientifically useful polling company to answer the specific question “What do people think about the word rationality?” I think it would still be useful for us to practice talking to people about it.
Writing everything down on a google doc might not actually be useful for the purpose of evaluating the information accurately, but it gets us into the practice of recording and checking over data.
Action can be way worse than inaction, if what you end up doing is misleading yourself or doing harm to your cause.
I don’t think what you’ve done is necessarily misleading or harmful, as long as you don’t consider it anything more than incomplete, qualitative research into the range of responses the word “rationality” gets from random people.
But you really, really need to decide what the point of this exercise is. Are you trying to gather useful data, or make people feel more positive about rationality, or just get comfortable talking to random people? It kind of seems like at the moment, you mainly want to find post-hoc reasons why the exercise was “useful”.
Here’s my suggestion: if you’re trying to do a survey, decide on your demographic(s) of interest. Get everyone on Less Wrong to ask around until they find a sympathiser who works in a branding/marketing survey organisation, and can slip in an extra question in a survey, asking how people respond to the term “rationality”.
Failing that, collaboratively draw up a proper survey protocol and get Less Wrongers to administer it to a random sample of a people. Think it through before you do it: e.g. stopping people outside on the street would be more representative than limiting it to a certain building. You could signal that you’re an official survey person by carrying a clipboard (not by wielding a recording device). You could improve participation by stating initially that you only have one question which will take 15 seconds, then not trying to start a discussion. You could improve participation among younger women by making it clear that you’re doing a survey, so they’re not concerned you’re trying to start an abstract philosophical conversation as a pretext to get them into bed.
I think this could have great potential, especially if you comparatively test alternative terms to “rationality”. Richard Dawkins tried to popularise the term “Brights” for people who don’t believe in the supernatural. If he’d done even the amount of field testing you have already done, he would have realised it sounds unsufferably smug. So I think your impulse to do market research is a good one.
I did use all of those reasons to justify why I thought I should do it beforehand. But I have noticed myself repeating those reasons to make myself feel more justified. (Also possible that my primary motivation in doing so in the first place was the social-skill development one)
In any case, I think your recommendations for how to proceed are good ones.
Another idea—if you can’t find someone skilled in market research to do this for you at a discount or free, read a textbook about how to assess potential new brands to help with designing the survey.
I agree that it looks useful in the sense of poking around to find out what sort of questions you want to ask in a more formal survey.