BTW, I’d also disallow 0 and 100, and give the option of giving log-odds instead of probability (and maybe encourage to do that for probabilities 99%). Someone’s “epsilon” might be 10^-4 whereas someone else’s might be 10^-30.
The point is not having to type lots of zeros (or of nines) with extreme probabilities (so that people won’t weasel out and use ‘epsilon’); having to type 1:999999999999999 is no improvement over having to type 0.000000000000001.
Is such precision meaningful? At least for me personally, 0.1% is about as low as I can meaningfully go—I can’t really discriminate between me having an estimate of 0.1%, 0.001%, or 0.0000000000001%.
Specifically, I would guess that you can distinguish the strength of your belief that a lottery ticket you might purchase will win the jackpot from one in a thousand (a.k.a. 0.1%). Am I mistaken?
That’s a very special case—in the case of the lottery, it is actually possible-in-principle to enumerate BIG_NUMBER equally likely mutually-exclusive outcomes. Same with getting the works of shakespeare out of your random number generator. The things under discussion don’t have that quality.
I agree in principle, but on the other hand the questions on the survey are nowhere as easy as “what’s the probability of winning such-and-such lottery”.
I’d force log odds, as they are the more natural representation and much less susceptible to irrational certainty and nonsense answers.
Someone has to actually try and comprehend what they are doing to troll logits; -INF seems a lot more out to lunch than p = 0.
I’d also like someone to go thru the math to figure out how to correctly take the mean of probability estimates. I see no obvious reason why you can simply average [0, 1] probability. The correct method would probably involve cooking up a hypothetical bayesian judge that takes everyones estimates as evidence.
Edit: since logits can be a bit unintuitive, I’d give a few calibration examples like odds of rolling a 6 on a die, odds of winning some lottery, fair odds, odds of surviving a car crash, etc.
I’d force log odds, as they are the more natural representation and much less susceptible to irrational certainty and nonsense answers.
Personally, for probabilities roughly between 20% and 80% I find probabilities (or non-log odds) easier than understand than log-odds.
Someone has to actually try and comprehend what they are doing to troll logits; -INF seems a lot more out to lunch than p = 0.
Yeah. One of the reason why I proposed this is the median answer of 0 in several probability questions. (I’d also require a rationale in order to enter probabilities more extreme than 1%/99%.)
I’d also like someone to go thru the math to figure out how to correctly take the mean of probability estimates. I see no obvious reason why you can simply average [0, 1] probability. The correct method would probably involve cooking up a hypothetical bayesian judge that takes everyones estimates as evidence.
I’d go with the average of log-odds, but this requires all of them to be finite...
yeah. that’s what I was getting at with the maxentropy judge.
On further thought, I really should look into figuring this out. Maybe I’ll do some work on it and post a discussion post. This could be a great group rationality tool.
I’d love a specific question on moral realism instead of leaving it as part of the normative ethics question. I’d also like to know about psychiatric diagnoses (autism spectrum, ADHD, depression, whatever else seems relevant)-- perhaps automatically remove those answers from a spreadsheet for privacy reasons.
You are aware that if you ask people for their sex but not their gender, and say something like “we have more women now”, you will be philosophized into a pulp, right?
You are aware that if you ask people for their sex but not their gender, and say something like “we have more women now”, you will be philosophized into a pulp, right?
Only if people here are less interested in applying probability theory than they are in philosophizing about gender… Oh.
Because the two are so highly correlated that having both would give us almost no extra information. One goal of the survey should be to maximize the useful-info-extracted / time-spent-on-it ratio, hence also the avoidance of write-ins for many questions (which make people spend more time on the survey, to get results that are less exploitable) (a write-in for gender works because people are less likely to write a manifesto for that than for politics).
I’m curious, what would you do with the results of such a question?
For my part, I suspect I would merely stare at them and be unsure what to make of a statistical result that aggregates “No, I already held the belief that the sequences attempted to convince me of” with “No, I held a contrary belief and the sequences failed to convince me otherwise.” (That it also aggregates “Yes, I held a contrary belief and the sequences convinced me otherwise.” and “Yes, I initially held the belief that the sequences attempted to convince me of, and the sequences convinced me otherwise” is less of a concern, since I expect the latter group to be pretty small.)
Originally I was going to suggest asking, “what were your religious beliefs before reading the sequences?”—and then I succumbed to the programmer’s urge to solve the general problem.
However, I guess measuring how effective the sequences are at causing people to change their mind is something that a LW survey can’t do, anyway (you’d need to also ask people who read the sequences but didn’t stick around to accurately answer that).
Mainly I was curious how many deconversions the sequences caused or hastened.
Ok, so use radio-buttons:
“believed before, still believe”
“believed before, changed my mind now”
“didn’t believe before, changed my mind now”
“never believed, still don’t”
I think the question is too vague as formulated. Does any probability update, no matter how small, count as changing your mind? But if you ask for precise probability changes, then the answers will likely be nonsense because most people (even most LWers, I’d guess) don’t keep track of numeric probabilities, just think “oh, this argument makes X a bit more believable” and such.
When asking for race/ethnicity, you should really drop the standard American classification into White—Hispanic—Black—Indian—Asian—Other. From a non-American perspective this looks weird, especially the “White Hispanic” category. A Spaniard is White Hispanic, or just White? If only White, how does the race change when one moves to another continent? And if White Hispanic, why not have also “Italic” or “Scandinavic” or “Arabic” or whatever other peninsula-ic races?
Since I believe the question was intended to determine the cultural background of LW readers, I am surprised that there was no question about country of origin, which would be more informative. There is certainly greater cultural difference between e.g. Turks (White, non-Hispanic I suppose) and White non-Hispanic Americans than between the latter and their Hispanic compatriots.
Also, making a statistic based on nationalities could help people determine whether there is a chance for a meetup in their country. And it would be nice to know whether LW has regular readers in Liechtenstein, of course.
I was also...well, not surprised per se, but certainly annoyed to see that “Native American” in any form wasn’t even an option. One could construe that as revealing, I suppose.
I don’t know how relevant the question actually is, but if we want to track ancestry and racial, ethnic or cultural group affiliation, the folowing scheme is pretty hard to mess up:
Country of origin: Country of residence: Primary Language: Native Language (if different): Heritage language (if different):
Note: A heritage language is one spoken by your family or identity group.
Heritage group:
Diaspora: Means your primary heritage and identity group moved to the country you live in within historical or living memory, as colonists, slaves, workers or settlers.
European diaspora (“white” North America, Australia, New Zealand, South Africa, etc) African diaspora (“black” in the US, West Indian, more recent African emigrant groups; also North African diaspora) Asian diaspora (includes, Turkic, Arab, Persian, Central and South Asian, Siberian native)
Indigenous: Means your primary heritage and identity group was resident to the following location prior to 1400, OR prior to the arrival of the majority culture in antiquity (for example: Ainu, Basque, Taiwanese native, etc):
-Africa -Asia -Europe -North America (between Panama and Canada, also includes Greenland and the Carribean) -Oceania (including Australia) -South America
Mixed: Select two or more:
European Diaspora African Diaspora Asian Diaspora African Indigenous American Indigenous Asian Indigenous European Indigenous Oceania Indigenous
What the US census calls “Non-white Hispanic” would be marked as “Mixed” > “European Diaspora” + “American Indigenous” with Spanish as either a Native or Heritage language. Someone who identifies as (say) Mexican-derived but doesn’t speak Spanish at all would be impossible to tell from someone who was Euro-American and Cherokee who doesn’t speak Cherokee, but no system is perfect...
Most LessWrong posters and readers are American, perhaps even the vast majority (I am not). Hispanic Americans differ from white Americans differ from black Americans culturally and socio-economically not just on average but in systemic ways regardless if the person in question defines himself as Irish American, Kenyan American, white American or just plain American. From the US we have robust sociological data that allows us to compare LWers based on this information. The same is true of race in Latin America, parts of Africa and more recently Western Europe.
Nationality is not the same thing as racial or even ethnic identity in multicultural societies.
Considering every now and then people bring up a desire to lower barriers to entry for “minorities” (whatever that means in a global forum), such stats are useful for those who argue on such issues and also for ascertaining certain biases.
Adding a nationality and/or citizenship question would probably be useful though.
Nationality is not the same thing as racial or even ethnic identity in multicultural societies.
I have not said that it is. I was objecting to arbitrariness of “Hispanic race”: I believe that the difference between Hispanic White Americans and non-Hispanic White Americans is not significantly higher than the difference between both two groups and non-Americans, and that the number of non-Americans among LW users would be higher than 3.8% reported for the Hispanics. I am not sure what exact sociological data we may extract from the survey, but in any case, the comparison to standard American sociological datasets will be problematic because the LW data are contaminated by presence of non-Americans and there is no way to say how much, because people were not asked about that.
I didn’t meant to imply you did, I just wanted to emphasise that data is gained by the racial breakdown. Especially in the American context, race sits at the strange junction of appearance, class, heritage, ethnicity, religion and subculture. And its hard to capture it by any of these metrics.
I am not sure what exact sociological data we may extract from the survey, but in any case, the comparison to standard American sociological datasets will be problematic because the LW data are contaminated by presence of non-Americans and there is no way to say how much, because people were not asked about that.
Once we have data on how many are American (and this is something we really should have) this will be easier to say.
If only White, how does the race change when one moves to another continent? And if White Hispanic, why not have also “Italic” or “Scandinavic” or “Arabic” or whatever other peninsula-ic races?
Because we don’t have as much useful sociological data on this. Obviously we can start collecting data on any of the proposed categories, but if we’re the only ones, it won’t much help us figure out how LW differs from what one might expect of a group that fits its demographic profile.
Since I believe the question was intended to determine the cultural background of LW readers, I am surprised that there was no question about country of origin, which would be more informative. There is certainly greater cultural difference between e.g. Turks (White, non-Hispanic I suppose) and White non-Hispanic Americans than between the latter and their Hispanic compatriots.
Much of the difference in the example of Turks is captured by the Muslim family background question.
Much of the difference in the example of Turks is captured by the Muslim family background question.
Much, but not most. Religion is easy to ascertain, but there are other cultural differences which are more difficult to classify, but still are signigicant *. Substitute Turks with Egyptian Christians and the example will still work. (And not because of theological differences between Coptic and Protestant Christianity.)
*) Among the culturally determined attributes are: political opinion, musical taste and general aesthetic preferences, favourite food, familiarity with different literature and films, ways of relaxation, knowledge of geography and history, language(s), moral code. Most of these things are independent of religion or only very indirectly influenced by it.
Offer a text field for race. You’ll get some distances, not to mention “human” or “other”, but you could always use that to find out whether having a contrary streak about race/ethnicity correlates with anything.
If you want people to estimate whether a meetup could be worth it, I recommend location rather than nationality—some nations are big enough that just knowing nationality isn’t useful.
I think using your stipulative definition of “supernatural” was a bad move. I would be very surprised if I asked a theologian to define “supernatural” and they replied “ontologically basic mental entities”. Even as a rational reconstruction of their reply, it would be quite a stretch. Using such specific definitions of contentious concepts isn’t a good idea, if you want to know what proportion of Less Wrongers self-identify as atheist/agnostic/deist/theist/polytheist.
OTOH, using a vague definition isn’t a good idea either, if you want to know something about what Less Wrongers believe about the world.
I had no problem with the question as worded; it was polling about LWers confidence in a specific belief, using terms from the LW Sequences. That the particular belief is irrelevant to what people who self-identify as various groups consider important about that identification is important to remember, but not in and of itself a problem with the question.
But, yeah… if we want to know what proportion of LWers self-identify as (e.g.) atheist, that question won’t tell us.
Yet another alternate, culture-neutral way of asking about politics:
Q: How involved are you in your region’s politics compared to other people in your region? A: [choose one] () I’m among the most involved () I’m more involved than average () I’m about as involved as average () I’m less involved than average () I’m among the least involved
Requires people to self assess next to a cultural baseline, and self assessments of this sort are notoriously inaccurate. (I predict everyone will think they have above-average involvement).
Within a US-specific context, I would eschew these comparisons to a notional average and use the following levels of participation:
0 = indifferent to politics and ignorant of current events 1 = attentive to current events, but does not vote 2 = votes in presidential elections, but irregularly otherwise 3 = always votes 4 = always votes and contributes to political causes 5 = always votes, contributes, and engages in political activism during election seasons 6 = always votes, contributes, and engages in political activism both during and between election seasons 7 = runs for public office
I suspect that the average US citizen of voting age is a 2, but I don’t have data to back that up, and I am not motivated to research it. I am a 4, so I do indeed think that I am above average.
Those categories could probably be modified pretty easily to match a parliamentary system by leaving out the reference to presidential elections and just having “votes irregularly” and “always votes”
Editing to add—for mandatory voting jurisdictions, include a caveat that “spoiled ballot = did not vote”
Personally, I’m not sure I necessarily consider the person who runs for public office to be at a higher level of participation than the person who works for them.
I agree denotationally with that estimate, but I think you’re putting too much emphasis on voting in at least the 0-4 range. Elections (in the US) only come up once or exceptionally twice a year, after all. If you’re looking for an estimate of politics’ significance to a person’s overall life, I think you’d be better off measuring degree of engagement with current events and involvement in political groups—the latter meaning not only directed activism, but also political blogs, non-activist societies with a partisan slant, and the like.
For example: do you now, or have you ever, owned a political bumper sticker?
There might be people who don’t always (or even usually) vote yet they contribute to political causes/engage in political activism, for certain values of “political” at least.
I had not before encountered this form of protest. If I were living in a place with mandatory voting and anonymous ballots, I would almost surely write my name on the ballot to spoil it.
I had not before encountered this form of protest. If I were living in a place with mandatory voting and anonymous ballots, I would almost surely write my name on the ballot to spoil it.
I have never actually spoiled a ballot in a municipality-or-higher-level election (though voting for a list with hardly any chance whatsoever of passing the election threshold has a very similar effect), but in high school I did vote for Homer Simpson as a students’ representative, and there were lots of similarly hilarious votes, including (IIRC) ones for God, Osama bin Laden, and Silvio Berlusconi.
Requires people to self assess next to a cultural baseline, and self assessments of this sort are notoriously inaccurate. (I predict everyone will think they have above-average involvement).
I’d actually have guessed an average of below average.
I predict everyone will think they have above-average involvement
Bad prediction. While it’s hard to say since so few people around here actually vote, my involvement in politics is close enough to 0 that I’d be very surprised if I was more involved than average.
For comparison what would you say the average persons level of involvement in politics consists of? (To avoid contamination, don’t research or overthink the question just give us the average you were comparing yourself to).
Edit: The intuitive average other commenters compared themselves to would also be of interest.
Good question. I don’t know what the average person’s involvement is, and I seem to know a lot of people (at least online) who are very politically involved, so I may be misestimating whether my political activity is above or below average.
On posting this I made the prediction that the average assumed by most lesswrong commenters would be above the actual average level of participation.
I hypothesise this is because most LW commenters come from relatively educated or affluent social groups, where political participation is quite high. Whereas there are large portions of the population who do not participate at all in politics (in the US and UK a significant percentage don’t even vote in the 4-yearly national elections).
Because of this I would be very sceptical of self reported participation levels, and would agree a quantifiable measure would be better.
I don’t think I’d describe that post as regretting asking “do you have a Y chromosome”. He’s apologizing for asking for data for one purpose (checking with colorblindness) and then using it for another (color names if you’re a guy/girl).
Everyone who’s suggesting changes: you are much more likely to get your way if you suggest a specific alternative. For example, instead of “handle politics better”, something like “your politics question should have these five options: a, b, c, d, and e.” Or instead of “use a more valid IQ measure”, something more like “Here’s a site with a quick and easy test that I think is valid”
In that case: use the exact ethics questions from the PhilPapers Survey (http://philpapers.org/surveys/), probably minus lean/accept distinction and the endless drop-down menu for “other.”
For IQ: maybe you could nudge people to greater honesty by splitting up the question: (1) have you ever taken an IQ test with [whatever features were specified on this year’s survey], yes or no? (2) if yes, what was your score?
Also, “ever” might be a bit too long. IQs and IQ tests can change over time, so maybe you should ask “have you taken an IQ test [with constraints] in the last 10 years?”
Strongly disagree with previous self here. I do not think replacing “gender” with “sex” avoids complaints or “philosophizing”, and “philosophizing” in context feels like a shorthand/epithet for “making this more complex than prevailing, mainstream views on gender.”
For a start, it seems like even “sex” in the sense used here is getting at a mainly-social phenomenon: that of sex assigned at birth. This is a judgement call by the doctors and parents. The biological correlates used to make that decision are just weighed in aggregate; some people are always going to throw an exception. If you’re not asking about the size of gametes and their delivery mechanism, the hormonal makeup of the person, their reproductive anatomy where applicable, or their secondary sexual characteristics, then “sex” is really just asking the “gender” question but hazily referring to biological characteristics instead.
Ultimately, gender is what you’re really asking for. Using “sex” as a synonym blurs the data into unintelligibility for some LWers; pragmatically, it also amounts to a tacit “screw you” to trans people. I suggest biting the bullet and dealing with the complexity involved in asking that question—in many situations people collecting that demographic info don’t actually need it, but it seems like useful information for LessWrong.
A suggested approach:
Two optional questions with something like the following phrasing:
Optional: Gender (pick what best describe how you identify):
-Male -Female -Genderqueer, genderfluid, other -None, neutrois, agender -Prefer not to say
Optional: Sex assigned at birth: -Male -Female -Intersex -Prefer not to say
A series of four questions on each Meyers-Briggs indicator would be good, although I’m sure the data would be woefully unsurprising. Perhaps link to an online test if people don’t know it already.
That list is way, way to short. I entirely gave up on the survey partway through because an actual majority of the questions were inapplicable or downright offensive to my sensibilities, or just incomprehensible, or I couldn’t answer them for some other reason.
Not that I can think of anything that WOULDN’T have that effect on me without being specifically tailored to me which sort of destroys the point of having a survey… Maybe I’m just incompatible with surveys in general.
Many of the questions were USA-centric, assuming people grew up with some religion or political climate common in the US. I didn’t get indoctrinated to republicans or democrats, I got indoctrinated to environmentalism, and there’s just no way to map that onto American politics where it’s an issue rather than a faction. And it might in some ways be the closest match on the religion question as well, being a question of fact that I later had to try to de-bias myself on.
The US-centricity is real problem, and probably worth a discussion post. Do political beliefs tell us something important about LW posters, and if so, are there general ways not tied to a particular country to ask about them. If there isn’t a general way, how can this be handled?
Question I’d like to see added: how much attention do you give to politics? That question should probably be split between attention to theory, attention to news, and attention to trying to make things happen.
I suggested in the survey thread to ask for Political Compass scores instead of a liberal/conservative/libertarian/socialist question. The Compass is slightly US-biased, but it contains enough questions for the end result to be significant even so. How much attention to politics would be an interesting question, I second that.
I suspect the compass is very US-based, though better than a short list or a single dimension.
there’s one more thing about interest in politics that I had trouble phrasing. There’s a thing that I call practical politics which I don’t do, but it’s working for particular candidates or being one yourself or knowing in some detail about the right place to push to get something to happen or not happen. It’s the step beyond voting and emailing your representative and signing petitions.
I’d be surprised if very many LWers do practical politics, but that might just be typical mind fallacy.
They do admit they’re biased, but the bias is not exactly American (indeed, they are British). And given that LW has lots of readers from non-US western countries but few from (say) China, while not ideal, it would be a lot better than the very US-centric answers in the last survey. (For example, I’d bet that a lot of people would have self-identified as socialist libertarians if given the chance.)
On politics I would like a way to say, I don’t identify with any political theory. To me this is like asking “what religion do you identify most with?” with options christianity, Islam, hinduism, other and the option to click no boxes. If, as an atheist with no religious ties you click other you’re in with shintoists and satanists and other other unmentioned religions. If you don’t answer you’re just not giving an answer. You could just not want to say. In any case the question frames things so that you have to subscribe to a questionable framework to answer it all
Solutions:
Option like the morality one, perhaps, 0 identification
options for “other” and “prefer not to say” retaining the ability to click no boxes, though there are probably other reasons to click no boxes.
or, as someone suggested in the original thread another question to gage how much you identify with something. The current way If I had to pick an answer I could probably dredge up some preference for one or another theories, but I’d be in the same box as someone actively promoting and a part of what option they clicked. Boxes for strongly identify, identify and weakly identify maybe. Or something.
Got to go.
edit: Could someone kindly explain the downvotes? I’m guessing Too esoteric? Personally not bothered? bothered that I’m bothered?
edit2: just realised some line breaks in the comment box haven’t translated to line breaks in the published comment. Is the post just hard to read?
Running list of changes for next year’s survey:
Ask who’s a poster versus a lurker!
A non-write-in “Other” for most questions
Replace “gender” with “sex” to avoid complaints/philosophizing.
Very very clear instructions to use percent probabilities and not decimal probabilities
Singularity year question should have explicit instructions for people who don’t believe in singularity
Separate out “relationship status” and “looking for new relationships” questions to account for polys
Clarify that research is allowed on the probability questions
Clarify possible destruction of humanity in cryonics/antiagathics questions.
What does it mean for aliens to “exist in the universe”? Light cone?
Make sure people write down “0” if they have 0 karma.
Add “want to sign up, but not available” as cryonics option.
Birth order.
Have children?
Country of origin?
Consider asking about SAT scores for Americans to have something to correlate IQs with.
Consider changing morality to PhilPapers version.
One about nationality (and/or native language)? I guess that would be much more relevant than e.g. birth order.
Regarding #4, you could just write a % symbol to the right of each input box.
BTW, I’d also disallow 0 and 100, and give the option of giving log-odds instead of probability (and maybe encourage to do that for probabilities 99%). Someone’s “epsilon” might be 10^-4 whereas someone else’s might be 10^-30.
I second that. See my post at http://lesswrong.com/r/discussion/lw/8lr/logodds_or_logits/ for a concise summary. Getting the LW survey to use log-odds would go a long way towards getting LW to start using log-odds in normal conversation.
People will mess up the log-odds, though. Non-log odds seem safer.
Two fields instead of one, but it seems cleaner than any of the other alternatives.
The point is not having to type lots of zeros (or of nines) with extreme probabilities (so that people won’t weasel out and use ‘epsilon’); having to type 1:999999999999999 is no improvement over having to type 0.000000000000001.
Is such precision meaningful? At least for me personally, 0.1% is about as low as I can meaningfully go—I can’t really discriminate between me having an estimate of 0.1%, 0.001%, or 0.0000000000001%.
I expect this is incorrect.
Specifically, I would guess that you can distinguish the strength of your belief that a lottery ticket you might purchase will win the jackpot from one in a thousand (a.k.a. 0.1%). Am I mistaken?
That’s a very special case—in the case of the lottery, it is actually possible-in-principle to enumerate BIG_NUMBER equally likely mutually-exclusive outcomes. Same with getting the works of shakespeare out of your random number generator. The things under discussion don’t have that quality.
I agree in principle, but on the other hand the questions on the survey are nowhere as easy as “what’s the probability of winning such-and-such lottery”.
You’re right, good point.
Just type 1:1e15 (or 1e-15 if you don’t want odd ratios).
I’d force log odds, as they are the more natural representation and much less susceptible to irrational certainty and nonsense answers.
Someone has to actually try and comprehend what they are doing to troll logits; -INF seems a lot more out to lunch than p = 0.
I’d also like someone to go thru the math to figure out how to correctly take the mean of probability estimates. I see no obvious reason why you can simply average [0, 1] probability. The correct method would probably involve cooking up a hypothetical bayesian judge that takes everyones estimates as evidence.
Edit: since logits can be a bit unintuitive, I’d give a few calibration examples like odds of rolling a 6 on a die, odds of winning some lottery, fair odds, odds of surviving a car crash, etc.
Personally, for probabilities roughly between 20% and 80% I find probabilities (or non-log odds) easier than understand than log-odds.
Yeah. One of the reason why I proposed this is the median answer of 0 in several probability questions. (I’d also require a rationale in order to enter probabilities more extreme than 1%/99%.)
I’d go with the average of log-odds, but this requires all of them to be finite...
Weighting, in part, by the calibration questions?
I dunno how you would weight it. I think you’d want to have a maxentropy ‘fair’ judge at least for comparison.
Calibration questions are probably the least controversial way of weighting. Compare to, say, trying to weight using karma.
This might be an interesting thing to develop. A voting system backed up by solid bayes-math could be useful for more than just LW surveys.
It might be interesting to see what results are produced by several weighting approaches.
yeah. that’s what I was getting at with the maxentropy judge.
On further thought, I really should look into figuring this out. Maybe I’ll do some work on it and post a discussion post. This could be a great group rationality tool.
Publish draft questions in advance, so we can spot issues before the survey goes live.
We should ask if people participated in the previous surveys.
I’d love a specific question on moral realism instead of leaving it as part of the normative ethics question. I’d also like to know about psychiatric diagnoses (autism spectrum, ADHD, depression, whatever else seems relevant)-- perhaps automatically remove those answers from a spreadsheet for privacy reasons.
I don’t care about moral realism, but psychiatric diagnoses (and whether they’re self-diagnosed or formally diagnosed) would be interesting.
You are aware that if you ask people for their sex but not their gender, and say something like “we have more women now”, you will be philosophized into a pulp, right?
Only if people here are less interested in applying probability theory than they are in philosophizing about gender… Oh.
Why not ask for both?
Because the two are so highly correlated that having both would give us almost no extra information. One goal of the survey should be to maximize the useful-info-extracted / time-spent-on-it ratio, hence also the avoidance of write-ins for many questions (which make people spend more time on the survey, to get results that are less exploitable) (a write-in for gender works because people are less likely to write a manifesto for that than for politics).
Because having a “gender” question causes complaints and philosophizing, which Yvain wants to avoid.
How about, “It’s highly likely that we have more women now”?
Suggestion: “Which of the following did you change your mind about after reading the sequences? (check all that apply)”
[] Religion
[] Cryonics
[] Politics
[] Nothing
[] et cetera.
Many other things could be listed here.
I’m curious, what would you do with the results of such a question?
For my part, I suspect I would merely stare at them and be unsure what to make of a statistical result that aggregates “No, I already held the belief that the sequences attempted to convince me of” with “No, I held a contrary belief and the sequences failed to convince me otherwise.” (That it also aggregates “Yes, I held a contrary belief and the sequences convinced me otherwise.” and “Yes, I initially held the belief that the sequences attempted to convince me of, and the sequences convinced me otherwise” is less of a concern, since I expect the latter group to be pretty small.)
Originally I was going to suggest asking, “what were your religious beliefs before reading the sequences?”—and then I succumbed to the programmer’s urge to solve the general problem.
However, I guess measuring how effective the sequences are at causing people to change their mind is something that a LW survey can’t do, anyway (you’d need to also ask people who read the sequences but didn’t stick around to accurately answer that).
Mainly I was curious how many deconversions the sequences caused or hastened.
Ok, so use radio-buttons: “believed before, still believe” “believed before, changed my mind now” “didn’t believe before, changed my mind now” “never believed, still don’t”
...and “believed something before, believe something different now”
I think the question is too vague as formulated. Does any probability update, no matter how small, count as changing your mind? But if you ask for precise probability changes, then the answers will likely be nonsense because most people (even most LWers, I’d guess) don’t keep track of numeric probabilities, just think “oh, this argument makes X a bit more believable” and such.
When asking for race/ethnicity, you should really drop the standard American classification into White—Hispanic—Black—Indian—Asian—Other. From a non-American perspective this looks weird, especially the “White Hispanic” category. A Spaniard is White Hispanic, or just White? If only White, how does the race change when one moves to another continent? And if White Hispanic, why not have also “Italic” or “Scandinavic” or “Arabic” or whatever other peninsula-ic races?
Since I believe the question was intended to determine the cultural background of LW readers, I am surprised that there was no question about country of origin, which would be more informative. There is certainly greater cultural difference between e.g. Turks (White, non-Hispanic I suppose) and White non-Hispanic Americans than between the latter and their Hispanic compatriots.
Also, making a statistic based on nationalities could help people determine whether there is a chance for a meetup in their country. And it would be nice to know whether LW has regular readers in Liechtenstein, of course.
I was also...well, not surprised per se, but certainly annoyed to see that “Native American” in any form wasn’t even an option. One could construe that as revealing, I suppose.
I don’t know how relevant the question actually is, but if we want to track ancestry and racial, ethnic or cultural group affiliation, the folowing scheme is pretty hard to mess up:
Country of origin:
Country of residence:
Primary Language:
Native Language (if different):
Heritage language (if different):
Note: A heritage language is one spoken by your family or identity group.
Heritage group:
Diaspora: Means your primary heritage and identity group moved to the country you live in within historical or living memory, as colonists, slaves, workers or settlers.
European diaspora (“white” North America, Australia, New Zealand, South Africa, etc)
African diaspora (“black” in the US, West Indian, more recent African emigrant groups; also North African diaspora)
Asian diaspora (includes, Turkic, Arab, Persian, Central and South Asian, Siberian native)
Indigenous: Means your primary heritage and identity group was resident to the following location prior to 1400, OR prior to the arrival of the majority culture in antiquity (for example: Ainu, Basque, Taiwanese native, etc):
-Africa
-Asia
-Europe
-North America (between Panama and Canada, also includes Greenland and the Carribean)
-Oceania (including Australia)
-South America
Mixed: Select two or more:
European Diaspora
African Diaspora
Asian Diaspora
African Indigenous
American Indigenous
Asian Indigenous
European Indigenous
Oceania Indigenous
What the US census calls “Non-white Hispanic” would be marked as “Mixed” > “European Diaspora” + “American Indigenous” with Spanish as either a Native or Heritage language. Someone who identifies as (say) Mexican-derived but doesn’t speak Spanish at all would be impossible to tell from someone who was Euro-American and Cherokee who doesn’t speak Cherokee, but no system is perfect...
Put two spaces after a line if you want a linebreak.
Most LessWrong posters and readers are American, perhaps even the vast majority (I am not). Hispanic Americans differ from white Americans differ from black Americans culturally and socio-economically not just on average but in systemic ways regardless if the person in question defines himself as Irish American, Kenyan American, white American or just plain American. From the US we have robust sociological data that allows us to compare LWers based on this information. The same is true of race in Latin America, parts of Africa and more recently Western Europe.
Nationality is not the same thing as racial or even ethnic identity in multicultural societies.
Considering every now and then people bring up a desire to lower barriers to entry for “minorities” (whatever that means in a global forum), such stats are useful for those who argue on such issues and also for ascertaining certain biases.
Adding a nationality and/or citizenship question would probably be useful though.
I have not said that it is. I was objecting to arbitrariness of “Hispanic race”: I believe that the difference between Hispanic White Americans and non-Hispanic White Americans is not significantly higher than the difference between both two groups and non-Americans, and that the number of non-Americans among LW users would be higher than 3.8% reported for the Hispanics. I am not sure what exact sociological data we may extract from the survey, but in any case, the comparison to standard American sociological datasets will be problematic because the LW data are contaminated by presence of non-Americans and there is no way to say how much, because people were not asked about that.
I didn’t meant to imply you did, I just wanted to emphasise that data is gained by the racial breakdown. Especially in the American context, race sits at the strange junction of appearance, class, heritage, ethnicity, religion and subculture. And its hard to capture it by any of these metrics.
Once we have data on how many are American (and this is something we really should have) this will be easier to say.
Because we don’t have as much useful sociological data on this. Obviously we can start collecting data on any of the proposed categories, but if we’re the only ones, it won’t much help us figure out how LW differs from what one might expect of a group that fits its demographic profile.
Much of the difference in the example of Turks is captured by the Muslim family background question.
Much, but not most. Religion is easy to ascertain, but there are other cultural differences which are more difficult to classify, but still are signigicant *. Substitute Turks with Egyptian Christians and the example will still work. (And not because of theological differences between Coptic and Protestant Christianity.)
*) Among the culturally determined attributes are: political opinion, musical taste and general aesthetic preferences, favourite food, familiarity with different literature and films, ways of relaxation, knowledge of geography and history, language(s), moral code. Most of these things are independent of religion or only very indirectly influenced by it.
Offer a text field for race. You’ll get some distances, not to mention “human” or “other”, but you could always use that to find out whether having a contrary streak about race/ethnicity correlates with anything.
If you want people to estimate whether a meetup could be worth it, I recommend location rather than nationality—some nations are big enough that just knowing nationality isn’t useful.
Suggestion: add “cryocrastinating” as a cryonics option.
I think using your stipulative definition of “supernatural” was a bad move. I would be very surprised if I asked a theologian to define “supernatural” and they replied “ontologically basic mental entities”. Even as a rational reconstruction of their reply, it would be quite a stretch. Using such specific definitions of contentious concepts isn’t a good idea, if you want to know what proportion of Less Wrongers self-identify as atheist/agnostic/deist/theist/polytheist.
OTOH, using a vague definition isn’t a good idea either, if you want to know something about what Less Wrongers believe about the world.
I had no problem with the question as worded; it was polling about LWers confidence in a specific belief, using terms from the LW Sequences. That the particular belief is irrelevant to what people who self-identify as various groups consider important about that identification is important to remember, but not in and of itself a problem with the question.
But, yeah… if we want to know what proportion of LWers self-identify as (e.g.) atheist, that question won’t tell us.
Yet another alternate, culture-neutral way of asking about politics:
Q: How involved are you in your region’s politics compared to other people in your region?
A: [choose one]
() I’m among the most involved
() I’m more involved than average
() I’m about as involved as average
() I’m less involved than average
() I’m among the least involved
Requires people to self assess next to a cultural baseline, and self assessments of this sort are notoriously inaccurate. (I predict everyone will think they have above-average involvement).
Within a US-specific context, I would eschew these comparisons to a notional average and use the following levels of participation:
0 = indifferent to politics and ignorant of current events
1 = attentive to current events, but does not vote
2 = votes in presidential elections, but irregularly otherwise
3 = always votes
4 = always votes and contributes to political causes
5 = always votes, contributes, and engages in political activism during election seasons
6 = always votes, contributes, and engages in political activism both during and between election seasons
7 = runs for public office
I suspect that the average US citizen of voting age is a 2, but I don’t have data to back that up, and I am not motivated to research it. I am a 4, so I do indeed think that I am above average.
Those categories could probably be modified pretty easily to match a parliamentary system by leaving out the reference to presidential elections and just having “votes irregularly” and “always votes”
Editing to add—for mandatory voting jurisdictions, include a caveat that “spoiled ballot = did not vote”
Personally, I’m not sure I necessarily consider the person who runs for public office to be at a higher level of participation than the person who works for them.
I agree denotationally with that estimate, but I think you’re putting too much emphasis on voting in at least the 0-4 range. Elections (in the US) only come up once or exceptionally twice a year, after all. If you’re looking for an estimate of politics’ significance to a person’s overall life, I think you’d be better off measuring degree of engagement with current events and involvement in political groups—the latter meaning not only directed activism, but also political blogs, non-activist societies with a partisan slant, and the like.
For example: do you now, or have you ever, owned a political bumper sticker?
Maybe: “How frequently do you visit websites/read media that have an explicit political slant?”
There might be people who don’t always (or even usually) vote yet they contribute to political causes/engage in political activism, for certain values of “political” at least.
I had not before encountered this form of protest. If I were living in a place with mandatory voting and anonymous ballots, I would almost surely write my name on the ballot to spoil it.
I do and I do. :)
I have never actually spoiled a ballot in a municipality-or-higher-level election (though voting for a list with hardly any chance whatsoever of passing the election threshold has a very similar effect), but in high school I did vote for Homer Simpson as a students’ representative, and there were lots of similarly hilarious votes, including (IIRC) ones for God, Osama bin Laden, and Silvio Berlusconi.
I’d actually have guessed an average of below average.
Bad prediction. While it’s hard to say since so few people around here actually vote, my involvement in politics is close enough to 0 that I’d be very surprised if I was more involved than average.
I have exactly zero involvement and so I’d never think that.
I think I have average or below-average involvement.
Maybe it would be better to ask about the hours/year spent on politics.
For comparison what would you say the average persons level of involvement in politics consists of? (To avoid contamination, don’t research or overthink the question just give us the average you were comparing yourself to).
Edit: The intuitive average other commenters compared themselves to would also be of interest.
Good question. I don’t know what the average person’s involvement is, and I seem to know a lot of people (at least online) who are very politically involved, so I may be misestimating whether my political activity is above or below average.
On posting this I made the prediction that the average assumed by most lesswrong commenters would be above the actual average level of participation.
I hypothesise this is because most LW commenters come from relatively educated or affluent social groups, where political participation is quite high. Whereas there are large portions of the population who do not participate at all in politics (in the US and UK a significant percentage don’t even vote in the 4-yearly national elections).
Because of this I would be very sceptical of self reported participation levels, and would agree a quantifiable measure would be better.
You should clarify in the antiagathics question that the person reaches the age of 1000 without the help of cryonics.
Replacing gender with sex seems like the wrong way to go to me. For example, note how Randall Munroe asked about sex, then regretted it.
I don’t think I’d describe that post as regretting asking “do you have a Y chromosome”. He’s apologizing for asking for data for one purpose (checking with colorblindness) and then using it for another (color names if you’re a guy/girl).
Everyone who’s suggesting changes: you are much more likely to get your way if you suggest a specific alternative. For example, instead of “handle politics better”, something like “your politics question should have these five options: a, b, c, d, and e.” Or instead of “use a more valid IQ measure”, something more like “Here’s a site with a quick and easy test that I think is valid”
In that case: use the exact ethics questions from the PhilPapers Survey (http://philpapers.org/surveys/), probably minus lean/accept distinction and the endless drop-down menu for “other.”
For IQ: maybe you could nudge people to greater honesty by splitting up the question: (1) have you ever taken an IQ test with [whatever features were specified on this year’s survey], yes or no? (2) if yes, what was your score?
Also, “ever” might be a bit too long. IQs and IQ tests can change over time, so maybe you should ask “have you taken an IQ test [with constraints] in the last 10 years?”
http://en.wikipedia.org/wiki/Intersex
Otherwise agreed.
Strongly disagree with previous self here. I do not think replacing “gender” with “sex” avoids complaints or “philosophizing”, and “philosophizing” in context feels like a shorthand/epithet for “making this more complex than prevailing, mainstream views on gender.”
For a start, it seems like even “sex” in the sense used here is getting at a mainly-social phenomenon: that of sex assigned at birth. This is a judgement call by the doctors and parents. The biological correlates used to make that decision are just weighed in aggregate; some people are always going to throw an exception. If you’re not asking about the size of gametes and their delivery mechanism, the hormonal makeup of the person, their reproductive anatomy where applicable, or their secondary sexual characteristics, then “sex” is really just asking the “gender” question but hazily referring to biological characteristics instead.
Ultimately, gender is what you’re really asking for. Using “sex” as a synonym blurs the data into unintelligibility for some LWers; pragmatically, it also amounts to a tacit “screw you” to trans people. I suggest biting the bullet and dealing with the complexity involved in asking that question—in many situations people collecting that demographic info don’t actually need it, but it seems like useful information for LessWrong.
A suggested approach:
Two optional questions with something like the following phrasing:
Optional: Gender (pick what best describe how you identify):
-Male
-Female
-Genderqueer, genderfluid, other
-None, neutrois, agender
-Prefer not to say
Optional: Sex assigned at birth:
-Male
-Female
-Intersex
-Prefer not to say
A series of four questions on each Meyers-Briggs indicator would be good, although I’m sure the data would be woefully unsurprising. Perhaps link to an online test if people don’t know it already.
You can accomplish this by adding a percent sign in the survey itself, to the right of to every textbox entry field.
Edit: sorry, already suggested.
As per my previous comments on this, separate out normative ethics and meta-ethics.
And maybe be extra-clear on not answering the IQ question unless you have official results? Or is that a lost cause?
I would much rather see a choice of units.
That list is way, way to short. I entirely gave up on the survey partway through because an actual majority of the questions were inapplicable or downright offensive to my sensibilities, or just incomprehensible, or I couldn’t answer them for some other reason.
Not that I can think of anything that WOULDN’T have that effect on me without being specifically tailored to me which sort of destroys the point of having a survey… Maybe I’m just incompatible with surveys in general.
Would you be willing to write a discussion post about the questions you want to answer?
No, because I fail utterly at writing things, and because my complaints are way to many so it’d take to much time typing them out.
Random sample of complaints?
Good idea!
Many of the questions were USA-centric, assuming people grew up with some religion or political climate common in the US. I didn’t get indoctrinated to republicans or democrats, I got indoctrinated to environmentalism, and there’s just no way to map that onto American politics where it’s an issue rather than a faction. And it might in some ways be the closest match on the religion question as well, being a question of fact that I later had to try to de-bias myself on.
The US-centricity is real problem, and probably worth a discussion post. Do political beliefs tell us something important about LW posters, and if so, are there general ways not tied to a particular country to ask about them. If there isn’t a general way, how can this be handled?
Question I’d like to see added: how much attention do you give to politics? That question should probably be split between attention to theory, attention to news, and attention to trying to make things happen.
I suggested in the survey thread to ask for Political Compass scores instead of a liberal/conservative/libertarian/socialist question. The Compass is slightly US-biased, but it contains enough questions for the end result to be significant even so. How much attention to politics would be an interesting question, I second that.
I suspect the compass is very US-based, though better than a short list or a single dimension.
there’s one more thing about interest in politics that I had trouble phrasing. There’s a thing that I call practical politics which I don’t do, but it’s working for particular candidates or being one yourself or knowing in some detail about the right place to push to get something to happen or not happen. It’s the step beyond voting and emailing your representative and signing petitions.
I’d be surprised if very many LWers do practical politics, but that might just be typical mind fallacy.
They do admit they’re biased, but the bias is not exactly American (indeed, they are British). And given that LW has lots of readers from non-US western countries but few from (say) China, while not ideal, it would be a lot better than the very US-centric answers in the last survey. (For example, I’d bet that a lot of people would have self-identified as socialist libertarians if given the chance.)
Amusingly, Yvain is not American. Though he probably absorbed US culture anyway.
I’m pretty sure Yvain is American, just studying abroad.
Uh, you’re right. I had him as Irish moving to the US, rather than the other way around.
On politics I would like a way to say, I don’t identify with any political theory. To me this is like asking “what religion do you identify most with?” with options christianity, Islam, hinduism, other and the option to click no boxes. If, as an atheist with no religious ties you click other you’re in with shintoists and satanists and other other unmentioned religions. If you don’t answer you’re just not giving an answer. You could just not want to say. In any case the question frames things so that you have to subscribe to a questionable framework to answer it all
Solutions:
Option like the morality one, perhaps, 0 identification options for “other” and “prefer not to say” retaining the ability to click no boxes, though there are probably other reasons to click no boxes. or, as someone suggested in the original thread another question to gage how much you identify with something. The current way If I had to pick an answer I could probably dredge up some preference for one or another theories, but I’d be in the same box as someone actively promoting and a part of what option they clicked. Boxes for strongly identify, identify and weakly identify maybe. Or something.
Got to go.
edit: Could someone kindly explain the downvotes? I’m guessing Too esoteric? Personally not bothered? bothered that I’m bothered?
edit2: just realised some line breaks in the comment box haven’t translated to line breaks in the published comment. Is the post just hard to read?