Calibration Question And Probability Question Analysis
Charity And Effective Altruism Analysis
When we started the survey in march I posted this graph showing the dropoff in question responses over time:
So it seems only reasonable to post the same graph with this years survey data:
(I should note that this analysis counts certain things as questions that the other chart does not, so it says there are many more questions than the previous survey when in reality where are about as many as last year.)
Almost two thousand eight hundred and fifty hours were spent surveying this year and you’ve all waited nearly two months from the first survey response to the results writeup. While the results have been available for over a week, they haven’t seen widespread dissemination in large part because they lacked a succinct summary of their contents. What you are reading now will of course be that summary. First, some statistics on the survey itself:
Survey hours spent in total: 2849.818888888889 Average number of minutes spent on survey: 102.14404619673437 Median number of minutes spent on survey: 39.775 Mode minutes spent on survey: 20.266666666666666
The takeaway here seems to be that some people take a long time with the survey, raising the average. However, most people’s survey time is somewhere below the forty five minute mark. LessWrong does a very long survey, and I wanted to make sure that investment was rewarded with a deep detailed analysis. Weighing in at over four thousand lines of python code, I hope the analysis I’ve put together is worth the wait. Before I continue I’d like to thank people who contributed to the analysis effort:
Bartosz Wroblewski (Bawr on #lesswrong)
Kuudes on #lesswrong
Obormot on #lesswrong
Two anonymous contributors
And anybody else who I may have forgotten. Thanks again to Scott Alexander, who wrote the majority of the survey and ran it in 2014, and who has also been generous enough to license his part of the survey under a creative commons license along with mine.
Demographics
The 2014 survey gave these numbers for age:
Age: 27.67 + 8.679 (22, 26, 31) [1490]
In 2016 the numbers were:
Mean: 28.108772669759592 Median: 26.0 Mode: 23.0
Most LWers are in their early to mid twenties, with some older LWers bringing up the average. The average is close enough to the former figure that we can probably say the LW demographic is in their 20′s or 30′s as a general rule.
In 2014 our gender ratio looked like this:
Female: 179, 11.9% Male: 1311, 87.2%
In 2016 the proportion of women in the community went up by over four percent:
Male: 2021 83.5% Female: 393 16.2%
One hypothesis on why this happened is that the 2016 survey focused on the diaspora rather than just LW. Diaspora communities plausibly have marginally higher rates of female membership. If I had more time I would write an analysis investigating the demographics of each diaspora community, but to answer this particular question I think a couple of SQL queries are illustrative:
(Note: ActiveMemberships one and two are 'LessWrong' and 'LessWrong Meetups' respectively.)
sqlite> select count(birthsex) from data where (ActiveMemberships_1 = "Yes" OR ActiveMemberships_2 = "Yes") AND birthsex="Male";
425
sqlite> select count(birthsex) from data where (ActiveMemberships_1 = "Yes" OR ActiveMemberships_2 = "Yes") AND birthsex="Female";
66
>>> 66 / 425
0.15529411764705883
Well, that hypothesis seems to be out. Of course, before we wring our hands too much on this question it pays to remember that assigned sex at birth isn’t the whole story. The gender question in 2014 had these results:
F (cisgender): 150, 10.0% F (transgender MtF): 24, 1.6% M (cisgender): 1245, 82.8% M (transgender FtM): 5, 0.3% Other: 64, 4.3%
In 2016:
F (cisgender): 321 13.3% F (transgender MtF): 65 2.7% M (cisgender): 1829 76% M (transgender FtM): 23 1% Other: 156 6.48%
Some things to note here. 16.2% of respondents were assigned female at birth but only 13.3% still identify as women. 1% are transmen, but where did the other 1.9% go? Presumably into the ‘Other’ field. Let’s find out.
sqlite> select count(birthsex) from data where birthsex = "Female" AND gender = "Other";
57
sqlite> select count(*) from data;
3083
>>> 57 / 3083
0.018488485241647746
Seems to be the case. In general the proportion of men is down 6.1% from 2014. We also gained 1.1% transwomen and .7% transmen in 2016. Moving away from binary genders, this surveys nonbinary gender count gained in proportion by nearly 2.2%. This means that over one in twenty LWers identified as a nonbinary gender, making it a larger demographic than binary transgender LWers! As exciting as that may sound to some ears the numbers tell one story and the write ins tell quite another.
It pays to keep in mind that nonbinary genders are a common troll option for people who want to write in criticism of the question. A quick look at the write ins accompanying the other option indicates that this is what many people used it for, but by no means all. At 156 responses, that’s small enough to be worth doing a quick manual tally.
Key = Agender, Esoteric, Female, Male, Male-to-Female, Nonbinary,
Objection on Basis Gender Doesn't Exist, Objection on Basis Gender Is Binary,
in Process of Transitioning, Refusal, Undecided
Sample Size: 156
A 35
E 6
F 6
M 21
MTF 1
NB 55
OBGDE 6
OBGIB 7
PT 2
R 7
U 10
So depending on your comfort zone as to what constitutes a countable gender, there are 90 to 96 valid ‘other’ answers in the survey dataset. (Labeled dataset)
>>> 90 / 3083
0.029192345118391177
With some cleanup the number trails behind the binary transgender one by the greater part of a percentage point, but only by. I bet that if you went through and did the same sort of tally on the 2014 survey results you’d find that the proportion of valid nonbinary gender write ins has gone up between then and now.
Some interesting ‘esoteric’ answers: Attack Helocopter, Blackstar, Elizer, spiderman, Agenderfluid
For the rest of this section I’m going to just focus on differences between the 2016 and 2014 surveys.
[Summing these all up to one shows that nearly 1% of change is unaccounted for. My hypothesis is that this 1% went into the other countries not in the list, this can’t be easily confirmed because the 2014 analysis does not list the other country percentage.]
[LessWrong got 5.3% more gay, 9.1% if you’re more loose with the definition. Before we start any wild speculation, the 2014 question included asexuality as an option and it got 3.9% of the responses, we spun this off into a separate question on the 2016 survey which should explain a significant portion of the change.]
Are you asexual?
Yes: 171 0.074 No: 2129 0.926
[Scott said in 2014 that he’d probably ‘vastly undercounted’ our asexual readers, a near doubling in our count would seem to support this.]
...and seeking more relationship partners: +0.200% 577 24.800% ...and possibly open to more relationship partners: −0.300% 716 30.800% ...and currently not looking for more relationship partners: +1.300% 1034 44.400%
Are you married?
Yes: 443 0.19 No: 1885 0.81
[This question appeared in a different form on the previous survey. Marriage went up by .8% from last year.]
Who do you currently live with most of the time?
Alone: −2.200% 487 20.800% With parents and/or guardians: +0.100% 476 20.300% With partner and/or children: +2.100% 687 29.400% With roommates: −2.000% 619 26.500%
[This would seem to line up with the result that single LWers went down by 2.3%]
[This is an interesting result, either nearly 4% of LWers are suddenly less enthusiastic about having kids, or new entrants to the survey are less likely and less sure if they want to. Possibly both.]
[Hm, the academic credentials of LWers seems to have gone down some since the last survey. As usual this may also be the result of new survey entrants.]
2016 LessWrong Diaspora Survey Analysis: Part One (Meta and Demographics)
2016 LessWrong Diaspora Survey Analysis
Overview
Meta
Demographics (You are here)
LessWrong Usage and Experience
LessWrong Criticism and Successorship
Diaspora Community Analysis
What it all means for LW 2.0
Transition into non-core/bonus sections.
Mental Health Section
Basilisk Section/Analysis
Blogs and Media analysis
Calibration Question And Probability Question Analysis
Charity And Effective Altruism Analysis
When we started the survey in march I posted this graph showing the dropoff in question responses over time:
So it seems only reasonable to post the same graph with this years survey data:
(I should note that this analysis counts certain things as questions that the other chart does not, so it says there are many more questions than the previous survey when in reality where are about as many as last year.)
Almost two thousand eight hundred and fifty hours were spent surveying this year and you’ve all waited nearly two months from the first survey response to the results writeup. While the results have been available for over a week, they haven’t seen widespread dissemination in large part because they lacked a succinct summary of their contents. What you are reading now will of course be that summary. First, some statistics on the survey itself:
Survey hours spent in total: 2849.818888888889
Average number of minutes spent on survey: 102.14404619673437
Median number of minutes spent on survey: 39.775
Mode minutes spent on survey: 20.266666666666666
The takeaway here seems to be that some people take a long time with the survey, raising the average. However, most people’s survey time is somewhere below the forty five minute mark. LessWrong does a very long survey, and I wanted to make sure that investment was rewarded with a deep detailed analysis. Weighing in at over four thousand lines of python code, I hope the analysis I’ve put together is worth the wait. Before I continue I’d like to thank people who contributed to the analysis effort:
Bartosz Wroblewski (Bawr on #lesswrong)
Kuudes on #lesswrong
Obormot on #lesswrong
Two anonymous contributors
And anybody else who I may have forgotten. Thanks again to Scott Alexander, who wrote the majority of the survey and ran it in 2014, and who has also been generous enough to license his part of the survey under a creative commons license along with mine.
Demographics
The 2014 survey gave these numbers for age:
Age: 27.67 + 8.679 (22, 26, 31) [1490]
In 2016 the numbers were:
Mean: 28.108772669759592
Median: 26.0
Mode: 23.0
Most LWers are in their early to mid twenties, with some older LWers bringing up the average. The average is close enough to the former figure that we can probably say the LW demographic is in their 20′s or 30′s as a general rule.
In 2014 our gender ratio looked like this:
Female: 179, 11.9%
Male: 1311, 87.2%
In 2016 the proportion of women in the community went up by over four percent:
Male: 2021 83.5%
Female: 393 16.2%
One hypothesis on why this happened is that the 2016 survey focused on the diaspora rather than just LW. Diaspora communities plausibly have marginally higher rates of female membership. If I had more time I would write an analysis investigating the demographics of each diaspora community, but to answer this particular question I think a couple of SQL queries are illustrative:
Well, that hypothesis seems to be out. Of course, before we wring our hands too much on this question it pays to remember that assigned sex at birth isn’t the whole story. The gender question in 2014 had these results:
F (cisgender): 150, 10.0%
F (transgender MtF): 24, 1.6%
M (cisgender): 1245, 82.8%
M (transgender FtM): 5, 0.3%
Other: 64, 4.3%
In 2016:
F (cisgender): 321 13.3%
F (transgender MtF): 65 2.7%
M (cisgender): 1829 76%
M (transgender FtM): 23 1%
Other: 156 6.48%
Some things to note here. 16.2% of respondents were assigned female at birth but only 13.3% still identify as women. 1% are transmen, but where did the other 1.9% go? Presumably into the ‘Other’ field. Let’s find out.
Seems to be the case. In general the proportion of men is down 6.1% from 2014. We also gained 1.1% transwomen and .7% transmen in 2016. Moving away from binary genders, this surveys nonbinary gender count gained in proportion by nearly 2.2%. This means that over one in twenty LWers identified as a nonbinary gender, making it a larger demographic than binary transgender LWers! As exciting as that may sound to some ears the numbers tell one story and the write ins tell quite another.
It pays to keep in mind that nonbinary genders are a common troll option for people who want to write in criticism of the question. A quick look at the write ins accompanying the other option indicates that this is what many people used it for, but by no means all. At 156 responses, that’s small enough to be worth doing a quick manual tally.
So depending on your comfort zone as to what constitutes a countable gender, there are 90 to 96 valid ‘other’ answers in the survey dataset. (Labeled dataset)
With some cleanup the number trails behind the binary transgender one by the greater part of a percentage point, but only by. I bet that if you went through and did the same sort of tally on the 2014 survey results you’d find that the proportion of valid nonbinary gender write ins has gone up between then and now.
Some interesting ‘esoteric’ answers: Attack Helocopter, Blackstar, Elizer, spiderman, Agenderfluid
For the rest of this section I’m going to just focus on differences between the 2016 and 2014 surveys.
Country
United States: −1.000% 1298 53.700%
United Kingdom: −0.100% 183 7.600%
Canada: +0.100% 144 6.000%
Australia: +0.300% 141 5.800%
Germany: −0.600% 85 3.500%
Russia: +0.700% 57 2.400%
Finland: −0.300% 25 1.000%
New Zealand: −0.200% 26 1.100%
India: −0.100% 24 1.000%
Brazil: −0.300% 16 0.700%
France: +0.400% 34 1.400%
Israel: +0.200% 29 1.200%
Other: 354 14.646%
[Summing these all up to one shows that nearly 1% of change is unaccounted for. My hypothesis is that this 1% went into the other countries not in the list, this can’t be easily confirmed because the 2014 analysis does not list the other country percentage.]
Race
Asian (East Asian): −0.600% 80 3.300%
Asian (Indian subcontinent): +0.300% 60 2.500%
Middle Eastern: 0.000% 14 0.600%
Black: −0.300% 12 0.500%
White (non-Hispanic): −0.300% 2059 85.800%
Hispanic: +0.300% 57 2.400%
Other: +1.200% 108 4.500%
Sexual Orientation
Heterosexual: −5.000% 1640 70.400%
Homosexual: +1.300% 103 4.400%
Bisexual: +4.000% 428 18.400%
Other: +3.880% 144 6.180%
[LessWrong got 5.3% more gay, 9.1% if you’re more loose with the definition. Before we start any wild speculation, the 2014 question included asexuality as an option and it got 3.9% of the responses, we spun this off into a separate question on the 2016 survey which should explain a significant portion of the change.]
Are you asexual?
Yes: 171 0.074
No: 2129 0.926
[Scott said in 2014 that he’d probably ‘vastly undercounted’ our asexual readers, a near doubling in our count would seem to support this.]
Relationship Style
Prefer monogomous: −0.900% 1190 50.900%
Prefer polyamorous: +3.100% 426 18.200%
Uncertain/no preference: −2.100% 673 28.800%
Other: +0.426% 45 1.926%
[Polyamorous gained three points, presumably the drop in uncertain people went into that bin.]
Number of Partners
0: −2.300% 1094 46.800%
1: −0.400% 1039 44.400%
2: +1.200% 107 4.600%
3: +0.900% 46 2.000%
4: +0.100% 15 0.600%
5: +0.200% 8 0.300%
Lots and lots: +1.000% 29 1.200%
Relationship Goals
...and seeking more relationship partners: +0.200% 577 24.800%
...and possibly open to more relationship partners: −0.300% 716 30.800%
...and currently not looking for more relationship partners: +1.300% 1034 44.400%
Are you married?
Yes: 443 0.19
No: 1885 0.81
[This question appeared in a different form on the previous survey. Marriage went up by .8% from last year.]
Who do you currently live with most of the time?
Alone: −2.200% 487 20.800%
With parents and/or guardians: +0.100% 476 20.300%
With partner and/or children: +2.100% 687 29.400%
With roommates: −2.000% 619 26.500%
[This would seem to line up with the result that single LWers went down by 2.3%]
How many children do you have?
0: +5.400% 2042 87.000%
1: +0.500% 115 4.900%
2: +0.100% 124 5.300%
3: +0.900% 48 2.000%
4: −0.100% 7 0.300%
5: +0.100% 6 0.300%
6: 0.000% 2 0.100%
Lots and lots: 0.000% 3 0.100%
[Interestingly enough, childless LWers went up by 5.4%. This would seem incongruous with the previous results. Not sure how to investigate though.]
Are you planning on having more children?
Yes: −5.400% 720 30.700%
Uncertain: +3.900% 755 32.200%
No: +2.800% 869 37.100%
[This is an interesting result, either nearly 4% of LWers are suddenly less enthusiastic about having kids, or new entrants to the survey are less likely and less sure if they want to. Possibly both.]
Work Status
Student: −5.402% 0 31.398%
Academics: +0.949% 0 6.649%
Self-employed: +4.223% 0 10.023%
Independently wealthy: +0.762% 0 1.362%
Non-profit work: +1.030% 0 4.930%
For-profit work: −1.756% 0 30.944%
Government work: +0.479% 0 4.379%
Homemaker: +1.024% 0 1.524%
Unemployed: +0.495% 0 7.395%
[Most interesting result here is that 5.4% of LWers are no longer students or new survey entrants aren’t.]
Profession
Art: +0.800% 51 2.300%
Biology: +0.300% 49 2.200%
Business: −0.800% 72 3.200%
Computers (AI): +0.700% 79 3.500%
Computers (other academic, computer science): −0.100% 156 7.000%
Computers (practical): −1.200% 681 30.500%
Engineering: +0.600% 150 6.700%
Finance / Economics: +0.500% 116 5.200%
Law: −0.300% 50 2.200%
Mathematics: −1.500% 147 6.600%
Medicine: +0.100% 49 2.200%
Neuroscience: +0.100% 28 1.300%
Philosophy: 0.000% 54 2.400%
Physics: −0.200% 91 4.100%
Psychology: 0.000% 48 2.100%
Other: +2.199% 277 12.399%
Other “hard science”: −0.500% 26 1.200%
Other “social science”: −0.200% 48 2.100%
[The largest profession growth for LWers in 2016 was art, that or this is a consequence of new survey entrants.]
What is your highest education credential earned?
None: −0.700% 96 4.200%
High School: +3.600% 617 26.700%
2 year degree: +0.200% 105 4.500%
Bachelor’s: −1.600% 815 35.300%
Master’s: −0.500% 415 18.000%
JD/MD/other professional degree: 0.000% 66 2.900%
PhD: −0.700% 145 6.300%
Other: +0.288% 39 1.688%
[Hm, the academic credentials of LWers seems to have gone down some since the last survey. As usual this may also be the result of new survey entrants.]