Hey Dan, thanks for responding. I wanted to ask a few questions:
You noted the non-response rate for the 20 randomly selected alumni. What about the non-response rate for the feedback survey?
“0 to 10, are you glad you came?” This is a biased question, because you frame that the person is glad. A similar negative question may say “0 to 10, are you dissatisfied that you came?” Would it be possible to anonymize and post the survey questions and data?
We also sent out a survey earlier this year to 20 randomly selected alumni who had attended workshops in the previous 3-18 months, and asked them the same question. 18 of the 20 filled out the survey, and their average response to that question was 9.6.
It’s great that you’re following up with people long after the workshops end. Why not survey all alumni? You have their emails.
I’ve read most of the blog posts about CFAR workshops that you linked to—they were one of my main motivations for attending a workshop. I notice that all reviews are from people who have already participated in LessWrong and related communities. (all refer to some prior CFAR, EA and rationality related topics before they attended camp). Also, it seems like in person conversations are majorly subjected to the availability bias, as the people who attended workshops || know people who work at MIRI/CFAR || are involved in LW meetups in Berkeley and surrounding areas would contribute to the positivity of these conversations.. Also, the evaporative cooling effect may also play a role, in that people who weren’t satisfied with the workshop would leave the group. Are there reviews from people who are not already familiar with LW/CFAR staff?
Also, I agree with MTGandP. It would be nice if CFAR could write a blog post or paper on how effective their teachings are, compared to a control group. Perhaps two one-day events, with subjects randomized across both days, should work well as a starting point.
Hey Dan, thanks for responding. I wanted to ask a few questions:
You noted the non-response rate for the 20 randomly selected alumni. What about the non-response rate for the feedback survey?
“0 to 10, are you glad you came?” This is a biased question, because you frame that the person is glad. A similar negative question may say “0 to 10, are you dissatisfied that you came?” Would it be possible to anonymize and post the survey questions and data?
It’s great that you’re following up with people long after the workshops end. Why not survey all alumni? You have their emails.
I’ve read most of the blog posts about CFAR workshops that you linked to—they were one of my main motivations for attending a workshop. I notice that all reviews are from people who have already participated in LessWrong and related communities. (all refer to some prior CFAR, EA and rationality related topics before they attended camp). Also, it seems like in person conversations are majorly subjected to the availability bias, as the people who attended workshops || know people who work at MIRI/CFAR || are involved in LW meetups in Berkeley and surrounding areas would contribute to the positivity of these conversations.. Also, the evaporative cooling effect may also play a role, in that people who weren’t satisfied with the workshop would leave the group. Are there reviews from people who are not already familiar with LW/CFAR staff?
Also, I agree with MTGandP. It would be nice if CFAR could write a blog post or paper on how effective their teachings are, compared to a control group. Perhaps two one-day events, with subjects randomized across both days, should work well as a starting point.