I’m glad I did this project and wrote this up. When your goal is to make a thing to make the AI alignment community wiser, it’s not really obvious how to tell if you’re succeeding, and this was a nice step forward in doing that in a way that “showed my work”. That said, it’s hard to draw super firm conclusions, because of bias in who takes the survey and some amount of vagueness in the questions. Also, if the survey says a small number of people used a resource and all found it very useful, it’s hard to tell if people who chose not to use the resource would find it useful.
That said, I’m not sure it’s had much of an impact. I don’t see people talking about it much, and the one link-back is an LTFF reviewer saying they didn’t really pay much attention to the survey in deciding whether to fund AXRP. Probably the biggest impact is that the process of figuring out how to make the survey helped me clarify how I would ask people in my day-to-day life whether a resource has been useful (my current favourite is “have you learned something by listening to / reading / partaking in the resource”, alongside whether they like it), but this is not conveyed well in the post itself.
I’m glad I did this project and wrote this up. When your goal is to make a thing to make the AI alignment community wiser, it’s not really obvious how to tell if you’re succeeding, and this was a nice step forward in doing that in a way that “showed my work”. That said, it’s hard to draw super firm conclusions, because of bias in who takes the survey and some amount of vagueness in the questions. Also, if the survey says a small number of people used a resource and all found it very useful, it’s hard to tell if people who chose not to use the resource would find it useful.
That said, I’m not sure it’s had much of an impact. I don’t see people talking about it much, and the one link-back is an LTFF reviewer saying they didn’t really pay much attention to the survey in deciding whether to fund AXRP. Probably the biggest impact is that the process of figuring out how to make the survey helped me clarify how I would ask people in my day-to-day life whether a resource has been useful (my current favourite is “have you learned something by listening to / reading / partaking in the resource”, alongside whether they like it), but this is not conveyed well in the post itself.