I don’t think many with monotonically increasing doom pay attention to current or any alignment research when they make their updates. I don’t think they’re using any model to base their estimates on. When you go and survey people, and pose the same timelines question phrased in two different ways, they give radically different answers[1].
I also think people usually feel the surprise at the fact we can get machines to do what they do at all at this stage in AI research, and not at the minorly unexpected deficiencies of alignment work.
Tangentially, it is also useful to remember this fact when trying to defer to experts. They gain no special powers in future prediction by virtue of their subject expertise, as has always been the case (“Progress is driven by peak knowledge, not average knowledge.”).
I don’t think many with monotonically increasing doom pay attention to current or any alignment research when they make their updates. I don’t think they’re using any model to base their estimates on. When you go and survey people, and pose the same timelines question phrased in two different ways, they give radically different answers.
I don’t think this survey is a good indicator, since it is very narrow in the questions.
If I was designing a survey to understand people’s predictions on AI progress, I would first do a bunch of qualitative questions asking them things like what models they are using, what they are paying attention to, what they expect to happen with AI, etc.. (And probably also other questions, such as whose analysis they know of and respect, etc.) This would help give a comprehensive list of perspectives that may inform people’s opinions.
Then I would take the qualitative perspectives and turn them into questions that more properly assess people’s perspectives. Only then can one really see whether the inconsistent answers people give really are so implausible, or if one is missing the bigger picture.
Ok, I went and looked at the survey (instead of just going based on memories of an 80k podcast that Ajeya was on where she stressed the existence of framing effects), and it is indeed very narrow, and looking at the effect of framing effects, I’m now less/not confident this is convincing to anyone who needs to be convinced that experts aren’t really using a model, but I still think that most likely don’t have a model, because people in general don’t usually have a model.
I don’t think many with monotonically increasing doom pay attention to current or any alignment research when they make their updates
Maybe I am just one of the “not many”. But I think this depends on how closely you track your timelines.
Personally, my timelines are uncertain enough that most of my substantial updates have been in the earlier direction (like from Median ~2050 to median 2030-2035). This probably happens to a lot of people who newly enter the field, because they naturally first put more emphasis on surveys like the one you mentioned.
I think my biggest ones were:
going from “taking the takes from capabilities researchers at face value, not having my own model and going with Metaculus” to “having my own views”.
GPT2 (…and the log loss still goes down) and then the same with GPT3. In the beginning, I still had substantial probability mass (30%) on this trend just not continuing.
Minverva (apparently getting language models to do math is not that hard (which was basically my last “trip wire” going off)).
I do think my P(doom) has slightly decreased from seeing everyone else finally freaking out.
Past me is trying to give himself too much credit here. Most of it was epistemic luck/high curiosity that lead him to join Søren Elverlin’s reading group in 2019 and then I just got exposed to the takes from the community.
I don’t think many with monotonically increasing doom pay attention to current or any alignment research when they make their updates. I don’t think they’re using any model to base their estimates on. When you go and survey people, and pose the same timelines question phrased in two different ways, they give radically different answers[1].
I also think people usually feel the surprise at the fact we can get machines to do what they do at all at this stage in AI research, and not at the minorly unexpected deficiencies of alignment work.
Tangentially, it is also useful to remember this fact when trying to defer to experts. They gain no special powers in future prediction by virtue of their subject expertise, as has always been the case (“Progress is driven by peak knowledge, not average knowledge.”).
I don’t think this survey is a good indicator, since it is very narrow in the questions.
If I was designing a survey to understand people’s predictions on AI progress, I would first do a bunch of qualitative questions asking them things like what models they are using, what they are paying attention to, what they expect to happen with AI, etc.. (And probably also other questions, such as whose analysis they know of and respect, etc.) This would help give a comprehensive list of perspectives that may inform people’s opinions.
Then I would take the qualitative perspectives and turn them into questions that more properly assess people’s perspectives. Only then can one really see whether the inconsistent answers people give really are so implausible, or if one is missing the bigger picture.
Ok, I went and looked at the survey (instead of just going based on memories of an 80k podcast that Ajeya was on where she stressed the existence of framing effects), and it is indeed very narrow, and looking at the effect of framing effects, I’m now less/not confident this is convincing to anyone who needs to be convinced that experts aren’t really using a model, but I still think that most likely don’t have a model, because people in general don’t usually have a model.
Maybe I am just one of the “not many”. But I think this depends on how closely you track your timelines. Personally, my timelines are uncertain enough that most of my substantial updates have been in the earlier direction (like from Median ~2050 to median 2030-2035). This probably happens to a lot of people who newly enter the field, because they naturally first put more emphasis on surveys like the one you mentioned. I think my biggest ones were:
going from “taking the takes from capabilities researchers at face value, not having my own model and going with Metaculus” to “having my own views”.
GPT2 (…and the log loss still goes down) and then the same with GPT3. In the beginning, I still had substantial probability mass (30%) on this trend just not continuing.
Minverva (apparently getting language models to do math is not that hard (which was basically my last “trip wire” going off)).
I do think my P(doom) has slightly decreased from seeing everyone else finally freaking out.
Past me is trying to give himself too much credit here. Most of it was epistemic luck/high curiosity that lead him to join Søren Elverlin’s reading group in 2019 and then I just got exposed to the takes from the community.