I guess I would say: Ajeya’s framework/model can incorporate this objection; this isn’t a “get rid of the whole framework” objection but rather a “tweak the model in the following way” objection.
Like, I agree that it would be bad if everyone who used Ajeya’s model had to put 100% of their probability mass into the six bio anchors she chose. That’s super misleading/biasing/ignores loads of other possible ways AGI might happen. But I don’t think of this as a necessary part of Ajeya’s model; when I use it, I throw out the six bio anchors and just directly input my probability distribution over OOMs of compute. My distribution is informed by the bio anchors, of course, but that’s not the only thing that informs it.
First, I want to clarify that I feel we’re going into a more interesting place, where there’s a better chance that you might find a point that invalidates Yudkowsky’s argument, and can thus convince him of the value of the model.
But it’s also important to realize that IMO, Yudkowsky is not just saying that biological anchors are bad. The more general problem (which is also developed in this post) is that predicting the Future is really hard. In his own model of AGI timelines, the factor that is basically impossible to predict until you can make AGI is the “how much resources are needed to build AGI”.
So saying “let’s just throw away the biological anchors” doesn’t evade the general counterargument that to predict timelines at all, you need to find information on “how much resources are needed to build AGI”, and that is incredibly hard. If you or Ajeya can argue for actual evidence in that last question, then yeah, I expect Yudkowsky would possibly update on the validity of the timeline estimates.
But at the moment, in this thread, I see no argument like that.
I guess I would say: Ajeya’s framework/model can incorporate this objection; this isn’t a “get rid of the whole framework” objection but rather a “tweak the model in the following way” objection.
Like, I agree that it would be bad if everyone who used Ajeya’s model had to put 100% of their probability mass into the six bio anchors she chose. That’s super misleading/biasing/ignores loads of other possible ways AGI might happen. But I don’t think of this as a necessary part of Ajeya’s model; when I use it, I throw out the six bio anchors and just directly input my probability distribution over OOMs of compute. My distribution is informed by the bio anchors, of course, but that’s not the only thing that informs it.
First, I want to clarify that I feel we’re going into a more interesting place, where there’s a better chance that you might find a point that invalidates Yudkowsky’s argument, and can thus convince him of the value of the model.
But it’s also important to realize that IMO, Yudkowsky is not just saying that biological anchors are bad. The more general problem (which is also developed in this post) is that predicting the Future is really hard. In his own model of AGI timelines, the factor that is basically impossible to predict until you can make AGI is the “how much resources are needed to build AGI”.
So saying “let’s just throw away the biological anchors” doesn’t evade the general counterargument that to predict timelines at all, you need to find information on “how much resources are needed to build AGI”, and that is incredibly hard. If you or Ajeya can argue for actual evidence in that last question, then yeah, I expect Yudkowsky would possibly update on the validity of the timeline estimates.
But at the moment, in this thread, I see no argument like that.