I don’t think I am following the argument here. You seem focused on the comparison with evolution, which is only a minor part of Bio Anchors, and used primarily as an upper bound. (You say “the number is so vastly large (and actually unknown due to the ‘level of details’ problem) that it’s not really relevant for timelines calculations,” but actually Bio Anchors still estimates that the evolution anchor implies a ~50% chance of transformative AI this century.)
Generally, I don’t see “A and B are very different” as a knockdown counterargument to “If A required ___ amount of compute, my guess is that B will require no more.” I’m not sure I have more to say on this point that hasn’t already been said—I acknowledge that the comparisons being made are not “tight” and that there’s a lot of guesswork, and the Bio Anchors argument doesn’t go through without some shared premises and intuitions, but I think the needed intuitions are reasonable and an update from Bio-Anchors-ignorant starting positions is warranted.
Unfortunately, I don’t have the time at the moment to answer in detail and have more of a conversation, as I’m fully focused on writing a long sequence about pushing for pluralism in alignment and extracting the core problem out of all the implementation details and additional assumption. I plan on going back to analyzing timeline research in the future, and will probably give better answers then.
That being said, here are quick fire thoughts:
I used the evolution case because I consider it the most obvious/straightforward case, in that it sounds so large that everyone instantly assumes that it gives you an upper bound.
My general impression about this report (and one I expect Yudkowsky to share) is that it didn’t made me update at all. I already updated from GPT and GPT3, and I didn’t find new bits of evidence in the report and the discussions around it, despite the length of it. My current impression (please bear in mind that I haven’t taken the time to study the report from that angle, so I might change my stance) is that this report, much like a lot of timeline work, seems like it takes as input a lot of assumption, and gives as output far less than was assumed. It’s the opposite of compression — a lot of assumptions are needed to conclude things that aren’t that strong and constraining.
I don’t think I am following the argument here. You seem focused on the comparison with evolution, which is only a minor part of Bio Anchors, and used primarily as an upper bound. (You say “the number is so vastly large (and actually unknown due to the ‘level of details’ problem) that it’s not really relevant for timelines calculations,” but actually Bio Anchors still estimates that the evolution anchor implies a ~50% chance of transformative AI this century.)
Generally, I don’t see “A and B are very different” as a knockdown counterargument to “If A required ___ amount of compute, my guess is that B will require no more.” I’m not sure I have more to say on this point that hasn’t already been said—I acknowledge that the comparisons being made are not “tight” and that there’s a lot of guesswork, and the Bio Anchors argument doesn’t go through without some shared premises and intuitions, but I think the needed intuitions are reasonable and an update from Bio-Anchors-ignorant starting positions is warranted.
Thanks for the answer!
Unfortunately, I don’t have the time at the moment to answer in detail and have more of a conversation, as I’m fully focused on writing a long sequence about pushing for pluralism in alignment and extracting the core problem out of all the implementation details and additional assumption. I plan on going back to analyzing timeline research in the future, and will probably give better answers then.
That being said, here are quick fire thoughts:
I used the evolution case because I consider it the most obvious/straightforward case, in that it sounds so large that everyone instantly assumes that it gives you an upper bound.
My general impression about this report (and one I expect Yudkowsky to share) is that it didn’t made me update at all. I already updated from GPT and GPT3, and I didn’t find new bits of evidence in the report and the discussions around it, despite the length of it. My current impression (please bear in mind that I haven’t taken the time to study the report from that angle, so I might change my stance) is that this report, much like a lot of timeline work, seems like it takes as input a lot of assumption, and gives as output far less than was assumed. It’s the opposite of compression — a lot of assumptions are needed to conclude things that aren’t that strong and constraining.