I’ve (briefly) addressed the compute bottleneck question on a different comment branch, and “hard-to-automate activities aren’t a problem” on another (confusion regarding the definition of various milestones).
[Dependence on Narrow Data Sets] is only applicable to the timeline to the superhuman coder milestone, not to takeoff speeds once we have a superhuman coder. (Or maybe you think a similar argument applies to the time between superhuman coder and SAR.)
I do think it applies, if indirectly. Most data relating to progress in AI capabilities comes from benchmarks of crisply encapsulated tasks. I worry this may skew our collective intuitions regarding progress toward broader capabilities, especially as I haven’t seen much attention paid to exploring the delta between things we currently benchmark and “everything”.
Hofstadter’s Law As Prior
Math: We’re talking about speed up relative to what the human researchers would have done by default, so this just divides both sides equally and cancels out.
This feels like one of those “the difference between theory and practice is smaller in theory than in practice” situations… Hofstadter’s Law would imply that Hofstadter’s Law applies here. :-)
For one concrete example of how that could manifest, perhaps there is a delay between “AI models exist that are superhuman at all activities involved in developing better models” and “those models have been fully adopted across the organization”. Interior to a frontier lab, that specific delay might be immaterial, it’s just meant as an existence proof that there’s room for us to be missing things.
I’ve (briefly) addressed the compute bottleneck question on a different comment branch, and “hard-to-automate activities aren’t a problem” on another (confusion regarding the definition of various milestones).
I do think it applies, if indirectly. Most data relating to progress in AI capabilities comes from benchmarks of crisply encapsulated tasks. I worry this may skew our collective intuitions regarding progress toward broader capabilities, especially as I haven’t seen much attention paid to exploring the delta between things we currently benchmark and “everything”.
This feels like one of those “the difference between theory and practice is smaller in theory than in practice” situations… Hofstadter’s Law would imply that Hofstadter’s Law applies here. :-)
For one concrete example of how that could manifest, perhaps there is a delay between “AI models exist that are superhuman at all activities involved in developing better models” and “those models have been fully adopted across the organization”. Interior to a frontier lab, that specific delay might be immaterial, it’s just meant as an existence proof that there’s room for us to be missing things.