Yeah, pretty much. Although I don’t expect this with super high confidence. Maybe 75%?
This is part of why I think a “pause” focused on large models / large training runs would actually dangerously accelerate progress towards AGI. I think a lot of well-resourced high-skill researchers would suddenly shift their focus onto breadth of exploration.
Another point:
I don’t think we’ll ever see AI agents that are exactly isomorphic to junior researchers. Why? Because of the weird spikiness of skills we see. In some ways the LLMs we have are much more skillful than junior researchers, in other ways they are pathetically bad. If you held their competencies constant except for improving the places where they are really bad, you’d suddenly have assistants much better than the median junior!
So when considering the details of how to apply the AI assistants we’re likely to get (based on extrapolating current spiky skill patterns), the set of affordances this offers to the top researchers is quite different from what having a bunch of juniors would be. I think this means we should expect things to weirder and less smooth than Ryan’s straightforward speed-up prediction.
If you look at the recent AI scientist work that’s been done you find this weird spiky portfolio. Having LLMs look through a bunch of papers and try to come up with new research directions? Mostly, but not entirely crap… But then since it’s relatively cheap to do, and quick to do, and not too costly to filter, the trade-off ends up seeming worthwhile?
As for new experiments in totally new regimes, yeah. That’s harder for current LLMs to help with than the well-trodden zones. But I think the specific skills currently beginning to be unlocked by the o1/o3 direction may be enough to make coding agents reliable enough to do a much larger share of this novel experiment setup.
So… It’s complicated. Can’t be sure of success. Can’t be sure of a wall.
Yeah, pretty much. Although I don’t expect this with super high confidence. Maybe 75%?
This is part of why I think a “pause” focused on large models / large training runs would actually dangerously accelerate progress towards AGI. I think a lot of well-resourced high-skill researchers would suddenly shift their focus onto breadth of exploration.
Another point:
I don’t think we’ll ever see AI agents that are exactly isomorphic to junior researchers. Why? Because of the weird spikiness of skills we see. In some ways the LLMs we have are much more skillful than junior researchers, in other ways they are pathetically bad. If you held their competencies constant except for improving the places where they are really bad, you’d suddenly have assistants much better than the median junior!
So when considering the details of how to apply the AI assistants we’re likely to get (based on extrapolating current spiky skill patterns), the set of affordances this offers to the top researchers is quite different from what having a bunch of juniors would be. I think this means we should expect things to weirder and less smooth than Ryan’s straightforward speed-up prediction.
If you look at the recent AI scientist work that’s been done you find this weird spiky portfolio. Having LLMs look through a bunch of papers and try to come up with new research directions? Mostly, but not entirely crap… But then since it’s relatively cheap to do, and quick to do, and not too costly to filter, the trade-off ends up seeming worthwhile?
As for new experiments in totally new regimes, yeah. That’s harder for current LLMs to help with than the well-trodden zones. But I think the specific skills currently beginning to be unlocked by the o1/o3 direction may be enough to make coding agents reliable enough to do a much larger share of this novel experiment setup.
So… It’s complicated. Can’t be sure of success. Can’t be sure of a wall.