“Non-trivial” is a pretty soft word to include in this sort of prediction, in my opinion.
I think I’d disagree if you had said “purely AI-written paper resolves an open millennium prize problem”, but as written I’m saying to myself “hrm, I don’t know how to engage with this in a way that will actually pin down the prediction”.
I think it’s well enough established that long form internally coherent content is within the capabilities of a sufficiently large language model. I think the bottleneck on it being scary (or rather, it being not long before The End) is the LLM being responsible for the inputs to the research.
Fair point, “non-trivial” is too subjective, the intuition that I meant to convey was that if we get to the point where LLMs can do the sort of pure-thinking research in math and physics at a level where the papers build on top of one another in a coherent way, then I’d expect us to be close to the end.
Said another way, if theoretical physicists and mathematicians get automated, then we ought to be fairly close to the end. If in addition to that the physical research itself gets automated, such that LLMs write their own code to do experiments (or run the robotic arms that manipulate real stuff) and publish the results, then we’re *really* close to the end.
“Non-trivial” is a pretty soft word to include in this sort of prediction, in my opinion.
I think I’d disagree if you had said “purely AI-written paper resolves an open millennium prize problem”, but as written I’m saying to myself “hrm, I don’t know how to engage with this in a way that will actually pin down the prediction”.
I think it’s well enough established that long form internally coherent content is within the capabilities of a sufficiently large language model. I think the bottleneck on it being scary (or rather, it being not long before The End) is the LLM being responsible for the inputs to the research.
Fair point, “non-trivial” is too subjective, the intuition that I meant to convey was that if we get to the point where LLMs can do the sort of pure-thinking research in math and physics at a level where the papers build on top of one another in a coherent way, then I’d expect us to be close to the end.
Said another way, if theoretical physicists and mathematicians get automated, then we ought to be fairly close to the end. If in addition to that the physical research itself gets automated, such that LLMs write their own code to do experiments (or run the robotic arms that manipulate real stuff) and publish the results, then we’re *really* close to the end.