I’ve run a few experiments of my own, trying to get it to contribute to some “philosophically challenging” mathematical research in agent foundations, and I think Terence’s take is pretty much correct. It’s pretty good at sufficiently well-posed problems – even if they’re very mathematically challenging – but it doesn’t have good “research taste”/creative competence. Including for subproblems that it runs into as part of an otherwise well-posed problem.
Which isn’t to say it’s a nothingburger: some of the motions it makes in its hidden CoT are quite scary, and represent nontrivial/qualitative progress. But the claims of “it reasons at the level of a PhD candidate!” are true only in a very limited sense.
(One particular pattern that impressed me could be seen in the Math section:
Similarly, since s(x) is of degree…
Let me compute the degree of s(x)
Consider: it started outputting a thought that was supposed to fetch some fact which it didn’t infer yet. Then, instead of hallucinating/inventing something on the fly to complete the thought, it realized that it didn’t figure that part out yet, then stopped itself, and swerved to actually computing the fact. Very non-LLM-like!)
I’ve run a few experiments of my own, trying to get it to contribute to some “philosophically challenging” mathematical research in agent foundations, and I think Terence’s take is pretty much correct. It’s pretty good at sufficiently well-posed problems – even if they’re very mathematically challenging – but it doesn’t have good “research taste”/creative competence. Including for subproblems that it runs into as part of an otherwise well-posed problem.
Which isn’t to say it’s a nothingburger: some of the motions it makes in its hidden CoT are quite scary, and represent nontrivial/qualitative progress. But the claims of “it reasons at the level of a PhD candidate!” are true only in a very limited sense.
(One particular pattern that impressed me could be seen in the Math section:
Consider: it started outputting a thought that was supposed to fetch some fact which it didn’t infer yet. Then, instead of hallucinating/inventing something on the fly to complete the thought, it realized that it didn’t figure that part out yet, then stopped itself, and swerved to actually computing the fact. Very non-LLM-like!)