o1 seems to make progress on this problem. Consider the following part of the CoT from the Math section here:
Similarly, since s(x) is of degree…
Let me compute the degree of s(x)
It starts a thought that’s supposed to complete in some statement of fact. The relevant fact happens to be something the model didn’t explicitly infer yet. Instead of inventing something on the fly to fill in the blank, as it’d do if it were mimicking a confidently-written document, it realizes it doesn’t know that fact yet, backpedals, and proceeds to infer it.
o1 seems to make progress on this problem. Consider the following part of the CoT from the Math section here:
It starts a thought that’s supposed to complete in some statement of fact. The relevant fact happens to be something the model didn’t explicitly infer yet. Instead of inventing something on the fly to fill in the blank, as it’d do if it were mimicking a confidently-written document, it realizes it doesn’t know that fact yet, backpedals, and proceeds to infer it.
Thoughts?