The estimate of the compute of their largest version ever (which is a very helpful way to phrase it) at only <=50x GPT-4 is quite relevant to many discussions (props to Nesov) and something Altman probably shouldn’t’ve said.
The estimate of test-time compute at 1000x effective-compute is confirmation of looser talk.
The scientific research part is of uncertain importance but we may well be referring back to this statement a year from now.
Good point regarding GPT-”4.5″. I guess I shouldn’t have assumed that everyone else has also read Nesov’s analyses and immediately (accurately) classified them as correct.
He says things that are advantageous, and sometimes they are even true. The benefit of not being known to be a liar usually keeps the correlation between claims and truth positive, but in his case it seems that ship has sailed.
(Checkably false claims are still pretty rare, and this may be one of those.)
Wow, that is a surprising amount of information. I wonder how reliable we should expect this to be.
Is it? What of this is new?
To my eyes, the only remotely new thing is the admission that “there’s a lot of research still to get to [a coding agent]”.
The estimate of the compute of their largest version ever (which is a very helpful way to phrase it) at only <=50x GPT-4 is quite relevant to many discussions (props to Nesov) and something Altman probably shouldn’t’ve said.
The estimate of test-time compute at 1000x effective-compute is confirmation of looser talk.
The scientific research part is of uncertain importance but we may well be referring back to this statement a year from now.
Good point regarding GPT-”4.5″. I guess I shouldn’t have assumed that everyone else has also read Nesov’s analyses and immediately (accurately) classified them as correct.
It’s just surprising that Sam is willing to say/confirm all of this given that AI companies normally at least try to be secretive.
He says things that are advantageous, and sometimes they are even true. The benefit of not being known to be a liar usually keeps the correlation between claims and truth positive, but in his case it seems that ship has sailed.
(Checkably false claims are still pretty rare, and this may be one of those.)
Would you update your timelines, if he is telling the truth ?