In contrast, Sakana’s AI scientist cost on average 15$/paper and .50$/review.
The Sakana AI stuff is basically total bogus, as I’ve pointed out on like 4 other threads (and also as Scott Alexander recently pointed out). It does not produce anything close to fully formed scientific papers. It’s output is really not better than just prompting o1 yourself. Of course, o1 and even Sonnet and GPT-4 are very impressive, but there is no update to be made after you’ve played around with that.
I agree that ML capabilities are under-elicited, but the Sakana AI stuff really is very little evidence on that, besides someone being good at marketing and setting up some scaffolding that produces fake prestige signals.
It does not produce anything close to fully formed scientific papers. It’s output is really not better than just prompting o1 yourself. Of course, o1 and even Sonnet and GPT-4 are very impressive, but there is no update to be made after you’ve played around with that.
(Again) I think this is missing the point that we’ve now (for the first time, to my knowledge) observed an early demo the full research workflow being automatable, as flawed as the outputs might be.
The Sakana AI stuff is basically total bogus, as I’ve pointed out on like 4 other threads (and also as Scott Alexander recently pointed out). It does not produce anything close to fully formed scientific papers. It’s output is really not better than just prompting o1 yourself. Of course, o1 and even Sonnet and GPT-4 are very impressive, but there is no update to be made after you’ve played around with that.
I agree that ML capabilities are under-elicited, but the Sakana AI stuff really is very little evidence on that, besides someone being good at marketing and setting up some scaffolding that produces fake prestige signals.
(Again) I think this is missing the point that we’ve now (for the first time, to my knowledge) observed an early demo the full research workflow being automatable, as flawed as the outputs might be.