As far as I can tell Strawberry is proving me right: it’s going beyond pre-training and scales inference—the obvious next step.
A lot of people said just scaling pre-trained transformers would scale to AGI. I think that’s silly and doesn’t make sense. But now you don’t have to believe me—you can just use OpenAIs latest model.
The next step is to do efficient long-horizon RL for data-sparse domains.
Strawberry working suggest that this might not be so hard. Don’t be fooled by the modest gains of Strawberry so far. This is a new paradigm that is heading us toward true AGI and superintelligence.
As far as I can tell Strawberry is proving me right: it’s going beyond pre-training and scales inference—the obvious next step.
A lot of people said just scaling pre-trained transformers would scale to AGI. I think that’s silly and doesn’t make sense. But now you don’t have to believe me—you can just use OpenAIs latest model.
The next step is to do efficient long-horizon RL for data-sparse domains.
Strawberry working suggest that this might not be so hard. Don’t be fooled by the modest gains of Strawberry so far. This is a new paradigm that is heading us toward true AGI and superintelligence.