I don’t understand how OpenAIs success at scaling GPT proves the universal learning model.
If I have a model which predicts “this simple architecture scales up to human intelligence with enough compute”, and that is tested and indeed shown to be correct, then the model is validated.
And it helps further rule out an entire space of theories about intelligence: namely all the theories that intelligence is very complicated and requires many complex interacting innate algorithms (evolved modularity, which EY seemed to subscribe to )
Couldn’t there be an as yet undiscovered algorithm for intelligence that is more efficient?
Sure there could be other algorithms for intelligence that are more efficient, and I already said I don’t think we are on quite on the final scaling curve with transfomers. But over time the probability mass remaining for these undiscovered algorithms continually diminishes as we explore ever more of the algorithmic search space.
Furthemore, evolution extensively explored the search space for architectures/algorithms for intelligent agents, and essentially found common variants of universal learning on NNs in multiple unrelated lineages, substantially adding to the evidence that yes this really is as good as it gets (at least for any near term conventional computers).
If I have a model which predicts “this simple architecture scales up to human intelligence with enough compute”, and that is tested and indeed shown to be correct, then the model is validated.
And it helps further rule out an entire space of theories about intelligence: namely all the theories that intelligence is very complicated and requires many complex interacting innate algorithms (evolved modularity, which EY seemed to subscribe to )
Sure there could be other algorithms for intelligence that are more efficient, and I already said I don’t think we are on quite on the final scaling curve with transfomers. But over time the probability mass remaining for these undiscovered algorithms continually diminishes as we explore ever more of the algorithmic search space.
Furthemore, evolution extensively explored the search space for architectures/algorithms for intelligent agents, and essentially found common variants of universal learning on NNs in multiple unrelated lineages, substantially adding to the evidence that yes this really is as good as it gets (at least for any near term conventional computers).
I see, thanks for clarifying.