And in particular, there’s a bitter lesson where more compute and data have fairly consistently improved AI, while specialization has not done nearly as well. GPT-3 is another case, and the perhaps GPT-4??? may be another example of a trend towards generality.
And in particular, there’s a bitter lesson where more compute and data have fairly consistently improved AI, while specialization has not done nearly as well. GPT-3 is another case, and the perhaps GPT-4??? may be another example of a trend towards generality.