Many models we are training currently already require orders of magnitude more data than a human sees in one lifetime.
Why I disagree: Again under the assumptions of Section 1, “many models we are training” are very different from human brain learning algorithms. Presumably human brain-like learning algorithms will have similar sample efficiency to actual human brain learning algorithms, for obvious reasons.
I updated heavily on data efficiency recently after compiling the data in this new AI timeline post. Basically it turns out that successful ANNs and BNNs follow a simple general rule where model capacity is similar to total input data capacity. I was actually surprised at how well this rule holds, across a wide variety of successful NNs. For example the adult human brain has on order 1e15 bit capacity and receives about 1e16 bits of retinal input by age 30, the Chinchilla LLM has 2e12 bit capacity vs 1e13 input bits, etc etc.
I updated heavily on data efficiency recently after compiling the data in this new AI timeline post. Basically it turns out that successful ANNs and BNNs follow a simple general rule where model capacity is similar to total input data capacity. I was actually surprised at how well this rule holds, across a wide variety of successful NNs. For example the adult human brain has on order 1e15 bit capacity and receives about 1e16 bits of retinal input by age 30, the Chinchilla LLM has 2e12 bit capacity vs 1e13 input bits, etc etc.