There’s a billion seconds in 30 years. Chinchilla was trained on 1.4 trillion tokens. So for a human adult to have as much data as chinchilla would require us to process the equivalent of ~1400 tokens per second. I think that’s something like 2 kilobyte per second.
Inputs to the human brain are probably dominated by vision. I’m not sure how many bytes per second we see, but I don’t think it’s many orders of magnitudes higher than 2kb.
(If 1 firing = 1 bit, that should be 34 megabit ~= 4 megabyte.)
This random article (which I haven’t fact-checked in the least) claims a bandwidth of 8.75 megabit ~= 1 megabyte. So that’s like 2.5 OOMs higher than the number I claimed for chinchilla. So yeah, it does seem like humans get more raw data.
(But I still suspect that chinchilla gets more data if you adjust for (un)interestingness. Where totally random data and easily predictable/compressible data are interesting, and data that is hard-but-possible to predict/compress is interesting.)
There’s a billion seconds in 30 years. Chinchilla was trained on 1.4 trillion tokens. So for a human adult to have as much data as chinchilla would require us to process the equivalent of ~1400 tokens per second. I think that’s something like 2 kilobyte per second.
Inputs to the human brain are probably dominated by vision. I’m not sure how many bytes per second we see, but I don’t think it’s many orders of magnitudes higher than 2kb.
That depends a lot on how you count. A quick Googling suggest that the optic nerve has 1.7 million nerve fibers.
If you think about a neuron firing rate of 20 hz that gives you 34 MB per second.
(If 1 firing = 1 bit, that should be 34 megabit ~= 4 megabyte.)
This random article (which I haven’t fact-checked in the least) claims a bandwidth of 8.75 megabit ~= 1 megabyte. So that’s like 2.5 OOMs higher than the number I claimed for chinchilla. So yeah, it does seem like humans get more raw data.
(But I still suspect that chinchilla gets more data if you adjust for (un)interestingness. Where totally random data and easily predictable/compressible data are interesting, and data that is hard-but-possible to predict/compress is interesting.)