Training run size has grown much faster than the world’s total supply of AI compute. If these near-frontier experiments were truly a bottleneck on progress, AI algorithmic progress would have slowed down over the past 10 years.
I think this history is consistent with near-frontier experiments being important, and labs continuing to do a large number of such experiments as part of the process of increasing lab spending on training compute.
ie: suppose OAI now spends $100m/model instead of $1m/model. There’s no reason that they couldn’t still be spending, say, 50% of their training compute on running 500 0.1%-scale experiments.
Caveat: This is at the firm level; you could argue that fewer near-frontier experiments are being done in total across the AI ecosystem, and certainly there’s less information flow between organizations conducting these experiments.
Neat post! The Europe-Africa ratio is especially striking, and will change my mental model of colonization a fair bit.
Also of interest for thinking about colonization / imperialism is the size of Japan’s population in that last map compared to the Southeast Asian territories it conquered during WWII.
Indeed Japan in general seems to have grown in population slower than just about any other major country I look at—a measly ~70% increase from 70m to 125m over 125 years. (“Russian Empire” then was bigger than Russia today, but Wikipedia has Russia proper at 70m then vs 144m today.)
The implied super-rapid relative population growth rates in Africa & parts of Asia in the 1900s also help me understand why people got freaked out about global overpopulation in the late 1900s, and why that pop growth needed innovations like Golden Rice to sustain it.