[...] I personally wouldn’t use the word ‘sequential’ for that—I prefer a more vertical metaphor like ‘things building upon other things’—but that’s a matter of taste I guess. Anyway, whatever we want to call it, humans can reliably do a great many steps, although that process unfolds over a long period of time.
…And not just smart humans. Just getting around in the world, using tools, etc., requires giant towers of concepts relying on other previously-learned concepts.
As a clarification for anyone wondering why I didn’t use a framing more like this in the post, it’s because I think these types of reasoning (horizontal and vertical/A and C) are related in an important way, even though I agree that C might be qualitatively harder than A (hence section §3.1). Or to put it differently, if one extreme position is “we can look entirely at A to extrapolate LLM performance into the future” and the other is “A and C are so different that progress on A is basically uninteresting”, then my view is somewhere near the middle.
As a clarification for anyone wondering why I didn’t use a framing more like this in the post, it’s because I think these types of reasoning (horizontal and vertical/A and C) are related in an important way, even though I agree that C might be qualitatively harder than A (hence section §3.1). Or to put it differently, if one extreme position is “we can look entirely at A to extrapolate LLM performance into the future” and the other is “A and C are so different that progress on A is basically uninteresting”, then my view is somewhere near the middle.