If we take this as the disagreement—will AI progress come from a handful of big insights, or many small ones—I think the world right looks a great deal more like Hanson’s view than Yudkowsky’s. In his interview with Lex Fridman, Sam Altman characterizes GPT-4 as improving on GPT-3 in a hundred little things rather than a few big things, and that’s… by far… my impression of current ML progress. So when I interpret their disagreement in terms of the kind of work you need to do before attaining AGI, I tend to agree that Hanson is right.
This also feels confused to me. Of course the key insight of the Transformer architecture was super simple, and as far as I can tell the primary difference between GPT-4 and GPT-3 is throwing a lot more compute at it, combined with a lot of engineering work to get it to work at larger scales and more GPUs (in a way that doesn’t substantially improve performance).
We don’t know how GPT-4 works, but I would currently bet that within 2-3 years we will see a system that gets GPT-4 performance and compute-efficiency whose source-code is extremely simple and does not require a lot of clever hacks, but whose difference from GPT-3 will be best characterized by “0 to 2 concrete insights that improved things”, since that is exactly what we’ve seen with GPT-2 and GPT-3. The first system to reach a capability threshold often has a bunch of hacks, usually stemming from a lack of polish or understanding or just bugs, which then iteratively get pared down as progress continues.
I agree I’m confused here. But it’s hard to come down to clear interpretations. I kinda think Hanson and Yudkowsky are also confused.
Like, here are some possible interpretations on this issue, and how I’d position Hanson and Yudkowsky on them based on my recollection and on vibes.
Improvements in our ways of making AI will be incremental. (Hanson pro, Yudkowsky maaaybe con, and we need some way to operationalize “incremental”, so probably just ambiguous)
Improvements in our ways of making AI will be made by lots of different people distributed over space and time. (Hanson pro, Yudkowsky maybe con, seems pretty Hanson favored)
AI in its final form will have elegant architecture (Hanson more con, Yudkowsky more pro, seems Yudkowsky favored, but I’m unhappy with what “elegant” means)
Or even
4. People know when they’re making a significant improvement to AI—the difference between “clever hack” and “deep insight” is something you see from beforehand just as much as afterwards. (Hanson vibes con, Yudkowsky vibes pro, gotta read 1000 pages of philosophy of progress before you call it, maybe depends on the technology, I tend to think people often don’t know)
Which is why this overall section is in the “hard to call” area.
This also feels confused to me. Of course the key insight of the Transformer architecture was super simple, and as far as I can tell the primary difference between GPT-4 and GPT-3 is throwing a lot more compute at it, combined with a lot of engineering work to get it to work at larger scales and more GPUs (in a way that doesn’t substantially improve performance).
We don’t know how GPT-4 works, but I would currently bet that within 2-3 years we will see a system that gets GPT-4 performance and compute-efficiency whose source-code is extremely simple and does not require a lot of clever hacks, but whose difference from GPT-3 will be best characterized by “0 to 2 concrete insights that improved things”, since that is exactly what we’ve seen with GPT-2 and GPT-3. The first system to reach a capability threshold often has a bunch of hacks, usually stemming from a lack of polish or understanding or just bugs, which then iteratively get pared down as progress continues.
I agree I’m confused here. But it’s hard to come down to clear interpretations. I kinda think Hanson and Yudkowsky are also confused.
Like, here are some possible interpretations on this issue, and how I’d position Hanson and Yudkowsky on them based on my recollection and on vibes.
Improvements in our ways of making AI will be incremental. (Hanson pro, Yudkowsky maaaybe con, and we need some way to operationalize “incremental”, so probably just ambiguous)
Improvements in our ways of making AI will be made by lots of different people distributed over space and time. (Hanson pro, Yudkowsky maybe con, seems pretty Hanson favored)
AI in its final form will have elegant architecture (Hanson more con, Yudkowsky more pro, seems Yudkowsky favored, but I’m unhappy with what “elegant” means)
Or even 4. People know when they’re making a significant improvement to AI—the difference between “clever hack” and “deep insight” is something you see from beforehand just as much as afterwards. (Hanson vibes con, Yudkowsky vibes pro, gotta read 1000 pages of philosophy of progress before you call it, maybe depends on the technology, I tend to think people often don’t know)
Which is why this overall section is in the “hard to call” area.
Here’s a market for your claim.
GPT-4 performance and compute efficiency from a simple architecture before 2026
How do I embed the market directly into the comment, instead of having a link to which people click through?
You just copy the link to the market, and if you paste it into an empty new paragraph it should automatically be replaced with an embed.