I agree I’m confused here. But it’s hard to come down to clear interpretations. I kinda think Hanson and Yudkowsky are also confused.
Like, here are some possible interpretations on this issue, and how I’d position Hanson and Yudkowsky on them based on my recollection and on vibes.
Improvements in our ways of making AI will be incremental. (Hanson pro, Yudkowsky maaaybe con, and we need some way to operationalize “incremental”, so probably just ambiguous)
Improvements in our ways of making AI will be made by lots of different people distributed over space and time. (Hanson pro, Yudkowsky maybe con, seems pretty Hanson favored)
AI in its final form will have elegant architecture (Hanson more con, Yudkowsky more pro, seems Yudkowsky favored, but I’m unhappy with what “elegant” means)
Or even
4. People know when they’re making a significant improvement to AI—the difference between “clever hack” and “deep insight” is something you see from beforehand just as much as afterwards. (Hanson vibes con, Yudkowsky vibes pro, gotta read 1000 pages of philosophy of progress before you call it, maybe depends on the technology, I tend to think people often don’t know)
Which is why this overall section is in the “hard to call” area.
I agree I’m confused here. But it’s hard to come down to clear interpretations. I kinda think Hanson and Yudkowsky are also confused.
Like, here are some possible interpretations on this issue, and how I’d position Hanson and Yudkowsky on them based on my recollection and on vibes.
Improvements in our ways of making AI will be incremental. (Hanson pro, Yudkowsky maaaybe con, and we need some way to operationalize “incremental”, so probably just ambiguous)
Improvements in our ways of making AI will be made by lots of different people distributed over space and time. (Hanson pro, Yudkowsky maybe con, seems pretty Hanson favored)
AI in its final form will have elegant architecture (Hanson more con, Yudkowsky more pro, seems Yudkowsky favored, but I’m unhappy with what “elegant” means)
Or even 4. People know when they’re making a significant improvement to AI—the difference between “clever hack” and “deep insight” is something you see from beforehand just as much as afterwards. (Hanson vibes con, Yudkowsky vibes pro, gotta read 1000 pages of philosophy of progress before you call it, maybe depends on the technology, I tend to think people often don’t know)
Which is why this overall section is in the “hard to call” area.