That seems a bit uncharitable to me. I doubt he rejects those heuristics wholesale. I’d guess that he thinks that e.g. recursive self improvement is one of those things where these heuristics don’t apply, and that this is foreseeable because of e.g. the nature of recursion. I’d love to hear more about what sort of knowledge about “operating these heuristics” you think he’s missing!
Anyway, it seems like he expects things to seem more-or-less gradual up until FOOM, so I think my original point still applies: I think his model would not be “shaken out” of his fast-takeoff view due to successful future predictions (until it’s too late).
He says things like AlphaGo or GPT-3 being really surprising to gradualists, suggesting he thinks that gradualism only works in hindsight.
I agree that after shaking out the other disagreements, we could just end up with Eliezer saying “yeah but automating AI R&D is just fundamentally unlike all the other tasks to which we’ve applied AI” (or “AI improving AI will be fundamentally unlike automating humans improving AI”) but I don’t think that’s the core of his position right now.
That seems a bit uncharitable to me. I doubt he rejects those heuristics wholesale. I’d guess that he thinks that e.g. recursive self improvement is one of those things where these heuristics don’t apply, and that this is foreseeable because of e.g. the nature of recursion. I’d love to hear more about what sort of knowledge about “operating these heuristics” you think he’s missing!
Anyway, it seems like he expects things to seem more-or-less gradual up until FOOM, so I think my original point still applies: I think his model would not be “shaken out” of his fast-takeoff view due to successful future predictions (until it’s too late).
He says things like AlphaGo or GPT-3 being really surprising to gradualists, suggesting he thinks that gradualism only works in hindsight.
I agree that after shaking out the other disagreements, we could just end up with Eliezer saying “yeah but automating AI R&D is just fundamentally unlike all the other tasks to which we’ve applied AI” (or “AI improving AI will be fundamentally unlike automating humans improving AI”) but I don’t think that’s the core of his position right now.