My reading was less that ‘this is unlikely to be the final paradigm’ and more that ‘a paradigm shift is likely within the 30 years roughly estimated for this to be the final paradigm’, and presumably most paradigm shifts would give us more progress rather than less to catch on in the field. With no prior knowledge of what that paradigm shift would be—maybe we manage to capture the ‘soul’ of a crow through emulation and infuse it into our starting points, or something similarly odd; maybe it is simple and obvious math in retrospect.
Ok, but Eliezer is saying that BOTH that his timelines are short (significantly less than 30 years) AND that he thinks ML isn’t likely to be the final paradigm (this judging from not just this conversation, but the other, real, ones in this sequence).
ML being the final paradigm would mean it would have to get ‘to the end’ before the next paradigm; the next paradigm will probably happen before 30 years; whatever the next paradigm is will be more impressive than the ML paradigm in some way—modest or dramatic. ML paradigm is pretty impressive, already, so anything notably more impressive than getting better at it seems likely to feel like a pretty sharp climb in capability.
My reading was less that ‘this is unlikely to be the final paradigm’ and more that ‘a paradigm shift is likely within the 30 years roughly estimated for this to be the final paradigm’, and presumably most paradigm shifts would give us more progress rather than less to catch on in the field. With no prior knowledge of what that paradigm shift would be—maybe we manage to capture the ‘soul’ of a crow through emulation and infuse it into our starting points, or something similarly odd; maybe it is simple and obvious math in retrospect.
Ok, but Eliezer is saying that BOTH that his timelines are short (significantly less than 30 years) AND that he thinks ML isn’t likely to be the final paradigm (this judging from not just this conversation, but the other, real, ones in this sequence).
ML being the final paradigm would mean it would have to get ‘to the end’ before the next paradigm; the next paradigm will probably happen before 30 years; whatever the next paradigm is will be more impressive than the ML paradigm in some way—modest or dramatic. ML paradigm is pretty impressive, already, so anything notably more impressive than getting better at it seems likely to feel like a pretty sharp climb in capability.