Perhaps, but as Eliezer has gone to some lengths to point out, the great majority of those working on AGI simply have no concept of how difficult the problem is, of the magnitude of the gulf between their knowledge and what they’d need to solve the problem.
There is an observational bias involved here. If you do look at the problem of AGI and come to understand it you realize just how difficult it is and you are likely to move to work on a less ambitious narrow-AI precursor. This leaves a much smaller remainder trying to work on AGI, including the bunch that doesn’t understand the difficulty.
I think you’re handwaving away issues that are dramatically more problematic than you give them credit for.
If you are talking about the technical issues, I think 1-100 billion and 5-20 years is a good cost estimate.
As for the danger issues, yes of course this will be the most powerful and thus most dangerous invention we ever make. The last, really.
There is an observational bias involved here. If you do look at the problem of AGI and come to understand it you realize just how difficult it is and you are likely to move to work on a less ambitious narrow-AI precursor. This leaves a much smaller remainder trying to work on AGI, including the bunch that doesn’t understand the difficulty.
If you are talking about the technical issues, I think 1-100 billion and 5-20 years is a good cost estimate.
As for the danger issues, yes of course this will be the most powerful and thus most dangerous invention we ever make. The last, really.