I’m reading through and catching up on this thread, and rather strongly agreed with your statement:
Eliezer and others at SIAI to assign (relatively) large amounts of probability mass to the scenario of a small set of people having some “insight” which allows them to suddenly invent AGI in a basement. In other words, they tend to view AGI as something like an unsolved math problem, like those on the Clay Millennium list, whereas it seems to me like a daunting engineering task analogous to colonizing Mars (or maybe Pluto).
However, pondering it again, I realize there is an epistemological spectrum ranging from math on the one side to engineering on the other. Key insights into new algorithms can undoubtedly speed up progress, and such new insights often can be expressed as pure math, but at the end of the day it is a grand engineering (or reverse engineering) challenge.
However, I’m somewhat taken aback when you say, “the notion that AGI is only decades away, as opposed to a century or two.”
I’m reading through and catching up on this thread, and rather strongly agreed with your statement:
However, pondering it again, I realize there is an epistemological spectrum ranging from math on the one side to engineering on the other. Key insights into new algorithms can undoubtedly speed up progress, and such new insights often can be expressed as pure math, but at the end of the day it is a grand engineering (or reverse engineering) challenge.
However, I’m somewhat taken aback when you say, “the notion that AGI is only decades away, as opposed to a century or two.”
A century or two?