Bambi, the same line of argument would cause any observers, standing off observing the course of Earthly evolution, to conclude that these so-called hypothesized “humans” were impossible.
Actually, the same @Hanson. If there weren’t deep keys to optimization, humans would be unable to achieve a sudden huge advantage over other lifeforms, using brains optimized in the entire absence of direct selective pressures for inventing guns or space shuttles. It would just be a matter of slowly evolving swords, slowly evolving guns, slowly evolving nuclear weapons...
It seems to me that the evidence available to us, in which to fight out this disagreement, consists mainly of the course of biological evolutionary history. Where we see that modern AI researchers are stuck in very much the mode of evolution producing prehuman lifeforms: They have to do lots of work to produce one special-purpose cognitive system.
So they conclude that general-purpose systems are impossible, based on historical experience. QED. Humans are a counterexample, but the actual disappointing laboratory experience seems so much more salient than that.
Bambi, the same line of argument would cause any observers, standing off observing the course of Earthly evolution, to conclude that these so-called hypothesized “humans” were impossible.
Actually, the same @Hanson. If there weren’t deep keys to optimization, humans would be unable to achieve a sudden huge advantage over other lifeforms, using brains optimized in the entire absence of direct selective pressures for inventing guns or space shuttles. It would just be a matter of slowly evolving swords, slowly evolving guns, slowly evolving nuclear weapons...
It seems to me that the evidence available to us, in which to fight out this disagreement, consists mainly of the course of biological evolutionary history. Where we see that modern AI researchers are stuck in very much the mode of evolution producing prehuman lifeforms: They have to do lots of work to produce one special-purpose cognitive system.
So they conclude that general-purpose systems are impossible, based on historical experience. QED. Humans are a counterexample, but the actual disappointing laboratory experience seems so much more salient than that.