You must be misunderstanding me, because what you just said seems like a total non-sequitur. What does pockets and deep conviction have to do with general intelligence indeed?
Starting with a rhetorical question was probably a bad idea. Let me try again:
But an AGI wouldn’t be an AGI if it wasn’t able to figure out how to solve the problem of getting from here to there and using in-situ resources to replicate itself.
I don’t think this is true. An AGI which—due to practical limitations—cannot eat its future light cone can still be generally intelligent. Humans are potentially an example (minus the “artificial”, but that isn’t the relevant part).
Claiming that general intelligence can eat the universe by definition seems to suffer the same problem as the Socrates/hemlock question. It would mean we can’t call something generally intelligent until we see it able to eat the universe, which would require not just theoretical knowledge but also the resources to pull it off. And if that’s the requirement, then human general intelligence is an open question and we’d have zero known examples of general intelligence.
This does not seem to fit how we’d like to use the term to point to “the kind of problem-solving humans can do”, so I think it’s a bad definition.
(AGI can probably eat the universe, but that’s more like a theorem than a definition.)
Ok there was an unstated assumption, but that assumption was that the AGI has physical effectors. Those effectors could be nearly anything, since with enough planning and time nearly any physical effector could be used to bootstrap your way to any other capability.
So many posts back I was asserting that even un augmented human beings have the capability to eat our collective future light cone. It’s a monumental project, yes, with probably hundreds of years before the first starships leave. But once they do our future descendents would be expanding into the cosmos at a fairly large fraction of the speed of light. I’ll point you to the various studies done by the British Interplanetary Society and others on interstellar colonization if you don’t want to take my word for it.
So if regular old homo sapiens can do it, but an AGI with physical effectors can’t, then I seriously question how general that intelligence is.
And remember that getting to this level of industrial capacity on earth followed from millions of years of biological evolution and thousands of years of cultural evolution in Earth’s biosphere. Why would one ship full of humans be able to replicate that success somewhere else?
Similarly, an AGI that can replicate itself with an industrial base at its disposal might not be able to when isolated from those resource (it’s still an AGI).
You must be misunderstanding me, because what you just said seems like a total non-sequitur. What does pockets and deep conviction have to do with general intelligence indeed?
Starting with a rhetorical question was probably a bad idea. Let me try again:
I don’t think this is true. An AGI which—due to practical limitations—cannot eat its future light cone can still be generally intelligent. Humans are potentially an example (minus the “artificial”, but that isn’t the relevant part).
Claiming that general intelligence can eat the universe by definition seems to suffer the same problem as the Socrates/hemlock question. It would mean we can’t call something generally intelligent until we see it able to eat the universe, which would require not just theoretical knowledge but also the resources to pull it off. And if that’s the requirement, then human general intelligence is an open question and we’d have zero known examples of general intelligence.
This does not seem to fit how we’d like to use the term to point to “the kind of problem-solving humans can do”, so I think it’s a bad definition.
(AGI can probably eat the universe, but that’s more like a theorem than a definition.)
Ok there was an unstated assumption, but that assumption was that the AGI has physical effectors. Those effectors could be nearly anything, since with enough planning and time nearly any physical effector could be used to bootstrap your way to any other capability.
So many posts back I was asserting that even un augmented human beings have the capability to eat our collective future light cone. It’s a monumental project, yes, with probably hundreds of years before the first starships leave. But once they do our future descendents would be expanding into the cosmos at a fairly large fraction of the speed of light. I’ll point you to the various studies done by the British Interplanetary Society and others on interstellar colonization if you don’t want to take my word for it.
So if regular old homo sapiens can do it, but an AGI with physical effectors can’t, then I seriously question how general that intelligence is.
Yes; please provide those links.
And remember that getting to this level of industrial capacity on earth followed from millions of years of biological evolution and thousands of years of cultural evolution in Earth’s biosphere. Why would one ship full of humans be able to replicate that success somewhere else?
Similarly, an AGI that can replicate itself with an industrial base at its disposal might not be able to when isolated from those resource (it’s still an AGI).