The space of behaviors of Turing Machines is not compressible, sub spaces are, but not the whole lot. What space do you expect the SeedAIs to be searching? If you could show that it is compressible and bound to have an uncountable number of better versions of the SeedAI, then you could convince me that I should worry about Fooming.
As such when I think of self-modifiers I think of them searching the space of Turing Machines, which just seems hard.
The space of possible gene combinations is not compressible—under the evolutionary mechanism.
The space of behaviours of Turing machines is not compressible, in the terms in which that compression has been envisaged.
The mechanism that compresses search space that Believer posits is something to do with brains; something to do with intelligence. And it works—we know it does; Kekule works on the structure of benzene without success; sleeps, dreams of a serpent biting its own tail, and waking, conceives of the benzene ring.
The mechanism (and everyone here believes that it is a mechanism) is currently mysterious. AI must possess this mechanism, or it will not be AI.
The space of behaviors of Turing Machines is not compressible, sub spaces are, but not the whole lot. What space do you expect the SeedAIs to be searching? If you could show that it is compressible and bound to have an uncountable number of better versions of the SeedAI, then you could convince me that I should worry about Fooming.
As such when I think of self-modifiers I think of them searching the space of Turing Machines, which just seems hard.
The space of possible gene combinations is not compressible—under the evolutionary mechanism.
The space of behaviours of Turing machines is not compressible, in the terms in which that compression has been envisaged.
The mechanism that compresses search space that Believer posits is something to do with brains; something to do with intelligence. And it works—we know it does; Kekule works on the structure of benzene without success; sleeps, dreams of a serpent biting its own tail, and waking, conceives of the benzene ring.
The mechanism (and everyone here believes that it is a mechanism) is currently mysterious. AI must possess this mechanism, or it will not be AI.