Maybe we won’t restart the inner algorithm from scratch every time we edit it, since it’s so expensive to do so. Instead, maybe once in a while we’ll restart the algorithm from scratch (“re-initialize to random weights” or something analogous), but most of the time, we’ll take whatever data structure holds the AI’s world-knowledge, and preserve it between one version of the inner algorithm and its successor. Doing that is perfectly fine and plausible, but again, the result doesn’t look like evolution; it looks like a hyperparameter search within a highly-constrained class of human-designed algorithms. Why? Because the world-knowledge data structure—a huge part of how an AGI works!—needs to be designed by humans and inserted into the AGI architecture in a modular way, for this approach to be possible at all.
Does it though? In “crystal nights” I described an AI-by-evolution scenario in which the ability to copy chunks of learned brain into your offspring is in the toolkit/genome for evolution to use if it wants. It sounds like you are saying this wouldn’t work, but I don’t see why.
EDIT: Also, the “Amp(GPT-7)” story seems to me to be an example of your Case 4 or Case 5 maybe, while also being an example of the evolutionary analogy being correct (see: the final step, where we evolve the chinese room bureaucracies).
Hmm, if you don’t know which bits are the learning algorithm and which are the learned content, and they’re freely intermingling, then I guess you could try randomizing different subsets of the bits in your algorithm, and see what happens, or something, and try to figure it out. This seems like a computationally-intensive and error-prone process, to me, although I suppose it’s hard to know. Also, which is which could be dynamic, and there could be bits that are not cleanly in either category. If you get it wrong, then you’re going to wind up updating the knowledge instead of the learning algorithm, or get bits of the learning algorithm that are stuck in a bad state but you’re not editing them because you think they’re knowledge. I dunno. I guess that’s not a disproof, but I’m going to stick with “unlikely”.
With enough compute, can’t rule anything out—you could do a blind search over assembly code! I tend to think that more compute-efficient paths to AGI are far likelier to happen than less compute-efficient paths to AGI, other things equal, because the less compute that’s needed, the faster you can run experiments, and the more people are able to develop and experiment with the algorithms. Maybe one giant government project can do a blind search over assembly code, but thousands of grad students and employees can run little experiments in less computationally expensive domains.
Does it though? In “crystal nights” I described an AI-by-evolution scenario in which the ability to copy chunks of learned brain into your offspring is in the toolkit/genome for evolution to use if it wants. It sounds like you are saying this wouldn’t work, but I don’t see why.
EDIT: Also, the “Amp(GPT-7)” story seems to me to be an example of your Case 4 or Case 5 maybe, while also being an example of the evolutionary analogy being correct (see: the final step, where we evolve the chinese room bureaucracies).
Hmm, if you don’t know which bits are the learning algorithm and which are the learned content, and they’re freely intermingling, then I guess you could try randomizing different subsets of the bits in your algorithm, and see what happens, or something, and try to figure it out. This seems like a computationally-intensive and error-prone process, to me, although I suppose it’s hard to know. Also, which is which could be dynamic, and there could be bits that are not cleanly in either category. If you get it wrong, then you’re going to wind up updating the knowledge instead of the learning algorithm, or get bits of the learning algorithm that are stuck in a bad state but you’re not editing them because you think they’re knowledge. I dunno. I guess that’s not a disproof, but I’m going to stick with “unlikely”.
With enough compute, can’t rule anything out—you could do a blind search over assembly code! I tend to think that more compute-efficient paths to AGI are far likelier to happen than less compute-efficient paths to AGI, other things equal, because the less compute that’s needed, the faster you can run experiments, and the more people are able to develop and experiment with the algorithms. Maybe one giant government project can do a blind search over assembly code, but thousands of grad students and employees can run little experiments in less computationally expensive domains.