The human brain is complex, but a lot of this is learned complexity, it’s not there in the genome.
The genome might still be spending tens or hundreds of kilobytes on preprogramming circuits in the cerebellum and brainstem (babies have to breathe from minute 0, that can’t be learned behavior.), but the majority of the complexity in the (human) brain is learned complexity. We can do calculus not because our genomes programmed calculus in directly, but because humans as they grow up are able to learn about the world in a very general way, and this incidentally teaches us to manipulate complex ideas. (Also see this post and its sequels that elaborates on this idea in more detail.)
Modern AI is sort of taking this to its logical endpoint. If an AI doesn’t need to breathe, or regulate its body temperature, or drink milk, or cry to wake up its parents from minute 0, then it can drop a lot of those preprogrammed behaviors and really just double down on learning about the world in a general way. It can have a short “genome” that tells it how to develop (even though the eventual “organism”—the trained AI—will be many many times the size of the “genome”).
Like, for example, it makes sense that a future LLM would be able to explain a mathematical concept that has been documented and previously discussed but I just can’t see it solving existing frontier problems in mathematical theory
You might be focusing too much on current experience with LLMs because LLMs are the new hotness. Present-day LLMs are learning systems that have only ever been trained on text in small snippets with no long-term pattern. They are a hundred times smaller than the human brain (if you’re very handwavy about estimating the “number of parameters” of a human brain). It’s an impressive emergent property that they can produce long-term coherent text at all.
If you took the same general learning algorithm, but trained it from the start to produce long-term coherent text in interaction with an environment, and made it 100x larger, I don’t think it’s unreasonable to think it might start learning to prove theorems.
The human brain is complex, but a lot of this is learned complexity, it’s not there in the genome.
The genome might still be spending tens or hundreds of kilobytes on preprogramming circuits in the cerebellum and brainstem (babies have to breathe from minute 0, that can’t be learned behavior.), but the majority of the complexity in the (human) brain is learned complexity. We can do calculus not because our genomes programmed calculus in directly, but because humans as they grow up are able to learn about the world in a very general way, and this incidentally teaches us to manipulate complex ideas. (Also see this post and its sequels that elaborates on this idea in more detail.)
Modern AI is sort of taking this to its logical endpoint. If an AI doesn’t need to breathe, or regulate its body temperature, or drink milk, or cry to wake up its parents from minute 0, then it can drop a lot of those preprogrammed behaviors and really just double down on learning about the world in a general way. It can have a short “genome” that tells it how to develop (even though the eventual “organism”—the trained AI—will be many many times the size of the “genome”).
You might be focusing too much on current experience with LLMs because LLMs are the new hotness. Present-day LLMs are learning systems that have only ever been trained on text in small snippets with no long-term pattern. They are a hundred times smaller than the human brain (if you’re very handwavy about estimating the “number of parameters” of a human brain). It’s an impressive emergent property that they can produce long-term coherent text at all.
If you took the same general learning algorithm, but trained it from the start to produce long-term coherent text in interaction with an environment, and made it 100x larger, I don’t think it’s unreasonable to think it might start learning to prove theorems.