Thanks for putting in the effort of writing this up.
Would you mind expanding on D:Prodigy? My impression is that most highly intelligent adults were impressive as children, but are more capable as adults than they were as children.
The phenomenon of child prodigies is indeed a real thing that exists. My impression of why that happens is that child and adult intellectual performance are not perfectly correlated, and thus the tails come apart. But I could be wrong about that, so if you have supporting material to that effect I’d be interested.
(as a note, I do agree that self-improvement is possible, but I think the shape of the curve is very important)
The point there was just that we don’t see an inverse relationship, with smarter humans having slower development during childhood. Yet, we do see that inverse relationship when we compare humans to other animals.
Regarding the other half of D:prodigy...I was making an empirical argument based on a large volume of literature, but if you consider the energy landscape of ANN systems, plateauing at bad performance means getting stuck in bad minima, and increasing the number and quality of good paths through the energy landscape is both what makes that less likely and what increases the speed of gradient descent.
As I noted, this raises the question of what’s different about human childhood development that requires it to be slow.
Its just biology so it isn’t applicable to AI. “Neoteny” is you want to dig deeper , to have a baby born with above 50% adult brain size would require another three months in the womb and the birthing of such a cranium would be pretty deadly to the mother.
Humans also have a few notable “pruning” episodes through childhood which correlate and are hypothesized to be involved with both autism spectrum disorder and schizophrenia , pruning that also has no logical bearing on how an LLM / ASI might develop.
Thanks for putting in the effort of writing this up.
Would you mind expanding on D:Prodigy? My impression is that most highly intelligent adults were impressive as children, but are more capable as adults than they were as children.
The phenomenon of child prodigies is indeed a real thing that exists. My impression of why that happens is that child and adult intellectual performance are not perfectly correlated, and thus the tails come apart. But I could be wrong about that, so if you have supporting material to that effect I’d be interested.
(as a note, I do agree that self-improvement is possible, but I think the shape of the curve is very important)
The point there was just that we don’t see an inverse relationship, with smarter humans having slower development during childhood. Yet, we do see that inverse relationship when we compare humans to other animals.
Regarding the other half of D:prodigy...I was making an empirical argument based on a large volume of literature, but if you consider the energy landscape of ANN systems, plateauing at bad performance means getting stuck in bad minima, and increasing the number and quality of good paths through the energy landscape is both what makes that less likely and what increases the speed of gradient descent.
As I noted, this raises the question of what’s different about human childhood development that requires it to be slow.
Its just biology so it isn’t applicable to AI. “Neoteny” is you want to dig deeper , to have a baby born with above 50% adult brain size would require another three months in the womb and the birthing of such a cranium would be pretty deadly to the mother.
Humans also have a few notable “pruning” episodes through childhood which correlate and are hypothesized to be involved with both autism spectrum disorder and schizophrenia , pruning that also has no logical bearing on how an LLM / ASI might develop.