It is a curious fact that child prodigies usually don’t end up contributing much to science or overall culture
Few people end up “contributing much to science or overall culture”. It’s like being surprised that most people who do sport regularly do not end up winning the Olympic Games.
The Myth of General Intelligence
Albert Einstein’s brain just happened to be the one in which these ideas recombined and mutated first to form “his” theories.
You don’t specify what exactly is the “myth” and what exactly is your alternative explanation. If your point is that Einstein could not have discovered general relativity if he was born 1000 years earlier, I agree. If your point is that any other person with university education living in the same era could have discovered the same, I disagree.
Knowledge is an inherent part of what we typically mean by intelligence
Again, what exactly do you mean? Yes, intelligence is about processing knowledge. No, you do not need any specific knowledge in order to be intelligent.
virtually everything you and I can do that we consider uniquely human are things we’ve copied from somewhere else, including “simple” things like counting or percentages. And, virtually none of the new things you and I do are improvements.
True. But it is also true that some people are way better at copying and repurposing these things than others.
In reality, before one succeeds, one must with overwhelming likelihood fail. Intelligence cannot be divorced from the reality of data collection through trial and error.
Nope. Depends on what kind of task you have in mind, and what kind of learning is available. Also, intelligence is related to how fast one learns.
Thank you for your response, I will try to address your comments.
Few people end up “contributing much to science or overall culture”. It’s like being surprised that most people who do sport regularly do not end up winning the Olympic Games.
Well, people often extrapolate that if a child prodigy, whether in sports or an intellectual pursuit, does much better than other children of the same age, that this difference will persist into adulthood. What I’m saying is that the reason it usually doesn’t, is that once you’ve absorbed existing techniques, you reach a plateau that’s extremely hard to break out of and even if you do, it’s only by a small amount and largely based on luck. And it doesn’t matter much if one reaches that plateau at age 15 (like, say, a child prodigy) or 25 (like, say, a “normal” person).
You don’t specify what exactly is the “myth” and what exactly is your alternative explanation.
The myth of general intelligence is that it is somehow different from “regular” intelligence. A human brain is not more general than any other primate brain. It’s actually the training methods that are different. There is nothing inherent in the model that is the human brain which makes it inherently more capable, à la Chomsky’s universal grammar. The only effective difference is that the human mind has a strong tendency towards imitation, which does not in itself make it more intelligent, but only if there are intelligent behaviors available to imitate, so what is in effect different about what’s called a “general” intelligence is not the model or agent itself, but the training method. There is nothing in principle that stops a chimpanzee from being able to read and write English, for example. It’s just that we haven’t figured out the methods to configure their brains into that state, because they don’t have a strong tendency to imitate, which human children do have, which makes training them much easier.
If your point is that Einstein could not have discovered general relativity if he was born 1000 years earlier, I agree. If your point is that any other person with university education living in the same era could have discovered the same, I disagree.
We agree on the first point. As for the second point, in hindsight we of course know that Einstein was able to discover what he did discover. However, before his discovery, it was not known what kind of person would be required. We did not even know exactly what was out there to discover. We are in that situation today with respect to discoveries we haven’t discovered yet. We don’t know what kind of person, with what kind of brain, in what configuration, will make what discoveries, so there is an element of chance. If in Einstein’s time the chips had fallen slightly differently, I don’t see why some other person couldn’t have made the same or very similar discoveries. It could have happened five years earlier or five years later, but it seems extremely unlikely to me that if Einstein had died as a baby that we would still be stuck with Newtonian mechanics today.
No, you do not need any specific knowledge in order to be intelligent.
Well, this comes down to what is a useful definition of intelligence. You indeed don’t need any specific knowledge if you define intelligence as something like IQ, but even a feral child with an IQ of 200 won’t outmatch a chimpanzee in any meaningful way; it won’t invent Hindu-Arabic numerals, language or even a hand axe. Likewise, ChatGPT is a useless pile of numbers before it sees the training data, and afterwards its behavior depends on what was in that training data. So in practice I would argue that to be intelligent in a specific domain you do need specific knowledge, whether that’s factual knowledge, or more implied knowledge that is absorbed by osmosis or practice.
But it is also true that some people are way better at copying and repurposing these things than others.
Some people are better at that, but I wouldn’t say way better. John von Neumann was perhaps at the apex of this, but that didn’t make him much more powerful in practice. He didn’t discover the Higgs boson. He didn’t build a gigahertz microprocessor. He didn’t cure his own cancer. He didn’t even put wheels under his suitcase. It’s impressive what he did do, but still pretty incremental. Why would an AI be much better at this?
Nope. Depends on what kind of task you have in mind, and what kind of learning is available.
What kind of practical task wouldn’t require trial and error? There are some tasks, such as predicting the motions of the planets, for which there turned out to be a method which works quite generally, but even then when you drop certain assumptions, the method becomes intensive or impossible to calculate.
Also, intelligence is related to how fast one learns.
Yes, a bigger model or a higher IQ can help you get more out of a given amount of data, but the scaling laws show diminishing returns. Pretty quickly, having more data starts to outweigh trying to squeeze more out of what you’ve already got.
There is nothing in principle that stops a chimpanzee from being able to read and write English, for example. It’s just that we haven’t figured out the methods to configure their brains into that state
Do you think there’s an upper bound on how well a chimp could read and write?
I’d say there virtually must be an upper bound. As to where this upper bound is, you could do the following back-of-the-napkin calculation.
ChatGPT is pretty good at reading and writing and it has something on the order of 100 billion to 1,000 billion parameters, one for each artificial synapse. A human brain has on the order of 100,000 billion natural synapses and a chimpanzee brain has about a third of that. If we could roughly equate artificial and natural synapses, it seems that a chimpanzee brain should in principle be able to model reading and writing as well as ChatGPT and then some. But then you’d have to devise a training method to set the strengths of natural synapses as desired.
Few people end up “contributing much to science or overall culture”. It’s like being surprised that most people who do sport regularly do not end up winning the Olympic Games.
You don’t specify what exactly is the “myth” and what exactly is your alternative explanation. If your point is that Einstein could not have discovered general relativity if he was born 1000 years earlier, I agree. If your point is that any other person with university education living in the same era could have discovered the same, I disagree.
Again, what exactly do you mean? Yes, intelligence is about processing knowledge. No, you do not need any specific knowledge in order to be intelligent.
True. But it is also true that some people are way better at copying and repurposing these things than others.
Nope. Depends on what kind of task you have in mind, and what kind of learning is available. Also, intelligence is related to how fast one learns.
Thank you for your response, I will try to address your comments.
Well, people often extrapolate that if a child prodigy, whether in sports or an intellectual pursuit, does much better than other children of the same age, that this difference will persist into adulthood. What I’m saying is that the reason it usually doesn’t, is that once you’ve absorbed existing techniques, you reach a plateau that’s extremely hard to break out of and even if you do, it’s only by a small amount and largely based on luck. And it doesn’t matter much if one reaches that plateau at age 15 (like, say, a child prodigy) or 25 (like, say, a “normal” person).
The myth of general intelligence is that it is somehow different from “regular” intelligence. A human brain is not more general than any other primate brain. It’s actually the training methods that are different. There is nothing inherent in the model that is the human brain which makes it inherently more capable, à la Chomsky’s universal grammar. The only effective difference is that the human mind has a strong tendency towards imitation, which does not in itself make it more intelligent, but only if there are intelligent behaviors available to imitate, so what is in effect different about what’s called a “general” intelligence is not the model or agent itself, but the training method. There is nothing in principle that stops a chimpanzee from being able to read and write English, for example. It’s just that we haven’t figured out the methods to configure their brains into that state, because they don’t have a strong tendency to imitate, which human children do have, which makes training them much easier.
We agree on the first point. As for the second point, in hindsight we of course know that Einstein was able to discover what he did discover. However, before his discovery, it was not known what kind of person would be required. We did not even know exactly what was out there to discover. We are in that situation today with respect to discoveries we haven’t discovered yet. We don’t know what kind of person, with what kind of brain, in what configuration, will make what discoveries, so there is an element of chance. If in Einstein’s time the chips had fallen slightly differently, I don’t see why some other person couldn’t have made the same or very similar discoveries. It could have happened five years earlier or five years later, but it seems extremely unlikely to me that if Einstein had died as a baby that we would still be stuck with Newtonian mechanics today.
Well, this comes down to what is a useful definition of intelligence. You indeed don’t need any specific knowledge if you define intelligence as something like IQ, but even a feral child with an IQ of 200 won’t outmatch a chimpanzee in any meaningful way; it won’t invent Hindu-Arabic numerals, language or even a hand axe. Likewise, ChatGPT is a useless pile of numbers before it sees the training data, and afterwards its behavior depends on what was in that training data. So in practice I would argue that to be intelligent in a specific domain you do need specific knowledge, whether that’s factual knowledge, or more implied knowledge that is absorbed by osmosis or practice.
Some people are better at that, but I wouldn’t say way better. John von Neumann was perhaps at the apex of this, but that didn’t make him much more powerful in practice. He didn’t discover the Higgs boson. He didn’t build a gigahertz microprocessor. He didn’t cure his own cancer. He didn’t even put wheels under his suitcase. It’s impressive what he did do, but still pretty incremental. Why would an AI be much better at this?
What kind of practical task wouldn’t require trial and error? There are some tasks, such as predicting the motions of the planets, for which there turned out to be a method which works quite generally, but even then when you drop certain assumptions, the method becomes intensive or impossible to calculate.
Yes, a bigger model or a higher IQ can help you get more out of a given amount of data, but the scaling laws show diminishing returns. Pretty quickly, having more data starts to outweigh trying to squeeze more out of what you’ve already got.
Do you think there’s an upper bound on how well a chimp could read and write?
I’d say there virtually must be an upper bound. As to where this upper bound is, you could do the following back-of-the-napkin calculation.
ChatGPT is pretty good at reading and writing and it has something on the order of 100 billion to 1,000 billion parameters, one for each artificial synapse. A human brain has on the order of 100,000 billion natural synapses and a chimpanzee brain has about a third of that. If we could roughly equate artificial and natural synapses, it seems that a chimpanzee brain should in principle be able to model reading and writing as well as ChatGPT and then some. But then you’d have to devise a training method to set the strengths of natural synapses as desired.