From a high-level perspective, it is clear that this is just wrong. Part of what human brains are doing is to minimise prediction error with regard to sensory inputs. Unbounded version of the task is basically of same generality and difficulty as what GPT is doing, and is roughly equivalent to understand everything what is understandable in the observable universe.
Yes, human brains can be regarded as trying to solve the problem of minimizing prediction error given their own sensory inputs, but no one is trying to push up the capabilities of an individual human brain as fast as possible to make it better at actually doing so. Lots of people are definitely trying this for GPTs, measuring their progress on harder and harder tasks as they do so, some of which humans already cannot do on their own.
Or, another way of putting it: during training, a GPT is asked to solve a concrete problem no human is capable of or expected to solve. When GPT fails to make an accurate prediction, it gets modified into something that might do better next time. No one performs brain surgery on a human any time they make a prediction error.
Upon opening your eyes, your visual cortex is asked to solve a concrete problem no brain is capable or expected to solve perfectly: predict sensory inputs. When the patterns of firing don’t predict the photoreceptor activations, your brain gets modified into something else, which may do better next time. Every time your brain fails to predict it’s visual field, there is a bit of modification, based on computing what’s locally a good update.
There is no fundamental difference in the nature of the task.
Where the actual difference is are the computational and architectural bounds of the systems.
The smartness of neither humans nor GPTs is bottlenecked by the difficulty of the task, and you can not say how smart the systems are by looking at the problems. To illustrate that fallacy with a very concrete example:
Please do this task: prove P ≠ NP in next 5 minutes. You will get $1M if you do.
Done?
Do you think you have become much smarter mind because of that? I doubt do—but you were given a very hard task, and a high reward.
The actual strategic difference and what’s scary isn’t the difficulty of the task, but the fact human brain’s don’t multiple their size every few months.
Do you think you have become much smarter mind because of that? I doubt do—but you were given a very hard task, and a high reward.
No, but I was able to predict my own sensory input pretty well, for those 5 minutes. (I was sitting in a quiet room, mostly pondering how I would respond to this comment, rather than the actual problem you posed. When I closed my eyes, the sensory prediction problem got even easier.)
You could probably also train a GPT on sensory inputs (suitably encoded) instead of text, and get pretty good predictions about future sensory inputs.
Stepping back, the fact that you can draw a high-level analogy between neuroplasticity in human brains ⇔ SGD in transformer networks, and sensory input prediction ⇔ next token prediction doesn’t mean you can declare there is “no fundamental difference” in the nature of these things, even if you are careful to avoid the type error in your last example.
In the limit (maybe) a sufficiently good predictor could perfectly predict both sensory input and tokens, but the point is that the analogy breaks down in the ordinary, limited case, on the kinds of concrete tasks that GPTs and humans are being asked to solve today. There are plenty of text manipulation and summarization problems that GPT-4 is already superhuman at, and SGD can already re-weight a transformer network much more than neuroplasticity can reshape a human brain.
Yes, human brains can be regarded as trying to solve the problem of minimizing prediction error given their own sensory inputs, but no one is trying to push up the capabilities of an individual human brain as fast as possible to make it better at actually doing so. Lots of people are definitely trying this for GPTs, measuring their progress on harder and harder tasks as they do so, some of which humans already cannot do on their own.
Or, another way of putting it: during training, a GPT is asked to solve a concrete problem no human is capable of or expected to solve. When GPT fails to make an accurate prediction, it gets modified into something that might do better next time. No one performs brain surgery on a human any time they make a prediction error.
This seems the same confusion again.
Upon opening your eyes, your visual cortex is asked to solve a concrete problem no brain is capable or expected to solve perfectly: predict sensory inputs. When the patterns of firing don’t predict the photoreceptor activations, your brain gets modified into something else, which may do better next time. Every time your brain fails to predict it’s visual field, there is a bit of modification, based on computing what’s locally a good update.
There is no fundamental difference in the nature of the task.
Where the actual difference is are the computational and architectural bounds of the systems.
The smartness of neither humans nor GPTs is bottlenecked by the difficulty of the task, and you can not say how smart the systems are by looking at the problems. To illustrate that fallacy with a very concrete example:
Please do this task: prove P ≠ NP in next 5 minutes. You will get $1M if you do.
Done?
Do you think you have become much smarter mind because of that? I doubt do—but you were given a very hard task, and a high reward.
The actual strategic difference and what’s scary isn’t the difficulty of the task, but the fact human brain’s don’t multiple their size every few months.
(edited for clarity)
No, but I was able to predict my own sensory input pretty well, for those 5 minutes. (I was sitting in a quiet room, mostly pondering how I would respond to this comment, rather than the actual problem you posed. When I closed my eyes, the sensory prediction problem got even easier.)
You could probably also train a GPT on sensory inputs (suitably encoded) instead of text, and get pretty good predictions about future sensory inputs.
Stepping back, the fact that you can draw a high-level analogy between neuroplasticity in human brains ⇔ SGD in transformer networks, and sensory input prediction ⇔ next token prediction doesn’t mean you can declare there is “no fundamental difference” in the nature of these things, even if you are careful to avoid the type error in your last example.
In the limit (maybe) a sufficiently good predictor could perfectly predict both sensory input and tokens, but the point is that the analogy breaks down in the ordinary, limited case, on the kinds of concrete tasks that GPTs and humans are being asked to solve today. There are plenty of text manipulation and summarization problems that GPT-4 is already superhuman at, and SGD can already re-weight a transformer network much more than neuroplasticity can reshape a human brain.