Well, I was trying to argue against the “statistical parrot” idea, because I think that unfairly downplays the significance and potential of these systems. That’s part of the purpose of the “submarine” metaphor: a submarine is actually a very impressive and useful device, even if it doesn’t swim like a fish.
I agree that there is some similarity between ANNs and brains, but the differences seem pretty stark to me.
I agree that there is some similarity between ANNs and brains, but the differences seem pretty stark to me.
There are enormous differences between an AMD EPYC processor and an RTX 4090, and yet within some performance constraints they can run the same code, and there are a near infinite ways they can instantiate programs that although vastly different in encoding details ultimately are very similar.
So obviously transformer based ANNs running on GPUs are very different physical systems than bio brains, but that is mostly irrelevant. What matters is similarity of the resulting learned software—the mindware. If you train hard enough on token prediction of the internet eventually to reach very low error the ANN must learn to simulate human minds, and a sufficient simulation of a mind simply … is .. a mind.
Well, I was trying to argue against the “statistical parrot” idea, because I think that unfairly downplays the significance and potential of these systems. That’s part of the purpose of the “submarine” metaphor: a submarine is actually a very impressive and useful device, even if it doesn’t swim like a fish.
I agree that there is some similarity between ANNs and brains, but the differences seem pretty stark to me.
There are enormous differences between an AMD EPYC processor and an RTX 4090, and yet within some performance constraints they can run the same code, and there are a near infinite ways they can instantiate programs that although vastly different in encoding details ultimately are very similar.
So obviously transformer based ANNs running on GPUs are very different physical systems than bio brains, but that is mostly irrelevant. What matters is similarity of the resulting learned software—the mindware. If you train hard enough on token prediction of the internet eventually to reach very low error the ANN must learn to simulate human minds, and a sufficient simulation of a mind simply … is .. a mind.