I think this is just a 21st century version of dualism. There’s two kinds of information, meaning and meaningless data. One can only exist in humans. Why? Humans are special. Why are humans special? Because we said so.
I said nothing about humans being special. This is an argument about LLMs, not about all possible artificial intelligences. Multi-modal devices have access to the physical world in a way that LLMS do not. That changes things.
I think that, particularly in the case of massively popular systems trained with RLHF (such as ChatGPT), these systems are “embodied” in cyberspace. They certainly are grounded in what humans want and don’t want, which isn’t the same as the physical world, but it’s still a source of meaning.
Here is my problem: I actually believe those old arguments about why machines can’t (possibly) think, the arguments from intention. That belief clashes with my appraisal of the behavior I’ve seen from ChatGPT in the last two months. Damn it! It looks like a duck.
How do I reconcile the two sides of that conflict? I don’t. Rather, I decide to hold the concept of thought, and a whole mess of allied concepts, in abeyance. I’m going to toss them between phenomenological brackets.
The people who insist that what these large language models are doing might as well be the work of thousands of drunken monkeys pounding on typewriters, they have no way of accounting for the coherent structure in ChatGPT’s output. Oh, they can pounce on its many mistakes – for it makes many – and say, “See, I told you, drunken monkeys!” But the AI boosters who insist that, yes, these guys can think, we’re on the way to AGI, they can’t tell us what it’s going on either. All they can say is that the models are “opaque” – almost a term of art by now – so we don’t know what’s going on, but we’re working on it. And indeed they are.
In this context, “think” is just a label that tells us nothing about what humans are doing that machines are not. That denial does not point the way to knowledge about how to improve these systems – for they surely need improving. I conclude, then, for certain purposes, such as discriminating between human behavior and that of advanced artificial intelligence, the idea of thought has little intellectual value.
Let me be clear. I am not denying that people think; of course we do. Nor am I asserting that advanced AI’s think. They (most likely) do not. But “to think” is an informal common-sense idea. It has no technical definition.[1] We are rapidly approaching an intellectual regime where the question of whether or not machines can think – reason, perceive, learn, feel, etc. – becomes a tractable technical issue. In this regime, common sense ideas about minds and mentation are at best of limited value. At worst, they are useless.
I take that as a sign that we are dealing with something new, really new. We have sailed into those waters where “Here be dragons” is written on the charts. It is time that we acknowledge that we don’t know what we’re doing, that the old ideas aren’t working very well, and get on with the business of creating new ones. Let us learn to fly with and talk with the dragons. It is time to be wild.
Perhaps we have to retire of the concept of meaning as well. As far as I know it doesn’t have a technical definition so perhaps we shouldn’t use it in technical conversations. What does “meaning” tell you about how LLMs work? Nothing. So what conceptual work is it doing? It seems to me it functions mostly as a way of keeping score in some ill-defined race to Mount AGI. But it doesn’t help you run the race.
[1] Some years ago I constructed a definition of the informal concept, “to think,” within a cognitive network. See, William Benzon, Cognitive Networks and Literary Semantics, MLN 91: 1976, pp. 961-964. For a similar approach to the common-sense notion, see William Benzon, First Person: Neuro-Cognitive Notes on the Self in Life and in Fiction, PsyArt: A Hyperlink Journal for Psychological Study of the Arts, August 21, 2000, pp. 23-25, https://www.academia.edu/8331456/First_Person_Neuro-Cognitive_Notes_on_the_Self_in_Life_and_in_Fiction.
I think this is just a 21st century version of dualism. There’s two kinds of information, meaning and meaningless data. One can only exist in humans. Why? Humans are special. Why are humans special? Because we said so.
I said nothing about humans being special. This is an argument about LLMs, not about all possible artificial intelligences. Multi-modal devices have access to the physical world in a way that LLMS do not. That changes things.
I think that, particularly in the case of massively popular systems trained with RLHF (such as ChatGPT), these systems are “embodied” in cyberspace. They certainly are grounded in what humans want and don’t want, which isn’t the same as the physical world, but it’s still a source of meaning.
Let me quote a passage from ChatGPT intimates a tantalizing future:
Perhaps we have to retire of the concept of meaning as well. As far as I know it doesn’t have a technical definition so perhaps we shouldn’t use it in technical conversations. What does “meaning” tell you about how LLMs work? Nothing. So what conceptual work is it doing? It seems to me it functions mostly as a way of keeping score in some ill-defined race to Mount AGI. But it doesn’t help you run the race.
[1] Some years ago I constructed a definition of the informal concept, “to think,” within a cognitive network. See, William Benzon, Cognitive Networks and Literary Semantics, MLN 91: 1976, pp. 961-964. For a similar approach to the common-sense notion, see William Benzon, First Person: Neuro-Cognitive Notes on the Self in Life and in Fiction, PsyArt: A Hyperlink Journal for Psychological Study of the Arts, August 21, 2000, pp. 23-25, https://www.academia.edu/8331456/First_Person_Neuro-Cognitive_Notes_on_the_Self_in_Life_and_in_Fiction.