most definitions of consciousness indicate that—if a being has independent cognition (i.e. a stream of consciousness)--then the being is conscious.
I don’t think that’s true. For instance, none of the definitions given in the LW wiki give that definition. And the whole argument rests on that claim—which rests on the meaning of “independent”. What is “independent”, anyway?
I’ll go over my best understanding of the words “consciousness”, “independent”, and “cognition” here.
Consciousness
I realize the concept of consciousness is extremely complex, and Wikipedia might be considered bottom-feeding. However, we have to start somewhere, and I don’t want to make it seem like I’m cherry-picking obscure or unaccepted definitions to suit my purposes.
Here is the first paragraph of Wikipedia’s explanation of consciousness:
“Consciousness, at its simplest, is awareness of internal and external existence.[1] However, its nature has led to millennia of analyses, explanations and debate by philosophers, theologians, and all of science. Opinions differ about what exactly needs to be studied or even considered consciousness. In some explanations, it is synonymous with the mind, and at other times, an aspect of mind. In the past, it was one’s “inner life”, the world of introspection, of private thought, imagination and volition.[2] Today, it often includes any kind of cognition, experience, feeling or perception. It may be awareness, awareness of awareness, or self-awareness either continuously changing or not.[3][4] The disparate range of research, notions and speculations raises a curiosity about whether the right questions are being asked.[5]”
Here are the footnotes corresponding to this paragraph:
Again, Wikipedia is hardly the ultimate source for this type of discussion, but I think there is some validity in the academic papers it cites.Footnotes three and four are the sources behind today today’s understanding of consciousness as including “any kind of cognition....” as well as “awareness”.
Cognition
I think of cognition as involving the process of reasoning. Additionally, a functional MRI should show circuits becoming active during the process of cognition by a human. Likewise, I imagine there would be physical signs if an LLM is reasoning, which is one of the points of interpretability research (as I understand it). The presence of any physical sign does not mean there’s reasoning, but any reasoning should necessarily have a physical component that might be detected. That is one aspect of cognition.
Independence/Independent (as it has been used in this post)
If I google “define Independent,” the first definition that comes up is “free from outside control; not depending on another’s authority.”
Technically, in my opinion, true cognition has an “independent” element to begin with, so the word “independent” is redundant. However, I added the word “independent” in my post for people who might define cognition more inclusively to say a clock or any other technology has cognition.
Right now, my computer is running programs, but that is based on programming from someone else’s cognition. The key here is that, if we dissect Chat-GPT4, I don’t believe you would find Python/Java/C++ or any known programming language that a programmer used in order to tell GPT4 how to solve the particular problems I gave it in the four sessions (from my original post and my own reply/addendum to my original post).
If such programming had been present and adequate, GPT4 would have gotten the problems right the first time. Yet in every example, GPT4 gets the problem wrong the first time. Then I explain the concept. Then it gets the problem right. Then GPT4 applies that same concept to different problems that require it, and knows not to use the concept to any “trick” follow-up problems that don’t require the concept.
Awareness (also worth noting)
In addition to the cognitive aspects of consciousness, the above sources for consciousness cite awareness. In the examples from my original post and my reply to my post, it appears GPT4 also has some apparent awareness in order to apply the newly-taught concepts to the other problems. To cite the easiest example, GPT4 seems be aware that the toddler is not “vulnerable,” but rather “predatory” because they pull on the animals’ ears. This awareness supersedes any existing word associations that would have existed in the training data between “toddler” and “vulnerable.” The same idea holds for the other examples.
If anything, there is both “independent cognition” and some awareness that seem apparent.
Final note: If cognition is a “chain of consciousness,” like I described it in my original post, then doesn’t that by definition involve consciousness? In any case, thanks for the feedback. It is useful to be forced to clarify definitions.
Footnotes three and four are the sources behind today today’s understanding of consciousness as including “any kind of cognition....” as well as “awareness”.
The wikipedia quote doesn’t to show that independence is necessary for consciousness, and your arguments from the behaviour of the LLM don’t to show that there is any awareness, or anything beyond forms of cognition.
I think of cognition as involving the process of reasoning.
The question is the relationship between cognition and consciousness, not reasoning. Your quotes show that, at best, cognition is necessary but insufficient for consciousness.
If I google “define Independent,” the first definition that comes up is “free from outside control; not depending on another’s authority.”
Independence in an absolute sense might be impossible: any deterministic system can be controlled if you know how it works , and you can set the initial conditions.
Right now, my computer is running programs, but that is based on programming from someone else’s cognition. The key here is that, if we dissect Chat-GPT4, I don’t believe you would find Python/Java/C++ or any known programming language that a programmer used in order to tell GPT4 how to solve the particular problems I gave it in the four sessions (from my original post and my own reply/addendum to my original post).
That seems to be the heart of the issue. No, its responses are not explictly programmed in.
Yes, its reponses show the ability to learn and synthesise. Which means...minimally...that’s it actually is an AI …. not a glorified search engine. That’s what AI is supposed to do.
The question is whether there is a slope from
*Shows learning and synthesis in cognition
*Has independent cognition
*Is conscious.
*(Has personhood?....should be a citizen...?)
If your think that learning and synthesis in cognition are sufficient for consciousness
conscious, you are effectively assuming that all AIs are conscious. But, historically, Artificial Consciousness has been regarded as a much higher bar than artificial intelligence.
I don’t think that’s true. For instance, none of the definitions given in the LW wiki give that definition. And the whole argument rests on that claim—which rests on the meaning of “independent”. What is “independent”, anyway?
I’ll go over my best understanding of the words “consciousness”, “independent”, and “cognition” here.
Consciousness
I realize the concept of consciousness is extremely complex, and Wikipedia might be considered bottom-feeding. However, we have to start somewhere, and I don’t want to make it seem like I’m cherry-picking obscure or unaccepted definitions to suit my purposes.
Here is the first paragraph of Wikipedia’s explanation of consciousness:
“Consciousness, at its simplest, is awareness of internal and external existence.[1] However, its nature has led to millennia of analyses, explanations and debate by philosophers, theologians, and all of science. Opinions differ about what exactly needs to be studied or even considered consciousness. In some explanations, it is synonymous with the mind, and at other times, an aspect of mind. In the past, it was one’s “inner life”, the world of introspection, of private thought, imagination and volition.[2] Today, it often includes any kind of cognition, experience, feeling or perception. It may be awareness, awareness of awareness, or self-awareness either continuously changing or not.[3][4] The disparate range of research, notions and speculations raises a curiosity about whether the right questions are being asked.[5]”
Here are the footnotes corresponding to this paragraph:
“consciousness”. Merriam-Webster. Retrieved June 4, 2012.
^ Jump up to:a b c d Jaynes J (2000) [1976]. The Origin of Consciousness in the Breakdown of the Bicameral Mind. Houghton Mifflin. ISBN 0-618-05707-2.
^ Rochat P (2003). “Five levels of self-awareness as they unfold early in life” (PDF). Consciousness and Cognition. 12 (4): 717–731. doi:10.1016/s1053-8100(03)00081-3. PMID 14656513. S2CID 10241157. Archived (PDF) from the original on 2022-10-09.
^ P.A. Guertin (2019). “A novel concept introducing the idea of continuously changing levels of consciousness”. Journal of Consciousness Exploration & Research. 10 (6): 406–412. Archived from the original on 2021-12-15. Retrieved 2021-08-19.
^ Hacker P (2012). “The Sad and Sorry History of Consciousness: being, among other things, a challenge to the “consciousness-studies community”″ (PDF). Royal Institute of Philosophy. supplementary volume 70. Archived (PDF) from the original on 2022-10-09.
Again, Wikipedia is hardly the ultimate source for this type of discussion, but I think there is some validity in the academic papers it cites. Footnotes three and four are the sources behind today today’s understanding of consciousness as including “any kind of cognition....” as well as “awareness”.
Cognition
I think of cognition as involving the process of reasoning. Additionally, a functional MRI should show circuits becoming active during the process of cognition by a human. Likewise, I imagine there would be physical signs if an LLM is reasoning, which is one of the points of interpretability research (as I understand it). The presence of any physical sign does not mean there’s reasoning, but any reasoning should necessarily have a physical component that might be detected. That is one aspect of cognition.
Independence/Independent (as it has been used in this post)
If I google “define Independent,” the first definition that comes up is “free from outside control; not depending on another’s authority.”
Technically, in my opinion, true cognition has an “independent” element to begin with, so the word “independent” is redundant. However, I added the word “independent” in my post for people who might define cognition more inclusively to say a clock or any other technology has cognition.
Right now, my computer is running programs, but that is based on programming from someone else’s cognition. The key here is that, if we dissect Chat-GPT4, I don’t believe you would find Python/Java/C++ or any known programming language that a programmer used in order to tell GPT4 how to solve the particular problems I gave it in the four sessions (from my original post and my own reply/addendum to my original post).
If such programming had been present and adequate, GPT4 would have gotten the problems right the first time. Yet in every example, GPT4 gets the problem wrong the first time. Then I explain the concept. Then it gets the problem right. Then GPT4 applies that same concept to different problems that require it, and knows not to use the concept to any “trick” follow-up problems that don’t require the concept.
Awareness (also worth noting)
In addition to the cognitive aspects of consciousness, the above sources for consciousness cite awareness. In the examples from my original post and my reply to my post, it appears GPT4 also has some apparent awareness in order to apply the newly-taught concepts to the other problems. To cite the easiest example, GPT4 seems be aware that the toddler is not “vulnerable,” but rather “predatory” because they pull on the animals’ ears. This awareness supersedes any existing word associations that would have existed in the training data between “toddler” and “vulnerable.” The same idea holds for the other examples.
If anything, there is both “independent cognition” and some awareness that seem apparent.
Final note: If cognition is a “chain of consciousness,” like I described it in my original post, then doesn’t that by definition involve consciousness? In any case, thanks for the feedback. It is useful to be forced to clarify definitions.
The wikipedia quote doesn’t to show that independence is necessary for consciousness, and your arguments from the behaviour of the LLM don’t to show that there is any awareness, or anything beyond forms of cognition.
The question is the relationship between cognition and consciousness, not reasoning. Your quotes show that, at best, cognition is necessary but insufficient for consciousness.
Independence in an absolute sense might be impossible: any deterministic system can be controlled if you know how it works , and you can set the initial conditions.
That seems to be the heart of the issue. No, its responses are not explictly programmed in. Yes, its reponses show the ability to learn and synthesise. Which means...minimally...that’s it actually is an AI …. not a glorified search engine. That’s what AI is supposed to do.
The question is whether there is a slope from
*Shows learning and synthesis in cognition *Has independent cognition *Is conscious. *(Has personhood?....should be a citizen...?)
If your think that learning and synthesis in cognition are sufficient for consciousness conscious, you are effectively assuming that all AIs are conscious. But, historically, Artificial Consciousness has been regarded as a much higher bar than artificial intelligence.