The key think to keep in mind is that EY is a physicalist. He doesn’t think that there is some special consciousness stuff. Instead, consciousness is just what it feels like to implement an algorithm capable of sophisticated social reasoning. An algorithm is conscious if and only if it is capable of sophisticated social reasoning and moreover it is conscious only when it applies that reasoning to itself. This is why EY doesn’t think that he himself is conscious when dreaming or in a flow state.
Additionally, EY does not think that chickens engage in sophisticated social games (others may disagree). This is why he is confident that neither GPT-3 (which reflectively predicts text) nor chickens are conscious. His criticism is not specifically against people who think chickens might be conscious, but only of people who think chickens might be conscious but notGPT-3. The implication is that any such theory would imply the existence of non-physical qualia which are possessed by chickens (because they have neurons) but not GPT-3 (because it is a computer program). Such meat-chauvanism is a parochial view which EY considers utterly unscientific.
Consider the types of evidence that might convince EY chickens (but not GPT-3) are conscious. Assuming his theory is correct, there would have to be evidence that chickens are self-aware and engage in complex social games. For example, if a chicken were to pass the mirror-test or if chickens were observed forming coalitions-of-coalitions.
On the other had, it would be much more difficult to produce evidence that would convince EY to abandon his current theory of consciousness, since he defines consciousness as “what an algorithm implementing complex social games feels like when reflecting on itself”. One possible piece of evidence would be if scientific evidence for the physical existence of qualia was discovered. Suppose, for example, that there was a particle (called perhaps a qualion) that was emitted whenever we experienced a conscious thought and that this particle could be scientifically studied and measured. If it was found that this particle is emitted both when we self-reflect and when we dream (but not by inanimate or mindless objects), then this could be considered evidence for a physical correlate of consciousness.
The key think to keep in mind is that EY is a physicalist. He doesn’t think that there is some special consciousness stuff.
Instead, consciousness is just what it feels like to implement an algorithm capable of sophisticated social reasoning.
The theory that consciousness is just what it feels like to be a sophisticated information processor has a number of attractive features ,but it is not a physicalist theory, in every sense of “physicalist”. In particular, physics does not predict that anything feels like anything from the inside, so that would need to be an additional posit.
Relatedly, his theory is in no way a reduction of of consciousness to physics (or computation). A reductive explanation of consciousness would allow you to predict specific subjective states from specific brain states (as in Mary’s Room); would allow you to reliably construct artificial consciousness; and so on. The “just what it feels.like from the inside” theory doesn’t do any of that.
Your 1 states EYs theory is physicalist in the sense of not being substance dualist …and that is true,as far as it goes...but it is far from the only issue, because there are many dualisms and many non-physiclaisms.
I think you can predict specific subjective states by observing that same computations result in same subjective states? I mean, in theory—do you mean that for a theory to be a reduction it must be practical to predict specific human’s qualia? By that standard we don’t have a physical reduction of billiard balls.
We do have a reductive explanation of billiard balls, in theory. If we don’t have a reductive explanation of billiard balls , we don’t have a reductive explanation of anything. Of course , the computations can be impractical, but that’s why Mary in Mary’s Room is a super scientist.
Say you had a system that implemented a sophisticated social reasoning algorith, and that was actually conscious. Now make a list of literally every sensory input and the behavioral output that the sensory input causes, and write it down in a very (very) long book. This book implements the same exact sophisticated social reasoning algorithm. To think that the book has sentience sounds to me like a statement of magical thinking, not of physicalism.
To think that the book has sentience sounds to me like a statement of magical thinking, not of physicalism.
I’m pretty sure this is because you’re defining “sentience” as some extra-physical property possessed by the algorithm, something with physicalism explicitly rejects.
Consciousness isn’t something that arises when algorithms compute complex social games. Consciousness is when some algorithm computes complex physical games. (under a purely physical theory of consciousness such as EY’s).
To understand how physicalism can talk about metaphysical categories, consider numbers. Some physical systems have the property of being “two of something” as understood by human beings. Two sheep standing in a field, for example. Or two rocks piled on of one another. There’s no magical thing that happens when “two” of something come into existence. They don’t suddenly send a glimmer of two-ness off into a pure platonic realm of numbers. They simply are “two”, and what makes them “two” is that being “two of something” is a category readily recognized by human beings (and presumably other intelligent beings).
Similarly, a physicalist theory of consciousness defines certain physical systems as conscious if they meet certain criteria. Specifically for EY, these criteria are self-recognition and complex social games. It matters no more whether they are implemented by a Chinese room or a computer or a bunch of meat. What matters is that they implement a particular algorithm.
When confronted with the Chinese-room consciousness, EY might say something like: “I recognize that this system is capable of self reflection and social reasoning in much the same way that I am, therefore I recognize that it is conscious in much the same way as I am.”
If I’m not mistaken, that book is behaviourally equivalent to the original algorithm but is not the same algorithm. From an outside view, they have different computational complexity. There are a number of different ways of defining program equivalence, but equivalence is different from identity. A is equivalent to B doesn’t mean A is B.
I don’t agree with Eliezer here. I don’t think we have a deep enough understanding of consciousness to make confident predictions about what is and isn’t conscious beyond “most humans are probably conscious sometimes”.
The hypothesis that consciousness is an emergent property of certain algorithms is plausible, but only that.
If that turns out to be the case, then whether or not humans, GPT-3, or sufficiently large books are capable of consciousness depends on the details of the requirements of the algorithm.
he defines consciousness as “what an algorithm implementing complex social games feels like when reflecting on itself”.
In that case I’ll not use the word consciousness and abstract away to “things which I ascribe moral weight to”, (which I think is a fair assumption given the later discussion of eating “BBQ GPT-3 wings” etc.)
Eliezer’s claim is therefore something along the lines of: “I only care about the suffering of algorithms which implement complex social games and reflect on themselves” or possibly “I only care about the suffering of algorithms which are capable of (and currently doing a form of) self-modelling”.
I’ve not seen nearly enough evidence to convince me of this.
I don’t expect to see a consciousness particle called a qualon. I more expect to see something like: “These particular brain activity patterns which are robustly detectable in an fMRI are extremely low in sleeping people, higher in dreaming people, higher still in awake people and really high in people on LSD and types of zen meditation.”
The key think to keep in mind is that EY is a physicalist. He doesn’t think that there is some special consciousness stuff. Instead, consciousness is just what it feels like to implement an algorithm capable of sophisticated social reasoning. An algorithm is conscious if and only if it is capable of sophisticated social reasoning and moreover it is conscious only when it applies that reasoning to itself. This is why EY doesn’t think that he himself is conscious when dreaming or in a flow state.
Additionally, EY does not think that chickens engage in sophisticated social games (others may disagree). This is why he is confident that neither GPT-3 (which reflectively predicts text) nor chickens are conscious. His criticism is not specifically against people who think chickens might be conscious, but only of people who think chickens might be conscious but not GPT-3. The implication is that any such theory would imply the existence of non-physical qualia which are possessed by chickens (because they have neurons) but not GPT-3 (because it is a computer program). Such meat-chauvanism is a parochial view which EY considers utterly unscientific.
Consider the types of evidence that might convince EY chickens (but not GPT-3) are conscious. Assuming his theory is correct, there would have to be evidence that chickens are self-aware and engage in complex social games. For example, if a chicken were to pass the mirror-test or if chickens were observed forming coalitions-of-coalitions.
On the other had, it would be much more difficult to produce evidence that would convince EY to abandon his current theory of consciousness, since he defines consciousness as “what an algorithm implementing complex social games feels like when reflecting on itself”. One possible piece of evidence would be if scientific evidence for the physical existence of qualia was discovered. Suppose, for example, that there was a particle (called perhaps a qualion) that was emitted whenever we experienced a conscious thought and that this particle could be scientifically studied and measured. If it was found that this particle is emitted both when we self-reflect and when we dream (but not by inanimate or mindless objects), then this could be considered evidence for a physical correlate of consciousness.
The theory that consciousness is just what it feels like to be a sophisticated information processor has a number of attractive features ,but it is not a physicalist theory, in every sense of “physicalist”. In particular, physics does not predict that anything feels like anything from the inside, so that would need to be an additional posit.
Relatedly, his theory is in no way a reduction of of consciousness to physics (or computation). A reductive explanation of consciousness would allow you to predict specific subjective states from specific brain states (as in Mary’s Room); would allow you to reliably construct artificial consciousness; and so on. The “just what it feels.like from the inside” theory doesn’t do any of that.
Your 1 states EYs theory is physicalist in the sense of not being substance dualist …and that is true,as far as it goes...but it is far from the only issue, because there are many dualisms and many non-physiclaisms.
I think you can predict specific subjective states by observing that same computations result in same subjective states? I mean, in theory—do you mean that for a theory to be a reduction it must be practical to predict specific human’s qualia? By that standard we don’t have a physical reduction of billiard balls.
We do have a reductive explanation of billiard balls, in theory. If we don’t have a reductive explanation of billiard balls , we don’t have a reductive explanation of anything. Of course , the computations can be impractical, but that’s why Mary in Mary’s Room is a super scientist.
Say you had a system that implemented a sophisticated social reasoning algorith, and that was actually conscious. Now make a list of literally every sensory input and the behavioral output that the sensory input causes, and write it down in a very (very) long book. This book implements the same exact sophisticated social reasoning algorithm. To think that the book has sentience sounds to me like a statement of magical thinking, not of physicalism.
I’m pretty sure this is because you’re defining “sentience” as some extra-physical property possessed by the algorithm, something with physicalism explicitly rejects.
Consciousness isn’t something that arises when algorithms compute complex social games. Consciousness is when some algorithm computes complex physical games. (under a purely physical theory of consciousness such as EY’s).
To understand how physicalism can talk about metaphysical categories, consider numbers. Some physical systems have the property of being “two of something” as understood by human beings. Two sheep standing in a field, for example. Or two rocks piled on of one another. There’s no magical thing that happens when “two” of something come into existence. They don’t suddenly send a glimmer of two-ness off into a pure platonic realm of numbers. They simply are “two”, and what makes them “two” is that being “two of something” is a category readily recognized by human beings (and presumably other intelligent beings).
Similarly, a physicalist theory of consciousness defines certain physical systems as conscious if they meet certain criteria. Specifically for EY, these criteria are self-recognition and complex social games. It matters no more whether they are implemented by a Chinese room or a computer or a bunch of meat. What matters is that they implement a particular algorithm.
When confronted with the Chinese-room consciousness, EY might say something like: “I recognize that this system is capable of self reflection and social reasoning in much the same way that I am, therefore I recognize that it is conscious in much the same way as I am.”
If I’m not mistaken, that book is behaviourally equivalent to the original algorithm but is not the same algorithm. From an outside view, they have different computational complexity. There are a number of different ways of defining program equivalence, but equivalence is different from identity. A is equivalent to B doesn’t mean A is B.
See also: Chinese Room Problem
I see, but in that case what is the claim about gpt3, that if it had behavioral equivalence to a complicated social being it would have consciousness?
I don’t agree with Eliezer here. I don’t think we have a deep enough understanding of consciousness to make confident predictions about what is and isn’t conscious beyond “most humans are probably conscious sometimes”.
The hypothesis that consciousness is an emergent property of certain algorithms is plausible, but only that.
If that turns out to be the case, then whether or not humans, GPT-3, or sufficiently large books are capable of consciousness depends on the details of the requirements of the algorithm.
In that case I’ll not use the word consciousness and abstract away to “things which I ascribe moral weight to”, (which I think is a fair assumption given the later discussion of eating “BBQ GPT-3 wings” etc.)
Eliezer’s claim is therefore something along the lines of: “I only care about the suffering of algorithms which implement complex social games and reflect on themselves” or possibly “I only care about the suffering of algorithms which are capable of (and currently doing a form of) self-modelling”.
I’ve not seen nearly enough evidence to convince me of this.
I don’t expect to see a consciousness particle called a qualon. I more expect to see something like: “These particular brain activity patterns which are robustly detectable in an fMRI are extremely low in sleeping people, higher in dreaming people, higher still in awake people and really high in people on LSD and types of zen meditation.”