Instrumental status: off-the-cuff reply, out of a wish that more people in this community understood what the sequences have to say about how to do philosophy correctly (according to me).
EY’s position seems to be that self-modelling is both necessary and sufficient for consciousness.
That is not how it seems to me. My read of his position is more like: “Don’t start by asking ‘what is consciousness’ or ‘what are qualia’; start by asking ‘what are the cognitive causes of people talking about consciousness and qualia’, because while abstractions like ‘consciousness’ and ‘qualia’ might turn out to be labels for our own confusions, the words people emit about them are physical observations that won’t disappear. Once one has figured out what is going on, they can plausibly rescue the notions of ‘qualia’ and ‘consciousness’, though their concepts might look fundamentally different, just as a physicist’s concept of ‘heat’ may differ from that of a layperson. Having done this exercise at least in part, I (Nate’s model of Eliezer) assert that consciousness/qualia can be more-or-less rescued, and that there is a long list of things an algorithm has to do to ‘be conscious’ / ‘have qualia’ in the rescued sense. The mirror test seems to me like a decent proxy for at least one item on that list (and the presence of one might correlate with a handful of others, especially among animals with similar architectures to ours).”
An ordering of consciousness as reported by humans might be:
Asleep Human < Awake Human < Human on Psychedelics/Zen Meditation
I don’t know if EY agrees with this.
My model of Eliezer says “Insofar as humans do report this, it’s a fine observation to write down in your list of ‘stuff people say about consciousness’, which your completed theory of consciousness should explain. However, it would be an error to take this as much evidence about ‘consciousness’, because it would be an error to act like ‘consciousness’ is a coherent concept when one is so confused about it that they cannot describe the cognitive antecedents of human insistence that there’s an ineffable redness to red.”
But what surprises me the most about EY’s position is his confidence in it.
My model of Eliezer says “The type of knowledge I claim to have, is knowledge of (at least many components of) a cognitive algorithm that looks to me like it codes for consciousness, in the sense that if you were to execute it then it would claim to have qualia for transparent reasons and for the same reasons that humans do, and to be correct about that claim in the same way that we are. From this epistemic vantage point, I can indeed see clearly that consciousness is not much intertwined with predictive processing, nor with the “binding problem”, etc. I have not named the long list of components that I have compiled, and you, who lack such a list, may well not be able to tell what consciousness is or isn’t intertwined with. However, you can still perhaps understand what it would feel like to believe you can see (at least a good part of) such an algorithm, and perhaps this will help you understand my confidence. Many things look a lot more certain, and a lot less confusing, once you begin to see how to program them.”
Your comments here and some comments Eliezer had made elsewhere seem to imply he believes he has at least in large party “solved” consciousness. Is this fair? And if so is there anywhere he has written up this theory/analysis in depth—because surely if correct this would be hugely important
I’m kind of assuming that whatever Eliezer’s model is, the bulk of the interestingness isn’t contained here and still needs to be cashed out, because the things you/he list (needing to examine consciousness through the lens of the cognitive algorithms causing our discussions of it, the centrality of self-modely reflexive things to consciousness etc.) are already pretty well explored and understood in mainstream philosophy, e.g Dennett.
Or is the idea here that Eliezer believes some of these existing treatments (maybe modulo some minor tweaks and gaps) are sufficient for him to feel like he has answered the question to his own satisfaction.
Basically struggling to understand which of the 3 below is wrong, because all three being jointly true seem crazy
Eliezer has a working theory of consciousness
This theory differs in important ways from existing attempts
Eliezer has judged that it is not worthwhile writing this up
I’m confident your model of Eliezer is more accurate than mine.
Neither the twitter thread or other writings originally gave me the impression that he had a model in that fine-grained detail. I was mentally comparing his writings on consciousness to his writings on free will. Reading the latter made me feel like I strongly understood free will as a concept, and since then I have never been confused, it genuinely reduced free will as a concept in my mind.
His writings on consciousness have not done anything more than raise that model to the same level of possibility as a bunch of other models I’m confused about. That was the primary motivation for this post. But now that you mention it, if he genuinely believes that he has knowledge which might bring him closer to (or might bring others closer to to) programming a conscious being, I can see why he wouldn’t share it in high detail.
While I agree with mostly everything your model of Eliezer said, I do not feel less confused about how Eliezer arrives to a conclusion that most animals are not conscious. Granted, I may, and probably actually am, lacking an important insight in the matter, but than it will be this insight that allows me to become less confused and I wish Eliezer shared it.
When I’m thinking about a thought process that allows to arrive to such a conclusion I imagine something like this. Consciousness is not fundamental but it feels like it is. That’s why we intuitively apply concepts such as quantity towards consciousness, thinking about more or less conscious creatures as being more or less filled with conscious-fluid as we previously though about flogiston or caloric fluid. But this intuition is confused and leads us astray. Consciousness is a result of a specific cognitive algorithm. This algorithm can either be executed or not. There are good reasons to assume that such algorithm would be developped by evolution only among highly social animals as such conditions lead to necessity to model other creatures modelling yourself.
And I see an obvious problem with this line of thoughts. Reversed confusion isn’t insight. Our confused intuition which leads us to quantifying consciousness may be wrong, but it isn’t necessary wrong. If anything, the idea that consciousness isn’t quantifiable is also originally based on the idea of consciousness being fundamental. Think about ancient hebrews who claimed that animals didn’t have souls. There are lots of bad reasons to think that farm animals are ethically irrelivant, indeed it would be super convinient, considered how tasty is their meat. That doesn’t automatically mean that they are ethically relevant, just hints at the possibility.
We can think about hearing, or vision, or sense of smell. They are not fundamental. They are the result of a specific algorithm executed by our brain. Yet we can quantify them. Quantifying them actually makes a lot of sense, considered that evolution works incrementally. Why can’t it be the same for consciousness?
I don’t think the thought process that allows one to arrive at (my model of) Eliezer’s model looks very much like your 2nd paragraph. Rather, I think it looks like writing down a whole big list of stuff people say about consciousness, and then doing a bunch of introspection in the vicinity, and then listing out a bunch of hypothesized things the cognitive algorithm is doing, and then looking at that algorithm and asking why it is “obviously not conscious”, and so on and so forth, all while being very careful not to shove the entire problem under the rug in any particular step (by being like “and then there’s a sensor inside the mind, which is the part that has feelings about the image of the world that’s painted inside the head” or whatever).
Assuming one has had success at this exercise, they may feel much better-equipped to answer questions like “is (the appropriate rescuing of) consciousness more like a gradient quantity or more like a binary property?” or “are chickens similarly-conscious in the rescued sense?”. But their confidence wouldn’t be coming from abstract arguments like “because it is an algorithm, it can either be executed or not” or “there are good reasons to assume it would be developed by evolution only among social animals”; their confidence would be coming from saying “look, look at the particular algorithm, look at things X, Y, and Z that it needs to do in particular, there are other highly-probable consequences of a mind being able to do X, Y, and Z, and we difinitively observe those consequences in humans, and observe their absence in chickens.”
You might well disbelieve that Eliezer has such insight into cognitive algorithms, or believe he made a mistake when he did his exercise! But hopefully this sheds some light on (what I believe is) the nature of his confidence.
what are the cognitive causes of people talking about consciousness and qualia
Based on the rest of your comment, I’m guessing you mean talk about consciousness and qualia in the abstract and attribute them to themselves, not just talk about specific experiences they’ve had.
a cognitive algorithm that looks to me like it codes for consciousness, in the sense that if you were to execute it then it would claim to have qualia for transparent reasons and for the same reasons that humans do, and to be correct about that claim in the same way that we are
Why use the standard of claiming to be conscious/have qualia? That is one answer that gets at something that might matter, but why isn’t that standard too high?
For example, he wrote:
I give serious consideration to the proposition that I am reflective enough for consciousness only during the moments I happen to wonder whether I am conscious.
If this proposition is false, we need to allow unsymbolized (non-verbal) ways to self-attribute consciousness for self-attributing consciousness to matter in itself, right? Would (solidly) passing the mirror test be (almost) sufficient at this point? There’s a visual self-representation, and an attribution of the perception of the mark to this self-representation. What else would be needed?
Would it need to non-symbolically self-attribute consciousness generally, not just particular experiences? How would this work?
If the proposion is true, doesn’t this just plainly contradict our everyday experiences of consciousness? I can direct my attention towards things other than wondering whether or not I’m conscious (and towards things other than and unrelated to my inner monologue), while still being conscious, at least in a way that still matters to me that I wouldn’t want to dismiss. We can describe our experiences without wondering whether or not we’re having (or had) them.
it would claim to have qualia for transparent reasons and for the same reasons that humans do, and to be correct about that claim in the same way that we are
What kinds of reasons? And what would being correct look like?
If unsymbolized self-attribution of consciousness is enough, how would we check just for it? The mirror test?
Based on the rest of your comment, I’m guessing you mean talk about consciousness and qualia in the abstract and attribute them to themselves, not just talk about specific experiences they’ve had.
If I were doing the exercise, all sorts of things would go in my “stuff people say about consciousness” list, including stuff Searl says about chinese rooms, stuff Chalmers says about p-zombies, stuff the person on the street says about the ineffable intransmissible redness of red, stuff schoolyard kids say about how they wouldn’t be able to tell if the color they saw as green was the one you saw as blue, and so on. You don’t need to be miserly about what you put on that list.
Why use the standard of claiming to be conscious/have qualia? That is one answer that gets at something that might matter, but why isn’t that standard too high?
Mostly (on my model) because it’s not at all clear from the getgo that it’s meaningful to “be conscious” or “have qualia”; the ability to write an algorithm that makes the same sort of observable-claims that we make, for the same cognitive reasons, demonstrates a mastery of the phenomenon even in situations where “being conscious” turns out to be a nonsense notion.
Note also that higher standards on the algorithm you’re supposed to produce are more conservative: if it is meanigful to say that an algorithm “is conscious”, then producing an algorithm that is both conscious, and claims to be so, for the same cognitive reasons we do, is a stronger demonstration of mastery than isolating just a subset of that algorithm (the “being conscious” part, assuming such a thing exists).
I’d be pretty suspicious of someone who claimed to have a “conscious algorithm” if they couldn’t also say “and if you inspect it, you can see how if you hook it up to this extra module here and initialize it this way, then it would output the Chinese Room argument for the same reasons Searl did, and if you instead initialize it that way, then it outputs the Mary’s Room thought experiment for the same reason people do”. Once someone demonstrated that sort of mastery (and once I’d verified it by inspection of the algorithm, and integrated the insights therefrom), I’d be much more willing to trust them (or to operate the newfound insights myself) on questions of how the ability to write philosophy papers about qualia relates to the ability of the mind to feel, but the qualifying bar for “do you have a reductionist explanation of consciousness” is “can you show me how to build something that produces the observations we set out to explain in the first place (people talking about ‘consciousness’) fo rthe same cognitive reasons?”.
Note further that demonstrating an algorithm that produces the same sort of claims humans do (eg, claims about the redness of red) for the same cognitive reasons, is not the same thing as asserting that everything “with consciousness/qualia” must make similar claims.
If this proposition is false, we need to allow unsymbolized (non-verbal) ways to self-attribute consciousness for self-attributing consciousness to matter in itself, right?
My model of Eliezer says “In lieu of an algorithmic account of the cognitive antecedents of people insisting they are conscious, that sort of claim is not even wrong.” (And similarly with various other claims in that section.) My model continues: “You seem to me to be trying to do far more with the word ‘consciousness’ than your understanding of the phenomenon permits. I recommend doing less abstract reasoning about how ‘consciousness’ must behave, and more thinking about the cognitive causes behind the creation of the Mary’s Room hypothetical.”
What kinds of reasons?
My model says: “The list of reasons is not particularly small, in this case.”
And what would being correct look like?
“The claim is correct if the actual cognitive reasons for Searl inventing the Chinese Room hypothetical, are analogous to the cognitive reasons that the alleged algorithm invents the Chinese Room hypothetical, and so on and so forth.
“This is of course difficult to check directly. However, fairly strong evidence of correctness can be attained by reading the algorithm and imagining its execution. Just as you can stare at the gears of a watch until you understand how their interactions makes the watch-hands tick, at which point you can be justifiably confident that you understand the watch, you should be able to stare at a cognitive algorithm explaining ‘consciousness’ until you understand how its execution makes things like ‘inner listeners’ ‘experiencing redness’ (in a suitably rescued sense), at which point you can be justifiably confident that you understand experience.
“Your fellow tribemembers, who have not understood how gears can drive the hands of a watch, might doubt your claim, saying ‘There are many theories of how the watch works, ranging from internal gears to external solar radiation to the whims of the spirits. How are you so confident that it is the turning of little gears, nevermind this specific mechanism that you claim you can sketch out in the dirt?’. And you could rightly reply, ’When we unscrew the back, we see gears. And there is an arrangement of gears, that I understand, that by inspection would tick the hands in just the way we observe the hands to tick. And while I have not fully taken the watch apart, the visible features of the gears we can see when we unscrew the back, match the corresponding properties of my simple gear mechanism. This is enough for me to be pretty confident that something like my mechanism, which I understand and which clealry by inspection ticks watch-hands, governs the watch before us.”
Shouldn’t mastery and self-awareness/self-modelling come in degrees? Is it necessary to be able to theorize and come up with all of the various thought experiments (even with limited augmentation from extra modules, different initializations)? Many nonhuman animals could make some of the kinds of claims we make about our particular conscious experiences for essentially similar reasons, and many demonstrate some self-awareness in ways other than by passing the mirror test (and some might pass a mirror test with a different sensory modality, or with some extra help, although some kinds of help would severely undermine a positive result), although I won’t claim the mirror test is the only one Eliezer cares about; I don’t know what else he has in mind. It would be helpful to see a list of the proxies he has in mind and what they’re proxies for.
EY: I give serious consideration to the proposition that I am reflective enough for consciousness only during the moments I happen to wonder whether I am conscious.
Me: If this proposition is false, we need to allow unsymbolized (non-verbal) ways to self-attribute consciousness for self-attributing consciousness to matter in itself, right?
You: My model of Eliezer says “In lieu of an algorithmic account of the cognitive antecedents of people insisting they are conscious, that sort of claim is not even wrong.” (And similarly with various other claims in that section.) My model continues: “You seem to me to be trying to do far more with the word ‘consciousness’ than your understanding of the phenomenon permits. I recommend doing less abstract reasoning about how ‘consciousness’ must behave, and more thinking about the cognitive causes behind the creation of the Mary’s Room hypothetical.”
To make sure I understand correctly, it’s not the self-attribution of consciousness and other talk of consciousness like Mary’s Room that matter in themselves (we can allow some limited extra modules for that), but their cognitive causes. And certain (kinds of) cognitive causes should be present when we’re “reflective enough for consciousness”, right? And Eliezer isn’t sure whether wondering whether or not he’s conscious is among them (or a proxy/correlate of a necessary cause)?
The mirror test seems to me like a decent proxy for at least one item on that list
This is merely a bias on our own part as humans. I think people are confusing consciousness with self-awareness. They are completely different things. Consciousness is the OS that runs on the meat machine. Self-awareness is an algorithm that runs on the OS. All meat machines that run this OS have different algorithms for different functions. Some may not have any self-awareness algorithm running, some may have something similar but not exactly the same as our own self-awareness algorithm. That’s where the mirror test fails. We can only observe the who-knows-how-many-levels of causality that lead to those animals to show or not show self-aware behaviors in front of a mirror. We can’t say anything consequential about the actual algorithm(s) running on their OS when they stand in front of a mirror. We are just running our own set of self-awareness algorithms when we stand in front of a mirror. It seems like these algorithms change according to evolution, just like other systems within the multicellular ecosystem that make up the individual organisms. We often see animals that demonstrate these “self-aware” traits because of similar evolutionary conditions, like cats and dogs have evolved to run a lot of socializing algorithms that mingle well with our own social algorithms.
Whether the self-reflective aspect of running these algorithms on our own OS makes one feel certain way about eating meat is in and of itself the result of the relationship between multi-threading the self-aware part and the self-preservation part in terms of labeling kins and such.
At this point we aren’t even conclusive about where to draw the boundary between hardware and software. We end up distinguishing between OS and simple firmware as conscious and unconscious. We mostly reduce the firmware down to simple physical reactions by the laws of physics while the OS exhibits something magical beyond those physical reactions in simpler systems. Is there something truly different that sets OS apart from firmware, or is it just our lack of understanding of the underlying mechanics? This of course touches upon the argument of determinism, which is just looking at the same systems differently.
Instrumental status: off-the-cuff reply, out of a wish that more people in this community understood what the sequences have to say about how to do philosophy correctly (according to me).
That is not how it seems to me. My read of his position is more like: “Don’t start by asking ‘what is consciousness’ or ‘what are qualia’; start by asking ‘what are the cognitive causes of people talking about consciousness and qualia’, because while abstractions like ‘consciousness’ and ‘qualia’ might turn out to be labels for our own confusions, the words people emit about them are physical observations that won’t disappear. Once one has figured out what is going on, they can plausibly rescue the notions of ‘qualia’ and ‘consciousness’, though their concepts might look fundamentally different, just as a physicist’s concept of ‘heat’ may differ from that of a layperson. Having done this exercise at least in part, I (Nate’s model of Eliezer) assert that consciousness/qualia can be more-or-less rescued, and that there is a long list of things an algorithm has to do to ‘be conscious’ / ‘have qualia’ in the rescued sense. The mirror test seems to me like a decent proxy for at least one item on that list (and the presence of one might correlate with a handful of others, especially among animals with similar architectures to ours).”
My model of Eliezer says “Insofar as humans do report this, it’s a fine observation to write down in your list of ‘stuff people say about consciousness’, which your completed theory of consciousness should explain. However, it would be an error to take this as much evidence about ‘consciousness’, because it would be an error to act like ‘consciousness’ is a coherent concept when one is so confused about it that they cannot describe the cognitive antecedents of human insistence that there’s an ineffable redness to red.”
My model of Eliezer says “The type of knowledge I claim to have, is knowledge of (at least many components of) a cognitive algorithm that looks to me like it codes for consciousness, in the sense that if you were to execute it then it would claim to have qualia for transparent reasons and for the same reasons that humans do, and to be correct about that claim in the same way that we are. From this epistemic vantage point, I can indeed see clearly that consciousness is not much intertwined with predictive processing, nor with the “binding problem”, etc. I have not named the long list of components that I have compiled, and you, who lack such a list, may well not be able to tell what consciousness is or isn’t intertwined with. However, you can still perhaps understand what it would feel like to believe you can see (at least a good part of) such an algorithm, and perhaps this will help you understand my confidence. Many things look a lot more certain, and a lot less confusing, once you begin to see how to program them.”
Your comments here and some comments Eliezer had made elsewhere seem to imply he believes he has at least in large party “solved” consciousness. Is this fair? And if so is there anywhere he has written up this theory/analysis in depth—because surely if correct this would be hugely important
I’m kind of assuming that whatever Eliezer’s model is, the bulk of the interestingness isn’t contained here and still needs to be cashed out, because the things you/he list (needing to examine consciousness through the lens of the cognitive algorithms causing our discussions of it, the centrality of self-modely reflexive things to consciousness etc.) are already pretty well explored and understood in mainstream philosophy, e.g Dennett.
Or is the idea here that Eliezer believes some of these existing treatments (maybe modulo some minor tweaks and gaps) are sufficient for him to feel like he has answered the question to his own satisfaction.
Basically struggling to understand which of the 3 below is wrong, because all three being jointly true seem crazy
Eliezer has a working theory of consciousness
This theory differs in important ways from existing attempts
Eliezer has judged that it is not worthwhile writing this up
I’m confident your model of Eliezer is more accurate than mine.
Neither the twitter thread or other writings originally gave me the impression that he had a model in that fine-grained detail. I was mentally comparing his writings on consciousness to his writings on free will. Reading the latter made me feel like I strongly understood free will as a concept, and since then I have never been confused, it genuinely reduced free will as a concept in my mind.
His writings on consciousness have not done anything more than raise that model to the same level of possibility as a bunch of other models I’m confused about. That was the primary motivation for this post. But now that you mention it, if he genuinely believes that he has knowledge which might bring him closer to (or might bring others closer to to) programming a conscious being, I can see why he wouldn’t share it in high detail.
While I agree with mostly everything your model of Eliezer said, I do not feel less confused about how Eliezer arrives to a conclusion that most animals are not conscious. Granted, I may, and probably actually am, lacking an important insight in the matter, but than it will be this insight that allows me to become less confused and I wish Eliezer shared it.
When I’m thinking about a thought process that allows to arrive to such a conclusion I imagine something like this. Consciousness is not fundamental but it feels like it is. That’s why we intuitively apply concepts such as quantity towards consciousness, thinking about more or less conscious creatures as being more or less filled with conscious-fluid as we previously though about flogiston or caloric fluid. But this intuition is confused and leads us astray. Consciousness is a result of a specific cognitive algorithm. This algorithm can either be executed or not. There are good reasons to assume that such algorithm would be developped by evolution only among highly social animals as such conditions lead to necessity to model other creatures modelling yourself.
And I see an obvious problem with this line of thoughts. Reversed confusion isn’t insight. Our confused intuition which leads us to quantifying consciousness may be wrong, but it isn’t necessary wrong. If anything, the idea that consciousness isn’t quantifiable is also originally based on the idea of consciousness being fundamental. Think about ancient hebrews who claimed that animals didn’t have souls. There are lots of bad reasons to think that farm animals are ethically irrelivant, indeed it would be super convinient, considered how tasty is their meat. That doesn’t automatically mean that they are ethically relevant, just hints at the possibility.
We can think about hearing, or vision, or sense of smell. They are not fundamental. They are the result of a specific algorithm executed by our brain. Yet we can quantify them. Quantifying them actually makes a lot of sense, considered that evolution works incrementally. Why can’t it be the same for consciousness?
I don’t think the thought process that allows one to arrive at (my model of) Eliezer’s model looks very much like your 2nd paragraph. Rather, I think it looks like writing down a whole big list of stuff people say about consciousness, and then doing a bunch of introspection in the vicinity, and then listing out a bunch of hypothesized things the cognitive algorithm is doing, and then looking at that algorithm and asking why it is “obviously not conscious”, and so on and so forth, all while being very careful not to shove the entire problem under the rug in any particular step (by being like “and then there’s a sensor inside the mind, which is the part that has feelings about the image of the world that’s painted inside the head” or whatever).
Assuming one has had success at this exercise, they may feel much better-equipped to answer questions like “is (the appropriate rescuing of) consciousness more like a gradient quantity or more like a binary property?” or “are chickens similarly-conscious in the rescued sense?”. But their confidence wouldn’t be coming from abstract arguments like “because it is an algorithm, it can either be executed or not” or “there are good reasons to assume it would be developed by evolution only among social animals”; their confidence would be coming from saying “look, look at the particular algorithm, look at things X, Y, and Z that it needs to do in particular, there are other highly-probable consequences of a mind being able to do X, Y, and Z, and we difinitively observe those consequences in humans, and observe their absence in chickens.”
You might well disbelieve that Eliezer has such insight into cognitive algorithms, or believe he made a mistake when he did his exercise! But hopefully this sheds some light on (what I believe is) the nature of his confidence.
Thanks, this is helpful.
Based on the rest of your comment, I’m guessing you mean talk about consciousness and qualia in the abstract and attribute them to themselves, not just talk about specific experiences they’ve had.
Why use the standard of claiming to be conscious/have qualia? That is one answer that gets at something that might matter, but why isn’t that standard too high?
For example, he wrote:
If this proposition is false, we need to allow unsymbolized (non-verbal) ways to self-attribute consciousness for self-attributing consciousness to matter in itself, right? Would (solidly) passing the mirror test be (almost) sufficient at this point? There’s a visual self-representation, and an attribution of the perception of the mark to this self-representation. What else would be needed?
Would it need to non-symbolically self-attribute consciousness generally, not just particular experiences? How would this work?
If the proposion is true, doesn’t this just plainly contradict our everyday experiences of consciousness? I can direct my attention towards things other than wondering whether or not I’m conscious (and towards things other than and unrelated to my inner monologue), while still being conscious, at least in a way that still matters to me that I wouldn’t want to dismiss. We can describe our experiences without wondering whether or not we’re having (or had) them.
What kinds of reasons? And what would being correct look like?
If unsymbolized self-attribution of consciousness is enough, how would we check just for it? The mirror test?
If I were doing the exercise, all sorts of things would go in my “stuff people say about consciousness” list, including stuff Searl says about chinese rooms, stuff Chalmers says about p-zombies, stuff the person on the street says about the ineffable intransmissible redness of red, stuff schoolyard kids say about how they wouldn’t be able to tell if the color they saw as green was the one you saw as blue, and so on. You don’t need to be miserly about what you put on that list.
Mostly (on my model) because it’s not at all clear from the getgo that it’s meaningful to “be conscious” or “have qualia”; the ability to write an algorithm that makes the same sort of observable-claims that we make, for the same cognitive reasons, demonstrates a mastery of the phenomenon even in situations where “being conscious” turns out to be a nonsense notion.
Note also that higher standards on the algorithm you’re supposed to produce are more conservative: if it is meanigful to say that an algorithm “is conscious”, then producing an algorithm that is both conscious, and claims to be so, for the same cognitive reasons we do, is a stronger demonstration of mastery than isolating just a subset of that algorithm (the “being conscious” part, assuming such a thing exists).
I’d be pretty suspicious of someone who claimed to have a “conscious algorithm” if they couldn’t also say “and if you inspect it, you can see how if you hook it up to this extra module here and initialize it this way, then it would output the Chinese Room argument for the same reasons Searl did, and if you instead initialize it that way, then it outputs the Mary’s Room thought experiment for the same reason people do”. Once someone demonstrated that sort of mastery (and once I’d verified it by inspection of the algorithm, and integrated the insights therefrom), I’d be much more willing to trust them (or to operate the newfound insights myself) on questions of how the ability to write philosophy papers about qualia relates to the ability of the mind to feel, but the qualifying bar for “do you have a reductionist explanation of consciousness” is “can you show me how to build something that produces the observations we set out to explain in the first place (people talking about ‘consciousness’) fo rthe same cognitive reasons?”.
Note further that demonstrating an algorithm that produces the same sort of claims humans do (eg, claims about the redness of red) for the same cognitive reasons, is not the same thing as asserting that everything “with consciousness/qualia” must make similar claims.
My model of Eliezer says “In lieu of an algorithmic account of the cognitive antecedents of people insisting they are conscious, that sort of claim is not even wrong.” (And similarly with various other claims in that section.) My model continues: “You seem to me to be trying to do far more with the word ‘consciousness’ than your understanding of the phenomenon permits. I recommend doing less abstract reasoning about how ‘consciousness’ must behave, and more thinking about the cognitive causes behind the creation of the Mary’s Room hypothetical.”
My model says: “The list of reasons is not particularly small, in this case.”
“The claim is correct if the actual cognitive reasons for Searl inventing the Chinese Room hypothetical, are analogous to the cognitive reasons that the alleged algorithm invents the Chinese Room hypothetical, and so on and so forth.
“This is of course difficult to check directly. However, fairly strong evidence of correctness can be attained by reading the algorithm and imagining its execution. Just as you can stare at the gears of a watch until you understand how their interactions makes the watch-hands tick, at which point you can be justifiably confident that you understand the watch, you should be able to stare at a cognitive algorithm explaining ‘consciousness’ until you understand how its execution makes things like ‘inner listeners’ ‘experiencing redness’ (in a suitably rescued sense), at which point you can be justifiably confident that you understand experience.
“Your fellow tribemembers, who have not understood how gears can drive the hands of a watch, might doubt your claim, saying ‘There are many theories of how the watch works, ranging from internal gears to external solar radiation to the whims of the spirits. How are you so confident that it is the turning of little gears, nevermind this specific mechanism that you claim you can sketch out in the dirt?’. And you could rightly reply, ’When we unscrew the back, we see gears. And there is an arrangement of gears, that I understand, that by inspection would tick the hands in just the way we observe the hands to tick. And while I have not fully taken the watch apart, the visible features of the gears we can see when we unscrew the back, match the corresponding properties of my simple gear mechanism. This is enough for me to be pretty confident that something like my mechanism, which I understand and which clealry by inspection ticks watch-hands, governs the watch before us.”
Shouldn’t mastery and self-awareness/self-modelling come in degrees? Is it necessary to be able to theorize and come up with all of the various thought experiments (even with limited augmentation from extra modules, different initializations)? Many nonhuman animals could make some of the kinds of claims we make about our particular conscious experiences for essentially similar reasons, and many demonstrate some self-awareness in ways other than by passing the mirror test (and some might pass a mirror test with a different sensory modality, or with some extra help, although some kinds of help would severely undermine a positive result), although I won’t claim the mirror test is the only one Eliezer cares about; I don’t know what else he has in mind. It would be helpful to see a list of the proxies he has in mind and what they’re proxies for.
To make sure I understand correctly, it’s not the self-attribution of consciousness and other talk of consciousness like Mary’s Room that matter in themselves (we can allow some limited extra modules for that), but their cognitive causes. And certain (kinds of) cognitive causes should be present when we’re “reflective enough for consciousness”, right? And Eliezer isn’t sure whether wondering whether or not he’s conscious is among them (or a proxy/correlate of a necessary cause)?
This is merely a bias on our own part as humans. I think people are confusing consciousness with self-awareness. They are completely different things. Consciousness is the OS that runs on the meat machine. Self-awareness is an algorithm that runs on the OS. All meat machines that run this OS have different algorithms for different functions. Some may not have any self-awareness algorithm running, some may have something similar but not exactly the same as our own self-awareness algorithm. That’s where the mirror test fails. We can only observe the who-knows-how-many-levels of causality that lead to those animals to show or not show self-aware behaviors in front of a mirror. We can’t say anything consequential about the actual algorithm(s) running on their OS when they stand in front of a mirror. We are just running our own set of self-awareness algorithms when we stand in front of a mirror. It seems like these algorithms change according to evolution, just like other systems within the multicellular ecosystem that make up the individual organisms. We often see animals that demonstrate these “self-aware” traits because of similar evolutionary conditions, like cats and dogs have evolved to run a lot of socializing algorithms that mingle well with our own social algorithms.
Whether the self-reflective aspect of running these algorithms on our own OS makes one feel certain way about eating meat is in and of itself the result of the relationship between multi-threading the self-aware part and the self-preservation part in terms of labeling kins and such.
At this point we aren’t even conclusive about where to draw the boundary between hardware and software. We end up distinguishing between OS and simple firmware as conscious and unconscious. We mostly reduce the firmware down to simple physical reactions by the laws of physics while the OS exhibits something magical beyond those physical reactions in simpler systems. Is there something truly different that sets OS apart from firmware, or is it just our lack of understanding of the underlying mechanics? This of course touches upon the argument of determinism, which is just looking at the same systems differently.