Intelligence without Consciousness

[Andrew Vlahos]

I sleep-talk. In fact, I don’t just talk in my sleep, I can carry on a conversation. Advice and insight I have is much lower quality than when I’m awake, but I’m mostly coherent. Once someone recorded a conversation with me, and it was pretty funny. For example: “A lot of people like Batman. We should buy stock in Batman”.

I’m not conscious, though. I’m not self-aware. If I have qualia, it’s a very tiny amount of qualia. I definitely wouldn’t wonder “do I have qualia?” If I didn’t keep waking up, there wouldn’t be a moral reason for others not to treat me how they find most convenient.

What does this mean? It means that Nonsentient Optimizers and Nonperson Predicates are definitely possible. There is some threshold of intelligence and/​or self-awareness needed to change a neural net into a person.

(Lucid dreaming is a separate issue. I’m talking about something that happens even when not lucid dreaming.)

Also, people in the comments keep saying maybe I did have consciousness and just don’t remember it. No: the conversation quality is sort of like this. When people talk to me, my response is usually connected to what they said, but I’m definitely not conscious.

[Dagon]

I’m not conscious, though. I’m not self-aware. If I have qualia, it’s a very tiny amount of qualia.

You don’t remember the experience, so how do you know how it feels? You may have less reflective capability than when waking, but even that’s hard to know, as without prompting (which would wake you up) you might just not be talking about that.

I don’t think this shows anything about your claims. Not only do we not know whether there was sentience involved, we don’t know if it was intelligence. Some well-formed sentences don’t show that—it’s far less impressive than modern language models, which nobody claims to be intelligent in a general sense.

[noggin-scratcher]

I’m not conscious, though. I’m not self-aware. If I have qualia, it’s a very tiny amount of qualia.

How do you know?

“Very near to sleep” seems to me like an altered state of consciousness, but not unconsciousness. With “asleep” not actually being a binary state, whereby you can end up in a near-sleep state either by being very nearly about to fall asleep, or by rising into a very shallow phase during sleep.

The brain can also stop forming long-term memories while still being conscious in the moment (like being “blackout” drunk) so that your remembering self has no recollection of that period, even though you were minimally awake for it.

[Charlie Steiner]

You have probably experienced the phenomenon of driving for miles while engrossed in conversation (or in silent soliloquy) and then discovering that you have utterly no memory of the road, the traffic, your car-driving activities. It is as if someone else had been driving. Many theorists (myself included, I admit) have cherished this as a favorite case of “unconscious perception and intelligent action.” But were you really unconscious of all those passing cars, stop lights, bends in the road at the time? You were paying attention to other things, but surely if you had been probed about what you had just seen at various moments on the drive, you would have had at least some sketchy details to report.

• Dan Dennett

What, then is Dennett’s alternative picture of consciousness? He calls it the multiple drafts model, and what it is is a wholehearted embracing of the distributed nature of human minds. If you probe someone’s consciousness different ways, like asking them to press a button now versus report what they remember later, you can sometimes get different answers, because the state of someone’s mind is distributed throughout their brain, and different probes can access different facts about that state. It’s like the thing that gets probed, which gets fixated into consciousness when we direct attention to it, has multiple drafts of itself available to different systems of the brain, and these drafts get passed around and edited as time passes.

• Me summarizing Dan Dennett

[Samuel Johnson]

I was struck by the words ‘there is some threshold of intelligence and/​or self awareness’ to qualify as a person? The addition of the words ‘or self-awareness’ seems to add some circularity to a commentary that aims to illuminate the concept of consciousness.

[kithpendragon]

I do this too. As does my 5yo, and at least one of my siblings. And, yes, the conversation seems most like a decent predictive text algorithm. I have no reason to expect that’s not exactly what’s going on in those moments: some parts of the brain are active, resulting in some of the interactive subroutines being available, but it isn’t anywhere near coordinated or sophisticated enough to add up to anything that resembles the usual thing people mean when they talk about “consciousness”.