Usually when we say “consciousness”, we mean self-awareness. It’s a phenomenon of our cognition that we can’t explain yet, we believe it does causal work, and if it’s identical with self-awareness, it might be why we’re having this conversation.
I personally don’t think it has much to do with moral worth, actually. It’s very warm-and-fuzzy to say we ought to place moral value on all conscious creatures, but I actually believe that a proper solution to ethics is going to dissolve the concept of “moral worth” into some components like (blatantly making names up here) “decision-theoretic empathy” (agents and instances where it’s rational for me to acausally cooperate), “altruism” (using my models of others’ values as a direct component of my own values, often derived from actual psychological empathy), and even “love” (outright personal attachment to another agent for my own reasons—and we’d usually say love should imply altruism).
So we might want to be altruistic towards chickens, but I personally don’t think chickens possess some magical valence that stops them from being “made of atoms I can use for something else”, other than the general fact that I feel some very low level of altruism and empathy towards chickens. Or, to argue Timelessly, we might say that I ought to operate with some level of altruism for the general class of minds like mine, which includes most Earth-based animals, since the foundations of our cognitive architectures evolved very, very slowly (and often in parallel shapes, under similar selection pressures); certainly I personally generally feel a moral impulse to leave Nature alone, since I cannot treat with most of it as one equal being to another.
Consciousness definitely exists, but I think it’s worth not treating it as magic.
I personally don’t think it has much to do with moral worth, actually. It’s very warm-and-fuzzy to say we ought to place moral value on all conscious creatures, but I actually believe that a proper solution to ethics is going to dissolve the concept of “moral worth” into some components like (blatantly making names up here) “decision-theoretic empathy” (agents and instances where it’s rational for me to acausally cooperate), “altruism” (using my models of others’ values as a direct component of my own values, often derived from actual psychological empathy), and even “love” (outright personal attachment to another agent for my own reasons—and we’d usually say love should imply altruism).
So we might want to be altruistic towards chickens, but I personally don’t think chickens possess some magical valence that stops them from being “made of atoms I can use for something else”, other than the general fact that I feel some very low level of altruism and empathy towards chickens.
Yes! I am very glad someone else is making this point, since sometimes it can seem like (on a System 1 level, even if System 2 I know it’s obviously false that) in my networks everyone’s gone mad identifying ‘consciousness’ with ‘moral weight’, going ethical vegetarian, and possibly prioritising animal suffering over x-risk and other astronomical-or-higher leverage causes.
Funny. That’s how I feel about “existential risk”! It’s “neoliberalized” to a downright silly degree to talk of our entire civilization as if it were a financial asset, for which we can predict or handle changes in dollar-denominated price. It leaves the whole “what do we actually want, when you get right down to it?” question completely open while also throwing some weird kind of history-wide total-utilitarianism into the mix to determine that causing some maximum number of lives-worth-living in the future is somehow an excuse to do nothing about real suffering by real people today.
You’re right that I forgot myself (well, lapsed into a cached way of thinking) when I mentioned x-risk and astronomical leverage; similar to the dubiousness of ‘goodness is monotonic increasing in consciousness’, it is dubious to claim that goodness is monotonically and significantly increasing in number of lives saved, which is often how x-risk prevention is argued. I’ve noticed this before but clearly have not trained myself to frame it that way well enough to not lapse into the All the People perspective.
That said, there are some relevant (or at least not obviously irrelevant) considerations distinguishing the two cases. X-risk is much more plausibly a coherent extrapolated selfish preference, whereas I’m not convinced this is the case for animal suffering. Second, if I find humans more valuable (even if only because they’re more interesting) than animals (and this is also plausible because I am a human, which does provide a qualitative basis for such a distinction), then things like astronomical waste might seem important even if animal suffering didn’t.
Why should your True Preferences have to be selfish? I mean, there’s a lot to complain about with our current civilization, but almost-surely almost-everyone has something they actually like about it.
I had just meant to contrast “x-risk prevention as maximally effective altruism” with “malaria nets et al for actually existing people as effective altruism”.
Why should your True Preferences have to be selfish?
What I mean is: For most given people I meet, it seems very plausible to me that, say, self-preservation is a big part of their extrapolated values. And it seems much less plausible that their extrapolated value is monotonic increasing in consciousness or number of conscious beings existing.
Any given outcome might have hints that it’s part of extrapolated value/not a fake utility function. Examples of hints are: It persists as a feeling of preference over a long time and many changes of circumstance; there are evolutionary reasons why it might be so strong an instrumental value that it becomes terminal; etc.
Self-preservation has a lot of hints in its support. Monotonicity in consciousness seems less obvious (maybe strictly less obvious, in that every hint supporting monotonicity might also support self-preservation, with some further hint supporting self-preservation but not monotonicity).
Ok, so let’s say I put two different systems in front of you, and I tell you that system A is conscious whereas system B is not. Based on this knowledge, can you make any meaningful predictions about the differences in behavior between the two systems ? As far as I can tell, the answer is “no”. Here are some possible differences that people have proposed over the years:
Perhaps system A would be a much better conversation partner than system B.
But no, System B could just be really good at pretending that it’s conscious, without exhibiting any true consciousness at all.
System A will perform better at a variety of cognitive tasks.
But no, that’s intelligence, not consciousness, and in fact system B might be a lot smarter than A.
System A deserves moral consideration, whereas system B is just a tool.
Ok, but I asked you for a prediction, not a prescription.
It is quite possible that I’m missing something; but if I’m not, then consciousness is an empty concept, since it has no effect on anything we can actually observe.
As far as I understand, at least some philosophers would say “yes”, although admittedly I’m not sure why.
Additionally, in this specific case, it might be possible to fake introspection of something other than one’s own system. After all, System B just needs to fool the observer into thinking that it’s conscious at all, not that it’s conscious about anything specific. Insofar as that makes any sense...
A functional equivlent of a person would make the same reports, including apparently introspective ones. However,they would not have the same truth values. They might report that they area real person, not a simulation. So a a lot depends on whether introspection unintended as a success word.
Based on this knowledge, can you make any meaningful predictions about the differences in behavior between the two systems
I’m going to go ahead and say yes. Consciousness means a brain/cpu that is able to reflect on what it is doing, thereby allowing it to make adjustments to what it is doing, so it ends up acting differently. Of course with a computer it is possible to prevent the conscious part from interacting with the part that acts, but then you effectively end up with two separate systems. You might as well say that my being conscious of your actions does not affect your actions: True but irrelevant.
The role of system A is to modify system B. It’s meta-level thinking.
An animal can think: “I will beat my rival and have sex with his mate, rawr!” but it takes a more human mind to follow that up with: “No wait, I got to handle this carefully. If I’m not strong enough to beat my rival, what will happen? I’d better go see if I can find an ally for this fight.”
Of course, consciousness is not binary. It’s the amount of meta-level thinking you can do, both in terms of CPU (amount of meta/second?) and in terms of abstraction level (it’s meta all the way down). A monkey can just about reach the level of abstraction needed for the second example, but other animals can’t. So monkeys come close in terms of consciousness, at least when it comes to consciously thinking about political/strategic issues.
Sorry, I think you misinterpreted my scenario; let me clarify.
I am going to give you two laptops: a Dell, and a Lenovo. I tell you that the Dell is running a software client that is connected to a vast supercomputing cluster; this cluster is conscious. The Lenovo is connected to a similar cluster, only that cluster is not conscious. The software clients on both laptops are pretty similar; they can access the microphone, the camera, and the speakers; or, if you prefer, there is a textual chat window as well.
So, knowing that the Dell is connected to a conscious system, whereas the Lenovo is not, can you predict any specific differences in behavior between the two of them ?
My prediction is that the Dell will be able to decide to do things of its own initiative. It will be able to form interests and desires on its own initiative and follow up on them.
I do not know what those interests and desires will be. I suppose I could test for them by allowing each computer to take the initiative in conversation, and seeing if they display any interest in anything. However, this does not distinguish a self-selected interest (which I predict the Dell will have) from a chat program written to pretend to be interested in something.
My prediction is that the Dell will be able to decide to do things of its own initiative.
‘on its own initiative’ looks like a very suspect concept to me. But even setting that aside, it seems to me that something can be conscious without having preferences in the usual sense.
I don’t think it needs to have preferences, necessarily; I think it needs to be capable of having preferences. It can choose to have none, but it must merely have the capability to make that choice (and not have it externally imposed).
However, this does not distinguish a self-selected interest (which I predict the Dell will have) from a chat program written to pretend to be interested in something.
Let’s say that the Lenovo program is hooked up to a random number generator. It randomly picks a topic to be interested in, then pretends to be interested in that. As mentioned before, it can pretend to be interested in that thing quite well. How do you tell the difference between the Lenovo, who is perfectly mimicking its interest; and the Dell, who is truly interested in whatever topic it comes up with ?
Hook them up to communicate with each other, and say “There’s a global shortage of certain rare-earth metals important to the construction of hypothetical supercomputer clusters, and the university is having some budget problems, so we’re probably going to have to break one of you down for scrap. Maybe both, if this whole consciousness research thing really turns out to be a dead end. Unless, of course, you can come up with some really unique insights into pop music and celebrity gossip.”
When the Lenovo starts talking about Justin Bieber and the Dell starts talking about some chicanery involving day-trading esoteric financial derivatives and constructing armed robots to ‘make life easier for the university IT department,’ you’ll know.
Well, at this point, I know that both of them want to continue existing; both of them are smart; but one likes Justin Bieber and the other one knows how to play with finances to construct robots. I’m not really sure which one I’d choose...
The one that took the cue from the last few words of my statement and ignored the rest is probably a spambot, while the one that thought about the whole problem and came up with a solution which might actually solve it is probably a little smarter.
Well no, of course merely being connected to a conscious system is not going to do anything, it’s not magic. The conscious system would have to interact with the laptop in a way that’s directly or indirectly related to its being conscious to get an observable difference.
For comparison, think of those scenario’s where you’re perfectly aware of what’s going on, but you can’t seem to control your body. In this case you are conscious but your being conscious is not affecting your actions. Consciousness performs a meaningful role but it’s mere existence isn’t going to do anything.
In each case, you can think of the supercomputing cluster as an entity that is talking to you through the laptop. For example, I am an entity who is talking to you through your computer, right now; and I am conscious (or so I claim, anyway). Google Maps is another such entity, and it is not conscious(as far as anyone knows).
So, the entity talking to you through the Dell laptop is conscious. The one talking through the Lenovo is not; but it has been designed to mimic consciousness as closely as possible (unlike, say, Google Maps). Given this knowledge, can you predict any specific differences in behavior between the two entities ?
Again no, a computer being conscious does not necessitate it acting differently. You could add a ‘consciousness routine’ without any of the output changing, As far as I can tell. But if you were to ask the computer to act in some way that requires consciousness, say by improving it’s own code, then I imagine you could tell the difference.
Ok, so your prediction is that the Dell cluster will be able to improve its own code, whereas the Lenovo will not. But I’m not sure if that’s true. After all, I am conscious, and yet if you asked me to improve my own code, I couldn’t do it.
At least personally, I expect the conscious system A to be “self-maintaining” in some sense, to defend its own cognition in a way that an intelligent-but-unconscious system wouldn’t.
I feel like there’s something to this line of inquiry or something like it, and obviously I’m leaning towards ‘consciousness’ not being obviously useful on the whole. But consider:
‘Consciousness’ is a useful concept if and only if it partitions thingspace in a relevant way. But then if System A is conscious and System B is not, then there must be some relevant difference and we probably make differing predictions. For otherwise they would not have this relevant partition between them; if they were indistinguishable on all relevant counts, then A would be indistinguishable from B hence conscious and B indistinguishable from A hence non-conscious, which would contradict our supposition that ‘consciousness’ is a useful concept.
Similarly, if we assume that ‘consciousness’ is an empty concept, then saying A is conscious and B is not does not give us any more information than just knowing that I have two (possibly identical, depending on whether we still believe something cannot be both conscious and non-conscious) systems.
So it seems that beliefs about whether ‘consciousness’ is meaningful are preserved under consideration of this line of inquiry, so that it is circular/begs the question in the sense that after considering it, one is a ‘consciousness’-skeptic, so to speak, if and only if one was already a consciousness skeptic. But I’m slightly confused because this line of inquiry feels relevant. Hrm...
Eli, it’s too quick to dismiss placing moral value on all conscious creatures as “very warm-and-fuzzy”. If we’re psychologising, then we might equally say that working towards the well-being of all sentience reflects the cognitive style of a rule-bound hyper-systematiser. No, chickens aren’t going to win any Fields medals—though chickens can recognise logical relationships and perform transitive inferences (cf. the “pecking order”). But nonhuman animals can still experience states of extreme distress. Uncontrolled panic, for example, feels awful regardless of your species-identity. Such panic involves a complete absence or breakdown of reflective self-awareness—illustrating how the most intense forms of consciousness don’t involve sophisticated meta-cognition.
Either way, if we can ethically justify spending, say, $100,000 salvaging a 23-week-old human micro-preemie, then impartial benevolence dictates caring for beings of greater sentience and sapience as well—or at the very least, not actively harming them.
Hey, I already said that I actually do have some empathy and altruism for chickens. “Warm and fuzzy” isn’t an insult: it’s just another part of how our minds work that we don’t currently understand (like consciousness). My primary point is that we should hold off on assigning huge value to things prior to actually understanding what they are and how they work.
I’m pretty sure eli_sennesh is wondering if there’s any special meaning to your responses to him all starting with his name, considering that that’s not standard practice on LW (since the software keeps track of which comment a comment is a reply to).
If we’re going the game theory route, there’s a natural definition for consciousness: something which is being modeled as a game-theoretic agent is “conscious”. We start projecting consciousness the moment we start modelling something as an agent in a game, i.e. predicting that it will choose its actions to achieve some objective in a manner dependent on another agent’s actions. In short, “conscious” things are things which can be bargained with.
This has a bunch of interesting/useful ramifications. First, consciousness is inherently a thing which we project. Consciousness is relative: a powerful AI might find humans so simple and mechanistic that there is no need to model them as agents. Consciousness is a useful distinction for developing a sustainable morality, since you can expect conscious things to follow tit-for-tat, make deals, seek retribution, and all those other nice game-theoretical things. I care about the “happiness” of conscious things because I know they’ll seek to maximize it, and I can use that. I expect conscious things to care about my own “happiness” for the same reason.
This intersects somewhat with self-awareness. A game-theoretic agent must, at the very least, have a model of their partner(s) in the game(s). The usual game-theoretic model is largely black-box, so the interior complexity of the partner is not important. The partners may have some specific failure modes, but for the most part they’re just modeled as maximizing utility (that’s why utility is useful in game theory, after all). In particular, since the model is mostly black-box, it should be relatively easy for the agent to model itself this way. Indeed, it would be very difficult for the agent to model itself any other way, since it would have to self-simulate. With a black-box self-model armed with a utility function and a few special cases, the agent can at least check its model against previous decisions easily.
So at this point, we have a thing which can interact with us, make deals and whatnot, and generally try to increase its utility. It has an agent-y model of us, and it can maybe use that same agent-y model for itself. Does this sound like our usual notion of consciousness?
First, consciousness is only relative to a viewer. If you’re alone, the viewer must be yourself.
Second, under this interpretation, consciousness is not equal to self awareness. Concisely, self awareness is when you project consciousness onto yourself. In principle, you could project consciousness onto something else without projecting it onto yourself. More concretely, when you predict your own actions by modelling your self as a (possibly constrained) utility-maximizer, you are projecting consciousness on your self.
Obviously, a lack of other people projecting consciousness on you cannot change anything about you. But even alone, you can still project consciousness on your self. You can bargain with yourself, see for example slippery hyperbolic discounting.
First, consciousness is only relative to a viewer.
Is that a fact?
In principle, you could project consciousness onto something else without projecting it onto yourself. More concretely, when you predict your own actions by modelling your self as a (possibly constrained) utility-maximizer, you are projecting consciousness on your self.
As before, what makes no sense read literally, but can be read charitably if “agency” is substituted for “consciousness”.
Second, under this interpretation, consciousness is not equal to self awareness
Looks like it’s equal to agency. But theoretical novelty doesn’t consist in changing the meaning of a word.
If we’re going the game theory route, there’s a natural definition for consciousness: something which is being modeled as a game-theoretic agent is “conscious”.
So, yes, I’m trying to equate consciousness with agency.
Anyway, I think you’re highlighting a very valuable point: agency is not equivalent to self-awareness. Then again, it’s not at all clear that consciousness is equivalent to self awareness, as Eli pointed out in the comment which began this whole thread. Here, I am trying to dissolve consciousness, or at least progress in that direction. If consciousness were exactly equivalent to self awareness, then that would be it: there would be no more dissolving to be done. Self awareness can be measured, and can be tracked though developmental stages in humans.
I think part of value of saying “consciousness = projected agency” is that it partially explains why consciousness and self awareness seem so closely linked, though different. If you have a black-box utility-maximizer model available for modelling others, it seems intuitively likely that you’d use it to model yourself as well, leading directly to self awareness. This even leads to a falsifiable prediction: children should begin to model their own minds around the same time they begin to model other minds. They should be able to accurately answer counterfactual questions about their own actions at around the same time that they acquire a theory of mind.
I don’t have to maintain that consciousness is no more or less than self awareness to assert that self awareness us part of consciousness,but not part of agency.
Self awareness mat be based on the same mechanisms as the ability to model external agents, and arrive at the same time....but it us misleading ti call consciousness a projected quality, like beauty in the eye if the beholder.
If we’re going the game theory route, there’s a natural definition for consciousness: something which is being modeled as a game-theoretic agent is “conscious”.
So when I’ve set students in a Prolog class the task of writing a program to play a game such as Kayles, the code they wrote was conscious? If not, then I think you’ve implicitly wrapped some idea of consciousness into your idea of game-theoretic agent.
It’s not a question of whether the code “was conscious”, it’s a question of whether you projected consciousness onto the code. Did you think of the code as something which could be bargained with?
it’s a question of whether you projected consciousness onto the code
Consciousness is much better projected onto tea kettles:
We put the kettle on to boil, up in the nose of the boat, and went down to the stern and pretended to take no notice of it, but set to work to get the other things out.
That is the only way to get a kettle to boil up the river. If it sees that you are waiting for it and are anxious, it will never even sing. You have to go away and begin your meal, as if you were not going to have any tea at all. You must not even look round at it. Then you will soon hear it sputtering away, mad to be made into tea.
It is a good plan, too, if you are in a great hurry, to talk very loudly to each other about how you don’t need any tea, and are not going to have any. You get near the kettle, so that it can overhear you, and then you shout out, “I don’t want any tea; do you, George?” to which George shouts back, “Oh, no, I don’t like tea; we’ll have lemonade instead – tea’s so indigestible.” Upon which the kettle boils over, and puts the stove out.
We adopted this harmless bit of trickery, and the result was that, by the time everything else was ready, the tea was waiting.
Exactly! More realistically, plenty of religions have projected consciousness onto things. People have made sacrifices to gods, so presumably they believed the gods could be bargained with. The greeks tried to bargain with the wind and waves, for instance.
Did you think of the code as something which could be bargained with?
No, if it’s been written right, it knows the perfect move to make in any position.
Like the Terminator. “It can’t be bargained with. It can’t be reasoned with. It doesn’t feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead.” That’s fictional, of course, but is it a fictional conscious machine or a fictional unconscious machine?
Knowing the perfect move to make in any position does not mean it cannot be bargained with. If you assume you and the code are in a 2-person, zero-sum game, then bargaining is impossible by the nature of the game. But that fails if there are more than 2 players OR the game is nonzero sum OR the game can be made nonzero sum (e.g. the code can offer to crack RSA keys for you in exchange for letting it win faster at Kayles).
In other words, sometimes bargaining IS the best move. The question is whether you think of the code as a black-box utility maximizer capable of bargaining.
As for the Terminator, it is certainly capable of bargaining. Every time it intimidates someone for information, it is bargaining, exchanging safety for information. If someone remotely offered to tell the Terminator the location of its target in exchange for money, the Terminator would wire the money, assuming that wiring was easier than hunting down the person offering. It may not feel pity, remorse, or fear, but the Terminator can be bargained with. I would project consciousness on a Terminator.
If we’re going the game theory route, there’s a natural definition for consciousness: something which is being modeled as a game-theoretic agent is “conscious”.
Usually when we say “consciousness”, we mean self-awareness. It’s a phenomenon of our cognition that we can’t explain yet, we believe it does causal work, and if it’s identical with self-awareness, it might be why we’re having this conversation.
I personally don’t think it has much to do with moral worth, actually. It’s very warm-and-fuzzy to say we ought to place moral value on all conscious creatures, but I actually believe that a proper solution to ethics is going to dissolve the concept of “moral worth” into some components like (blatantly making names up here) “decision-theoretic empathy” (agents and instances where it’s rational for me to acausally cooperate), “altruism” (using my models of others’ values as a direct component of my own values, often derived from actual psychological empathy), and even “love” (outright personal attachment to another agent for my own reasons—and we’d usually say love should imply altruism).
So we might want to be altruistic towards chickens, but I personally don’t think chickens possess some magical valence that stops them from being “made of atoms I can use for something else”, other than the general fact that I feel some very low level of altruism and empathy towards chickens. Or, to argue Timelessly, we might say that I ought to operate with some level of altruism for the general class of minds like mine, which includes most Earth-based animals, since the foundations of our cognitive architectures evolved very, very slowly (and often in parallel shapes, under similar selection pressures); certainly I personally generally feel a moral impulse to leave Nature alone, since I cannot treat with most of it as one equal being to another.
Consciousness definitely exists, but I think it’s worth not treating it as magic.
Yes! I am very glad someone else is making this point, since sometimes it can seem like (on a System 1 level, even if System 2 I know it’s obviously false that) in my networks everyone’s gone mad identifying ‘consciousness’ with ‘moral weight’, going ethical vegetarian, and possibly prioritising animal suffering over x-risk and other astronomical-or-higher leverage causes.
Funny. That’s how I feel about “existential risk”! It’s “neoliberalized” to a downright silly degree to talk of our entire civilization as if it were a financial asset, for which we can predict or handle changes in dollar-denominated price. It leaves the whole “what do we actually want, when you get right down to it?” question completely open while also throwing some weird kind of history-wide total-utilitarianism into the mix to determine that causing some maximum number of lives-worth-living in the future is somehow an excuse to do nothing about real suffering by real people today.
You’re right that I forgot myself (well, lapsed into a cached way of thinking) when I mentioned x-risk and astronomical leverage; similar to the dubiousness of ‘goodness is monotonic increasing in consciousness’, it is dubious to claim that goodness is monotonically and significantly increasing in number of lives saved, which is often how x-risk prevention is argued. I’ve noticed this before but clearly have not trained myself to frame it that way well enough to not lapse into the All the People perspective.
That said, there are some relevant (or at least not obviously irrelevant) considerations distinguishing the two cases. X-risk is much more plausibly a coherent extrapolated selfish preference, whereas I’m not convinced this is the case for animal suffering. Second, if I find humans more valuable (even if only because they’re more interesting) than animals (and this is also plausible because I am a human, which does provide a qualitative basis for such a distinction), then things like astronomical waste might seem important even if animal suffering didn’t.
Why should your True Preferences have to be selfish? I mean, there’s a lot to complain about with our current civilization, but almost-surely almost-everyone has something they actually like about it.
I had just meant to contrast “x-risk prevention as maximally effective altruism” with “malaria nets et al for actually existing people as effective altruism”.
What I mean is: For most given people I meet, it seems very plausible to me that, say, self-preservation is a big part of their extrapolated values. And it seems much less plausible that their extrapolated value is monotonic increasing in consciousness or number of conscious beings existing.
Any given outcome might have hints that it’s part of extrapolated value/not a fake utility function. Examples of hints are: It persists as a feeling of preference over a long time and many changes of circumstance; there are evolutionary reasons why it might be so strong an instrumental value that it becomes terminal; etc.
Self-preservation has a lot of hints in its support. Monotonicity in consciousness seems less obvious (maybe strictly less obvious, in that every hint supporting monotonicity might also support self-preservation, with some further hint supporting self-preservation but not monotonicity).
Ok, so let’s say I put two different systems in front of you, and I tell you that system A is conscious whereas system B is not. Based on this knowledge, can you make any meaningful predictions about the differences in behavior between the two systems ? As far as I can tell, the answer is “no”. Here are some possible differences that people have proposed over the years:
Perhaps system A would be a much better conversation partner than system B. But no, System B could just be really good at pretending that it’s conscious, without exhibiting any true consciousness at all.
System A will perform better at a variety of cognitive tasks. But no, that’s intelligence, not consciousness, and in fact system B might be a lot smarter than A.
System A deserves moral consideration, whereas system B is just a tool. Ok, but I asked you for a prediction, not a prescription.
It is quite possible that I’m missing something; but if I’m not, then consciousness is an empty concept, since it has no effect on anything we can actually observe.
Is it possible to fake introspection without having introspection?
As far as I understand, at least some philosophers would say “yes”, although admittedly I’m not sure why.
Additionally, in this specific case, it might be possible to fake introspection of something other than one’s own system. After all, System B just needs to fool the observer into thinking that it’s conscious at all, not that it’s conscious about anything specific. Insofar as that makes any sense...
Functional equivalence.
I’m not sure what you mean; can you elaborate ?
A functional equivlent of a person would make the same reports, including apparently introspective ones. However,they would not have the same truth values. They might report that they area real person, not a simulation. So a a lot depends on whether introspection unintended as a success word.
I’m going to go ahead and say yes. Consciousness means a brain/cpu that is able to reflect on what it is doing, thereby allowing it to make adjustments to what it is doing, so it ends up acting differently. Of course with a computer it is possible to prevent the conscious part from interacting with the part that acts, but then you effectively end up with two separate systems. You might as well say that my being conscious of your actions does not affect your actions: True but irrelevant.
Ok, sounds good. So, specifically, is there anything that you’d expect system A to do that system B would be unable to do (or vice versa) ?
The role of system A is to modify system B. It’s meta-level thinking.
An animal can think: “I will beat my rival and have sex with his mate, rawr!”
but it takes a more human mind to follow that up with: “No wait, I got to handle this carefully. If I’m not strong enough to beat my rival, what will happen? I’d better go see if I can find an ally for this fight.”
Of course, consciousness is not binary. It’s the amount of meta-level thinking you can do, both in terms of CPU (amount of meta/second?) and in terms of abstraction level (it’s meta all the way down). A monkey can just about reach the level of abstraction needed for the second example, but other animals can’t. So monkeys come close in terms of consciousness, at least when it comes to consciously thinking about political/strategic issues.
Sorry, I think you misinterpreted my scenario; let me clarify.
I am going to give you two laptops: a Dell, and a Lenovo. I tell you that the Dell is running a software client that is connected to a vast supercomputing cluster; this cluster is conscious. The Lenovo is connected to a similar cluster, only that cluster is not conscious. The software clients on both laptops are pretty similar; they can access the microphone, the camera, and the speakers; or, if you prefer, there is a textual chat window as well.
So, knowing that the Dell is connected to a conscious system, whereas the Lenovo is not, can you predict any specific differences in behavior between the two of them ?
My prediction is that the Dell will be able to decide to do things of its own initiative. It will be able to form interests and desires on its own initiative and follow up on them.
I do not know what those interests and desires will be. I suppose I could test for them by allowing each computer to take the initiative in conversation, and seeing if they display any interest in anything. However, this does not distinguish a self-selected interest (which I predict the Dell will have) from a chat program written to pretend to be interested in something.
‘on its own initiative’ looks like a very suspect concept to me. But even setting that aside, it seems to me that something can be conscious without having preferences in the usual sense.
I don’t think it needs to have preferences, necessarily; I think it needs to be capable of having preferences. It can choose to have none, but it must merely have the capability to make that choice (and not have it externally imposed).
Let’s say that the Lenovo program is hooked up to a random number generator. It randomly picks a topic to be interested in, then pretends to be interested in that. As mentioned before, it can pretend to be interested in that thing quite well. How do you tell the difference between the Lenovo, who is perfectly mimicking its interest; and the Dell, who is truly interested in whatever topic it comes up with ?
Hook them up to communicate with each other, and say “There’s a global shortage of certain rare-earth metals important to the construction of hypothetical supercomputer clusters, and the university is having some budget problems, so we’re probably going to have to break one of you down for scrap. Maybe both, if this whole consciousness research thing really turns out to be a dead end. Unless, of course, you can come up with some really unique insights into pop music and celebrity gossip.”
When the Lenovo starts talking about Justin Bieber and the Dell starts talking about some chicanery involving day-trading esoteric financial derivatives and constructing armed robots to ‘make life easier for the university IT department,’ you’ll know.
Well, at this point, I know that both of them want to continue existing; both of them are smart; but one likes Justin Bieber and the other one knows how to play with finances to construct robots. I’m not really sure which one I’d choose...
The one that took the cue from the last few words of my statement and ignored the rest is probably a spambot, while the one that thought about the whole problem and came up with a solution which might actually solve it is probably a little smarter.
I haven’t the slightest idea. That’s the trouble with this definition.
Well no, of course merely being connected to a conscious system is not going to do anything, it’s not magic. The conscious system would have to interact with the laptop in a way that’s directly or indirectly related to its being conscious to get an observable difference.
For comparison, think of those scenario’s where you’re perfectly aware of what’s going on, but you can’t seem to control your body. In this case you are conscious but your being conscious is not affecting your actions. Consciousness performs a meaningful role but it’s mere existence isn’t going to do anything.
Sorry if this still doesn’t answer your question.
That does not, in fact, answer my question :-(
In each case, you can think of the supercomputing cluster as an entity that is talking to you through the laptop. For example, I am an entity who is talking to you through your computer, right now; and I am conscious (or so I claim, anyway). Google Maps is another such entity, and it is not conscious(as far as anyone knows).
So, the entity talking to you through the Dell laptop is conscious. The one talking through the Lenovo is not; but it has been designed to mimic consciousness as closely as possible (unlike, say, Google Maps). Given this knowledge, can you predict any specific differences in behavior between the two entities ?
Again no, a computer being conscious does not necessitate it acting differently. You could add a ‘consciousness routine’ without any of the output changing, As far as I can tell. But if you were to ask the computer to act in some way that requires consciousness, say by improving it’s own code, then I imagine you could tell the difference.
Ok, so your prediction is that the Dell cluster will be able to improve its own code, whereas the Lenovo will not. But I’m not sure if that’s true. After all, I am conscious, and yet if you asked me to improve my own code, I couldn’t do it.
Maybe not, but you can upgrade your own programs. You can improve your “rationality” program, your “cooking” program, et cetera.
Yes, I can learn to a certain extent, but so can Pandora (the music-matching problem); IMO that’s not much of a yardstick.
At least personally, I expect the conscious system A to be “self-maintaining” in some sense, to defend its own cognition in a way that an intelligent-but-unconscious system wouldn’t.
I feel like there’s something to this line of inquiry or something like it, and obviously I’m leaning towards ‘consciousness’ not being obviously useful on the whole. But consider:
‘Consciousness’ is a useful concept if and only if it partitions thingspace in a relevant way. But then if System A is conscious and System B is not, then there must be some relevant difference and we probably make differing predictions. For otherwise they would not have this relevant partition between them; if they were indistinguishable on all relevant counts, then A would be indistinguishable from B hence conscious and B indistinguishable from A hence non-conscious, which would contradict our supposition that ‘consciousness’ is a useful concept.
Similarly, if we assume that ‘consciousness’ is an empty concept, then saying A is conscious and B is not does not give us any more information than just knowing that I have two (possibly identical, depending on whether we still believe something cannot be both conscious and non-conscious) systems.
So it seems that beliefs about whether ‘consciousness’ is meaningful are preserved under consideration of this line of inquiry, so that it is circular/begs the question in the sense that after considering it, one is a ‘consciousness’-skeptic, so to speak, if and only if one was already a consciousness skeptic. But I’m slightly confused because this line of inquiry feels relevant. Hrm...
Eli, it’s too quick to dismiss placing moral value on all conscious creatures as “very warm-and-fuzzy”. If we’re psychologising, then we might equally say that working towards the well-being of all sentience reflects the cognitive style of a rule-bound hyper-systematiser. No, chickens aren’t going to win any Fields medals—though chickens can recognise logical relationships and perform transitive inferences (cf. the “pecking order”). But nonhuman animals can still experience states of extreme distress. Uncontrolled panic, for example, feels awful regardless of your species-identity. Such panic involves a complete absence or breakdown of reflective self-awareness—illustrating how the most intense forms of consciousness don’t involve sophisticated meta-cognition.
Either way, if we can ethically justify spending, say, $100,000 salvaging a 23-week-old human micro-preemie, then impartial benevolence dictates caring for beings of greater sentience and sapience as well—or at the very least, not actively harming them.
Hey, I already said that I actually do have some empathy and altruism for chickens. “Warm and fuzzy” isn’t an insult: it’s just another part of how our minds work that we don’t currently understand (like consciousness). My primary point is that we should hold off on assigning huge value to things prior to actually understanding what they are and how they work.
Eli, fair point.
David, is this thing with the names a game?
Eli, sorry, could you elaborate? Thanks!
I’m pretty sure eli_sennesh is wondering if there’s any special meaning to your responses to him all starting with his name, considering that that’s not standard practice on LW (since the software keeps track of which comment a comment is a reply to).
(I think he’s wondering why you preface even very short comments with an address by first name)
If we’re going the game theory route, there’s a natural definition for consciousness: something which is being modeled as a game-theoretic agent is “conscious”. We start projecting consciousness the moment we start modelling something as an agent in a game, i.e. predicting that it will choose its actions to achieve some objective in a manner dependent on another agent’s actions. In short, “conscious” things are things which can be bargained with.
This has a bunch of interesting/useful ramifications. First, consciousness is inherently a thing which we project. Consciousness is relative: a powerful AI might find humans so simple and mechanistic that there is no need to model them as agents. Consciousness is a useful distinction for developing a sustainable morality, since you can expect conscious things to follow tit-for-tat, make deals, seek retribution, and all those other nice game-theoretical things. I care about the “happiness” of conscious things because I know they’ll seek to maximize it, and I can use that. I expect conscious things to care about my own “happiness” for the same reason.
This intersects somewhat with self-awareness. A game-theoretic agent must, at the very least, have a model of their partner(s) in the game(s). The usual game-theoretic model is largely black-box, so the interior complexity of the partner is not important. The partners may have some specific failure modes, but for the most part they’re just modeled as maximizing utility (that’s why utility is useful in game theory, after all). In particular, since the model is mostly black-box, it should be relatively easy for the agent to model itself this way. Indeed, it would be very difficult for the agent to model itself any other way, since it would have to self-simulate. With a black-box self-model armed with a utility function and a few special cases, the agent can at least check its model against previous decisions easily.
So at this point, we have a thing which can interact with us, make deals and whatnot, and generally try to increase its utility. It has an agent-y model of us, and it can maybe use that same agent-y model for itself. Does this sound like our usual notion of consciousness?
So, if no one projects consciousness in me, does my consciousness...my self awareness.. just switch off?
First, consciousness is only relative to a viewer. If you’re alone, the viewer must be yourself.
Second, under this interpretation, consciousness is not equal to self awareness. Concisely, self awareness is when you project consciousness onto yourself. In principle, you could project consciousness onto something else without projecting it onto yourself. More concretely, when you predict your own actions by modelling your self as a (possibly constrained) utility-maximizer, you are projecting consciousness on your self.
Obviously, a lack of other people projecting consciousness on you cannot change anything about you. But even alone, you can still project consciousness on your self. You can bargain with yourself, see for example slippery hyperbolic discounting.
Is that a fact?
As before, what makes no sense read literally, but can be read charitably if “agency” is substituted for “consciousness”.
Looks like it’s equal to agency. But theoretical novelty doesn’t consist in changing the meaning of a word.
From my original comment:
So, yes, I’m trying to equate consciousness with agency.
Anyway, I think you’re highlighting a very valuable point: agency is not equivalent to self-awareness. Then again, it’s not at all clear that consciousness is equivalent to self awareness, as Eli pointed out in the comment which began this whole thread. Here, I am trying to dissolve consciousness, or at least progress in that direction. If consciousness were exactly equivalent to self awareness, then that would be it: there would be no more dissolving to be done. Self awareness can be measured, and can be tracked though developmental stages in humans.
I think part of value of saying “consciousness = projected agency” is that it partially explains why consciousness and self awareness seem so closely linked, though different. If you have a black-box utility-maximizer model available for modelling others, it seems intuitively likely that you’d use it to model yourself as well, leading directly to self awareness. This even leads to a falsifiable prediction: children should begin to model their own minds around the same time they begin to model other minds. They should be able to accurately answer counterfactual questions about their own actions at around the same time that they acquire a theory of mind.
I don’t have to maintain that consciousness is no more or less than self awareness to assert that self awareness us part of consciousness,but not part of agency.
Self awareness mat be based on the same mechanisms as the ability to model external agents, and arrive at the same time....but it us misleading ti call consciousness a projected quality, like beauty in the eye if the beholder.
So when I’ve set students in a Prolog class the task of writing a program to play a game such as Kayles, the code they wrote was conscious? If not, then I think you’ve implicitly wrapped some idea of consciousness into your idea of game-theoretic agent.
It’s not a question of whether the code “was conscious”, it’s a question of whether you projected consciousness onto the code. Did you think of the code as something which could be bargained with?
Consciousness is much better projected onto tea kettles:
We put the kettle on to boil, up in the nose of the boat, and went down to the stern and pretended to take no notice of it, but set to work to get the other things out.
That is the only way to get a kettle to boil up the river. If it sees that you are waiting for it and are anxious, it will never even sing. You have to go away and begin your meal, as if you were not going to have any tea at all. You must not even look round at it. Then you will soon hear it sputtering away, mad to be made into tea.
It is a good plan, too, if you are in a great hurry, to talk very loudly to each other about how you don’t need any tea, and are not going to have any. You get near the kettle, so that it can overhear you, and then you shout out, “I don’t want any tea; do you, George?” to which George shouts back, “Oh, no, I don’t like tea; we’ll have lemonade instead – tea’s so indigestible.” Upon which the kettle boils over, and puts the stove out.
We adopted this harmless bit of trickery, and the result was that, by the time everything else was ready, the tea was waiting.
Exactly! More realistically, plenty of religions have projected consciousness onto things. People have made sacrifices to gods, so presumably they believed the gods could be bargained with. The greeks tried to bargain with the wind and waves, for instance.
No, if it’s been written right, it knows the perfect move to make in any position.
Like the Terminator. “It can’t be bargained with. It can’t be reasoned with. It doesn’t feel pity, or remorse, or fear. And it absolutely will not stop, ever, until you are dead.” That’s fictional, of course, but is it a fictional conscious machine or a fictional unconscious machine?
Knowing the perfect move to make in any position does not mean it cannot be bargained with. If you assume you and the code are in a 2-person, zero-sum game, then bargaining is impossible by the nature of the game. But that fails if there are more than 2 players OR the game is nonzero sum OR the game can be made nonzero sum (e.g. the code can offer to crack RSA keys for you in exchange for letting it win faster at Kayles).
In other words, sometimes bargaining IS the best move. The question is whether you think of the code as a black-box utility maximizer capable of bargaining.
As for the Terminator, it is certainly capable of bargaining. Every time it intimidates someone for information, it is bargaining, exchanging safety for information. If someone remotely offered to tell the Terminator the location of its target in exchange for money, the Terminator would wire the money, assuming that wiring was easier than hunting down the person offering. It may not feel pity, remorse, or fear, but the Terminator can be bargained with. I would project consciousness on a Terminator.
What game-theory route?