I’m a bit late on this, obviously, but I’ve had a question that I’ve always felt was a bit too nonsensical (and no doubt addressed somewhere in the sequences that I haven’t found) to bring up but it kinda bugs me.
Do we have any ideas/guesses/starting points about whether or not “self-awareness” is some kind of weird quirk of our biology and evolution or if would be be an inevitable consequence of any general AI?
I realize that’s not a super clear definition- I guess I’m talking about that feeling of “existing is going on here” and you can’t take it away- even if it turned out that all the evidence I thought I was getting was really just artificial stimulation of a culture of neurons, even if I’m just a whole brain emulation on some computer, even if I’m really locked up in a psych ward somewhere on antipsychotics? Because my first-hand experience of existing is irrefutable evidence for existence, even if I’m completely wrong about everything besides that?
Since I assume that basically everything about me has a physical correlate, I assume there’s some section of my brain that’s responsible for processing that. I imagine it would be useful to have awareness of myself in order to simulate future situations, etc- building models in our heads is something human brains seem quite good at. So could an AI be built without that? Obviously it would have access to its own source code and such… but do we have any information on whether self-awareness/sense of self is just a trick our brains play on us and an accident of evolution or whether that would be a basic feature of basically any general AI?
The question makes sense, but the answers probably won’t.
Questions like this are usually approached in an upside-down way. People assume, as you are doing, that reality is “just neurons” or “just atoms” or “just information”, then they imagine that what they are experiencing is somehow “just that”, and then they try to live with that belief. They will even construct odd ways of speaking, in which elements of the supposed “objective reality” are substituted for subjective or mentalistic terms, in order to affirm the belief.
You’re noticing that “self-awareness” or “the feeling that something is happening” or “the feeling that I exist” doesn’t feel like it’s the same thing as “neurons”; though perhaps you will tell yourself—as Wittgenstein may have done—that you don’t actually know what being a pack of neurons should feel like, so how do you know that it wouldn’t feel exactly like this?
But if you pay attention to the subjective component of your thought, even when you’re thinking objectively or scientifically, you’ll notice that the reduction actually goes in the other direction. You don’t have any direct evidence of the “objective existence” of neurons or atoms or “information”. The part of reality that you do know about is always some “experience” that is “happening”, which may include thoughts about an objective world, that match up in some way with elements of the experience. In other words, you don’t know that there are neurons or atoms, but you can know that you are having thoughts about these hypothetical objects. If you’re really good at observing and analyzing your thoughts, you may even be able to say a lot about the conscious mental activity which goes into making the thought and applying it to experience.
The fundamental problem is that the physical concept of reality is obtained by taking these conscious states and amputating the subjective part, leaving only the “object” end. Clearly there is a sense in which the conscious subject is itself an “object”, it’s something that exists. But all the dimly apprehended ontological peculiarities which make consciousness what it is, and which give rise to these nebulous names like intentionality and qualia, are not part of the ontology of “objects”. If you think about another person, and think of them as possessing a consciousness just as you do, then your object-of-thought is still ontologically a subject, but of course that’s not what we do in physics. We just focus on a few attributes which we have learnt to think about rigorously, like space, time, quantity, and then we assume that they are the whole of reality.
That is the essence of the upside-down approach to the problem of physics and consciousness. You have to go in the other direction, acknowledge that consciousness is what it is, and then try to understand physics so that something in physics has all those properties. This isn’t easy and it’s why I promote certain kinds of quantum brain theories, because quantum mechanics, under certain interpretations, can contain complex “wholes” that might be consciousness. But even if that were true, you would still have one more challenge: find a way to think of the usual mathematical ontology of physics as the superficial functional description, and the lived experience of existing as a glimpse of the true reality, rather than the other way around.
Your labeling of physicalism as an “upside-down” approach reminded me of this quote from Schopenhauer, which you would no doubt approve of:
Of all systems of philosophy which start from the object, the most consistent, and that which may be carried furthest, is simple materialism. It regards matter, and t with it time and space, as existing absolutely, and ignores the relation to the subject in which alone all this really exists. It then lays hold of the law of causality as a guiding principle or clue, regarding it as a self-existent order (or arrangement) of things, Veritas aeterna, and so fails to take account of the understanding, in which and for which alone causality is. It seeks the primary and most simple state of matter, and then tries to develop all the others from it; ascending from mere mechanism, to chemism, to polarity, to the vegetable and to the animal kingdom. And if we suppose this to have been done, the last link in the chain would be animal sensibility —
that is knowledge — which would consequently now appear as a mere modification or state of matter produced by causality. Now if we had followed materialism thus far with clear ideas, when we reached its highest point we would suddenly be seized with a fit of the inextinguishable laughter of the Olympians. As if waking from a dream, we would all at once become aware that its final result — knowledge, which it reached so laboriously, was presupposed as the indispensable condition of its very starting-point, mere matter; and when we imagined that we thought matter, we really thought only the subject that perceives matter ; the eye that sees it, the hand that feels it, the understanding that knows it
Thus the tremendous petitio principii reveals itself unexpectedly; for suddenly the last link is seen to be the starting-point, the chain a circle, and the materialist is like Baron Munchausen who, when swimming in water on horseback, drew the horse into the air with his legs.
I still think it is a confused philosophy, but it is a memorable and powerful passage.
Although Schopenhauer set himself against people like Hegel, his outlook still seems to have been a sort of theistic Berkeleyan idealism, in which everything that exists owes its existence to being the object of a universal consciousness; the difference between Hegel and Schopenhauer being, that Hegel calls this universal consciousness rational and good, whereas Schopenhauer calls it irrational and evil, a cosmic Will whose local manifestation in oneself should be annulled through the pursuit of indifference.
But I’m much more like a materialist, in that I think of the world as consisting of external causal interactions between multiple entities, some of which have a mindlike interior, but which don’t owe their existence to their being posited by an overarching cosmic mind. I say a large part of the problem is just that physicalism employs an insufficiently rich ontology. Its categories don’t include the possibility of “entity with a mindlike interior”.
To put it another way: Among all the entities that the world contains, are entities which we can call subjects or persons or thinking beings, and these entities themselves “contain” “ideas of objects” and “experiences of objects”. My problem with physicalism is not that it refuses to treat all actual objects as ideas, or otherwise embed all objects into subjects; it is just that it tries to do without the ontological knowledge obtained by self-reflection, which is the only way we know that there are such things as conscious beings, with their specific properties.
Somehow, we possess the capacity to conceive of a self, as well as the capacity to conceive of objects independent of the self. Physicalism tries to understand everything using only this second capacity, and as such is methodologically blind to the true nature of anything to do with consciousness, which can only be approached through the first capacity. This bias produces a “mechanistic, materialistic” concept of the universe, and then we wonder where the ghost in the machine is hiding. The “amputation of the subjective part” really refers to the attempt to do without knowledge of the first kind, when understanding reality.
I’m a bit late on this, obviously, but I’ve had a question that I’ve always felt was a bit too nonsensical (and no doubt addressed somewhere in the sequences that I haven’t found) to bring up but it kinda bugs me.
Do we have any ideas/guesses/starting points about whether or not “self-awareness” is some kind of weird quirk of our biology and evolution or if would be be an inevitable consequence of any general AI?
I realize that’s not a super clear definition- I guess I’m talking about that feeling of “existing is going on here” and you can’t take it away- even if it turned out that all the evidence I thought I was getting was really just artificial stimulation of a culture of neurons, even if I’m just a whole brain emulation on some computer, even if I’m really locked up in a psych ward somewhere on antipsychotics? Because my first-hand experience of existing is irrefutable evidence for existence, even if I’m completely wrong about everything besides that?
Since I assume that basically everything about me has a physical correlate, I assume there’s some section of my brain that’s responsible for processing that. I imagine it would be useful to have awareness of myself in order to simulate future situations, etc- building models in our heads is something human brains seem quite good at. So could an AI be built without that? Obviously it would have access to its own source code and such… but do we have any information on whether self-awareness/sense of self is just a trick our brains play on us and an accident of evolution or whether that would be a basic feature of basically any general AI?
Sorry if this question doesn’t really make sense!
The question makes sense, but the answers probably won’t.
Questions like this are usually approached in an upside-down way. People assume, as you are doing, that reality is “just neurons” or “just atoms” or “just information”, then they imagine that what they are experiencing is somehow “just that”, and then they try to live with that belief. They will even construct odd ways of speaking, in which elements of the supposed “objective reality” are substituted for subjective or mentalistic terms, in order to affirm the belief.
You’re noticing that “self-awareness” or “the feeling that something is happening” or “the feeling that I exist” doesn’t feel like it’s the same thing as “neurons”; though perhaps you will tell yourself—as Wittgenstein may have done—that you don’t actually know what being a pack of neurons should feel like, so how do you know that it wouldn’t feel exactly like this?
But if you pay attention to the subjective component of your thought, even when you’re thinking objectively or scientifically, you’ll notice that the reduction actually goes in the other direction. You don’t have any direct evidence of the “objective existence” of neurons or atoms or “information”. The part of reality that you do know about is always some “experience” that is “happening”, which may include thoughts about an objective world, that match up in some way with elements of the experience. In other words, you don’t know that there are neurons or atoms, but you can know that you are having thoughts about these hypothetical objects. If you’re really good at observing and analyzing your thoughts, you may even be able to say a lot about the conscious mental activity which goes into making the thought and applying it to experience.
The fundamental problem is that the physical concept of reality is obtained by taking these conscious states and amputating the subjective part, leaving only the “object” end. Clearly there is a sense in which the conscious subject is itself an “object”, it’s something that exists. But all the dimly apprehended ontological peculiarities which make consciousness what it is, and which give rise to these nebulous names like intentionality and qualia, are not part of the ontology of “objects”. If you think about another person, and think of them as possessing a consciousness just as you do, then your object-of-thought is still ontologically a subject, but of course that’s not what we do in physics. We just focus on a few attributes which we have learnt to think about rigorously, like space, time, quantity, and then we assume that they are the whole of reality.
That is the essence of the upside-down approach to the problem of physics and consciousness. You have to go in the other direction, acknowledge that consciousness is what it is, and then try to understand physics so that something in physics has all those properties. This isn’t easy and it’s why I promote certain kinds of quantum brain theories, because quantum mechanics, under certain interpretations, can contain complex “wholes” that might be consciousness. But even if that were true, you would still have one more challenge: find a way to think of the usual mathematical ontology of physics as the superficial functional description, and the lived experience of existing as a glimpse of the true reality, rather than the other way around.
Your labeling of physicalism as an “upside-down” approach reminded me of this quote from Schopenhauer, which you would no doubt approve of:
I still think it is a confused philosophy, but it is a memorable and powerful passage.
Although Schopenhauer set himself against people like Hegel, his outlook still seems to have been a sort of theistic Berkeleyan idealism, in which everything that exists owes its existence to being the object of a universal consciousness; the difference between Hegel and Schopenhauer being, that Hegel calls this universal consciousness rational and good, whereas Schopenhauer calls it irrational and evil, a cosmic Will whose local manifestation in oneself should be annulled through the pursuit of indifference.
But I’m much more like a materialist, in that I think of the world as consisting of external causal interactions between multiple entities, some of which have a mindlike interior, but which don’t owe their existence to their being posited by an overarching cosmic mind. I say a large part of the problem is just that physicalism employs an insufficiently rich ontology. Its categories don’t include the possibility of “entity with a mindlike interior”.
To put it another way: Among all the entities that the world contains, are entities which we can call subjects or persons or thinking beings, and these entities themselves “contain” “ideas of objects” and “experiences of objects”. My problem with physicalism is not that it refuses to treat all actual objects as ideas, or otherwise embed all objects into subjects; it is just that it tries to do without the ontological knowledge obtained by self-reflection, which is the only way we know that there are such things as conscious beings, with their specific properties.
Somehow, we possess the capacity to conceive of a self, as well as the capacity to conceive of objects independent of the self. Physicalism tries to understand everything using only this second capacity, and as such is methodologically blind to the true nature of anything to do with consciousness, which can only be approached through the first capacity. This bias produces a “mechanistic, materialistic” concept of the universe, and then we wonder where the ghost in the machine is hiding. The “amputation of the subjective part” really refers to the attempt to do without knowledge of the first kind, when understanding reality.
Have you read Zen and the Art of Motorcycle Maintenance?
The climactic realization is
gung vzzrqvngr rkcrevrapr vf havgnel, ohg gur zvaq dhvpxyl qvivqrf vg vagb jung vf vafvqr gur frys naq bhgfvqr gur frys.
...That’s the sound made by a poorly maintained motorcycle.
No, I haven’t read it; thanks for the recommendation.