Which Parts Are “Me”?

Eliezer Yudkowsky22 Oct 2008 18:15 UTC
66 points
117 comments5 min readLW linkArchive

Everything I am, is surely my brain; but I don’t accept everything my brain does, as “me”.

Though I don’t like the fact, it does seem in my case to be true, that reflecting upon a passion can diminish it. It does seem that in taking a step back from myself to look at what my brain is doing, that this introduces a greater emotional distance from the world, than when I am not reflecting. Though I am extremely cautious when it comes to the moral issue of how to modify human nature, this is one thing that I would change, I think.

The point at which you start thinking that not everything that goes on your brain is to be identified with “you”, is something of a Rubicon, I fear. And those who prize the intensity of a simple passionate existence, above success in life or a complex layered personality, should not cross that Rubicon.

(Though in this case you should accept advice from rationalists about practical policies for guarding yourself, because this is a dangerous world in which to be unreflective. For example: if you don’t follow the link to the rest of this post, then sign the #$!@ up for cryonics regardless of what your simple passions say on the subject.)

Still here? Then here is the Rubicon of which I speak:

(1) Never seeing ex-girlfriend-X again is so terribly unbearably sad!
->
(2) Why am I so terribly sad?
->
(3) Damn, my stupid brain is still obsessing about this when I just want to get over it.

The first scenario is what I would call the intensity of simple passion; nothing between the world and the emotions. The sentence contains no “I” to get in the way. There is nothing to think about except the world itself, the sadness and the missing person.

In the second scenario it is not the world that is sad, but some particular person, an “I”; and the sadness of this “I” is something that can be called into question.

And in the third scenario, the borders of “I” have been drawn in a way that doesn’t include everything in the brain, so that “I” is the victim of the brain, struggling against it. And this is not paradoxical. Everything that I am, has to be in my brain somewhere, because there is nowhere else for it to be. But that doesn’t mean I have to identify with everything that my brain does. Just as I draw the border of “me” to include my brain but exclude my computer’s CPU—which is still a sensible decision at least for now—I can define the border of myself to exclude certain events that happen in my brain, which I do not control, do not want to happen, and do not agree with.

That time I faced down the power-corrupts circuitry, I thought, “my brain is dumping this huge dose of unwanted positive reinforcement”, and I sat there waiting for the surge to go away and trying not to let it affect anything.

Thinking “I am being tempted” wouldn’t have quite described it, since the deliberate process that I usually think of as “me”—the little voice inside my own head—was not even close to being swayed by the attempted dose of reward that neural circuit was dumping. I wasn’t tempted by power; I’d already made my decision, and the question was enforcing it.

But a dangerous state of mind indeed it would have been, to think “How tempting!” without an “I” to be tempted. From there it would only be a short step to thinking “How wonderful it is to exercise power!” This, so far as I can guess, is what the brain circuit is supposed to do to a human.

So it was a fine thing that I was reflective, on this particular occasion.

The problem is when I find myself getting in the way of even the parts I call “me”. The joy of helping someone, or for that matter, the sadness of death—these emotions that I judge right and proper, which must be me if anything is me—I don’t want those feelings diminished.

And I do better at this, now that my metaethics are straightened out, and I know that I have no specific grounds left for doubting my feelings.

But I still suspect that there’s a little distance there, that wouldn’t be there otherwise, and I wish my brain would stop doing that.

I have always been inside and outside myself, for as long as I can remember. To my memory, I have always been reflective. But I have witnessed the growth of others, and in at least one case I’ve taken someone across that Rubicon. The one now possesses a more complex and layered personality—seems more to me now like a real person, even—but also a greater emotional distance. Life’s lows have been smoothed out, but also the highs. That’s a sad tradeoff and I wish it didn’t exist.

I don’t want to have to choose between sanity and passion. I don’t want to smooth out life’s highs or even life’s lows, if those highs and lows make sense. I wish to feel the emotion appropriate to the event. If death is horrible then I should fight death, not fight my own grief.

But if I am forced to choose, I will choose stability and deliberation, for the sake of what I protect. And my personality does reflect that. What you are willing to trade off, will sometimes get traded away—a dire warning in full generality.

This post is filed under “morality” because the question “Which parts of my brain are ‘me’?” is a moral question—it’s not predicted so much as chosen. You can’t perform a test on neural tissue to find whether it’s in or out. You have to accept or reject any particular part, based on what you think humans in general, and yourself particularly, ought to be.

The technique does have its advantages: It brings greater stability, being less subject to sudden changes of mind in the winds of momentary passion. I was unsettled the first time I met an unreflective person because they changed so fast, seemingly without anchors. Reflection conveys a visibly greater depth and complexity of personality, and opens a realm of thought that otherwise would not exist. It makes you more moral (at least in my experience and observation) because it gives you the opportunity to make moral choices about things that would otherwise be taken for granted, or decided for you by your brain. Waking up to reflection is like the difference between being an entirely helpless prisoner and victim of yourself, versus becoming aware of the prison and getting a chance to escape it sometimes. Not that you are departing your brain entirely, but the you that is the little voice in your own head may get a chance to fight back against some circuits that it doesn’t want to be influenced by.

And the technique’s use, to awaken the unreflective, is as I have described: First you must cross the gap between events-in-the-world just being terribly sad or terribly important or whatever, of themselves; and say, “I feel X”. Then you must begin to judge the feeling, saying, “I do not want to feel this—I feel this way, but I wish I didn’t.” Justifying yourself with “This is not what a human should be”, or “the emotion does not seem appropriate to the event”.

And finally there is the Rubicon of “I wish my brain wouldn’t do this”, at which point you are thinking as if the feeling comes from outside the inner you, imposed upon you by your brain. (Which does not say that you are something other than your brain, but which does say that not every brain event will be accepted by you as you.)

After crossing this Rubicon you have set your feet fully upon the reflective Way; and I’ve yet to hear of anyone turning back successfully, though I think some have tried, or wished they could.

And once your feet are set on walking down that path, there is nothing left but to follow it forward, and try not to be emotionally distanced from the parts of yourself that you accept as you—an effort that a mind of simple passion would not need to make in the first place. And an effort which can easily backfire by drawing your attention to the layered depths of your selfhood, away from the event and the emotion.

Somewhere at the end of this, I think, is a mastery of techniques that are Zenlike but not Zen, so that you have full passion in the parts of yourself that you identify with, and distance from the pieces of your brain that you reject; and a complex layered personality with a stable inner core, without smoothing out those highs or lows of life that you accept as appropriate to the event.

And if not, then screw it, let’s hack the brain so that it works that way. I have no confidence in my ability to judge how human nature should change, and would sooner leave it up to a more powerful mind in the same metamoral reference frame. But if I had to guess, I think that’s the right thing to do.

Eliezer Yudkowsky22 Oct 2008 18:15 UTC
66 points
117 comments5 min readLW linkArchive