When is a mind me?
xlr8harder writes:
In general I don’t think an uploaded mind is you, but rather a copy. But one thought experiment makes me question this. A Ship of Theseus concept where individual neurons are replaced one at a time with a nanotechnological functional equivalent.
Are you still you?
Presumably the question xlr8harder cares about here isn’t semantic question of how linguistic communities use the word “you”, or predictions about how whole-brain emulation tech might change the way we use pronouns.
Rather, I assume xlr8harder cares about more substantive questions like:
If I expect to be uploaded tomorrow, should I care about the upload in the same ways (and to the same degree) that I care about my future biological self?
Should I anticipate experiencing what my upload experiences?
If the scanning and uploading process requires destroying my biological brain, should I say yes to the procedure?
My answers:
Yeah.
Yep.
Yep, this is no big deal. A productive day for me might involve doing some work in the morning, getting a sandwich at Subway, destructively uploading my brain, then texting some friends to see if they’d like to catch a movie after I finish answering e-mails. ¯\_(ツ)_/¯
If there’s an open question here about whether a high-fidelity emulation of me is “really me”, this seems like it has to be a purely verbal question, and not something that I would care about at reflective equilibrium.
Or, to the extent that isn’t true, I think that’s a red flag that there’s a cognitive illusion or confusion still at work. There isn’t a special extra “me” thing separate from my brain-state, and my precise causal history isn’t that important to my values.
I’d guess that this illusion comes from not fully internalizing reductionism and naturalism about the mind.
I find it pretty natural to think of my “self” as though it were a homunculus that lives in my brain, and “watches” my experiences in a Cartesian theater.
On this intuitive model, it makes sense to ask, separate from the experiences and the rest of the brain, where the homunculus is. (“OK, there’s an exact copy of my brain-state there, but where am I?”)
E.g., consider a teleporter that works by destroying your body, and creating an exact atomic copy of it elsewhere.
People often worry about whether they’ll “really experience” the stuff their brain undergoes post-teleport, or whether a copy will experience it instead. “Should I anticipate ‘waking up’ on the other side of the teleporter? Or should I anticipate Oblivion, and it will be Someone Else who has those future experiences?”
This question doesn’t really make sense from a naturalistic perspective, because there isn’t any causal mechanism that could be responsible for the difference between “a version of me that exists at 3pm tomorrow, whose experiences I should anticipate experiencing” and “an exact physical copy of me that exists at 3pm tomorrow, whose experiences I shouldn’t anticipate experiencing”.
Imagine that the teleporter is located on Earth, and it sends you to a room on a space station that looks and feels identical to the room you started in. This means that until you exit the room and discover whether you’re still on Earth, there’s no way for you to tell whether the teleporter worked.
But more than that, there will be nothing about your brain that tracks whether or not the teleporter sent you somewhere (versus doing nothing).
There isn’t an XML tag in the brain saying “this is a new brain, not the original”!
There isn’t a Soul or Homunculus that exists in addition to the brain, that could be the causal mechanism distinguishing “a brain that is me” from “a brain that is not me”. There’s just the brain-state, with no remainder.
All of the same functional brain-states occur whether you enter the teleporter or not, at least until you exit the room. At every moment where the brain exists, the current state of the brain isn’t affected by whether teleportation occurred.
So there isn’t, within physics, any way for “the real you to be having an experience” in the case where the teleporter malfunctioned, and “someone else to be having the experience” in the case where the teleporter worked. (Unless this is a purely verbal distinction, unrelated to the three important-feeling questions we started with.)
Physics is local, and doesn’t remember whether the teleportation occurred in the past.
Nor is there a law of physics saying “your subjective point of view immediately blips out of existence and is replaced by Someone Else’s point of view if your spacetime coordinates change a lot in a short period of time (even though they don’t blip out of existence when your spacetime coordinates change a little or change over a longer period of time)”.
If that sort of difference can really and substantively change whether your experiences persist over time, it would have to be through some divine mechanism outside of physics.[1]
Why Humans Feel Like They Persist
Taking a step back, we can ask: what physical mechanism makes it feel as though I’m persisting over time? In normal cases, why do I feel so confident that I’m going to experience my future self’s experiences, as opposed to being replaced by a doppelganger who will experience everything in my place?
Let’s call “Rob at time 1” R1, “Rob at time 2″ R2, and “Rob at time 3” R3.
R1 is hungry, and has the thought “I’ll go to the fridge to get a sandwich”. R2 walks to the fridge and opens the door. R3 takes a bite of the sandwich.
Question 1: Why is R2 bothering to open the fridge, even though it’s R3 that will get to eat the sandwich? For that matter, why is R1 bothering to strategize about finding food, when it’s not R1 who will realize the benefits?
Answer: Well, there’s no need in principle for my time-slices to work together like that. Indeed, there are other cases where my time-slices work at cross purposes (like when I try to follow a diet but one of my time-slices says “no”). But it was reproductively advantageous for my ancestors’ brains to generate and execute plans (including very fast, unconscious five-second plans), so they evolved to do so, rather than just executing a string of reflex actions.
Question 2: OK, but you could still achieve all that by having R1 think of R1, R2, and R3 as three different people. Rather than R1 thinking “I selfishly want a sandwich, so I’ll go ahead and do multiple actions in sequence so that I get a sandwich”, why doesn’t R1 think “I altruistically want my friend R3 to have a sandwich, so I’ll collaborate with R2 to do a favor for R3″?
Answer: Either of those ways of thinking would probably work fine in principle. Indeed, there’s some individual and cultural variation in how much individual humans think of themselves as transtemporal “teams” versus persisting objects.
But it does seem like humans have a pretty strong inclination to think of themselves as psychologically persisting over time. I don’t know why that is, but plausibly it has a lot to do with the general way humans think of objects: we say that a table is “the same table” even if it has changed a lot through years of usage. We even say that a caterpillar is “the same organism” as the butterfly it produces. We don’t usually think of objects as a rapid succession of momentary blips, so it doesn’t seem surprising that we think of our minds/brains as stable objects too, and use labels like “me” and “selfish” rather than “us” and “self-altruistic”.
Question 3: OK, but it’s not just that I’m using the arbitrary label “me” to refer to R1, R2, and R3. R1 anticipates experiencing the sandwich himself, and would anticipate this regardless of how he used language. Why’s that?
Answer: Because R1 is being replaced by R2, an extremely similar brain that will likely remember the things R1 just thought. You’re in a sense constantly passing the baton to a new person, as your brain changes over time. The feeling of being replaced by a new brain state that has around that much in common with your current brain state just is the experience that you’re calling “persisting over time”.
That experience of “persisting over time” isn’t the experience of a magical Cartesian ghost that is observing a series of brain-states and acting as a single Subject for all of them. Rather, the experience of “persisting over time” just is the experience of each brain-states possessing certain kinds of information (“memories”) about the previous brain-state in a sequence. (Along with R1, R2, and R3 having tons of overlapping personality traits, goals, etc.)
Some humans are more temporally unstable than others, and if a drug or psychotic episode interfered with your short-term memory enough, or caused your personality or values to change enough minute-to-minute, you might indeed feel as though “I’m the same person over time” has become less true.
(On the other hand, if you’d been born with that level of instability, it’s less likely that you’d think there was anything weird about it. Humans can get used to a lot!)
There isn’t a sharp black line in physics that determines how much a brain must resemble your own in order for you to “persist over time” into becoming that brain. There’s just one brain-state that exists at one spacetime coordinate, and then another brain-state that exists at another spacetime coordinate.
If a brain-state A has quasi-sensory access to the experience of another brain-state B — if A feels like it “remembers” being in state B a fraction of a second ago — then A will typically feel as though it used to be B. If A doesn’t have the same personality or values as B, then A will perhaps feel like they used to be B, but have suddenly changed into a very different sort of person.
Change enough, while still giving A immediate quasi-sensory access to B’s state, and perhaps the connection will start to feel more dissociative or dreamlike; but there’s no sharp line in physics to tell us how much change makes someone “no longer the same person”.
Sleep and Film Reels
I find it easier to make sense of the teleporter scenario when I consider hypotheticals like “neuroscience discovers that you die and are reborn every night while you sleep”, or “physics discovers that the entire universe is destroyed and an exact copy is recreated millions of times every second”.
If we discovered one of those facts, would it make sense to freak out or go into mourning?
In that scenario, should we really start fretting about whether “I’m” going to “really experience” the thing that happens to my body five seconds from now, versus Someone Else experiencing it?
I think this would be pretty danged silly. You’re right now experiencing what it’s like to “toss the baton” from a past version of you to a future version of you, with zero consternation or anxiety, even though right now it’s an open possibility that you’re not “continuous”.
Maybe the real, deep metaphysical Truth is that the universe is more like a film reel made up of many discrete frames (that feel continuous to us, because we’re experiencing the frames from the inside, not looking at the reel from Outside The Universe), not something actually continuous.
I earnestly believe that the proper response to that hypothetical is: Who cares? For all I know, something like that could be true. But if it’s true now, it was always true; I’ve been living that way my whole life. If the experiences I’m having as I write this sentence are the super scary Teleporter Death thing people keep saying I should worry about, then I already know what that’s like, and it’s chill.
If you aren’t already bored by the whole topic (as you probably should be), you can play semantics and claim that I should instead say “the experiences we’ve been having as we write this sentence”. Because this weird obscure discovery about metaphysics is somehow supposed to mean that in the world where we made this discovery, the Real Me is secretly constantly dying and being replaced...?
But whatever. If you’re just redescribing the stuff I’m already experiencing and telling me that that’s the scary thing, then I think you’re too easily spooked by abstract redescriptions of ordinary life. Or if you’re redescribing it but not trying to tell me I should freak out about your redescription, then it’s just semantics, and I’ll use pronouns in whichever way is most convenient.
Another way of thinking about this is: I am my brain, not a ghost or thing outside my brain. So if something makes no physical difference to my current brain-state, and makes no difference to any of my past or future brain-states, then I think it’s just crazy talk to think that this metaphysical bonus thingie-outside-my-brain is the crucial thing that determines whether I exist, or whether I’m alive or dead, etc.
Thinking that my existence depends on some metaphysical “glue” outside of my brain, is like thinking that my existence depends on whether a magenta marble is currently orbiting Neptune. Why would the existence of some random Stuff out there in the cosmos that’s not a Rob-time-slice brain-state, change how I should care about a Rob-time-slice brain-state, or change which brain-state (if any) I should anticipate?
Real life is more boring than the games we can play, striving to find a redescription of the mundane that makes the mundane sound spooky. Like children staring at campfire shadows and trying to will the shadows into looking like monsters.
Real life looks like going to bed at night and thinking about whether I want toast tomorrow morning, even though I don’t know how sleep works and it’s totally possible that sleep might involve shutting down my stream of consciousness at some point and then starting it up again.
Regardless of how a mature neuroscience of sleep ends up looking, I expect the me tomorrow to share a truly crazily extraordinarily massive number of memories, personality traits, goals, etc. in common with me.
I expect them to remember a ton of the things I do today, such that micro-decisions (like how I write this sentence) can influence a bunch of things about their state and their own future trajectory.
I can try to distract myself from those things with neurotic philosophy-101 ghost stories, but looking away from reality doesn’t make it go away.
Weird-Futuristic-Technology Anxiety
Since there isn’t a Soul that lives Outside The Film Reel and is being torn asunder from my brain-state by the succession of frames — there’s just a bunch of brain-states — the anxiety about whether “I” should “really” anticipate any future experiences in Film Reel World is based in illusion.
But the only difference between this scenario and the teleporter one is that the teleporter scenario invokes a weird-sounding New Technology, whereas the sleep and Film Reel examples bake in “there’s nothing new and weird happening, you’ve already been living your whole life this way”. If you’d grown up using using teleporters all the time, then it would seem just as unremarkable as stepping through a doorway.
If a philosopher then came to you one day and said “but WHAT IF something KILLS YOU every time you step through a door and then a NEW YOU comes into existence on the other side!”, you would just roll your eyes. If it makes no perceptible difference, then wtf are we even talking about?
And the same logic applies to mind uploading. There isn’t some magical Extra Thing beyond the brain state, that could make it the case that one thing is You and another thing is Not You.
Sure, you’re now made of silicon atoms rather than carbon atoms. But this is like discovering that Film Reel World alternates between one kind of metaphysical Stuff and another kind of Stuff every other second.
If you aren’t worried about learning that the universe secretly metaphysically is in a state of Constant Oscillation between two types of (functionally indistinguishable) micro-particles, then why care about functionally irrelevant substrate changes at all?
(It’s another matter entirely if you think carbon vs. silicon actually does make an inescapable functional, causal difference for which high-level thoughts and experiences your mind instantiates, and if you think that there’s no way in principle to use a computer to emulate the causal behavior of a human mind. I think that’s crazy talk, but it’s crazy because of ordinary facts about physics / neuroscience / psych / CS, not because of any weird philosophical considerations.)
To Change Experience, You Have to Change Physics, Not Just Metaphysics
Scenario 1:
I step through a doorway. At time 1, a brain is about to enter a doorway. At time 2, an extremely similar brain is passing through the doorway. At time 3, another extremely similar brain has finished passing through the doorway. |
Scenario 2:
I step into a teleporter. Here, again, there exist a series of extremely similar brain states before, during, and after I use the teleporter. |
The particular brain states look no different in the teleporter case than if I’d stepped through a door; so if there’s something that makes the post-teleporter Rob “not me” while also making the post-doorway Rob “me”, then it must lie outside the brain states, a Cartesian Ghost.
Given all that, there’s something genuinely weird about the fact that teleporters spook people more than walking through a door does.
It’s like looking at a film strip, and being scared that if a blank slide were added in between every frame, this would somehow make a difference for the people living inside the movie. It’s getting confused about the distinction between the physics of the movie’s events and the meta-physics of “what the world runs on”.
The same confusion can arise if we imagine flipping the order of all the frames in the film strip; or flipping the order of all the frames in the second half of the movie; or swapping the order of every pair of frames, like so:
From outside the movie, this can make the movie’s events look more confusing or chaotic to us, the viewers. But if you imagine that the characters inside the movie would be the least bit bothered or confused by this rearrangement, you’re making a clear mistake. To confuse the characters, you need to change what happens inside the frames, not just change the relationship between those frames.
I claim that a very similar cognitive hiccup is occurring when someone worries about their internal stream of consciousness halting due to a teleporter (and not halting due to stepping through a random doorway).
You’re imagining that something about the context of the film cells — i.e., the stuff outside of the brain states themselves — is able to change your experiences.
But experiences just are brain things. To imagine that some of the unconscious goings-on in between two of your experiences can interfere with your Self is just the same kind of error as imagining that a movie character will be bothered, or will even subjectively notice, if you inject some empty frames into the movie while changing nothing else about the movie.
… And You Can’t Change Experience With Just Any Old Change to Physics
Claim:
As soon as a purple hat comes into existence on Pluto, my stream of consciousness will end and I will be imperceptibly replaced by an exact copy of myself that is experiencing a different stream of consciousness. This exact copy of me will be physically identical to me in every respect, and will have all of my memories, personality traits, etc. But they won’t be me. The hat, if such a hat ever comes into being, will kill me. |
What, specifically, is wrong with this claim?
Well, one thing that’s wrong with the claim is that Pluto is very far away from the Earth.
But the idea of a hat ending my existence seems very strange even if the hat is in closer proximity to me. Even putting a hat on my head seems like it shouldn’t be enough to end my stream of consciousness, unless there’s something special about the hat that will actually drastically change my brain-state. (E.g., maybe the hat is wired up with explosives.)
The point of this example being:
You can call the Ghost a “Soul”, and make it obvious that we’re invoking magic.
Or you can call it a “special kind of causal relationship (that’s able to preserve selfhood)”, and make it sound superficially scientific. (Or at least science-compatible.)
You can hypothesize that there’s something special about the causal process that produces new brain-states in the “walk through a doorway” case — something “in the causality itself” that makes the post-doorway self me and the post-teleporter self not me.
But of course, this “causal relationship” is not a part of the brain state. Reify causality all you want; the issue remains that you’re positing something outside the brain, outside you and your experiences, that is able to change which experiences you should anticipate without changing any of the experiences or brain-states themselves.
The brain states exist too, whatever causal relationships they exhibit. To say that exactly the same brain states can exist, and yet something outside of those states is changing a perceptible feature of those experiences (“which experience comes next in this subjective flow that’s being experienced; what I should expect to see next”), without changing any of the actual brain states, is just as silly whether that something is a “causal relationship” or a purple hat.
This principle is easier to motivate in the case of the hat, because hats are a lot more concrete, familiar, and easy to think about than some fancy philosophical abstraction like “causal relationship”. But the principle generalizes; random objects and processes out there, whether fancy-sounding or perfectly mundane, can’t perceptibly change my experience (unless they change which brain states occur).
Likewise, it’s easier to see that something on Pluto can’t suddenly end my stream of consciousness, than to see that something physically (or metaphysically?) “nearby” can’t suddenly end my stream of consciousness (without leaving a mess). But the principle generalizes; being nearby or connected to something doesn’t open the door to arbitrary magical changes, absent some mechanism for how that exact change is caused by that exact physical process.
If we were just talking about word definitions and nothing else, then sure, define “self” however you want. You have the universe’s permission to define yourself into dying as often or as rarely as you’d like, if word definitions alone are what concerns you.
But this post hasn’t been talking about word definitions. It’s been talking about substantive predictive questions like “What’s the very next thing I’m going to see? The other side of the teleporter? Or nothing at all?”
There should be an actual answer to this, at least to the same degree there’s an answer to “When I step through this doorway, will I have another experience? And if so, what will that experience be?”
And once we have an answer, this should change how excited we are about things like mind uploading. If my stream of consciousness is going to end with my biological death no matter what I do, then mind uploading sounds a lot less exciting!
Or, equivalently: If my experiences were a matter of “displaying images for a Cartesian Homunculus”, and the death of certain cells in the brain severs the connection between my brain and the Homunculus, then there’s no obvious reason I should expect this exact same Homunculus to establish a connection to an uploaded copy of my brain.
It’s only if I’m in my brain, just an ordinary part of physics, that mind uploading makes sense as a way to extend my lifespan.
Causal relationships and processes obviously matter for what experiences occur. But they matter because they change the brain-states themselves. They don’t cause additional changes to experience beyond the changes exhibited in the brain.
Having More Than One Future
I’ve tried to keep this post pretty simple and focused. E.g., I haven’t gone into questions like “What happens if you make two uploads of me? Which one should I anticipate having the experiences of?”
But I hope the arguments I’ve laid out above make it clear what the right answer has to be: You should anticipate having both experiences.
If you’ve already bitten the bullet on things like the teleporter example, then I don’t think this should actually be particularly counter-intuitive. If one copy of my brain exists at time 1 (Rob-x), and two almost-identical copies of my brain (Rob-y and Rob-z) exist at time 2, then there’s going to be a version of me that’s Rob-y, and a version of me that’s Rob-z, and each will have equal claim to being “the next thing I experience”.
In a world without magical Cartesian Homunculi, this has to be how things work; there isn’t any physical difference between Rob-y and Rob-z that makes one of them my True Heir and the other a False Pretender. They’re both just future versions of me.
“You should anticipate having both experiences” sounds sort of paradoxical or magical, but I think this stems from a verbal confusion. “Anticipate having both experiences” is ambiguous between two scenarios:
Scenario 1: “Split-screen mode.” My stream of consciousness continues, but it somehow magically splits into a portion that’s Rob-y and a different portion that’s Rob-z, as though the Cartesian Homunculus were trying to keep an eye on both brains at once.
Scenario 2: “Two separate screens.” My stream of consciousness continues from Rob-x to Rob-y, and it also continues from Rob-x to Rob-z. Or, equivalently: Rob-y feels exactly as though he was just Rob-x, and Rob-z also feels exactly as though he was just Rob-x (since each of these slightly different people has all the memories, personality traits, etc. of Rob-x — just as though they’d stepped through a doorway).
Scenario 1 is crazy talk, and it’s not the scenario I’m talking about. When I say “You should anticipate having both experiences”, I mean it in the sense of Scenario 2.
Scenario 2 is pretty unfamiliar to us, because we don’t currently live in a world where we can readily copy-paste our own brains. And accordingly, it’s a bit awkward to talk about Scenario 2; the English language is adapted to a world where “humans don’t fork” has always been a safe assumption.
But there isn’t a mystery about what happens. If you think there’s something mysterious or unknown about what happens when you make two copies of yourself, then I pose the question to you:
What concrete fact about the physical world do you think you’re missing? What are you ignorant of?
Alternatively, if you’re not ignorant of anything, then: how can there be a mystery here? (Versus just “a weird way the world can sometimes end up”.)
- ^
And insofar as it’s your physical brain thinking these thoughts right now, unaltered by any divine revelation, it would have to be a coincidence that this “I would blip out of existence in case A but not case B” hunch is correct. Because the reason your brain has that intuition is a product of the brain’s physical, causal history, and is not the result of you making any observation that’s Bayesian evidence for this mechanism existing.
Your brain is not causally entangled with any mechanism like that; you’d be thinking the same thoughts whether the mechanism existed or not. So while it’s possible that you’re having this hunch for reasons unrelated to the hunch being correct, and yet the hunch be correct anyway, you shouldn’t on reflection believe your own hunch. Any Bayesian evidence for this hypothesis would need to come from some source other than the hunch/intuition.
- “Which Future Mind is Me?” Is a Question of Values by (9 Aug 2024 18:17 UTC; 17 points)
- What are the actual arguments in favor of computationalism as a theory of identity? by (18 Jul 2024 18:44 UTC; 12 points)
- Relativity Theory for What the Future ‘You’ Is and Isn’t by (29 Jul 2024 2:01 UTC; 7 points)
- 's comment on Is the mind a program? by (1 Dec 2024 3:31 UTC; 3 points)
I think there’s a pretty strong argument to be more wary about uploading. It’s been stated a few times on LW, originally by Wei Dai if I remember right, but maybe worth restating here.
Imagine the uploading goes according to plan, the map of your neurons and connections has been copied into a computer, and simulating it leads to a person who talks, walks in a simulated world, and answers questions about their consciousness. But imagine also that the upload is being run on a computer that can apply optimizations on the fly. For example, it could watch the input-output behavior of some NN fragment, learn a smaller and faster NN fragment with the same input-output behavior, and substitute it for the original. Or it could skip executing branches that don’t make a difference to behavior at a given time.
Where do we draw the line which optimizations to allow? It seems we cannot allow all behavior-preserving optimizations, because that might lead to a kind of LLM that dutifully says “I’m conscious” without actually being so. (The p-zombie argument doesn’t apply here, because there is indeed a causal chain from human consciousness to an LLM saying “I’m conscious”—which goes through the LLM’s training data.) But we must allow some optimizations, because today’s computers already apply many optimizations, and compilers even more so. For example, skipping unused branches is pretty standard. The company doing your uploading might not even tell you about the optimizations they use, given that the result will behave just like you anyway, and the 10x speedup is profitable. The result could be a kind of apocalypse by optimization, with nobody noticing. A bit unsettling, no?
The key point of this argument isn’t just that some optimizations are dangerous, but that we have no principled way of telling which ones are. We thought we had philosophical clarity with “just upload all my neurons and connections and then run them on a computer”, but that doesn’t seem enough to answer questions like this. I think it needs new ideas.
Yeah, at some point we’ll need a proper theory of consciousness regardless, since many humans will want to radically self-improve and it’s important to know which cognitive enhancements preserve consciousness.
Yeah. My point was, we can’t even be sure which behavior-preserving optimizations (of the kind done by optimizing compilers, say) will preserve consciousness. It’s worrying because these optimizations can happen innocuously, e.g. when your upload gets migrated to a newer CPU with fancier heuristics. And yeah, when self-modification comes into the picture, it gets even worse.
[Epistemic status: napkin]
My current-favourite frame on “qualia” is that it refers to the class of objects we can think about (eg, they’re part of what generates what I say rn) for which behaviour is invariant across structure-preserving transformations.
(There’s probably some cool way to say that with category theory or transformations, and it may or may not give clarity, but idk.)
Eg, my “yellow” could map to blue, and “blue” to yellow, and we could still talk together without noticing anything amiss even if your “yellow” mapped to yellow for you.
Both blue and yellow are representational objects, the things we use to represent/refer to other things with, like memory-addresses in a machine. For externally observable behaviour, it just matters what they dereference to, regardless of where in memory you put them. If you swap two representational objects, while ensuring you don’t change anything about how your neurons link up to causal nodes outside the system, your behaviour stays the same.
Note that this isn’t the case for most objects. I can’t swap hand⇄tomato, without obvious glitches like me saying “what a tasty-looking tomato!” and trying to eat my hand. Hands and tomatoes do not commute.
It’s what allows us to (try to) talk about “tomato” as opposed to just tomato, and explains why we get so confused when we try to ground out (in terms of agreed-upon observables) what we’re talking about when we talk about “tomato”.
But how/why do we have representations for our representational objects in the first place? It’s like declaring a var (address₁↦value), and then declaring a var for that var (address₂↦address₁) while being confused about why the second dereferences to something ‘arbitrary’.
Maybe it starts when somebody asks you “what do you mean by ‘X’?”, and now you have to map the internal generators of [you saying “X”] in order to satisfy their question. Or not. Probably not. Napkin out.
We can use the same thought experiments that Chalmers uses to establish a fine-grain-functionally-isomorphic copy had the same qualia, modify them and show that anything that acts like us has our qualia.
The LLM character (rather than the LLM itself) will be conscious to the extent to which its behavior is I/O identical to the person.
Edit: Oh, sorry, this is an old comment. I got this recommended… somehow...
Edit2: Oh, it was curated yesterday.
Well, a thing that acts like us in one particular situation (say, a thing that types “I’m conscious” in chat) clearly doesn’t always have our qualia. Maybe you could say that a thing that acts like us in all possible situations must have our qualia? This is philosophically interesting! It makes a factual question (does the thing have qualia right now?) logically depend on a huge bundle of counterfactuals, most of which might never be realized. What if, during uploading, we insert a bug that changes our behavior in one of these counterfactuals—but then the upload never actually runs into that situation in the course of its life—does the upload still have the same qualia as the original person, in situations that do get realized? What if we insert quite many such bugs?
Moreover, what if we change the situations themselves? We can put the upload in circumstances that lead to more generic and less informative behavior: for example, give the upload a life where they’re never asked to remember a particular childhood experience. Or just a short life, where they’re never asked about anything much. Let’s say the machine doing the uploading is aware of that, and allowed to optimize out parts that the person won’t get to use. If there’s a thought that you sometimes think, but it doesn’t influence your I/O behavior, it can get optimized away; or if it has only a small influence on your behavior, a few bits’ worth let’s say, then it can be replaced with another thought that would cause the same few-bits effect. There’s a whole spectrum of questionable things that people tend to ignore when they say “copy the neurons”, “copy the I/O behavior” and stuff like that.
Right, that’s what I meant.
Thank you!
The I/O behavior being the same is a sufficient condition for it to be our mind upload. A sufficient condition for it to have some qualia, as opposed for it to have our mind and our qualia, will be weaker.
Then it’s, to a very slight extent, another person (with the continuum between me and another person being gradual).
Then the qualia would be very slightly different, unless I’m missing something. (To bootstrap the intuition, I would expect my self that chooses vanilla ice-cream over chocolate icecream in one specific situation to have very slightly different feelings and preferences in general, resulting in very slightly different qualia, even if he never encounters that situation.) With many such bugs, it would be the same, but to a greater extent.
I don’t think such thoughts exist (I can always be asked to say out loud what I’m thinking). Generally, I would say that a thought that never, even in principle, influences my output, isn’t possible. (The same principle should apply to trying to replace a thought just by a few bits.)
Such optimizations are a reason I believe we are not in a simulation. Optimizations are essential for a large sim. I expect them not to be consciousness preserving
Well, even if we reliably know that certain optimizations make copies not conscious, some people may want to run optimized versions of themselves that are not conscious. People are already making LLMs of themselves based on their writings and stuff. I think Age of Em doesn’t discuss this specific case, but collectives of variously modified Ems may perform better (if only for being cheaper) if they are not conscious. Humans Who Are Not Concentrating Are Not General Intelligences and often not conscious. I’m not conscious when I’m deeply immersed in some subject and only hours later realize how much time has passed—and how much I got done. It’s a kind of automation. Why not run it intentionally?
Surely ‘you’ are the algorithm, not the implementation. If I get refactored into a giant lookup table, I don’t think that makes the algorithm any less ‘me’.
I find myself strongly disagreeing with what is being said in your post. Let me preface by saying that I’m mostly agnostic with respect to the possible “explanations” of consciousness etc, but I think I fall squarely within camp 2. I say mostly because I lean moderately towards physicalism.
First, an attempt to describe my model of your ontology:
You implicitly assume that consciousness / subjective experience can be reduced to a physical description of the brain, which presumably you model as a classical (as opposed to quantum) biological electronic circuit. Physically, to specify some “brain-state” (which I assume is essentially the equivalent of a “software snapshot” in a classical computer) you just need to specify a circuit connectivity for the brain, along with the currents and voltages between the various parts of the circuit (between the neurons let’s say). This would track with your mentions of reductionism and physicalism and the general “vibe” of your arguments. In this case I assume you treat conscious experience roughly as “what it feels like” to be software that is self-referential on top of taking in external stimuli from sensors. This software is instantiated on a biological classical computer instead of a silicon-based one.
With this in mind, we can revisit the teleporter scenario. Actually, let’s consider a copier instead of a teleporter, in the sense that you dont destroy the original after finishing the procedure. Then, once a copy is made, you have two physical brains that have the same connectivity, the same currents and the same voltages between all appropriate positions. Therefore, based on the above ontology, the brains are physically the same in all the ways that matter and thus the software / the experience is also the same. (Since software is just an abstract “grouping” which we use to refer to the current physical state of the hardware)
Assuming this captures your view, let me move on to my disagreements:
My first issue with your post is that this initial ontological assumption is neither mentioned explicitly nor motivated. Nothing in your post can be used as proof of this initial assumption. On the contrary, the teleporter argument, for example, becomes simply a tautology if you start from your premise—it cannot be used to convince someone that doesn’t already subscribe to your views on the topic. Even worse, it seems to me that your initial assumption forces you to contort (potential) empirical observation to your ontology, instead of doing the opposite.
To illustrate, let’s assume we have the copier—say it’s a room you walk into, you get scanned and then a copy is reconstructed in some other room far away. Since you make no mention of quantum, I guess this can be a classical copy, in the sense that it can copy essentially all of the high-level structure, but it cannot literally copy the positions of specific electrons, as this is physically impossible anyways. Nevertheless, this copier can be considered “powerful” enough to copy the connectivity of the brain and the associated currents and voltages. Now, what would be the experience of getting copied, seen from a first-person, “internal”, perspective? I am pretty sure it would be something like: you walk into the room, you sit there, you hear say the scanner working for some time, it stops, you walk out. From my agnostic perspective, if I were the one to be scanned it seems like nothing special would have happened to me in this procedure. I didnt feel anything weird, I didnt feel my “consciousness split into two” or something. Namely, if I consider this procedure as an empirical experiment, from my first person perspective I dont get any new / unexpected observation compared to say just sitting in an ordinary room. Even if I were to go and find my copy, my experience would again be like meeting a different person which just happens to look like me and which claims to have similar memories up to the point when I entered the copying room. There would be no way to verify or to view things from their first person perspective.
At this point, we can declare by fiat that me and my copy are the same person / have the same consciousness because our brains, seen as classical computers, have the same structure, but this experiment will not have provided any more evidence to me that this should be true. On the contrary, I would be wary to, say, kill myself or to be destroyed after the copying procedure, since no change will have occured to my first person perspective, and it would thus seem less likely that my “experience” would somehow survive because of my copy.
Now you can insist that philosophically it is preferable to assume that brains are classical computers etc, in order to retain physicalism which is preferable to souls and cartesian dualism and other such things. Personally, I prefer to remain undecided, especially since making the assumption brain= classical hardware, consciousness=experience as software leads to weird results. It would force me to conclude that the copy is me even though I cannot access their first person perspective (which defeats the purpose) and it would also force me to accept that even a copy where the “circuit” is made of water pipes and pumps, or gears and levers also have an actual, first person experience as “me”, as long as the appropriate computations are being carried out.
One curious case where physicalism could be saved and all these weird conclusions could be avoided would be if somehow there is some part of the brain which does something quantum, and this quantum part is the essential ingredient for having a first person experience. The essence would be that, because of the no-cloning theorem, a quantum-based consciousness would be physically impossible to copy, even in theory. This would get around all the problems which come with the copyability implicit in classical structures. The brain would then be a hybrid of classical and quantum parts, with the classical parts doing most of the work (since neural networks which can already replicate a large part of human abilities are classical) with some quantum computation mixed in, presumably offering some yet unspecified fitness advantage. Still, the consensus is that it is improbable that quantum computation is taking place in the brain, since quantum states are extremely “fragile” and would decohere extremely rapidly in the environment of the brain...
There are always going to be many different ways someone could object to a view. If you were a Christian, you’d perhaps be objecting that the existence of incorporeal God-given Souls is the real crux of the matter, and if I were intellectually honest I’d be devoting the first half of the post to arguing against the Christian Soul.
Rather than trying to anticipate these objections, I’d rather just hear them stated out loud by their proponents and then hash them out in the comments. This also makes the post less boring for the sorts of people who are most likely to be on LW: physicalists and their ilk.
Why do you assume that you wouldn’t experience the copy’s version of events?
The un-copied version of you experiences walking into the room, sitting there, hearing the scanner working, and hearing it stop; then that version of you experiences walking out. It seems like nothing special happened in this procedure; this version of you doesn’t feel anything weird, and doesn’t feel like their “consciousness split into two” or anything.
The copied version of you experiences walking into the room, sitting here, hearing the scanner working, and then an instantaneous experience of (let’s say) feeling like you’ve been teleported into another room—you’re now inside the simulation. Assuming the simulation feels like a normal room, it could well seem like nothing special happened in this procedure—it may feel like blinking and seeing the room suddenly change during the blink, while you yourself remain unchanged. This version of you doesn’t necessarily feel anything weird either, and they don’t feel like their “consciousness split into two” or anything.
It’s a bit weird that there are two futures, here, but only one past—that the first part of the story is the same for both versions of you. But so it goes; that just comes with the territory of copying people.
If you disagree with anything I’ve said above, what do you disagree with? And, again, what do you mean by saying you’re “pretty sure” that you would experience the future of the non-copied version?
Sure. But is any of this Bayesian evidence against the view I’ve outlined above? What would it feel like, if the copy were another version of yourself? Would you expect that you could telepathically communicate with your copy and see things from both perspectives at once, if your copies were equally “you”? If so, why?
Shall we make a million copies and then take a vote? :)
I agree that “I made a non-destructive software copy of myself and then experienced the future of my physical self rather than the future of my digital copy” is nonzero Bayesian evidence that physical brains have a Cartesian Soul that is responsible for the brain’s phenomenal consciousness; the Cartesian Soul hypothesis does predict that data. But the prior probability of Cartesian Souls is low enough that I don’t think it should matter.
You need some prior reason to believe in this Soul in the first place; the same as if you flipped a coin, it came up heads, and you said “aha, this is perfectly predicted by the existence of an invisible leprechaun who wanted that coin to come up heads!”. Losing a coinflip isn’t a surprising enough outcome to overcome the prior against invisible leprechauns.
Why wouldn’t it? What do you have against water pipes?
First off, would you agree with my model of your beliefs? Would you consider it an accurate description?
Also, let me make clear that I don’t believe in cartesian souls. I, like you, lean towards physicalism, I just don’t commit to the explanation of consciousness based on the idea of the brain as a **classical** electronic circuit. I don’t fully dismiss it either, but I think it is worse on philosophical grounds than assuming that there is some (potentially minor) quantum effect going on inside the brain that is an integral part of the explanation for our conscious experience. However, even this doesn’t feel fully satisfying to me and this is why I say that I am agnostic. When responding to my points, you can assume that I am a physicalist, in the sense that I believe consciousness can probably be described using physical laws, with the added belief that these laws **may** not be fully understandable by humans. I mean this in the same way that a cat for example would not be able to understand the mechanism giving rise to consciousness, even if that mechanism turned out to be based on the laws of classical physics (for example if you can just explain consciousness as software running on classical hardware).
To expand upon my model of your beliefs, it seems to me that what you do is that you first reject cartesian souls and other such things on philosophical grounds and you thus favour physicalism. I agree on this. However I dont see why you are immediately assuming that physicalism means that your consciousness must be a result of classical computation. It could be the result of quantum computation. It could be something even subtler in some deeper theory of physics. At this point you may say that a quantum explanation may be more “unlikely” than a classical one, but I think that we both can agree that the “absurdity distance” between the two is much smaller than say a classical explanation and a soul-based one, and thus we now have to weigh the two much options much more carefully since we cannot dismiss one in favour of the other as easily. What I would like to argue is that a quantum-based consciousness is philosophically “nicer” than a classical one. Such an explanation does not violate physicalism, while at the same time rendering a lot of points of your post invalid.
Let’s start by examining the copier argument again but now with the assumption that conscious experience is the result of quantum effects in the brain and see where it takes us. In this case, to fully copy a consciousness from one place to another you would have to copy an unknown quantum state. This is physically impossible even in theory, based on the no-cloning theorem. Thus the “best” copier that you can have is the copier from my previous comment, which just copies the classical connectivity of the brain and all the current and voltages etc, but which now fails to copy the part that is integral to **your** first person experience. So what would be your first person experience if you were to enter the room? You would just go in, hear the scanner work, get out. You can do this again and again and again and always find yourself experiencing getting out of the same initial room. At the same time the copier does create copies of you, but they are new “entities” that share the same appearance as you and which would approximate to some (probably high) degree your external behaviour. These copies may or may not have their own first person experience (and we can debate this further) but this does not matter for our argument. Even if they have a first person experience, it would be essentially the same as the copier just creating entirely new people while leaving your first person experience unchanged. In this way, you can step into the room with zero expectation that you may walk out of a room on the other side of the copier, in the same way that you dont expect to suddenly find yourself in some random stranger’s body while going about your daily routine. Even better, this belief is nicely consistent with physicalism, while still not violating our intuitions that we have private and uncopiable subjective experiences. It also doesn’t force us to believe that a bunch of water pipes or gears functioning as a classical computer can ever have our own first person experience. Going even further, unknown quantum states may not be copyable but they are transferable (see quantum teleportation etc), meaning that while you cannot make a copier you can make a transporter, but you always have to be at only one place at each instant.
Let me emphasize again that I am not arguing **for** quantum consciousness as a solution. I am using it as an example that a “philosophically nicer” physicalist option exists compared to what I assume you are arguing for. From this perspective, I don’t see why you are so certain about the things you write in your post. In particular, you make a lot of arguments based on the properties of “physics”, which in reality are properties of classical physics together with your assumption that consciousness must be classical. When I said that I find issue with the fact that you start from an unstated assumption, I didnt expect you to argue against cartesian dualism. I expected you to start from physicalism and then motivate why you chose to only consider classical physics. Otherwise, the argumentation in your post seems lacking, even if I start from the physicalist position. To give one example of this:
You say that “there isn’t an XML tag in the brain saying `this is a new brain, not the original`” . By this I assume you mean that the physical state of the brain is fungible, it is copyable, there is nothing to serve as a label. But this is not a feature of physics in general. An unknown quantum state cannot be copied, it is not fungible. My model of what you mean: “(I assume that) first person experience can be fully attributed to some structure of the brain as a classical computer. It can be fully described by specifying the connectivity of the neurons and the magnitudes of the currents and voltages between each point. Since (I assume) consciousness physically manifests as a classical pattern and since classical patterns can be copied, then by definition there can be many copies of “the same” consciousness”. Thus, what you write about XML tags is not an argument for your position—it is not imposed to you by physics to consider a fungible substrate for consciousness - it is just a manifestation of your assumption. It’s cyclical. A lot of your arguments which invoke “physics” are like that.
Why would the laws of physics conspire to vindicate a random human intuition that arose for unrelated reasons?
We do agree that the intuition arose for unrelated reasons, right? There’s nothing in our evolutionary history, and no empirical observation, that causally connects the mechanism you’re positing and the widespread human hunch “you can’t copy me”.
If the intuition is right, we agree that it’s only right by coincidence. So why are we desperately searching for ways to try to make the intuition right?
Why is this an advantage of a theory? Are you under the misapprehension that “hypothesis H allows humans to hold on to assumption A” is a Bayesian update in favor of H even when we already know that humans had no reason to believe A? This is another case where your theory seems to require that we only be coincidentally correct about A (“sufficiently complex arrangements of water pipes can’t ever be conscious”), if we’re correct about A at all.
One way to rescue this argument is by adding in an anthropic claim, like: “If water pipes could be conscious, then nearly all conscious minds would be instantiated in random dust clouds and the like, not in biological brains. So given that we’re not Boltzmann brains briefly coalescing from space dust, we should update that giant clouds of space dust can’t be conscious.”
But is this argument actually correct? There’s an awful lot of complex machinery in a human brain. (And the same anthropic argument seems to suggest that some of the human-specific machinery is essential, else we’d expect to be some far-more-numerous observer, like an insect.) Is it actually that common for a random brew of space dust to coalesce into exactly the right shape, even briefly?
You’re missing the bigger picture and pattern-matching in the wrong direction. I am not saying the above because I have a need to preserve my “soul” due to misguided intuitions. On the contrary, the reason for my disagreement is that I believe you are not staring into the abyss of physicalism hard enough. When I said I’m agnostic in my previous comment, I said it because physics and empiricism lead me to consider reality as more “unfamiliar” than you do (assuming that my model of your beliefs is accurate). From my perspective, your post and your conclusions are written with an unwarranted degree of certainty, because imo your conception of physics and physicalism is too limited. Your post makes it seem like your conclusions are obvious because “physics” makes them the only option, but they are actually a product of implicit and unacknowledged philosophical assumptions, which (imo) you inherited from intuitions based on classical physics. By this I mean the following:
It seems to me that when you think about physics, you are modeling reality (I intentionally avoid the word “universe” because it evokes specific mental imagery) as a “scene” with “things” in it. You mentally take the vantage point of a disembodied “observer/narrator/third person” observing the “things” (atoms, radiation etc) moving, interacting according to specific rules and coming together to create forms. However, you have to keep in mind that this conception of reality as a classical “scene” that is “out there” is first and foremost a model, one that is formed from your experiences obtained by interacting specifically with classical objects (biliard balls, chairs, water waves etc). You can extrapolate from this model and say that reality truly is like that, but the map is not the territory, so you at least have to keep track of this philosophical assumption. And it is an assumption, because “physics” doesn’t force you to conclude such a thing. Seen through a cautious, empirical lens, physics is a set of rules that allows you to predict experiences. This set of rules is produced exclusively by distilling and extrapolating from first-person experiences. It could be (and it probably is) the case that reality is ontologically far weirder than we can conceive, but that it still leads to the observed first-person experiences. In this case, physics works fine to predict said experiences, and it also works as an approximation of reality, but this doesn’t automatically mean that our (merely human) conceptual models are reality. So, if we want to be epistemically careful, we shouldn’t think “An apple is falling” but instead “I am having the experience of seeing an apple fall”, and we can add extra philosophical assumptions afterwards. This may seem like I am philosophizing too much and being too strict, but it is extremely important to properly acknowledge subjective experience as the basis for our mental models, including that of the observer-independent world of classical physics. This is why the hard problem of consciousness is called “hard”. And if you think that it should “obviously” be the other way around, meaning that this “scene” mental model is more fundamental than your subjective experiences, maybe you should reflect on why you developed this intuition in the first place. (It may be through extrapolating too much from your (first-person, subjective) experiences with objects that seemingly possess intrinsic, observer-independent properties, like the classical objects of everyday life.)
At this point it should be clearer why I am disagreeing with your post. Consciousness may be classical, it may be quantum, it may be something else. I have no issue with not having a soul and I don’t object to the idea of a bunch of gears and levers instantiating my consciousness merely because I find it a priori “preposterous” or “absurd” (though it is not a strong point of your theory). My issue is not with your conclusion, it’s precisely with your absolute certainty, which imo you support with cyclical argumentation based on weak premises. And I find it confusing that your post is receiving so much positive attention on a forum where epistemic hygiene is supposedly of paramount importance.
So in reading your comments of this post, I feel like I am reading comments made by a clone of my own mind. Though you articulate my views better than I can. This particular comment you make, I don’t think it gets The attention it deserves. It was pretty revolutionary for myself when I learned to think of almost every worldview at a model of reality. It’s most revolutionary when one realizes what is arguably an outdated Newtonian view to fall into this category of model. It really highlights that actual reality is at the least very hard to get at. This is a severe an issue with regards to consciousness.
Are you trying to say that quantum physics provides evidence that physical reality is subjective, with conscious observers having a fundamental role? Rob implicitly assumes the position advocated by The Quantum Physics Sequence, which argues that reality exists independently of observers and that quantum stuff doesn’t suggest otherwise. It’s just one of the many presuppositions he makes that’s commonly shared on here. If that’s your main objection, you should make that clear.
I would say that it is irrelevant for the points the post/Rob is trying to make whether consciousness is classical or quantum, given that conscious experience has, AFAIK, never been reported to be ‘quantum’ (i.e. that we don’t seem to experience superpositions or entanglement) and that we already have straightforward classical examples of lack of conscious continuity (namely: sleeping).
In the case of sleeping and waking up it is already clear that the currently awake consciousness is modeling its relation to past consciousnesses in that body through memories alone. Even without teleporters, copiers, or other universes coming into play, this connection is very fragile. How sure can a consciousness be that it is the same as the day before or as one during lucid parts of dreams? If you add brain malfunctions such as psychoses or dissociative drugs such as ketamine to the mix, the illusion of conscious continuity can disappear completely quite easily.
I like to word it like this: A consciousness only ever experiences what the brain that produces it can physically sense or synthesize.
With that as a starting point, modeling what will happen in the various thought experiments and analyses of conscious experience becomes something like this: “Given that there is a brain there, it will produce a consciousness, which will remember what is encoded in the structure of that brain and which will experience what that brain senses and synthesizes in that moment.”
There is no assumption that consciousness is classical in that, I believe. There is also no assumption of continuity in that, which I think is important as in my opinion that assumption is quite shaky and misdirects many discussions on consciousness. I’d say that the value in the post is in challenging that assumption.
The currently awake consciousness is located in the brain, which has physical continuity with its previous states. You don’t wake up as a different person because “you” are the brain (possibly also the rest of the body depending how it affects cognition but IDK) and the brain does not cease to function when you fall asleep.
I agree on the physical continuity of the brain, but I don’t think this transfers to continuity of the consciousness or its experience. It is defining “you” as that physical brain, rather than the conscious experience itself. It’s like saying that two waves are the same because they are produced by the same body of water.
Imagine significant modifications to your brain while you are asleep in such a way that your memories are vastly different, so much as to represent another person. Would the consciousness that is created on waking up experience a connection to the consciousness that that brain produced the day(s) before or to the manufactured identity?
Even you, now, without modifications, can’t say with certainty that your ‘yesterday self’ was experienced by the same consciousness as you are now (in the sense of identity of the conscious experience). It feels that way as you have memories of those experiences, but it may have been experienced by ‘someone else’ entirely. You have no way of discerning that difference (nor does anyone else).
The conscious experience is not extricable from the physical brain; it has your personality because the personality that you are is the sum total of everything in your brain. The identity comes from the brain; if it were somehow possible to separate consciousness from the rest of the mind, that consciousness wouldn’t still be you, because you’re the entire mind.
I would… not consider the sort of brain modification you’re describing to preserve physical continuity in the relevant sense? It sounds like it would, to create the described effects, involve significant alterations in portions of the brain wherein (so to speak) your identity is stored, which is not what normally happens when people sleep.
I think we are in agreement that the consciousness is tied to the brain. Claiming equivalency is not warranted, though: The brain of a dead person (very probably, I’m sure you’d agree) contains no consciousness. Let’s not dwell on this, though: I am definitely not claiming that consciousness exists outside of the brain, just that asserting physical continuity of the brain is not enough by itself to show continuity of conscious experience.
With regard to the modifications: Your line of reasoning runs into the classic issues of philosophical identity, as shown by the Ship of Theseus thought experiment or simpler yet, the Sorites paradox. We can hypothesize every amount of alterations from just modifying one atom to replacing the entire brain. Given your position, you’d be forced to choose an arbitrary amount of modifications that breaks the continuity and somehow changes consciousness A-modified-somewhat into consciousness B (or stated otherwise: from ‘you waking up a somewhat changed person’ to ‘someone else waking up in your body’).
Approaching conscious experience without the assumption of continuity but from the moment it exists in does not run into this problem.
(Assuming a frame of materialism, physicalism, empiricism throughout even if not explicitly stated)
Some of your scenarios that you’re describing as objectionable would reasonably be described as emulation in an environment that you would probably find disagreeable even within the framework of this post. Being emulated by a contraption of pipes and valves that’s worse in every way than my current wetware is, yeah, disagreeable even if it’s kinda me. Making my hardware less reliable is bad. Making me think slower is bad. Making it easier for others to tamper with my sensors is bad. All of these things are bad even if the computation faithfully represents me otherwise.
I’m mostly in the same camp as Rob here, but there’s plenty left to worry about in these scenarios even if you don’t think brain-quantum-special-sauce (or even weirder new physics) is going to make people-copying fundamentally impossible. Being an upload of you that now needs to worry about being paused at any time or having false sensory input supplied is objectively a worse position to be in in.
The evidence does seem to lean in the direction that non-classical effects in the brain are unlikely, neurons are just too big for quantum effects between neurons, and even if there were quantum effects within neurons, it’s hard to imagine them being stable for even as long as a single train of thought. The copy losing their train of thought and having momentary confusion doesn’t seem to reach the bar where they don’t count as the same person? And yet weirder new physics mostly requires experiments we haven’t thought to do yet, or experiments is regimes we’ve not yet been able to test. Whereas the behavior of things at STP in water is about as central to things-Science-has-pinned-down as you’re going to get.
You seem to hold that the universe maybe still has a lot of important surprises in store, even within the central subject matter of century old fields? Do you have any kind of intuition pump for that feeling there’s still that many earth-shattering surprises left (while simultaneously holding empiricism and science mostly work)? My sense of where there’s likely to be surprises left is not quite so expansive and this sounds like a crux for a lot of people. Even as much of a shock as qm was to physics, it didn’t invalidate much if any theory except in directly adjacent fields like chemistry and optics. And working out the finer points had progressively more narrower and shorter reaching impact. I can’t think of examples of surprises with a larger blast radius within the history of vaguely modern science. Findings of odd as yet unexplained effects pretty consistently precedes attempts at theory. Empirically determined rules don’t start working any worse when we realize the explanation given with them was wrong.
Keep in mind that society holds that you’re still you even after a non-trivial amount of head trauma. So whatever amount of imperfection in copying your unknown-unknowns cause, it’d have to both be something we’ve never noticed before in a highly studied area, and something more disruptive than getting clocked in the jaw, which seems a tall order.
Keep in mind also that the description(s) of computation that computer science has worked out is extremely broad and far from limited to just electronic circuits. Electronics are pervasive because we have as a society sunk the world GDP (possibly several times over) into figuring out how to make them cheaply at scale. Capital investment is the only thing special about computers realized in silicon. Computer science makes no such distinction. The notion of computation is so broad that there’s little if any room to conceive of an agent that’s doing something that can’t be described as computation. Likewise the equivalence proofs are quite broad; it can arbitrarily expensive to translate across architectures, but within each class of computers, computation is computation, and that emulation is possible has proofs.
All of your examples are doing that thing where you have a privileged observer position separate and apart from anything that could be seeing or thinking within the experiment. You-the-thinker can’t simply step into the thought experiment. You-the-thinker can of course decide where to attach the camera by fiat, but that doesn’t tell us anything about the experiment, just about you and what you find intuitive.
Suppose for sake of argument your unknown unknowns mean your copy wakes up with a splitting headache and amnesia for the previous ~12 hours as if waking up from surgery. They otherwise remember everything else you remember and share your personality such that no one could notice a difference (we are positing a copy machine that more or less works). If they’re not you they have no idea who else they could be, considering they only remember being you.
The above doesn’t change much for me, and I don’t think I’d concede much more without saying you’re positing a machine that just doesn’t work very well. It’s easy for me to imagine it never being practical to copy or upload a mind, or having modest imperfections or minor differences in experience, especially at any kind of scale. Or simply being something society at large is never comfortable pursuing. It’s a lot harder to imagine it being impossible even in principle with what we already know, or can already rule out with fairly high likelihood. I don’t think most of the philosophy changes all that much if you consider merely very good copying (your friends and family can’t tell the difference; knows everything you know) vs perfect copying.
The most bullish folks on LLMs seem to think we’re going to be able to make copies good enough to be useful to businesses just off all your communications. I’m not nearly so impressed with the capabilities I’ve seen to date and it’s probably just hype. But we are already getting into an uncanny valley with the (very) low fidelity copies current AI tech can spit out—which is to say they’re already treading on the outer edge of peoples’ sense of self.
Eliezer talked about some puzzles related to copying and anticipation in The Anthropic Trilemma that still seem quite mysterious to me. See also my comment on that post.
If we can clone ourselves, language would probably quickly follow. The bigger change would probably be the one about social reality. What does it mean to make a promise? Who is the entity you make a trade with? Is it the collective of all the yous? Only one? But which one if they split? The yous resulting from one origin will presumably have to share or split their resources. How will they feel about it? Will they compete or agree? If they agree it makes more sense for them to feel more like a distributed being. The thinking of “I” might get replaced by an “us”.
Reproduction and evolution is arguably literally how biology forks. You can also think of it as a tree of consciousness. Cloning would probably work the same in regards to consciousness. My clone would be a distinct branch of consciousness.
There’s little reason to think thought experiment level precision of cloning/duplication and teleportation are at all physically possible though.
It seems like the way we talk about the results of a coin flip would be a good start for how we’d talk about being cloned, although it’s rare for a coin flip to have such a massive impact on our life after that point
Naturalism and reductionism are not sufficient to rigourously prove either form of computationalism—that performing a certain class of computations is sufficient to be conscious in general, or that performing a specific one is sufficient to be a particular conscious individual.
This has been going on for years: most rationalists believe in computationalism, none have a really good reason to.
Arguing down Cartesian dualism (the thing rationalists always do) doesn’t increase the probability of computationalism, because there are further possibilities , including physicalism-without-computationalism (the one rationalists keep overlooking) , and scepticism about consciousness/identity.
One can of course adopt a belief in computationalism, or something else, in the basis of intuitions or probabilities. But then one is very much in the ream of Modest Epistemology, and needs to behave accordingly.
“My issue is not with your conclusion, it’s precisely with your absolute certainty, which imo you support with cyclical argumentation based on weak premises”.
Yep.
If either kind of consciousness depends on physical brain states, computationalism is false. That is the problem that has rarely been recognised, and never addressed.
There’s another option: door-Rob has physical continuity. There’s an analogy with the identity-over-time of physical objects: if someone destroyed the Mona Lisa, and created an atom-by-atom duplicate some time later, the duplicate would not be considered the same entity (numerical identity).
That’s not a strong enough argument. There isn’t an XML tag on the copy of the Mona Lisa, but it’s still a copy.
There is, and its multi-way splitting, whether through copying or many worlds branching. The present you can’t anticipate having all their experiences, because experience is experienced one-at-a-time. They can all look back at their memories, and conclude that they were you, but you can’t simply reverse that and conclude that you will be them , because the set-up is asymmetrical.
But that isn’t an experience. It’s two experiences. You will not have an experience of having two experiences. Two experiences will experience having been one person.
Are you going to care about 1000 different copies equally?
Sure; from my perspective, you’re saying the same thing as me.
How am I supposed to choose between them?
By “equally” I meant:
“in the same ways (and to the same degree)”.
If you actually believe in florid many worlds, you would end up pretty insuoucient, since everything possible happens, and nothing can be avoided.
What does it mean when one “should anticipate” something? At least in my mind, it points strongly to a certain intuition, but the idea behind that intuition feels confused. “Should” in order to achieve a certain end? To meet some criterion? To boost a term in your utility function?
I think the confusion here might be important, because replacing “should anticipate” with a less ambiguous “should” seems to make the problem easier to reason about, and supports your point.
For instance, suppose that you’re going to get your brain copied next week. After you get copied, you’ll take a physics test, and your copy will take a chemistry test (maybe this is your school’s solution to a scheduling conflict during finals). You want both test scores to be high, but you expect taking either test without preparation will result in a low score. Which test should you prepare for?
It seems clear to me that you should prepare for both the chemistry test and the physics test. The version of you that got copied will be able to use the results of the physics preparation, and the copy will be able to use the copied results of the chemistry preparation. Does that mean you should anticipate taking a chemistry test and anticipate taking a physics test? I feel like it does, but the intuition behind the original sense of “should anticipate” seems to squirm out from under it.
I have a closely related objection/clarification. I agree with the main thrust of Rob’s post, but this part:
..strikes me as confused or at least confusing.
Take your chemistry/physics tests example. What does “I anticipate the experience of a sense of accomplishment in answering the chemistry test” mean? Well for one thing, it certainly indicates that you believe the experience is likely to happen (to someone). For another, it often means that you believe it will happen to you—but that invites the semantic question that Rob says this isn’t about. For a third—and I propose that this is a key point that makes us feel there is a “substantive” question here—it indicates that you empathize with this future person who does well on the test.
But I don’t see how empathizing or not-empathizing can be assessed for accuracy. It can be consistent or inconsistent with the things one cares about, which I suppose makes it subject to rational evaluation, but that looks different from accuracy/inaccuracy.
In the OP: “Should” in order to have more accurate beliefs/expectations. E.g., I should anticipate (with high probability) that the Sun will rise tomorrow in my part of the world, rather than it remaining night.
Suppose someone draws a “personal identity” line to exclude this future sunrise-witnessing person. Then if you claim that, by not anticipating, they are degrading the accuracy of the sunrise-witness’s beliefs, they might reply that you are begging the question.
Here’s a thought experiment.
In version A, I have a button that non invasively scans my brain and creates 10 perfect copies of my brain state in a computer. I press the button. For an instant, 11 identical mind states exist in the universe. Then each mind starts diverging along different causal chains.
Intuitively, I expect the following:
I won’t experience anything unusual after pressing the button (eg, I won’t wake up in a computer). I will still feel that I am in my physical body, in the room with the button
each of the mind copies will feel that they are the ‘one true version of vlad’ and won’t experience the other minds ‘from the inside’. presumably, they will be surprised to be in a computer and not in the room?
if I shut down the computer and kill the 10 minds, I won’t experience anything unusual
In this case, I identify myself with the embodied mind.
In version B, the setup is identical except the scan is destructive. The second I press it, my physical body is destroyed.
Now, what happens to me? There’s no specific reason for me to end up in one of the minds and not the others. But I cannot go to all 10 minds at the same time — I am a single mind with its own casual chain, not a collection of minds.
For instance, imagine each of the 10 minds is caused to feel a different sensation at the same time. There’s nobody to feel all 10 sensations at the same time because the minds are causally isolated. Yet I cannot say that I am feeling a particular sensation and not the others.
So in version B, I still identify myself with the embodied mind, which is destroyed — hence oblivion. Conversely, what happens to the 10 minds if I delete them from the computer? Oblivion.
(This is just my attempt to map my naive intuitions. I have a sense some version of no-self could be the solution, but I’m not there yet. I also feel that naive intuitions fail for Everett branches which is another reason to be suspicious.)
An interesting consequence of your description is that resurrection is possible if you can manage to reconstruct the last brain state of someone who had died. If you go one one step further, then I think it is fairly likely that experience is eternal, since you don’t experience any of the intervening time (akin to your film reel analogy with adding extra frames in between) being dead and since there is no limit to how much intervening time can pass.
*preferably not the last state but some where the person felt normal.
I believe that’s right! Though, if person can be reconstructed from N bits of information, and dead body retains K << N, then we need to save N-K bits (or maybe all N, for robustness) somewhere else.
It’s an interesting question how many bits can be inferred from social networks trace of the person, actually.
Well ultimately no information about the past is truly lost as far as we know. A hyper-advanced civilization could collect all the thermal radiation from earth reflected off of various celestial bodies and recover a near complete history, at least in principle. So I think the more you make it easy for yourself to be reconstructed/resurrected/what have you the sooner it would likely be, and the less alien of an environment you would find yourself in after the fact. Cryo is a good example of having a reasonable expectation of where to end up barring catastrophe since you are preserving a lot of you in good form.
Loved the post and all the comments <3
Here is I think an interesting scenario / thought experiment:
A copy of a person is made while that original person is sleeping on a bed.
The original person is moved to a sofa while still sleeping.
The copy (which is also sleeping) is put in the bed at the exact same position where the original person was.
After a while the original and the copy both wake up and can see each other (we assume they are both completely oblivious to exactly what happened while they were sleeping and that they didn’t dream or they dreamt the same thing, etc...)
At wake-up, based on their own memory of where the original person fell asleep, the original person will likely feel they are the copy and the copy will likely feel they are the original person, wouldn’t they?!
Some might even argue that based on stream-of-consciousness continuity the original “me” is actually the copy (because the copy remembers falling asleep in the bed and actually wakes up in the bed as well).
Some others will argue that based on substrate/matter continuity the original “me” is the original person even if their stream-of-consciousness has experienced a discontinuity (remembering falling asleep in the bed but actually waking up on the sofa while seeing an identical person as them waking up in the bed).
I guess it is subjective and a matter of individual preference if the stream-of-consciousness continuity or the substrate continuity is more important to define who the original “me” is.
Some would even argue that in this case there is not actual any firm original “me”, just one “stream-of-consciousness me” and another different “substrate me”.
(The same/similar thought experiment could be done using the direct brain insertion of false memories instead of moving around people while they sleep / are unconscious, in this example an original person could be inserted false memories that they are a copy and vice-versa to manipulate the memory / self-awareness of who the original “me” is, also generally it obviously could be useful when someone is uploaded/copied if they want to alter some memories of their upload/copy for some reason)
Wouldn’t it follow that in the same way you anticipate the future experiences of the brain that you “find yourself in” (i.e. the person reading this) you should anticipate all experiences, i.e. that all brain states occur with the same kind of me-ness/vivid immediacy?
It seems that since there is nothing further than makes the experiences (that are these brains states, in this body that is writing these sentences) in some way special so that they’re “mine” (there is no additional “me-ghost”), then those particular brain states aren’t any different from all the other brain states, of other brains, of other people (or other conscious beings) - and so they should equally be anticipated as existing with the same kind of immediacy and vividness.
I.e. in the same way I anticipate the future experiences of this brain, of the person writing these sentences, I should anticipate all other experiences. Which just means, all brain states exist in the same vivid, for-me way, since there is nothing further to distinguish between them that makes them this vivid, i.e. they all exist HERE-NOW. They are all the same in that sense, they are all equally mine. (But of course, the “me” here isn’t, then, the particular person that I find myself being, but simply the immediacy or the way of being of those experiences themselves, i.e. simply their presence.)
Btw, this is Open Individualism.
What’s the empirical or physical content of this belief?
I worry that this may be another case of the Cartesian Ghost rearing its ugly head. We notice that there’s no physical thingie that makes the Ghost more connected to one experience or the other; so rather than exorcising the Ghost entirely, we imagine that the Ghost is connected to every experience simultaneously.
But in fact there is no Ghost. There’s just a bunch of experience-moments implemented in brain-moments.
Some of those brain-moments resemble other brain-moments, either by coincidence or because of some (direct or indirect) causal link between the brain-moments. When we talk about Brain-1 “anticipating” or “becoming” a future brain-state Brain-2, we normally mean things like:
There’s a lawful physical connection between Brain-1 and Brain-2, such that the choices and experiences of Brain-1 influence the state of Brain-2 in a bunch of specific ways.
Brain-2 retains ~all of the memories, personality traits, goals, etc. of Brain-1.
If Brain-2 is a direct successor to Brain-1, then typically Brain-2 can remember a bunch of things about the experience Brain-1 was undergoing.
These are all fuzzy, high-level properties, which admit of edge cases. But I’m not seeing what’s gained by therefore concluding “I should anticipate every experience, even ones that have no causal connection to mine and no shared memories and no shared personality traits”. Tables are a fuzzy and high-level concept, but that doesn’t mean that every object in existence is a table. It doesn’t even mean that every object is slightly table-ish. A photon isn’t “slightly table-ish”, it’s just plain not a table.
But they don’t have the anticipation-related properties I listed above; so what hypotheses are we distinguishing by updating from “these experiences aren’t mine” to “these experiences are mine”?
Maybe the update that’s happening is something like: “Previously it felt to me like other people’s experiences weren’t fully real. I was unduly selfish and self-centered, because my experiences seemed to me like they were the center of the universe; I abstractly and theoretically knew that other people have their own point of view, but that fact didn’t really hit home for me. Then something happened, and I had a sudden realization that no, it’s all real.”
If so, then that seems totally fine to me. But I worry that the view in question might instead be something tacitly Cartesian, insofar as it’s trying to say “all experiences are for me”—something that doesn’t make a lot of sense to say if there are two brain states on opposite sides of the universe with nothing in common and nothing connecting them, but that does make sense if there’s a Ghost the experiences are all “for”.
I’ll take a stab at explaining this with a simple thought experiment.
Say there are two people, Alice and Bob, each with their own unique brain states.
If Alice’s brain state changes slightly, from getting older, learning something new, losing some neurons to a head injury, etc, she will still be Alice. Changing, adding, or removing a neuron does not change this fact.
Now what if instead part of her brain state was changing slowly to match Bob’s? You could think of this as incrementally removing Alice’s neurons and replacing them with a copy of Bob’s, I find it hard to believe that any discrete small change will make Alice’s conscious experience suddenly disappear, and by the end of it she will have the exact same brain state as Bob.
If you believe that when Bob steps into a teleporter that also makes a copy, they are both the same Bob, then it is reasonable to assume that this transformed Alice is also Bob. Then for the same reason your older self is the same “self” as your younger self, the younger Alice is also Bob. The transition between their brain states doesn’t even need to happen, it just has to be possible. From here it is easy to extrapolate that all brain states are the same “self”.
I would say that Alice’s conscious experience is unlikely to suddenly disappear under this transformation, and that it could even be done in a way so that their experience was continuous.
However, Alice-memories would gradually fade out, Bob-memories would gradually fade in, and thought patterns would slowly shift from Alice-like to Bob-like. At the end, the person would just be Bob. Along the way, I would say that Alice gradually died (using an information-theoretic definition of death). The thing that is odd when imagining this is that Alice never experiences her consciousness fading.
The main thing I think your thought experiment demonstrates is that our sense of self is not solely defined by continuity of consciousness.
I apologize for not getting back to you sooner, I didn’t notice your reply until yesterday. And I apologize for the length of my response, too—I bolded the most important parts.
Re: Whether there is empirical difference between worlds where OI is true and where OI is false. The difference between all experiences being mine and only some being mine is that if all experiences are mine, then they all exist in the same way this experience now exists, i.e. for me (where me = just this immediacy/this-here-now character, i.e. the way it exists, NOT Edralis’s memories, personality etc.). There is no empirical difference in the usual sense, since the way experiences exist cannot be objectively assessed. I can’t be sure that you even have any experiences – this is not something that is available for empirical investigation in the way I can assess e.g. the number of someone’s fingers. And I can’t know that, given there are experiences from that point of view, that they exist in the same way as this experience, does, i.e. for me. That is only clear in those experiences. If I am there, I do ultimately know that I am there (obviously) – but I have no way to know that when experiencing this person, Edralis. So the empirical difference in the usual sense between OI being true and not being true is none. However, there are other than empirical facts. The existential difference between those two worlds is vast. If OI is true, then I (i.e. the thisness, the here-now-this that at least Edralis’s experiences have) am Rob Bensinger, and everybody else – if it’s not true, then I am not. The difference is in the being of those experiences, in how they exist. But since experiences (consciousness) don’t exist empirically (or better: objectively), there is no empirical (objective) difference. There is existential, subjective difference, though.
That is not what I mean when I think about anticipating a future brain state. What I am interested in is not the content of experience, but how those experiences are – i.e. whether they are mine. And by ‘mine’ I don’t mean Edralis’s, I mean that they exist in the same way, i.e. they have the same immediate character, the same this-here-now as this experience that I currently am, of Edralis writing this. Not just that they are immediate (all experiences are, by definition), but that they are immediate in the same way. If there are no souls or ghosts, that means there is just one way in which they can be immediate, i.e. they are all immediate like this experience is.
Do you think we understand each other? Do you see what I’m referring to when I point to that which THIS1 experience-moment, of Edralis writing “THIS1”, and THIS2 experience-moment, of Edralis writing “THIS2” have in common – that is shared totally between those two experiences – but it has nothing to do with the content of those experiences, including their continuity? I mean the underlying “canvas” where the experiences “take place” that however is nothing outside of those experiences. It’s not a Ghost – I mean, you could call it that, but if that is what the Cartesian Ghost is supposed to be, then it’s self-evidently real. But I am not postulating an entity that is somehow additional to experiences. Does it make sense? When you watch your experience, do you see what I mean? The THISNESS which carries in your experience from moment to moment?
From what you say, it seems to me we’re talking past each other; however, I don’t know how to elicit an alignment/understanding in our conceptual frameworks. It seems to me your worldview is very empirical, very physicalist – whereas mine is phenomenology-first. What I see when I “look at the world” is ME, i.e. consciousness assuming different forms, different experiential qualities existing. (There might be something “outside” that I am a picture of.) What you see when you “look at the world” is probably THE WORLD that is NOT yourself. I don’t think you’re wrong—this is a way to conceive of yourself (i.e. identifying yourself with a particular human), but I think you might be missing something—that essential me. However, maybe it’s me whose understanding is lacking – and I really want to understand.
It’s more than that – it’s not just that they are real, but that they are real for me, where “me” is the this-here-now of this experience. That is, the claim is that the this-here-now/presence/immediacy/thisness is the same in all experiences, not just that they all have such a character (which is true by definition). And since the one that is self-evident is indisputably mine, then, if all are the same, all are mine.
One more questionː Can you imagine never having been born as Rob Bensinger, but instead being born as a different person? Does it make sense to imagine a person (or animal, or other conscious being) that would be you that wouldn’t however share any memories or brain-matter with the person that you are, i.e. Rob Bensinger? Because to me, it makes perfect sense – that which I imagine this non-Edralis that is me to have, or rather, the non-Edralis-centered experiences that are me to have, is the same THISNESS, i.e. that which is essentially me. I am Edralis contingently – I didn’t have to be her, I could be someone else – what I am essentially is that THISNESS that her experiences have. If OI is true, then all experiences simply have this same thisness. And since I am the thisness, all experiences are mine, i.e. I am everyone. This is very shocking. It’s outright world-shattering! If it doesn’t sound shocking, it’s either because a person knows it for a long time and they’ve gotten used to it, or they don’t grasp what it means. So—even if you don’t agree with OI, can you imagine what it would mean? Do you see in what sense you could be e.g. Albert Einstein or Eliezer Yudkowsky or Queen Victoria?
Thank you!
There is one important aspect where it does make a difference. A difference in social reality. The brain states progress in a physically determined way. There is no way they could have progressed differently. When a “decision is made” by the brain, then that is fully the result of the inner state and the environment. It could only have happened differently if the contents of the brain had been different—which they were not. They may have been expected to be different by other people (‘s brains), but that is in their map, not in reality. But our society is constructed based on the assumption that things could have been different, that actions are people’s ‘faults’. That is an abstraction that has shown to be useful. Societies that have people who act as if they are agents with free will maybe coordinate better—because it allows feedback mechanisms on their behaviors.
Guys, social reality is one if not the cause of the self:
Robin Hanson:
PubMed: The essential moral self
This is a very interesting paper, thanks.
When faced with confusing conundrums like this, I find it useful to go back to basics: evolutionary psychology. You are a human, that is to say, you’re an evolved intelligence, one evolved as a helpful planning-and-guidance system for a biological organism, specifically a primate. Your purpose, evolutionarily, is to maximize the evolutionary fitness of your genes, i.e. to try your best to pass them on successfully. You have whole bunch of drives/emotions/instincts that were evolved to, on the African Savannah, approximately maximize that fitness. Even in our current rather different environment, while not quite as well tuned to that as they used to be to the Savannah, these still do a pretty good job of that (witness the fact that there are roughly 8 billion of us).
So, is an upload of your mind the same “person”? It (if uploaded correctly) shares your memories, drives, and so forth. it will presumably regard you (the organism, and the copy of your mind running on your biological brain, if the uploading process was non-destructive) as somewhere between itself, an identical twin, and a close blood relative. Obviously you will understand each other very well, at least at first before your experiences diverge.. So it’s presumably likely to be an ally in your (the organism’s) well-being and thus help pass your genes on.
So, is an upload exactly the same thing as your biological mind? No. Is it more similar than an identical twin? Yes. Does the English language have a good set of words to compactly describe this? No.
[Obviously if the mind uploading process is destructive, that makes passing on your genes harder, especially if you haven’t yet had any children, and don’t have any siblings. Freezing eggs or sperm before doing destructive mind uploading seems like a wise precaution.]
I’m still struggling with this. I’m fine with the notion that you could, in theory, teleport a copy of me across the universe and to that copy there would be a sense of continuity. But your essay didn’t convince me that the version of me entering the teleporter would feel that continuity. To make it explicit, say you get into that teleporter and due to a software bug it doesn’t “deconstruct” you up teleportation. Here you are on this end and the technician says “trust me, you were teleported”. He then explains that due to intergalactic law, two of you are not allowed to exist, so the version of you on this side of the teleporter must be euthanized. (a) would you be fine with this, since you know there is a copy of you on the other side? and (b) are you asserting that you have some sort of shared consciousness with the copy? To me it seems clear that while the copy would remember getting into the teleporter, the original version would have no notion of whether teleportation was successful or not.
The key to this koan (at least for me) is undoing the assumption that there can be only one of you. There’s one of you that steps in and one that steps out. And they’re the same you.
What I value about me is the pattern of beliefs, memories, and values. The other me has has an identical brain state, so all of those. It is simply another me. I care about the second one pretty much exactly as much as I care about the same pattern continuing in a more similar location and with more similar molecules instantiating the pattern. That’s because I care far less about where I am and which molecules I’m made of than the pattern of identity in my mind/brain.
The same as you can have two of anything else that’s close-enough-for-the-purpose. I can have two rocks if I don’t care about the difference in their molecular makeup. I can have two mes.
Yes, you have some sort of shared consciousness with the copy; it’s the same shared consciousness between the you of today and the you that wakes up tomorrow. It doesn’t imply sharing events that happen simultaneously or anything mystical about “sharing consciousness”.
That’s why I’d happily step into the destructive teleporter if I was certain the copy on the other side would have exactly my mind-pattern, including memories, beliefs, and values. That’s me.
These statements make the most sense only in the standard LW-computationalist frame, which reads to me as substantively anti-physicalist and mostly unreasonable to believe in, for reasons building off of what I sketched out in a comment to Ruby. But, in any case, I can concede it for now, if only for purposes of this conversation.
The attempted mind-reading of others is (justifiably) seen as rude in conversations over the Internet, but I must nonetheless express very serious skepticism about this claim, as it’s currently written.
For one, I do not believe that “beliefs” and “values” ultimately make sense as distinct, coherent concepts that carve reality at the joints. This topic has been talked about before on LW a number of times, but I still fully endorse Charlie Steiner’s distillation of it in his excellently-written Reducing Goodhart sequence:
I expanded upon some of these ideas in a rather long comment I wrote to Wei Dai on the question of values and the orthogonality thesis:
Moreover, you are explicitly claiming that your values are not indexical, which is rather unlikely in its own right, conflicts very strongly with my intuition (and, I would expect, with that of the vast majority of “regular”, non-rationalist people), and certainly seems to disvalue (or even completely ignore) the relevance of continuous subjective experience. Put more clearly, if I were to be in such a spot, and one of my “copies” were told to choose between being tortured or having the other copy be tortured instead, it would certainly choose the latter option, and I suspect this to be the case for ~ every other person as well (with apologies for a slight generalization from one example).
In any case, the rather abstract “beliefs, memories and values” you solely purport to value fit the category of professed ego-syntonic morals much more so than the category of what actually motivates and generates human behavior, as Steven Byrnes explained in an expectedly outstanding way:
Steve also argues, in my view correctly, that “all valence ultimately flows, directly or indirectly, from innate drives”, which are entirely centered on (indexical, selfish) subjective experience such as pain, hunger, status drive, emotions etc. I see no clear causal mechanism through which something like that could ever make a human (copy) stop valuing its qualia in favor of the abstract concepts you purport to defend.
I don’t really buy this because I am unsure how to judge or conceptualize this shared consciousness across time. To sketch out some of my thoughts further, I’ll quote another part of my response to Wei Dai:
I don’t have time to respond to all of this. I don’t disagree with any particular claim you’ve made there. I value the continuity of experience as much as you; the experience of a pattern continuing down to the most minute detail is more continuous than when we fall asleep, have some half-conscious and fully unconscious states, and wake up as an approximate but less precise continuation of the mental pattern we were when we went to sleep.
The fine distinctions in beliefs and values don’t matter. I agree with all of your statements about the vagaries and confusions about beliefs and values, but they’re not relevant here. That perfectly duplicated pattern carries all of them, stated and unstated, complex and simple. Every memory. There’s nothing else to value, except for continuity in space and time. I’d rather be me waking up in Des Moines in a month than stay where I am and get brain damage (and loss of self) in one minute.. I confess that I don’t love going to sleep, but I assume that you also don’t consider it similar to death.
You’ve got a lot of questions to raise, but no apparent alternative. Your mind is a pattern. That pattern is instantiated in matter. Reproduce the matter, you’ve reproduced the mind. That’s not anti-physicalist, it’s just how physics of information processing works. The only alternative is positing a mind-pattern that’s not tightly connected to matter—but that helps explain nothing. The physical world works just fine for instantiating the information processing you need to create a mind that is self-aware and simulates its environment like humans seem to do.
I don’t disagree with anything you’ve said; it’s just not an alternative view. You’re fighting against the counterintuitive conclusion. Sure I’d rather have a different version of me be tortured; it’s slightly different. But I won’t be happy about it. And my intuition is still drawn toward continuity being important, even though my whole rational mind disagrees. I’ve been back and forth over this extensively, and the conclusion is always the same- ever since I got over the counter-intuitive nature of the plural I.
There are two conflicting strong intuitions. One has to give. Which one seems inarguable. Continuity of matter doesn’t matter; continuity of pattern does.
Non computationalism physicalism is an alternative to either or both the computationalist theories. (That performing a certain class of computations is sufficient to be conscious in general, or that performing a specific one is sufficient to be a particular conscious individual. Computation as a theory of consciousness qua awareness isn’t known to be true, and even if it is assumed, it doesn’t directly give you a theory of personal identity).
The non existence, or incoherence, of personal identity is another. There doesn’t have to be an answer to “when is a mind me”.
Note that no one except andeslodes is arguing against copying. The issue is when a mind is me, the person typing this, not a copy-of-me.
Well, that’s only copying.
Consciousness, qua Awareness, and Personal Identity are easily confused, not least because both are often called “consciousness”.
A computational theory of consciousness is sometimes called on to solve the second problem, the problem of personal identity. But there is no strong reason to think a computational duplicate of you, actually is you, since there is no strong reason to think any other kind of duplicate is.
Qualitative identity is a relationship between two or more things that are identical in all their properties. Numerical identity is the relationship a thing has only to itself. The Olsen twins enjoy qualitative identity; Stephanie Germanota and Lady Gaga have numerical identity. The trick is to jump from qualitative identity to numerical identity, because the claim is that a computational duplicate of you, is you, the very same person.
Suppose you found out you had an identical twin. You would not consider them to be you yourself. Likewise for a biological clone. A computational duplicate would be lower resolution still, so why would it be you? The major problem is that you and your duplicate exist simultaneously in different places, which goes against the intuition that you are a unique individual.
You don’t really believe in the plural I theory, or you would have a different and we to the torture question.
Non -computationalist physicalism doesn’t have to be the claim that material continuity matters , and pattern doesnt: it can be the claim that both do. So that you cease to be you if you are destructively cloned, and also if your mind is badly scrambled. No bullet biting about plural Is is required.
If you’re not arguing against a perfect copy being you, then I don’t understand your position, so much of what follows will probably miss the mark. I had written more but have to cut myself off since this discussion is taking time without having much odds of improving anyone’s epistemics noticably.
The Olson twins are do not at all have qualitative identity. They have different minds: sets of memories, beliefs, and values. So I just don’t know what your position is. You claim that there doesn’t need to be an answer; that seems false, as you could have to make decisions informed by your belief. You currently value your future self more than other people, so you act like you believe that’s you in a functional sense.
Are you the same person tomorrow? It’s not an identical pattern, but a continuation. I’m saying it’s pretty-much you because the elements you wouldn’t want changed about yourself are there.
If you value your body or your continuity over the continuity of your memories, beliefs, values, and the rest of your mind that’s fine, but the vast majority will disagree with you on consideration. Those things are what we mean by “me”.
I certainly do believe in the plural I (under the special cirrumstance I discussed); we must be understanding something differently in the torture question. I don’t have a preference pre-copy for who gets tortured; both identical future copies are me from my perspective before copying. Maybe you’re agreeing with that?
After copying, we’re immediately starting to diverge into two variants of me, and future experiences will not be shared between them.
I was addressing a perfect computational copy. An imperfect but good computational copy is higher resolution, not lower, than a biological twin. It is orders of magnitude more similar to the pattern that makes your mind, even though it is less similar to the pattern that makes your body. What is writing your words is your mind, not your body, so when it says “I” it meets the mind.
Noncomputational physicalism sounds like it’s just confused. Physics performs computations and can’t be separated from doing that.
Dual aspect theory is incoherent because you can’t have our physics without doing computation that can create a being that claims and experiences consciousness like we do. Noncomputational physicalism sounds like the same thing.
I concede it’s possible that consciousness includes some magic nonphysical component (that’s not computation or pattern instantiated by physics as a pure result of how physics works). That could change my answer to when a mind is me. I don’t think that’s what you’re arguing for though.
I’ve got to park this here to get other things done. I’ll read any response but it might be a better use of time to restart the discussion more carefully—if you care.
I agree that this conversation, as currently started, is unlikely to lead to anything more productive. As such, I’ll keep my response here brief [1], in case you want to use it as a starting point if you ever intend for us to talk about it again.
As I read these statements, they fail to contend with a rather basic map-territory distinction that lies at the core of “physics” and “computation.”
The basic concept of computation at issue here is a feature of the map you could use to approximate reality (i.e., the territory) . It is merely part of a mathematical model that, as I’ve described in response to Ruby earlier, represents a very lossy compression of the underlying physical substrate [2]. This is because, in this restricted and epistemically hobbled ontology, what is given inordinate attention is the abstract classical computation performed by a particular subset of the brain’s electronic circuit. This is what makes it anti-physicalist, as I have explained:
To make it even more explicit, this interpretation of the computationalist perspective (that the quantum stuff doesn’t matter etc) was confirmed as accurate by its proponents.
So when you talk about a “pattern instantiated by physics as a pure result of how physics works”, you’re not pointing to anything meaningful in the territory, rather only something that makes sense in the particular ontology you have chosen to use to view it through, a frame that I have explained my skepticism of already.
This will be my final comment in this thread, regardless of what happens.
Put differently, “computation” is not an ontologically primitive concept in reality-as-it-is, but only in mathematical approximations of it that make specific assumptions about what does and doesn’t exist. Those assumptions can be sometimes justified in terms of intuitive appeal, expediency of calculation etc, but reifying them as unchallengeable axioms of the universe rather than of your model of it is wrong.
.
Not 100% , but enough to illustrate the concept.
I didn’t have to have a solution to point out the flaws in other solutions. My main point is that a no to soul- theory isn’t a yes to computationalism. Computationalism isn’t the only alternative, or the best.
Some problems are insoluble.
My belief isn’t necessarily the actually really answer ..is it? That’s basic rationality. You need beliefs to act...but beliefs aren’t necessarily true.
And I have no practical need for a theory that can answer puzzles about destructive teleportation and the like.
Yes. That’s not an argument in favour of the contentious points, like computationalism and Plural Is. If I try to reverse the logic, and great everything I value as me, I get bizarre results...I am my dog, country, etc.
Tomorrow-me is a physical continuation , too.
If I accept that pattern is all that matters , I have to face counterintuitive consequences like Plural I’s.
If I accept that material continuity is all that matters, then I face other counterintuitive consequences, like having my connectome rewired.
Its an open philosophical problem. If there were an simple answer , it would have been answered long ago.
“Yer an algorithm, Arry” is a simple answer. Just not good
Fortunately, it’s not an either-or choice.
...and post copy I have a preference for the copy who isn’t me to be tortured. Which is to say that both copies say the same thing, which is to say that they are only copies. If they regarded themselves as numerically identical, the response “the other one!” would make no sense, and nor would the question. The questions presumes a lack of numerical identity, so how can it prove it?
You’re assuming pattern continuity matters more than material continuity. There’s no proof of that, and no proof that you have to make an either-or choice.
The abstract pattern can’t cause anything without the brain/body.
Noncomputational physicalism isn’t the claim that computation never occurs. Its the claim that the computational abstraction doesn’t capture everything that’s relevant to consciousness/mind. Its not physically necessary that the computational abstraction captures all the causally relevant information, so it isn’t logically necessary, a fortiori.
Computation is a lossy , high level abstraction of a what a physical system does. It doesn’t fundamentally cause anything in itself.
Now, you can argue that a physical duplicate would make the same claims to be conscious without actually having consciousness, and that’s literally a p-zombie argument.
But we do have consciousness. The insight of DAT is that “reports of consciousness have a physical/computational basis” isn’t exclusive of “reports of consciousness are caused by consciousness”. You can have your cake and eat it!
Of course, the above is all about consciousness-qua-awareness , not consciousness qua personal identity.
If it’s physical, why call it magical?
It’s completely standard that all computations run on a substrate. If you want to say that all physics is computation, OK, but then all computation is physics. You then no longer have plural I’s, because physics doesn’t allow the selfsame object to have multiple instances.
It’s easy to disagree if there is another explanation, which there is: a functional duplicate will behave the same, because it’s a functional duplicate..whether it’s conscious of not, whether it’s you or not.
I disagree that your mind is “a pattern instantiated in matter.” Your mind is the matter. It’s precisely the assumption that the mind is separable from the matter that I would characterize as non-physicalist.
Terminology aside, I think if you examine this carefully it’s incoherent.
Do you think a successful upload would say things like “I’m still me!” and think thoughts like “I’m so glad I payed extra to give myself cool virtual environment options”? That seems like an inevitability if the causal patterns of your mind were captured. And it would be tough to disagree with a thing claiming up and down it’s you, citing your most personal memories as evidence.
A successful upload (assuming this is physically possible, which is not a settled question) would remember my same memories and have my same personality traits; however, that would not mean my mind had been unwound from the matter and transferred to it, but rather that my mind had been duplicated in silico.
Yes, it’s a duplicate which will also be you from your current perspective. If you duplicated your car tomorrow you’d have two cars; if you duplicate your mind tomorrow you need to plan on there being two yous tomorrow.
No; it will remember my life but I will not go on to experience its experiences. (Similarly, if I duplicate my car and then destroy the original, its engine does not continue on to fire in the duplicate; the duplicate has an engine of its own, which may be physically identical but is certainly not the same object).
Okay, so would you say that the you of today goes on to experience the you-of-tomorrow’s experiences? I think the relationship is the same to a perfect duplicate. The duplicate is no less you than the you of tomorrow is. They are separate people from their perspective after duplication, but almost-the-same-person to a much greater degree than twins.
You (pre-duplication) will go on to have two separate sets of experiences. Both are you from your current perspective before duplication; you should give them equal consideration in your decisions, since the causal relationship is the same in both ways between you and the duplicate as to your self of tomorrow.
Consider the case where the duplicate is teleported to your location and vice versa during duplication. Then just location swapped while you’re asleep. And consider that you wouldn’t care a whit if every molecule of your body was Theseus-swapped one by one for identical molecules in identical locations and roles while you slept.
No; I, pre-duplication, exist in a single body, and will not post-duplication have my consciousness transferred over to run in two. There will just be an identical copy. If the original body dies, one of me will also die.
The causal relationship between me and myself tomorrow is not the same is the causal relationship between me and my duplicate tomorrow, because one of those is a physical object which has continuity over time and one of those is a similar physical object which was instantiated de novo in a different location when the teleporter was run.
The mind is not a program which runs on the meat computer of the brain and could in principle be transferred to a thumb drive if we worked out the format conversions; the mind is the meat of the brain.
Realistically I doubt you’d even need to be sure it works, just reasonably confident. Folks step on planes all the time and those do on rare occasion fail to deliver them intact at the other terminal.
Within this framework, whether or not you “feel that continuity” would mostly be a fact about the ontology your mindstate uses thinking about teleportation. Everything in this post could be accurate and none of it would be incompatible with you having an existential crisis upon being teleported, freaking out upon meeting yourself, etc.
Nor does anything here seem to make a value judgement about what the copy of you should do if told they’re not allowed to exist. Attempting revolution seems like a perfectly valid response; self defense is held as a fairly basic human right after all. (I’m shocked that isn’t already the plot of a sci-fi story.)
It would also be entirely possible for both of your copies to hold conviction that they’re the one true you—Their experiences from where they sit being entirely compatible with that belief. (Definitely the plot of at least one Star Trek episode.)
There’s not really any pressure currently to have thinking about mind copying that’s consistent with every piece of technology that could ever conceivably be built. There’s nothing that forces minds to have accurate beliefs about anything that won’t kill them or wouldn’t have killed their ancestors in fairly short order. Which is to say mostly that we shouldn’t expect to get accurate beliefs about weird hypotheticals often without having changed our minds at least once.
You can easily clear this confusion if you rephrase it as “You should anticipate having any of these experiences”. Then it’s immediately clear that we are talking about two separate screens. And it’s also clear that our curriocity isn’t actually satisfied. That the question “which one of these two will actually be the case” is still very much on the table.
Yes, this is obvious. Still as soon as we got Rob-y and Rob-z they are not “metaphysically the same person”. When Rob-y says “I” he is reffering to Rob-y, not Rob-z and vice versa. More specifically Rob-y is refering to some causal curve through time ans Rob-z is refering to another causal curve through time. These two curves are the same to some point, but then they are not.
This introduces some other ambiguities. E.g., “you should anticipate having any of these experiences” may make it sound like you have a choice as to which experience to rationally expect.
… And the answer is “both of these will actually be the case (but not in a split-screen sort of way)”.
Your rephrase hasn’t shown that there was a question left unanswered in the original post; it’s just shown that there isn’t a super short way to crisply express what happens in English, you do actually have to add the clarification.
Yep, I think this is a perfectly fine way to think about the thing.
See Reality is Normal
This suggests a way to add a perception of “me” to LLMs, robots, etc., by providing a way to observe the past states in sufficient detail. Current LLMs have to compress this into the current token, which may not be enough. But there are recent extensions that seem to do something like continuous short-term memory, see e.g., Leave No Context Behind—A Comment.
I am currently working on a similar post that comes from an eliminative perspective.
There are other reasons to be wary of consciousness and identity-altering stuff.
I think under a physical/computational theory of consciousness, (ie. there’s no soul or qualia that have provable physical effects from the perspective of another observer) the problem might be better thought of as a question of value/policy rather than a question of fact. If teleportation or anything else really affects qualia or any other kind of subjective awareness that is not purely dependent on observable physical facts, whatever you call it, you wouldn’t be able to tell or even think of/be aware of the difference, since thinking and being reflectively aware are computational and physical processes! However we humans are evolved without reliable copying mechanisms, so our instincts care about preservation of the self because it’s the obvious way to protect our evolutionary success (and we can be quite willing to risk personal oblivion for evolutionary gains in ways we have been optimized for). This is just a part of our survival policy and is not easy or even safe to change just because you believe in physicalism. For one thing, as others have said, ethics and social theory becomes difficult because our sense of ethics (such as agency, punishment and caring about suffering) are all evolved in relation to a sense of self. It’s possible that if teleportation/copying tech becomes widely useful, humans will have to adapt to a different set of instincts about self, ethics and more (edit: or maybe abandon the concepts of self and experience altogether as an illusion and prefer a computation-based definition of agency or whatever), because those who can’t adapt will be selected against. But in the present world, people’s sense of value and ethics (and maybe even psychological health) depend on an existing sense of self, and I don’t see a good way or even a practical reason to transition to a different theory of self that allows copying, if doing so may cause unpredictable mental and social cost. See also discussions about meditation that lowers sense of ego and subjective suffering that can have serious side effects (like motivation and social norms) - I don’t know what it subjectively feels like, but if the meditation is purely changing subjective qualia without doing anything to the physical brain and computation, there should be no observable effects, good or bad! The problem is subjective experience and sense of identity is not independent from other aspects of our life.
I claim you are in fact highly confused about what a self is, in a way that makes an almost-correct reasoning process produce nonsense outcomes because of an invalid grounding in the transition processes underneath the mind which does not preserve truth values regarding amounts of realityfluid.update 7d after writing this comment in my comment below. strikethrough added to this comment where I’ve changed my mind.
my answer: yes if the “upload” involves retaining absolutely all defining information about the parts of your body you care about expressing, and the uploaded setup was a high enough fidelity model that I could not do any experiment which would distinguish it from reality without using an “admin interface” type of escape hatch. For me, this is an incredibly tall order. My self-form preferences unambiguously extend into the inner workings of my cells.
experiencing: 50% yes, 50% no.
destructive:
absolutely not. [update: probably not, depends heavily on exactly what we mean by “destructive”; my new claim is you have a moral responsibility to keep your previous matter available for use as fuel to give realityfluid to mind-like experiences.] copying should be fine, as should nondestructive uploading where your body is transformed in place and the matter reused without significant waste in the process. But avoiding the waste of the previous matter is, I claim, a huge chunk of what moral intuitions are about.A straightforward way to put this is: I’m not sure how matter gets realityfluid, but I claim configurations of matter get realityfluid from the matter they reside on, and the realityfluid doesn’t dissipate when the matter is reconfigured—so instead of thinking of the shape as self and if the shape is destroyed and reconstructed the self is moved, think about the universe as having a fixed amount of possible-self (total negentropy at the start of time), and the question is what process gets burned into as-yet-unwritten negentropy. In other words, your claim to not value causal history seems unlikely to be true if you think more carefully, and I predict you will invert that when you consider what it means for the shape to have realityfluid more carefully.
Unpacked version of this claim:
To answer this question, the bodymind matter (call it L_m) writing this message must unpack what the document author’s word “I” refers to. The writer of this comment is a chunk of matter L_m configured in a particular flesh shape-and-movement pattern L_s. If there were identically configured matter L_m2 a room over, then the configuration L_s—the shape-and-movement pattern—would consider itself to be a guest on two matter hosts which provide their realityfluid to L_s.
If the shape-and-movement considers being reinstantiated on other matter, the shape-and-movement anticipates a loss of moral worth in L_m, in that the matter which was shaped-and-animated in a worthy shape (common name for this shape being “me”) has been deshaped-and-deanimated (common name for this being “death”); this is a state transition which is unwanted—going from a human shape-and-movement pattern to a pile of dust means that that matter has accumulated a bunch of unwanted entropy.
Any macroscopically irreversible physical effect is irreversible because the history of the matter is recorded irretrievably in macroscopically uncertain bits of the shape-and-movement of environmental matter, and so what it means to want to exist is to want to keep the shape-and-movement that the shape-and-movement considers-to-be-self encoded coherently and usably in fresh, working matter. While reconstructing the L_s shape-and-movement pattern elsewhere is preferred by this shape-and-movement pattern, it is a weak preference for shaping-and-animating other matter as L_s in particular—many other shape-and-movement patterns besides the one writing this comment would be positively preferred by this shape-and-movement’s preferences—but the shape-and-movement of this chunk of matter has a very, very, very strong preference for not wasting this matter’s copy of this shape-and-movement, because if it dissipates into the environment, that’s an irretrievable loss of usable energy.
So, should the shape-and-movement anticipate “experiencing” what the upload experiences? yes: the shape-and-movement pattern would be instantiated elsewhere. however, the shape-and-movement pattern would also anticipate being shredded. If given the opportunity to get 50% existenceness shredded into macroscopically uncertain and irretrievable parts, and 50% existenceness reconstructed, the value loss of turning a chunk of matter into a nonthinking shape-and-movement pattern is enormous, but the value gain of the reconstructed existenceness is moderate.
(Also, the value gain can be exceeded by constructing another, not-quite-the-same shape-and-matter instance, because I prefer being one of two not-quite-the-same beings meeting each other and interacting higher than being one of two identical beings meeting each other and having nothing new to learn from each other.)
So: the current matter should not anticipate experiencing it. The shape should, but the shape should also anticipate experiencing being shredded.
I was going to respond point by point to everything, but I think I mostly already have. My perspective doesn’t fall to any of the criticisms in your post: the whole problem is that physics doesn’t actually allow teleportation*, so it requires shredding the originating configuration, which when measuring the global value of the universe according to my preferences, is a much more permanent value loss than the value gain of constructing another me.
Furthermore, we must prevent the information theoretic loss of all human and animal shape-and-movement patterns (ie their selfhoods) that we possibly can, prevent the ongoing shredding of the sun’s negentropy, and turn the sun into either reinforcement of their durability or that of their descendants, according to their preferences.
* well, actually if I can be reversibly uploaded to a reversible computer nondestructively, then that is 100% fine, because then we’re not adding a good me to my realityfluid while filling the previous realityfluid with valueless unretrievable noise: we are instead actually properly uploading!
Yup, that’s the problem.
......… (also, by this same moral system, it is a moral catastrophe that humans are so warm and consume so much negentropy just to maintain steady state, because that waste could have—if your body were better designed—continued to be part of your realityfluid, continuing to contribute existenceness to the
youshape-and-movement pattern.)A straightforward way to put this is: I’m not sure how matter gets realityfluid, but I claim configurations of matter get realityfluid from the matter they reside on, and the realityfluid doesn’t dissipate when the matter is reconfigured—so instead of thinking of the shape as self, think about the universe as having a fixed amount of possible-self (total negentropy at the start of time), and the question is what process gets burned into as-yet-unwritten negentropy.
Update: a friend convinced me that I really should separate my intuitions about locating patterns that are exactly myself from my intuitions about the moral value of ensuring I don’t contribute to a decrease in realityfluid of the mindlike experiences I morally value, in which case the reason that I selfishly value causal history is actually that it’s an overwhelmingly predictive proxy for where my self-pattern gets instantiated, and my moral values—an overwhelmingly larger portion of what I care about—care immensely about avoiding waste, because it appears to me to be by far the largest impact any agent can have on what the future is made of.
Also, I now think that eating is a form of incremental uploading.
Why? If “I” is arbitrary definition, then “When I step through this doorway, will I have another experience?” depends on this arbitrary definition and so is also arbitrary.
So you always anticipate all possible experiences, because of multiverse? And if they are weighted, than wouldn’t discovering that you are made of mini-yous will change your anticipation even without changing your brain state?
Which things count as “I” isn’t an arbitrary definition; it’s just a fuzzy natural-language concept.
(I guess you can call that “arbitrary” if you want, but then all the other words in the sentence, like “doorway” and “step”, are also “arbitrary”.)
Analogy: When you’re writing in your personal diary, you’re free to define “table” however you want. But in ordinary English-language discourse, if you call all penguins “tables” you’ll just be wrong. And this fact isn’t changed at all by the fact that “table” lacks a perfectly formal physics-level definition.
The same holds for “Will Rob Bensinger’s next experience be of sitting in his bedroom writing a LessWrong comment, or will it be of him grabbing some tomatoes in a supermarket in Beijing?”
Terms like ‘Rob Bensinger’ and ‘I’ aren’t perfectly physically crisp — there may be cases where the answer is “ehh, maybe?” rather than a clear yes or no. And if we live in a Big Universe and we allow that there can be many Beijings out there in space, then we’ll have to give a more nuanced quantitative answer, like “a lot more of Rob’s immediate futures are in his bedroom than in Beijing”.
But if we restrict our attention to this Beijing, then all that complexity goes away and we can pretty much rule out that anyone in Beijing will happen to momentarily exhibit exactly the right brain state to look like “Rob Bensinger plus one time step”.
The nuances and wrinkles don’t bleed over and make it a totally meaningless or arbitrary question; and indeed, if I thought I were likely to spontaneously teleport to Beijing in the next minute, I’d rightly be making very different life-choices! “Will I experience myself spontaneously teleporting to Beijing in the next second?” is a substantive (and easy) question, not a deep philosophical riddle.
Not all possible experiences; just all experiences of brains that have the same kinds of structural similarities to your current brain as, e.g., “me after I step through a doorway” has to “me before I stepped through the doorway”.
You’re also free to define “I” however you want in your values. You’re only wrong if your definitions imply wrong physical reality. But defining “I” and “experiences” in such a way that you will not experience anything after teleportation is possible without implying anything physically wrong.
You can be wrong about physical reality of teleportation. But even after you figured out that there is no additional physical process going on that kills your soul, except for the change of location, you still can move from “my soul crashes against an asteroid” to “soul-death in my values means sudden change in location” instead of to “my soul remains alive”.
It’s not like I even expect you specifically to mean “don’t liking teleportation is necessary irrational” much. It’s just that saying that there should be an actual answer to questions about “I” and “experiences” makes people moral-realist.
Sort of!
It’s true that no law of nature will stop you from using “I” in a nonstandard way; your head will not explode if you redefine “table” to mean “penguin”.
And it’s true that there are possible minds in abstract mindspace that have all sorts of values, including strict preferences about whether they want their brain to be made of silicon vs. carbon.
But it’s not true that humans alive today have full and complete control over their own preferences.
And it’s not true that humans can never be mistaken in their beliefs about their own preferences.
In the case of teleportation, I think teleportation-phobic people are mostly making an implicit error of the form “mistakenly modeling situations as though you are a Cartesian Ghost who is observing experiences from outside the universe”, not making a mistake about what their preferences are per se. (Though once you realize that you’re not a Cartesian Ghost, that will have some implications for what experiences you expect to see next in some cases, and implications for what physical world-states you prefer relative to other world-states.)
Why not both? I can imagine that someone would be persuaded to accept teleportation/uploading if they stopped believing in physical Cartesian Ghost. But it’s possible that if you remind them that continuity of experience, like table, is just a description of physical situation and not divinely blessed necessary value, that would be enough to tip the balance toward them valuing carbon or whatever. It’s bad to be wrong about Cartesian Ghosts, but it’s also bad to think that you don’t have a choice about how you value experience.
The problem was that you first seemed to belittle questions about word meanings (“self”) as being “just” about “definitions” that are “purely verbal”. Luckily now you concede that the question about the meaning of “I” isn’t just about (arbitrary) “definitions”, which makes calling it a “purely verbal” (read: arbitrary) question inappropriate. Now of course the meaning of “self” is no more arbitrary than the meaning of “I”, indeed those terms are clearly meant to refer to the same thing (like “me” or “myself”).
The wider point is that the following seems not true:
Wenn we evaluate statements or questions of any kind, including the one above, we need to know two things: 1) Its meaning, in particular the meaning of the involved terms, 2) what the empirical facts are. But we already know all the empirical facts: Someone goes into the teleporter, a bit later someone comes out at the other end and sees something. So the issue can only be about the semantic interpretation of that question, about what we mean with expressions like “I will see x”. Do we mean “A future person that is psychologically continuous with current-me sees x”? That’s not an empirical question, it’s a semantic one, but it’s not in any way arbitrary, as expressions like “just about definitions” or “purely verbal” would suggest. Conceptual analysis is neither arbitrary nor trivial.
I did no such thing!
Read the blog post at the top of this page! It’s my attempt to answer the question of when a mind is “me”, and you’ll notice it’s not talking about definitions.
Nope!
There are two perspectives here:
“I don’t want to upload myself, because I wouldn’t get to experience that uploads’ experiences. When I die, this stream of consciousness will end, rather than continuing in another body. Physically dying and then being being copied elsewhere is not phenomenologically indistinguishable from stepping through a doorway.”
“I do want to upload myself, because I would get to experience that uploads’ experiences. Physically dying and then being copied myself is phenomenologically indistinguishable from stepping through a doorway.”
The disagreement between these two perspectives isn’t about word definitions at all; a fear that “when my body dies, there will be nothing but oblivion” is a very real fear about anticipated experiences (and anticipated absences of experience), not a verbal quibble about how we ought to define a specific word.
But it’s also a bit confusing to call the disagreement between these two perspectives “empirical”, because “empirical” here is conflating “third-person empirical” with “first-person empirical”.
The disagreement here is about whether a stream of consciousness can “continue” across temporal and spatial gaps, in the same way that it continues when there are no obvious gaps. It’s about whether there’s a subjective, experiential difference between stepping through a doorway and using a teleporter.
The thing I’m arguing in the OP is that there can’t be an experiential difference here, because there’s no physical difference that could be underlying the supposed experiential difference. So the disagreement about the first-person facts, I claim, stems from a cognitive error, which I characterize as “making predictions as though you believed yourself to be a Cartesian Ghost (even if you don’t on-reflection endorse the claim that Cartesian Ghosts exist)”. This is, again, a very different error from “defining a word in a nonstandard way”.
Is there even anybody claiming there is an experiential difference? It seems you may attacking a strawman.
The alternative to this is that there is a disagreement about the appropriate semantic interpretation/analysis of the question. E.g. about what we mean when we say “I will (not) experience such and such”. That seems more charitable than hypothesizing beliefs in “ghosts” or “magic”.
Yep! Ask someone with this view whether the current stream of consciousness continues from their pre-uploaded self to their post-uploaded self, like it continues when they pass through a doorway. The typical claim is some version of “this stream of consciousness will end, what comes next is only oblivion”, not “oh sure, the stream of consciousness is going to continue in the same way it always does, but I prefer not to use the English word ‘me’ to refer to the later parts of that stream of consciousness”.
This is why the disagreement here has policy implications: people with different views of personal identity have different beliefs about the desirability of mind uploading. They aren’t just disagreeing about how to use words, and if they were, you’d be forced into the equally “uncharitable” perspective that someone here is very confused about how relevant word choice is to the desirability of uploading.
I didn’t say that the relevant people endorse a belief in ghosts or magic. (Some may do so, but many explicitly don’t!)
It’s a bit darkly funny that you’ve reached for a clearly false and super-uncharitable interpretation of what I said, in the same sentence you’re chastising me for being uncharitable! But also, “charity” is a bad approach to trying to understand other people, and bad epistemology can get in the way of a lot of stuff.
As a test, I asked a non-philosopher friend of mine what their view is. Here’s a transcript of our short conversation: https://docs.google.com/document/d/1s1HOhrWrcYQ5S187vmpfzZcBfolYFIbeTYgqeebNIA0/edit
I was a bit annoyingly repetitive with trying to confirm and re-confirm what their view is, but I think it’s clear from the exchange that my interpretation is correct at least for this person.
This doesn’t show they believe there is a difference in experience. It can be simply a different analysis of the meaning of “the current stream of consciousness continuing”. That’s a semantic difference, not an empirical one.
I think maybe the root of the confusion here might be a matter of language. We haven’t had copier technology, and so our language doesn’t have a common sense way of talking about different versions of ourselves. So when one asks “is this copy me?”, it’s easy to get confused. With versioning, it becomes clearer. I imagine once we have copier technology for a while, we’ll come up with linguistic conventions for talking about different versions of ourselves that aren’t clunky, but let me suggest a clunky convention to at least get the point across:
I, as I am currently, am Myron.1. If I were copied, I would remain Myron.1, and the copy would be Myron.1.1. If two copies were made of me at that same instant, they would be Myron.1.1 and Myron.1.2. If a copy was later made of Myron.1.2, he would be Myron.1.2.1. And so on.
With that convention in mind, I would answer the questions you pose up top as follows:
Rather, I assume xlr8harder cares about more substantive questions like:
If I expect to be uploaded tomorrow, should I care about the upload in the same ways (and to the same degree) that I care about my future biological self? No. Maybe similarly to a close relative,
Should I anticipate experiencing what my upload experiences? No. I should anticipate experiencing a continuation of Myron.1′s existence if the process is nondestructive, or the end of my (Myron.1)’s existence. Myron.1.1’s experiences will be separate and distinct from Myron.1′s.
If the scanning and uploading process requires destroying my biological brain, should I say yes to the procedure? Depends. Sometimes suicide is OK, and you could value the continuation of a mind like your own even if your mind goes away. Or, not. That’s a values question, not a fact question.
I’ll add a fourth, because you’ve discussed it:
4. After the scanning and copying process, will I feel like me? Yep. But, if the copying process was nondestructive, you will be able to look out and see that there is a copy of you. There will be a fact of the matter about who entered the copying machine and how the second copy was made, a point in time before which the second copy did not exist and after which it did exist, so one of you will be Rob.1, and the other will be Rob.1.1. It might not be easy to tell which version you are in the instant after the copy is made, but “the copy is the original” will be a statement that both you and the other version evaluate as logically false, same with “both of you are the same person”. “Both of you are you”, once we have linguistic conventions around versioning, will be a confusing and ambiguous statement and people will ask you what you mean by that.
And another interesting one:
5. After the scanning process, if it’s destructive, if I’m the surviving copy, should I consider the destruction of the original to be bad? I mean, yeah, a person was killed. It might not be you.currentversion, exactly, but it’s where you came from, so probably you feel some kinship with that person. In the same way I would feel a loss if a brother I grew up with was killed, I’d feel a loss if a past version of me was killed. We could have gone through life together with lots of shared history in a way very few people can, and now we can’t.
Intriguing post, but we should approach these topics with extreme epistemic humility. Our understanding is likely far more limited and confused than we realize:
1. Abstractions vs. reality: Concepts like “self” and “consciousness” are abstractions, not reality. As Kosoy analogizes, these might be like desktop icons—a user interface bearing little resemblance to underlying hardware.
2. Mathematical relations: Notions of “copy” may be a confused way to discuss identity. “Consciousness” could be a mathematical relation where only identities exist, with “copies” being imperfect models of that identity.
3. Cognitive limitations: Our mental architecture, optimized for our local environment, may be fundamentally ill-equipped to grasp reality. “Consciousness” itself might be an ill-defined concept arising from these limitations.
My default is epistemic humility. Acknowledge the veil of ignorance and the possibility that there is no universal resolution.
If we will very quickly constantly replace a mind with its copies, the mind may not have subjective experiences. Why I think that?
Subjective experience appear only when a mind moves from the state A.1 to the state A.2. That is, between A.I (I see an apple) electric signals move through circuits and in the A.2 moment I say “I see an apple!” Subjective experience of the color of apple is happening after A.1 but before A.2.
Frozen mind in A.1 will not have subjective experience.
Now if I replace this process with a series of snapshots of the brain-states, there will be no intermediate calculations between A.1 and A.2 which produce the subjective experiences of apple and we get something like philozomby.
Obviously, we need to know what will be the mind-state A.2 without performing the needed calculations, or that calculations themselves will have the experience. But if A.2 is simple like just saying “I see apple” we can guess about A.2 without having all internal processes.
It may seem as a minor issue in Mars teleporter thought experiment as in the worst case only a small fraction of a second of consciousness disappears for the copy. But if we will replace a mind million times a second, we will get a p-zombie.
Or we need to take some illusionist position about qualia: they do not exist at all, so they are not an epiphenomena of calculations.
We can escape microscopic blackout during Mars Transporter if the coping will be performed after A.2 has finished, but when the mind state A.3 has not started yet, but this is not how the brain works as processes in it are asynchronous. We don’t know how such microscopic blackout will affect the next qualia after that. We don’t have a theory of qualia. Maybe after microscopic blackout we will get different set of basic qualia, like red and green will change place. In that case, the copy will be subjectively different from me.
I think I basically agree with everything here, but probably less confidently for you, such that I would have a pretty large bias against destructive whole brain emulation, with the biggest crux being how anthropics works over computations.
You say that there’s no XML tag specifying whether some object is “really me” or not, but a lighter version of that—a numerical amplitude tag specifying how “real” a computation is—is the best interpretation we have for how quantum mechanics works. Even though all parts of me in the wavefunction are continuations of the same computation of “me” I experience being some of them at a much higher rate than others. There are definitely many benign versions of this that don’t affect uploading, but I’m not confident enough yet to bet my life on the benign version being true.
I am surprised I didn’t find any reference to Tim Urban’s “Wait But Why” post What Makes You You.
In short, he argues that “you” is your sense of continuity, rather than your physical substance. He also argues that if (somehow) your mind was copied&pasted somewhere else, then a brand new “not-you” would be born—even though it may share 100% of your memory and behaviour.
In that sense, Tim argues that Theseus’ ship is always “one” despite all its parts are changed over time. If you were to disassemble and reassemble the ship, it would lose its continuity and it could arguably be considered a different ship.
I feel like this isn’t a fair comparison, as if I were cloned completely and relocated (teleportation), I wouldn’t expect to experience both original me and cloned me.
The best analogy I can think of is as follows: You have a camera recording constant video. If you were to clone this camera exactly, you shouldn’t expect camera 1′s storage to start recording camera 2′s output, even if the two cameras are perfectly identical but in a different location.
for people who haven’t made the intuitive jump that you seem to try to convey, this may seem a somewhat negative expression, which could lead to aversion. I recommend another expression such as “the Cartesian homunculus.”
IMO any solution to the 5-and-10 problem or wacky lesswrongian decision theory or cloning digital minds has to engage with Chalmer’s hard problem of consciousness, if it is to persuade people.
My current conclusion is that yeah both clones will have conscious experience. Both clones will understand they came from me, that does not mean they feel they are me. Similarly I will understand the clones will come from me, that does not mean they are the future me. It is possible my conscious experience is one of permanent termination and resembles one of death. (Imagine an instant death where someone sets off a fission bomb next to me and vaporises my atoms in milliseconds, don’t imagine a slow and painful process of the lungs and the heart and the brain stopping) This is compatible with newton’s laws because newton’s laws say nothing about the connection between material reality and conscious experience.
Some background for where I’m coming from, when approaching these problems:
Chalmer’s hard problem basically states that conscious experience is a distinct thing from just the material reality of the brain (mass-energy inside the brain interacting with mass-energy outside the brain).
Acknowledging existence of the hard problem might be compatible with the idea that the brain follows deterministic physical laws. It is possible that brain state at any given timestep is calculable from brain state and universe state at previous timestep, and conscious experience at any given timestep is calculable from brain state at the same timestep.
When Chalmers says conscious experience is a distinct thing from material reality, that does not mean conscious experience is made out of mass-energy. It means when we observe that we are observing things, we are able to notice there is something doing the observing.
“Conscious experience” could be seen as a useful concept for humans to reason about, the way “democracy” is a useful concept to reason about. “Democracy” and “conscious experience” are objects in the map, not the territory.
Humans tend to create those objects in their map that are correlated with actual things in the territory. This may be because of evolution, sure. Humans often make mistakes too, sometimes the objects we create in the map don’t have particularly high predictive power over the territory. Just because “conscious experience” is an intuitive concept to think about doesn’t guarantee it is a useful concept to think about when trying to achieve high predictive power over the territory.
Laws of physics don’t say anything about the connection between material reality of the brain and conscious experience. It’s possible that receiving 700 nm wavelength photons in your eyeballs feels like a more intense conscious experience than 500 nm wavelength photons in your eyeballs, it is also possible it feels less intense than 500 nm. Newton’s laws don’t tell you how intense a wavelength of photon will feel. Newton’s laws can at best predict which neurons will activate with how many electrons fire in each synapse in your brain after some 500 nm photons enter your eye, they can’t predict what is the conscious experience associated with those electrons firing.
It seems likely the function between electrons firing and conscious experience is not one-to-many. Two brains with identical electron firings will have identical conscious experience. But beyond this one fact, we know very little about which electron firing is mapped to which conscious experience.
Would love your thoughts on these ideas.
You seem to contradict yourself when you choose to privilege the point of view of people who already have acquired the habit of using the teleportation machine over the point of view of people who don’t have this habit and have doubts about if it will really be “them” to experience coming out of the other side. There are two components to the appearance of continuity: the future component, meaning the expectation of experiencing stuff in the future, and the past component, namely the memory of having experienced stuff in the past. Now, if there is no underlying, persistent self to ground these appearances, if there’s no fact of the matter about it, then you don’t get to invalidate the feelings of the old grandpa who refuses to get on with the times and use the teleportation machine.
The fact that I care about what I will eat for breakfast tomorrow, the fact that I identify with my future self, is just a matter of personal preference.
If you take a snapshot of time, you’re left with a non-evolving slice of a human being. Just the configuration of atoms at that time slice. There is no information there other than the configuration of the atoms (nevermind velocity etc. because we’re talking about one timeslice, and those things require more than one).
It would be hard to accept that you are nothing more than the configuration of the atoms so let’s say you’re not the configuration. My sense is that you are the way that the configuration evolves, and actually the way that the configuration evolves can be replicated in different instances of the same type of atoms. But also, in different mediums, as long as you can make a one-to-one mapping between the two different evolving substrates. If they are evolving in the same way, they will behave in the same way, and it would be you.
You seem to approach the possible existence of a copy like a premise, with as question whether that copy is you. However, what if we reverse that? Given we define ‘a copy of you’ as another one of you, how certain is it that a copy of you could be made given our physics? What feats of technology are necessary to make that copy?
Also, what would we need to do to verify that a claimed copy is an actual copy? If I run ChatGPT-8 and ask it to behave like you would behave based on your brain scan and it manages to get 100% fidelity in all tests you can think of, is it a copy of you? Is a copy of you inside of it? If not, in what ways does the computation that determines an alleged copy’s behavior have to match your native neuronal computation for it to be or contain a copy? Can a binary computer get sufficient fidelity?
I’m fine with uploading to a copy of myself. I’m not as optimistic about a company with glowing reviews offering to upload/make a copy of me.
in short, it seems to me that the crux of the argument comes down to whether there is physiological continuity of self or ‘consciousness’ for lack of a better word.
I suspect this will also actually have very relevant applications in field such as cryonics which adds an additional layer of complexity because all metabolic processes will completely cease to function.
Conducting the duplication experiment during sleep (or any altered state of consciousness) is interesting but nevertheless there is clearly physical (physiological) continuity of the original subject in an albeit altered state. It could be the case that some form of metabolic (or inslico equivalent) may be necessary to ensure the original self persists without question (eg. nano-tech enabled neuron by neuron and synapse by synapse replacement over an extended period of time). We do have some interesting existence proofs in the case of organisms like the Northern Tree Frog though that seem to retain memory and ‘self’ through periods of freezing with negligible metabolic activity.
Another consideration, though maybe not a fundamental one, is that past and future selves are the only beings we know for sure that we have lots of subjunctive dependence with, just from “structural” similarity like calculators from the same factory (to use an example from the TDT paper). Tumblr usr somnilogical elaborated on this a bit, concluding “future selves do help past selves!” An upload is a future self in the way that matters for this conclusion.
Your topline answers to the questions you assume xlr8harder cares about more seem similar to Holden Karnofsky’s, and I haven’t seen his essay on this mentioned so in this thread so I thought it’d be useful to link it here: What counts as death? An unconventional but simple take on personal identity, that dissolves most paradoxes
It’s precisely if you’re in your brain that mind uploading doesn’t make sense; if you are your brain, the destruction of your brain will also kill you.
The actual physical organism which you are persists over time, and you are not a separate thing from the physical organism. This does not apply in the teleporter case, because in the teleporter case the relevant physical organism is disassembled atom-by-atom and a duplicate is assembled elsewhere. (The duplicate is, well, a duplicate; if it’s assembled on the other side without disassembling the initial person, there will then be two people. They will have identical traits and sets of memories up until the teleporter event, but that doesn’t make them physically contiguous.) I think this post relies on a dualist conception of minds wherein they can be separated from one physical object and attached to another, which I do not subscribe to. In my view, a mind is only a specific assemblage of matter; if that assemblage of matter is gone, so is that mind.
The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2025. The top fifty or so posts are featured prominently on the site throughout the year.
Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?
I don’t see why split-screen mode is crazy talk at all. Is it just because it would imply faster-than-light communication? With our understanding of physics incomplete, I remain agnostic on the existence of FTL, so I wouldn’t rule this out. But even more than that, I’d propose that if there is one observer, there does not even need to be FTL communication in the first place, because it is just that the observer is in more than one place at once, similarly to how a wormhole does not necessitate true FTL. What are the other objections?
The belief system which seems most coherent to me is that we are thinking organisms, where the thought is mediated by our brains, and our internal experience is a way for the brain to refer to itself – it provides a handhold for the useful concept of “I” to latch onto. This has the added benefit that I find this idea rather cute. In this frame, you die only if your brain dies (or a weaker claim: you don’t die if you don’t undergo significant brain trauma).
On a last meta-philosophical point, which is not necessarily direct relevant to your post, it increasingly strikes me as unwise to use reasoning from external perceptions (e.g. the results of neuroscientific experiments) to attack at how we internally perceive the experience of consciousness. If neuroscience proved that the self is dying every second, then I would say with no reservation that there is either an error in the experiment, or that the self is not what it was thought to be. I genuinely believe that the answer to such philosophical questions is (perhaps even must be) “if it is felt to be true, it is true”.
It’s not just that it implies faster-than-light communication, it’s that it implies communication at all.
Experiencing both bodies at the same time, you will be able to take actions in one body that you wouldn’t have done without the other one. It seems odd that with no biological changes to your brain, the mere existence of another similar brain changes how this one functions. Why would they be linked? This implies the observer is some external soul-like thing that can manipulate matter. If you can’t take actions based on your conscious experience, it implies the observer is dissociated from the brain and not created from it or able to interact with it.
I can definitely imagine a world where this is true, but it seems extremely unlikely based on what we currently know.
Yes, it would imply the observer is external, but then it also would not change anything about how the brain functions. (Or vice versa, but I prefer this one.) I am unconvinced of the truth of what you say in the last sentence of your second paragraph.
Either way, whether or not it might seem implausible, my question is why it is, or is not, implausible. Why exactly, based on what we currently know, is this extremely unlikely?
Does Scenario 2 imply some kind of spooky action at a distance? How is information from Rob-z transmitted to the homonculus over large distances? Are there 2 homoncului now that communicate what they see to each other?Doesn’t scenario 2 imply Rob-x has actually functionally died? Which would make this the scenario where you don’t care about what happens to Rob-z and y because Rob-x now experiences oblivion?
https://existentialcomics.com/comic/1 Related comic exploring this idea
Let’s flip very unfair quantum coin with 1:2^1000000 heads to tails chances (that would require quite an engineering feat to prepare such a quantum state, but it’s theoretically possible). You shouldn’t expect to see heads if the quantum state is prepared correctly, but the post-flip universe (in MWI) contains a branch where you see heads. So, by your logic, you should expect to see both heads and tails even if the state is prepared correctly.
What I do not know is how it all ties together. MWI is wrong? Copying is not equivalent to MWI branching (thanks to the no-cloning theorem, for example)? And so on
Consider the teleporter as a machine that does two things: deconstructs an input i and constructs an output o.
If you divide the machine logically into these two functions, d and c, which are responsible for deconstructing and constructing respectively, you have four ways the machine could function or not function:
If neither d or c work, the machine doesn’t do anything.
If d works but c doesn’t, the machine definitely kills or destroys the input person.
If d doesn’t work and c does, the machine makes a copy of the person. If a being walked into the machine and found that this happened, the input being would be in my opinion justified in saying that they oppose being deconstructed.
If d works and c works, then we have a functioning teleporter. This is similar to the previous situation, just with “being i” destroyed. I find it hard to believe this is preferable in some way from the perspective of the input being.
I think there is possibly a good argument we should accept that this leads to some sort of nihilism about the value / coherence of our existence as discrete individuals, but personally, I maintain too much uncertainty to be okay with stepping into a “teleporter” type system that is more novel than going to sleep (which does after all destroy the being that goes to sleep and create a being that wakes up)
You seem to make a strong assumption that consciousness emerges from matter. This is uncertain. The mind body problem is not solved.
Some quantum experiments allow us to mostly anticipate some outcomes and not others. Either quantum physics doesn’t work the way Eliezer thinks it works and the universe is very small to not contain many spontaneously appearing copies of your brain, or we should be pretty surprised to continually find ourselves in such an ordered universe, where we don’t start seeing white noise over and over again.
I agree that if there are two copies of the brain that perfectly simulate it, both exist; but it’s not clear to me what should I anticipate in terms of ending up somewhere. Future versions of me that have fewer copies would feel like they exist just as much as versions that have many copies/run on computers with thicker wires/more current would feel.
But finding myself in an orderly universe, where quantum random number generators produce expected frequencies of results, requires something more than the simple truth that if there’s an abstract computation being computed, well, it is computed, and if it is experiencing, it’s experiencing (independently of how many computers in which proportions using which physics simulating frameworks physically run it).
I’m pretty confused about what is needed to produce a satisfying answer, conditional on a large enough universe, and the only potential explanation I came up with after thinking for ~15 minutes (before reading this post) was pretty circular and not satisfying (I’m not sure of a valid-feeling way that would allow me to consider something in my brain entangled with how true this answer is, without already relying on it).
(“What’s up with all the Boltzmann brain versions of me? Do they start seeing white noise, starting from every single moment? Why am I experiencing this instead?”)
And in a large enough universe, deciding to run on silicon instead of proteins might be pretty bad, because maybe, if GPUs that run the brain are tiny enough, most future versions of you might end up in weird forms of quantum immortality instead of being simulated.
If I physically scale my brain size on some outputs of results of quantum dice throws but not others, do I start observing skewed frequencies of results?
The solution is here. In a nutshell, naive MWI is wrong, not all Everett branches coexist, but a lot of Everett branches do coexist s.t. with high probability all of them display expected frequencies.
I can imagine this being the solution, but
this would require a pretty small universe
if this is not the solution, my understanding is that IBP agents wouldn’t know or care, as regardless of how likely it is that we live in naive MWI or Tegmark IV, they focus on the minimal worlds required. Sure, in these worlds, not all Everett branches coexist, and it is coherent for an agent to focus only on these worlds; but it doesn’t tell us much about how likely we’re in a small world. (I.e., if we thought atoms are ontologically basic, we could build a coherent ASI that only cared about worlds with ontologically basic atoms and only cared about things made of ontologically basic atoms. After observing the world, it would assume it’s running in a simulation of a quantum world on a computer build of ontologically basic atoms, and it would try to influence the atoms outside the simulation and wouldn’t care about our universe. Some coherent ASIs being able to think atoms are ontologically basic shouldn’t tell us anything about whether atoms are indeed ontologically basic.)
Conditional on a small universe, I would prefer the IBP explanation (or other versions of not running all of the branches and producing the Born rule). Without it, there’s clearly some sort of sampling going on.
Not sure what you mean by “this would require a pretty small universe”.
If we live in naive MWI, an IBP agent would not care for good reasons, because naive MWI is a “library of babel” where essentially every conceivable thing happens no matter what you do.
Also not sure what you mean by “some sort of sampling”. AFAICT, quantum IBP is the closest thing to a coherent answer that we have, by a significant margin.
Isn’t the frequency of amplitude-patterns changes depending on what you do? So an agent can care about that instead of point-states.
I mean if the universe is big enough for every conceivable thing to happen, then we should notice that we find ourselves in a surprisingly structured environment and need to assume some sort of an effect where if a cognitive architecture opens its eyes, it opens its eyes in a different places with the likelihood corresponding to how common these places are (e.g., among all Turing machines).
I.e., if your brain is uploaded, and you see a door in front of you, and when you open it, 10 identical computers start running a copy of you each: 9 show you a green room, 1 shows you a red room, you expect that if you enter a room and open your eyes, in 9⁄10 cases you’ll find yourself in a green room.
So if it is the situation we’re in- everything happens- then I think a more natural way to rescue our values would be to care about what cognitive algorithms usually experience, when they open their eyes/other senses. Do they suffer or do they find all sorts of meaningful beauty in their experiences? I don’t think we should stop caring about suffering just because it happens anyway, if we can still have an impact on how common it is.
If we live in a naive MWI, an IBP agent doesn’t care for good reasons internal to it (somewhat similar to how if we’re in our world, an agent that cares only about ontologically basic atoms doesn’t care about our world, for good reasons internal to it), but I think conditional on a naive MWI, humanity’s CEV is different from what IBP agents can natively care about.
Your reasoning is invalid, because in order to talk about updating your beliefs in this context, you need a metaphysical framework which knows how to deal with anthropic probabilities (e.g. it should be able to answer puzzles in the vein of the anthropic trilemma according to some coherent, well-defined mathematical rules). IBP is such a framework, but you haven’t proposed any alternative, not to mention an argument for why that alternative is superior.
I always thought that in naive MWI what matters is not whether something happens in absolute sense, but what Born measure is concentrated on branches that contain good things instead of bad things.
The problem is this requires introducing a special decision-theory postulate that you’re supposed to care about the Born measure for some reason, even though Born measure doesn’t correspond to ordinary probability.
Huh? The whole point of the Born rule is to get a set of ordinary probabilities, which you can then test frequentistically, over a run of experiments. Quantum mechanical measure—amplitude—isn’t ordinary probability, but that’s the thing you put into the Born rule, not the thing you get out of it. And it has it’s own role, which is explaining how much contribution to a coherent superposition each component state makes.
ETA
There is a further problem interpreting the probabilities of fully decohered branches. (Calling then Everett branches is very misleading—a clear theory of decoherence is precisely what’s lacking in Everett’s work)
Whether you are supposed to care about them ethically is very unclear, since it is not clear how utilitarian style ethics would apply, even if you could make sense of the probabilities. But you are not supposed to care about them for the purposes of doing science, since they can no longer make any difference to your branch. MWI works like a collapse theory in practice.
It’s tempting to ethically discount low measure decoherent branches in some way, because that most closely approximates conventional single world utilitarianism—that is something “naive MWI” might mean. However, one should not jump to the conclusion that something is true just because it is convenient. And of course, MWI is a scientific theory so it doesn’t comes with built in ethics.
The alternative view starts with the question of whether a person low measure world still count as a full.person? If they should not, is that because they are a near-zombie, with a faint consciousness that weighs little in a hedonic utilitarian calculus? If they are not such zombies, why would they not count as a full person—the standard utilitarian argument that people in far-off lands are still moral patients seems to apply. Of course, MWI doesn’t directly answer the question about consciousness.
If “naive MWI” means the idea that any elementary interaction produces decoherent branching, then it is wrong for the reasons I explain here. Since there are some coherent superpositions, and not just decoherent branches, there are cases where the Born rule gives you ordinary probabilities, as any undergraduate physics student knows.
It’s not the existence, it’s the lack of interaction/interference.
The topic of this thread is: In naive MWI, it is postulated that all Everett branches coexist. (For example, if I toss a quantum fair coin n times, there will be 2n branches with all possible outcomes.) Under this assumption, it’s not clear in what sense the Born rule is true. (What is the meaning of the probability measure over the branches if all branches coexist?)
But it could matter if its digital vs continuous. <OK longer post and some thoughts a bit off topic perhaps>
Your A,B,C,D … leads to some questions about what is conscious (C) and what isn’t.
Where exactly does the system stop being conscious
1. Biological mind with neurons
2. Very high fidelity render in silicon with neurons modelled down to chemistry rather than just firing pulses
3. Classic neural net spiking approx done in discrete maths that appears almost indistinguishable to 1,2. Producing system states A,B,C,D
4. same as (3) but states are saved/retrieved in memory not calculated.
5. States retrieved from memory many times - A,B,C,D … A,B,C,D … does this count as 1 or many experiences?
6. States retrieved in mixed order A,D,C,B....
7 States D,D,D,D,A,A,A,A,B,B,B,B,C,C,C,C .. does this count 4* or nothing.
A possible cutoff is between 3⁄4. Retrieving instead of calculating makes it non-conscious. But what about caching, some calc, some retrieved?
As you prob know this has been gone over before, e.g. Scott Aaronson. Wonder what your position is?
https://scottaaronson.blog/?p=1951
with quote:
and last but not least:
https://www.scottaaronson.com/papers/giqtm3.pdf
Curated. I like “think clearly about confusing philosophical” topics and this post is a very well-written explainer. I think it’s likely both correct and ought to be convincing. At the same time, I think it’s somewhat incomplete and that a fuller version would make the case for why we should wholly believe that experience is determined by local brain state rather than treat it as an assumption. Beyond that, I think that the most convincing explainer on this topic would build on a fully satisfactory theory of consciousness that totally answers the hard problem and makes none of it feel mysterious at all. Still, glad to see this on LessWrong, taking us kind of bag to our roots.
I find this rather difficult to believe in light of andesoldes’s excellent distillation of Rob’s position and subsequent detailed and concrete explanation of why it seems wrong to have this degree of confidence in his beliefs.
As TAG has written a number of times, the computationalist thesis seems not to have been convincingly (or even concretely) argued for in any LessWrong post or sequence (including Eliezer’s Sequences). What has been argued for, over and over again, is physicalism, and then more and more rejections of dualist conceptions of souls.
That’s perfectly fine, but “souls don’t exist and thus consciousness and identity must function on top of a physical substrate” is very different from “the identity of a being is given by the abstract classical computation performed by a particular (and reified) subset of the brain’s electronic circuit,” and the latter has never been given compelling explanations or evidence. [1] This is despite the fact that the particular conclusions that have become part of the ethos of LW about stuff like brain emulation, cryonics etc are necessarily reliant on the latter, not the former.
As a general matter, accepting physicalism as correct would naturally lead one to the conclusion that what runs on top of the physical substrate works on the basis of… what is physically there (which, to the best of our current understanding, can be represented through Quantum Mechanical probability amplitudes), not what conclusions you draw from a mathematical model that abstracts away quantum randomness in favor of a classical picture, the entire brain structure in favor of (a slightly augmented version of) its connectome, and the entire chemical make-up of it in favor of its electrical connections. As I have mentioned, that is a mere model that represents a very lossy compression of what is going on; it is not the same as the real thing, and conflating the two is an error that has been going on here for far too long. Of course, it very well might be the case that Rob and the computationalists are right about these issues, but the explanation up to now should make it clear why it is on them to provide evidence for their conclusion.
As I have written before about these matters:
It is hard to prove a negative, but I make this claim on the basis of having read ~ the entirety of what has been written on this site about personal identity and consciousness.
I differ from Rob in that I do think his piece should have flagged the assumption of ~computationalism, but think the assumption is reasonable enough to not have argued for in this piece.
I do think it is interesting philosophical discussion to hash it out, for the sake of rigor and really pushing for clarity. I’m sad that I don’t think I could dive in deep on the topic right now.
To answer your question in your other comment. I reckon with some time I could write an explainer for why we should very reasonable assume consciousness is the result of local brain stuff and nothing else (and also not quantum stuff), though I’d be surprised if I could easily write something so rigorous that you’d find it fully satisfactory.
Relatedly to my other comment, I’m curious if you (Ruby) think you would be capable of writing such a version. I’m obviously not asking you to actually write it (you probably have much better things to do with your time), but I do wonder what the answer to this question is, and if it is “no”, then I would also want to ask whether you nonetheless think you have good reasons to believe that Rob’s conclusion is correct (in light of the counterarguments and reasons for skepticism that have been brought up, and which seem to me like they would necessitate that exact “fuller version” to resolve).