Thanks. By the way, the “chatification” of the mind is a real problem. It’s an example of reverse alignment: humans are more alignable than AI (we are gullible), so during interactions with AI, human goals will drift more quickly than AI goals. In the end, we get perfect alignment: humans will want paperclips.
avturchin
Karma: 3,914
For the outside view: Imagine that an outside observer uses a fair coin to observe one of two rooms (assuming merging in the red room has happened). They will observe either a red room or a green room, with a copy in each. However, the observer who was copied has different chances of observing the green and red rooms. Even if the outside observer has access to the entire current state of the world (but not the character of mixing of the paths in the past), they can’t determine the copied observer’s subjective chances. This implies that subjective unmeasurable probabilities are real.
Even without merging, an outside observer will observe three rooms with equal 1⁄3 probability for each, while an insider will observe room 1 with 1⁄2 probability. In cases of multiple sequential copying events, the subjective probability for the last copy becomes extremely small, making the difference between outside and inside perspectives significant.
When I spoke about the similarity with the Sleeping Beauty problem, I meant its typical interpretation. It’s an important contribution to recognize that Monday-tails and Tuesday-tails are not independent events.
However, I have an impression that this may result in a paradoxical two-thirder solution: In it, Sleeping Beauty updates only once – recognizing that there are two more chances to be in tails. But she doesn’t update again upon knowing it’s Monday, as Monday-tails and Tuesday-tails are the same event. In that case, despite knowing it’s Monday, she maintains a 2⁄3 credence that she’s in the tails world. This is technically equivalent to the ‘future anthropic shadow’ or anti-doomsday argument – the belief that one is now in the world with the longest possible survival.
Thanks for your thoughtful answer.
To achieve magic, we need the ability to merge minds, which can be easily done with programs and doesn’t require anything quantum. If we merge 21 and 1, both will be in the same red room after awakening. If awakening in the red room means getting 100 USD, and the green room means losing it, then the machine will be profitable from the subjective point of view of the mind which enters it. Or we can just turn off 21 without awakening, in which case we will get 1⁄3 and 2⁄3 chances for green and red.
The interesting question here is whether this can be replicated at the quantum level (we know there is a way to get quantum magic in MWI, and it is quantum suicide with money prizes, but I am interested in a more subtle probability shift where all variants remain). If yes, such ability may naturally evolve via quantum Darwinism because it would give an enormous fitness advantage – I will write a separate post about this.
Now the next interesting thing: If I look at the experiment from outside, I will give all three variants 1⁄3, but from inside it will be 1⁄4, 1⁄4, and 1⁄2. The probability distribution is exactly the same as in Sleeping Beauty, and likely both experiments are isomorphic. In the SB experiment, there are two different ways of “copying”: first is the coin and second is awakenings with amnesia, which complicates things.
Identity is indeed confusing. Interestingly, in the art world, path-based identity is used to define identity, that is, the provenance of artworks = history of ownership. Blockchain is also an example of path-based identity. Also, in path-based identity, the Ship of Theseus remains the same.
There is a strange correlation between paradox of young Sun (it had lower luminosity) and stable Earth temperature which was provided by higher greenhouse effect. As sun goes brighter, CO2 declined. It was even analyses as evidence of anthropic effects.
In his article “The Anthropic Principle in Cosmology and Geology” [Shcherbitsky, 1999], A. S. Shcherbakov thoroughly examines the anthropic principle’s effect using the historical dynamics of Earth’s atmosphere as an example. He writes: “It is known that geological evolution proceeds within an oscillatory regime. Its extreme points correspond to two states, known as the ‘hot planet’ and ‘white planet’… The ‘hot planet’ situation occurs when large volumes of gaseous components, primarily carbon dioxide, are released from Earth’s mantle...As calculations show, the gradual evaporation of ocean water just 10 meters deep can create such greenhouse conditions that water begins to boil. This process continues without additional heat input. The endpoint of this process is the boiling away of the oceans, with near-surface temperatures and pressures rising to hundreds of atmospheres and degrees… Geological evidence indicates that Earth has four times come very close to total glaciation. An equal number of times, it has stopped short of ocean evaporation. Why did neither occur? There seems to be no common and unified saving cause. Instead, each time reveals a single and always unique circumstance. It is precisely when attempting to explain these that geological texts begin to show familiar phrases like ‘...extremely low probability,’ ‘if this geological factor had varied by a small fraction,’ etc...
In the fundamental monograph ‘History of the Atmosphere’ [Budyko, 1985], there is discussion of an inexplicable correlation between three phenomena: solar activity rhythms, mantle degassing stages, and the evolution of life. ‘The correspondence between atmospheric physicochemical regime fluctuations and biosphere development needs can only be explained by random coordination of direction and speed of unrelated processes—solar evolution and Earth’s evolution. Since the probability of such coordination is exceptionally small, this leads to the conclusion about the exceptional rarity of life (especially its higher forms) in the Universe.’”
Quantum immortality and gun jammed do not contradict each other: for example, if we survive 10 rounds failures because of QI, we most likely survive only on those timelines where gun is broken. So both QI and gun jamming can be true and support one another and there is no contradiction.
One problem here is that quantum immortality and angel immortality eventually merges: for example, if we survive 10 LHC failures because of QI, we most likely survive only on those timelines where some alien stops LHC. So both QI and angel immortality can be true and support one another and there is no contradiction.
I know this post and have two problems with it: what they call ’anthropic shadow” is not proper term as Bostrom defined anthropic shadow as underestimation of past risks based on the fact of survival in his article this the same name. But it’s ok.
The more serious problem is that quantum immortality and angel immortality eventually merges: for example, if we survive 10 LHC failures because of QI, we most likely survive only on those timeline where some alien stops LHC. So both QI and angel immortality can be true and support one another and there is no contradiction.
Check my new post which favors the longest and thickest timelines https://www.lesswrong.com/posts/hB2CTaxqJAeh5jdfF/quantum-immortality-a-perspective-if-ai-doomers-are-probably?commentId=aAzrogWBqtFDqMMpp
A sad thing is that most of life moments are like this 30-minutes intervals—we forget most life events, they are like dead ends.
More generally, type-copies of me still matter for me.
Under them the chance for you to find yourself in a branch where all coins are Heads is 1⁄128, but your over chance to survive is 100%. Therefore the low chance of failed execution doesn’t matter, quantum immortality will “increase” the probability to 1
You are right, and it’s a serious counterargument to consider. Actually, I invented path-dependent identity as a counterargument to Miller’s thought experiment.
You are also right that the Anthropic Trilemma and Magic by Forgetting do not work with path-dependent identity.
However, we can almost recreate the magic machine from the Anthropic Trilemma using path-based identity:
Imagine that I want to guess in which room I will be if there are two copies of me in the future, red or green.
I go into a dream. A machine creates my copy and then one more copy of that copy, which will result in 1⁄4 and 1⁄4 chances each. The second copy then merges with the first one, so we end up with only two copies, but I have a 3⁄4 chance to be the first one and 1⁄4 to be the second. So we’ve basically recreated a machine that can manipulate probabilities and got magic back.
The main problem of path-dependent identity is that we assume the existence of a “global hidden variable” for any observer. It is hidden as it can’t be measured by an outside viewer and only represents the subjective chances of the observer to be one copy and not another. And it is global as it depends on the observer’s path, not their current state. It therefore contradicts the view that mind is equal to a Turing computer (functionalism) and requires the existence of some identity carrier which moves through paths (qualia, quantum continuity, or soul).
Also, path-dependent identity opens the door to back-causation and premonition, because if we normalize outputs of some black box where paths are mixed, similar to the magic machine discussed above, we get a shift in its input probability distribution in the past. This becomes similar to the ‘timeline selection principle’ (which I discussed in a longer version of this blog post but cut to fit format) in which not observer-moments are selected, but the whole timelines without updating on my position in the timeline. This idea formalizes the future anthropic shadow as I am more likely to be in the timeline that is fattest and longest in the future.
As we assume that coin tosses are quantum, and I will be killed if (I didn’t guess pi) or (coin toss is not heads) there is always a branch with 1⁄128 measure where all coins are heads, and they are more probable than surviving via some errors in the setup.
All hell breaks loose” refers here to a hypothetical ability to manipulate perceived probability—that is, magic. The idea is that I can manipulate such probability by changing my measure.One way to do this is described in Yudkowsky’s ” The Anthropic Trilemma,” where an observer temporarily boosts their measure by increasing the number of their copies in an uploaded computer.
I described a similar idea in “Magic by forgetting,” where the observer boosts their measure by forgetting some information and thus becoming similar to a larger group of observers.
Hidden variables also appear depending on the order in which I make copies: if each copy is made from subsequent copies, the original will have a 0.5 probability, the first copy 0.25, the next 0.125, and so on.
“Anthropic shadow” appear only because the number of observers changes in different branches.
Orthogonality between goals and DT makes sense only if I don’t have preferences about the type of DT or the outcomes which one of them necessitates.
In the case of QI, orthogonality works if we use QI to earn money or to care about relatives.
However, humans have preferences about existence and non-existence beyond normal money utility. In general, people strongly don’t want to die. It means that I have a strong preference that some of my copies survive anyway, even if it is not very useful for some other preferences under some other DT.
Another point is the difference between Quantum suicide and QI. QS is an action, but QI is just a prediction of future observations and because of that it is less affected by decision theories. We can say that those copies of me who survive [high chance of death event] will say that they survived because of QI.
But if I use quantum coin to make a life choice, there will be splitting, right?
Wei· 3h
This post touches on several issues I’ve been thinking about since my early work on anthropic decision theory and UDT. Let me break this down:
1. The measure-decline problem is actually more general than just quantum mechanics. It appears in any situation where your decision algorithm gets instantiated multiple times, including classical copying, simulation, or indexical uncertainty. See my old posts on anthropic probabilities and probability-as-preference.
2. The “functional identity” argument being used here to dismiss certain types of splitting is problematic. What counts as “functionally identical” depends on your decision theory’s level of grain. UDT1.1 would treat seemingly identical copies differently if they’re in different computational states, while CDT might lump them together.
Some relevant questions that aren’t addressed:
- How do we handle preference aggregation across different versions of yourself with different measures?
- Should we treat quantum branching differently from other forms of splitting? (I lean towards “no” these days)
- How does this interact with questions of personal identity continuity?
- What happens when we consider infinite branches? (This relates to my work on infinite ethics)The real issue here isn’t about measure per se, but about how to aggregate preferences across different instances of your decision algorithm. This connects to some open problems in decision theory:
1. The problem of preference aggregation across copies
2. How to handle logical uncertainty in the context of anthropics
3. Whether “caring about measure” can be coherently formalizedI explored some of these issues in my paper on UDT, but I now think the framework needs significant revision to handle these cases properly.
Stuart · 2h
> The problem of preference aggregation across copies
This seems key. Have you made any progress on formalizing this since your 2019 posts?Wei · 2h
Some progress on the math, but still hitting fundamental issues with infinity. Might post about this soon.Abram · 1h
Curious about your current thoughts on treating decision-theoretic identical copies differently. Seems related to logical causation?Wei · 45m
Yes—this connects to some ideas about logical coordination I’ve been developing. The key insight is that even “identical” copies might have different logical roles...[Edit: For those interested in following up, I recommend starting with my sequence on decision theory and anthropics, then moving to the more recent work on logical uncertainty.]
Vladimir_N 3h
(This is a rather technical comment that attempts to clarify some decision-theoretic confusions.)
Your treatment of measure requires more formal specification. Let’s be precise about what we mean by “caring about measure” in decision-theoretic terms.
Consider a formalization where we have:
1. A space of possible outcomes Ω
2. A measure μ on this space
3. A utility function U: Ω → ℝ
4. A decision function D that maps available choices to distributions over ΩThe issue isn’t about “spending” measure, but about how we aggregate utility across branches. The standard formulation already handles this correctly through expected utility:
E[U] = ∫_Ω U(ω)dμ(ω)
Your concern about “measure decline” seems to conflate the measure μ with the utility U. These are fundamentally different mathematical objects serving different purposes in the formalism.
If we try to modify this to “care about measure directly,” we’d need something like:
U’(ω) = U(ω) * f(μ(ω))
But this leads to problematic decision-theoretic behavior, violating basic consistency requirements like dynamic consistency. It’s not clear how to specify f in a way that doesn’t lead to contradictions.
The apparent paradox dissolves when we properly separate:
1. Measure as probability measure (μ)
2. Utility as preference ordering over outcomes (U)
3. Decision-theoretic aggregation (E[U])[Technical note: This relates to my work on logical uncertainty and reflection principles. See my 2011 paper on decision theory in anthropic contexts.]
orthonormal · 2h
> U’(ω) = U(ω) * f(μ(ω))
This is a very clean way of showing why “caring about measure” leads to problems.Vladimir_N · 2h
Yes, though there are even deeper issues with updateless treatment of anthropic measure that I haven’t addressed here for brevity.Wei_D · 1h
Interesting formalization. How would this handle cases where the agent’s preferences include preferences over the measure itself?Vladimir_N · 45m
That would require extending the outcome space Ω to include descriptions of measures, which brings additional technical complications...[Note: This comment assumes familiarity with measure theory and decision theory fundamentals.]
Eli · 2h
*sigh*
I feel like I need to step in here because people are once again getting confused about measure, identity, and decision theory in ways I thought we cleared up circa 2008-2009.
First: The whole “measure declining by choice” framing is confused. You’re not “spending” measure like some kind of quantum currency. The measure *describes* the Born probabilities; it’s not something you optimize for directly any more than you should optimize for having higher probabilities in your belief distribution.
Second: The apparent “splitting” of worlds isn’t fundamentally different between quantum events, daily choices, and life-changing decisions. It’s all part of the same unified wavefunction evolving according to the same physics. The distinction being drawn here is anthropocentric and not particularly meaningful from the perspective of quantum mechanics.
What *is* relevant is how you handle subjective anticipation of future experiences. But note that “caring about measure” in the way described would lead to obviously wrong decisions—like refusing to make any choices at all to “preserve measure,” which would itself be a choice (!).
If you’re actually trying to maximize expected utility across the multiverse (which is what you should be doing), then the Born probabilities handle everything correctly without need for additional complexity. The framework I laid out in Quantum Ethics handles this cleanly.
And please, can we stop with the quantum suicide thought experiments? They’re actively harmful to clear thinking about decision theory and anthropics. I literally wrote “Don’t Un-think the Quantum” to address exactly these kinds of confusions.
(Though I suppose I should be somewhat grateful that at least nobody in this thread has brought up p-zombies or consciousness crystals yet...)
[Edit: To be clear, this isn’t meant to discourage exploration of these ideas. But we should build on existing work rather than repeatedly discovering the same confusions.]
RationalSkeptic · 1h
> like refusing to make any choices at all to “preserve measure,”
This made me laugh out loud. Talk about Pascal’s Mugging via quantum mechanics...Eli · 45m
Indeed. Though I’d note that proper handling of Pascal’s Mugging itself requires getting anthropics right first...
In replies to this comment I will post other Sonnet3.5-generated replies by known LW people. If it is against the rules please let me know and I will delete. I will slightly change the names, so they will not contaminate future search and AI training
My point was that only 3 is relevant. How it improves average decision making?
In big world immortality there are causally disconnected copies which survive in very remote regions of the universe. But if we don’t need continuity, but only similarity of minds, for identity, it is enough.
We care still alive
No GPT-5 yet
Rumors of hitting the wall