The idea of observer’s stability is fundamental for our understanding of reality (and also constantly supported by our experience) – any physical experiment assumes that the observer (or experimenter) remains the same during the experiment.
avturchin
Karma: 4,326
The same is valid for life extension research. It requires decades, and many, including Brian Johnson, say that AI will solve aging and therefore human research in aging is not relevant. However, most of aging research is about collecting data about very slow processes. The more longitudinal data we collect, the easier it will be for AI to “take up the torch.”
The problem with the subjective choice view is that I can’t become Britney Spears. :) If I continue to sit at the table, I will find myself there every next moment even if I try to become someone else. So mapping into the next moments is an objective fact.
Moreover, even a single moment of experience is a mapping between two states of the brain, A and B. For example, moment A is before I see a rose, and moment B is after I see it and say: “A rose!” The experience of a red rose happens after A but before B.
The rainbow of qualia theory is objective but it assumes the existence of a hypothetical instrument: a qualiascope. A qualiascope is a mind which can connect to other minds and compare their experiences. This works the same way as my mind can compare qualia of colors and sounds without being any of them. Whether a qualiascope is physically possible is not obvious, as its observations may disturb the original qualia.
I think there is more to consider. For example, we can imagine a “qualia rainbow” theory of identity. I don’t necessarily endorse it, but it illustrates why understanding qualia is important for identity.
Imagine that infinitely many different qualia of “reds” could denote one real red. Each person, when born, is randomly initialized with a unique set of qualia for all colors and other sensations. This set can be called a “rainbow of qualia,” and continuous computing in the brain maintains it throughout a person’s life. A copy of me with a different set of qualia, though behaviorally indistinguishable, is not me. Only future mind states with the same set of qualia as mine are truly me, even if my memories were replaced with those of a rat.
Anthropic Trilemma is masterpiece.
Generally, I agree with what you said above—there is no (with some caveats—see below) soul-like identity, and we should use informational identity instead. Informational identity is objective, measurable sameness of memory and allows existence of many copies. It can be used to survive the end of the universe. I just care about the existence of a copy of me in another universe.
The main caveat is that the no-soul view ignores the existence of qualia. Qualia and the nature of consciousness are not solved yet, and we can’t claim that the identity problem is solved without first solving qualia and consciousness.
The theory of quantum immortality depends on the theory of identity which—as you correctly pointed out—is difficult.
There are two objective facts about identity:
I will be in my next observer-moment in the next moment of time. There is an objective process which makes mind states follow one another.
I can recognize myself as me or not.
In your thought experiment, these two properties are deliberately made to contradict each other.
A simple answer here is that you should not anticipate becoming a pig because a pig can’t think about personal identity. Anticipation assumes comparison between expectation and reality. A pig can’t perform such an operation. But this is not a satisfactory model.
It can be solved if we assume that we have two types of identity—informational (me or not me) and continuous. This seems paradoxical. But if we then assume that continuous identity passes through all possible minds eventually, then any pig will eventually become me again in some multiverse timelines, and I can calculate a share of my future copies which have a memory of being a pig.
This thought experiment can be done without any supertechnology, just using dreaming as an example: what if some of my copies will have a dream that they are pigs, and others have a dream about being themselves. The idea of anticipation produces error in that case, as in one way it assumes the existence of a mind capable of comparison, but in another way it assumes natural consequences of mind states.
In short, a correct identity theory allows one to compute correct probabilities of future observations in the situation of many different copies. See also my post The Quantum Mars Teleporter: An Empirical Test Of Personal Identity Theories.
We can define “me” as an observer, who has the same set of the memories M. In that case the theory of QI is formally correct.
Quantum immortality moves one into s-risk eventually, if it is not used properly. For example, in your scenario, I can change my internal clock so my next observer-moments will be in another universe which didn’t reached heat death yet. This works for state-based identity theory.
For continuity-based identity theory, I can master (via universal wish-fulfilling machine based on quantum immortality) an explosion of a new universe via quantum fluctuation, jump into it and survive (because of quantum immortality) all difficulties of the initial period.
One more way is to expect my survival via quantum immortality. This, however, increases the chances of observing s-risks. In a broader view, the share of future worlds with me where AI is aligned or non-existent is larger than the share of s-risks worlds.
Thus, I will observe myself surviving AI, but it will be bad or good depending on the exact theory of personal identity: whether it counts states or continuity.
Paragraphs are also getting shorter. And music is getting louder.
I think that the scenario of the war between several ASI (each merged with its origin country) is underexplored. Yes, there can be a value handshake between ASIs, but their creators will work to prevent this and see it as a type of misalignment.
Somehow, this may help some groups of people survive, as those ASI which preserve their people will look more trustworthy in the eyes of other ASIs, and this will help them form temporary unions.The final outcome will be highly unstable: either one ASI will win, or several ASIs will start space exploration in different directions.
Ok
[Deleted]
If we know the correct answers to decision theory problems, we have some internal instrument: either a theory or a vibe meter, to learn the correct answers.
Claude seems to learn to mimic our internal vibe meter.
The problem is that it will not work outside the distribution.
Yes, great variant of the universal answer-improving prompt and it can be applied several times to any content.
If the simulation argument is valid and dreams are simulations of reality, can we apply the simulation argument to dreams? If not, is this an argument against the simulation argument? If yes, why am I not now in a dream?
If I see something, is it more likely to be dream or reality?
Sleeping takes only one-third of my time, and REM takes even less.
But:Some dreams occur even in other phases of sleep
Dreams are much more eventful than normal life. There is always something happening. Also, the distribution of events in dreams is skewed toward expensive, dangerous, adventurous content, full of social interactions.
There is an eraser of dream memory, which cleans memories of dreams after every 15 minutes and also after awakening and during the day. As a result, we underestimate the number of dreams we have had.
As a result, the number of important events in dreams may be several orders of magnitude more than in real life. I think a good estimate is 100 times, but it depends on the types of events. For recurrent dreams—like big waves and war for me—it can be much higher.
So why am I not in a dream now? Because writing coherent dream-conscious (lucid) text is not the dominant type of content in dreams. But if I were chased by a monster or big waves, I should give higher a priori chances that I am actually dreaming.
Conclusion: The simulation argument works for dreams, but selectively, as dream content is different from most normal life content.
Yes, but it knows all Bostrom articles, maybe because it has seen the list a hundred times.
Most LLMs’ replies can be improved by repeatedly asking “Improve the answer above” and it is similar to the test-time compute idea and diffusion.
In most cases, I can get better answers from LLMs just by asking “Improve the answer above.”
In my experience, the improvements are observable for around 5 cycles, but after that the result either stops improving or gets stuck in some error mode and can’t jump to a new level of thinking. My typical test subject: “draw a world map as text art.” In good improvement sessions with Sonnet, it eventually adds grids and correct positions for continents.
One person on Twitter (I lost the link, maybe @goodside) automated this process and got much better code for a game after 100 cycles of improvements during an entire night using many credits. He asked Claude to write code for automated prompting first. I repeated this experiment with my tasks.
I tried different variants of “improve it,” like adding critiques or generating several answers within one reply. I also tried a meta-level approach, where I asked to improve not only the answer but also the prompt for improvements.
I started these experiments before the test-time compute idea went mainstream, and it looks like a type of test-time compute use. The process also resembles diffusion.
The main question here: in which cases does the process quickly get stuck, and in which does it produce unbounded improvements? It seems to get stuck in local minima and in situations where the model’s intelligence isn’t sufficient to see ways to improve or discern better or worse versions. It also can’t jump to another valley: if it started improving in some direction, it will continue to push in that direction, ignoring other possibilities. Only running another chat window manually helps to change valleys.
Iterative improvement of images also works in GPT-4o. But not for Gemini Pro 2.5, and o1 is also bad at improving, progressing very slowly. It seems that test-time improving contradicts test-time reasoning.
Results for “Improve it”: https://poe.com/s/aqk8BuIoaRZ7eDqgKAN6
Variants of the main prompt: “Criticize the result above and iteratively improve it” https://poe.com/s/A2yFioj6e6IFHz68hdDx
This prompt—“Create a prompt X for iterative improvement of the answer above. Apply the generated prompt X.”—converges quickly to extraordinary results but overshoots, like creating games instead of drawings. It also uses thinking: https://poe.com/s/cLoB7gyGXHNtwj0yQfPf
The trick is that the improving prompt should be content-independent and mechanically copy-pasted after each reply.
It looks like (based on the article published a few days ago by Anthropic about the microscope) Claude Sonnet was trained to distinguish facts from hallucinations, so it’s not surprising that it knows when it hallucinates.
I can also use functional identity theory, where I care about the next steps of agents functionally similar to my current thought-line in logical time.