In contrast the fourth process is literally insane. [Its mental processes] correspond to nothing in reality (or at least, nothing in its reality).
Rather, it’s not thinking about your reality. It does think about its own reality, for which you’ve even proposed a model (the actual battle). If you argue with it, using a Boltzmann brain that spontaneously appeared at Waterloo as your avatar, it’ll say that it’s you, Boltzmann-Stuart, who is insane, who’s saying that it, the actual Napoleon, sprang into existence from the thermal radiation of a black hole.
Its reality is not morally relevant to you, so its thought about its reality is misguided for the goal of thinking about things that are morally relevant to you. But your reality is morally irrelevant to it, so thinking about your reality is similarly misguided from its point of view. The symmetry arises from different preferences (that are not trivially relocatable for the same reasons as hedonium isn’t), with moral relevance of various worlds an aspect of preference, different near-copies of the same agent valuing different worlds to different extents, because these copies are related to these worlds in different ways.
I get the idea in principle, but not sure I agree with the symmetry in practice.
If we take me and Boltzmann Stuarts, we would have the same values. And, assuming we could somehow communicate, Boltzmann Stuart would defer to current me’s decisions (because our preferences would be over the current me’s world).
I’m guessing you are talking about a different Boltzmann-Stuart than the one from my comment (the one I was talking about has the same info as you, or maybe it is just you, it only acts as an avatar at Waterloo, so there is nothing for you two to argue about).
I think you mean a Stuart Armstrong from another world who you observe as a Boltzmann brain, closer to Boltzmann-Napoleon from the post, except that it’s Stuart and not Napoleon (correct me if I’m wrong). In that case I don’t see why you’d expect it to defer to your decisions more than you’d expect yourself to defer to its decisions, since at least the observations will be analogous. You see a Boltzmann brain that thinks it’s a Stuart from another world (its world), and similarly Boltzmann-Stuart sees a Boltzmann brain that thinks (as you do) that it’s a Stuart from another world (your world). There seems to be no reason to expect one of them to be more persuasive than the other. I’d guess they could cooperate on an equal footing instead.
because our preferences would be over the current me’s world
This is probably the root of the intended disagreement. As with hedonium (simpler versions with major flaws) in your post, preferences may refer to environment, point to it without specifying what it is. As a result, placing the same person in a different world changes their preferences, because the person doesn’t know what they prefer, they need to observe the world to learn which world they care about. So if Boltzmann-Stuart remembers always living in a different world, that’s the world they care about.
There are multiple reasons for not caring about a world that are similar in practice. One is the world having no morally relevant features in it. Another is not being able to affect it, which is usually the case when it’s counterfactual (when you’ve observed events absent in that world, evidence that you are now acting in a different world and so probably don’t affect the counterfactual world). Not discarding hard-to-affect worlds as morally irrelevant is one thing that gives UDT an advantage, as it can look for unusual means of control such as prediction of agent’s decisions made in a world that doesn’t contain the agent in the usual form. Yet another reason to disregard a world is not being able to predict (or compare) consequences of decisions, so that there is no point in caring about which decisions are enacted in that world.
Above these reasons seems to be a global measure over the worlds, which should be part of preference, but it’s not clear if the world where an agent originated shouldn’t leave large bias on which worlds are preferred in this sense. For this thought experiment, this seems to be the only relevant distinction, assuming Boltzmann avatars can coordinate decisions across worlds, so that the factors of unpredictability and inability to control don’t apply (and we know that the worlds have similar features of moral relevance, the same human civilization).
So one way to break the symmetry is to formulate your preference while in the original world, then wipe your memories and replace them with those of another world, while somehow keeping the preference (which threatens to become epiphenomenal in this setup, so perhaps that doesn’t make sense). Otherwise, if you are only human, your preference is not yet formulated sufficiently to become relocatable between worlds without changes in measure of how much various worlds are valued, and so having memories of always living in another world also changes the preference that you’d formulate on reflection.
Why? As I understand it, Boltzmann-Stuart affects its own world, in the same sense as Harry Potter affects Hogwarts but doesn’t affect Mordor, it’s control in a hypothetical situation. We don’t require that any part of the setup is real, instead we see whether the agent controls the environment inside the hypothetical. And its own world has the same kind of valuable people as your world, in the same way as Albus Dumbledore is a morally relevant feature in his hypothetical world, while a non-magical chair in Hogwarts isn’t. The problem with Boltzmann-Stuart’s world is not that it’s not affected by Boltzmann-Stuart, or that it doesn’t have anything of relative value in it (compared to the value of the world itself).
The problem, from your perspective, is that the whole world is in some sense “not real”. For the purposes of decision making, being “not real” seems to be the same thing as not being morally relevant, hence the enumeration of possible reasons for not being considered relevant in my comment. The reason that seems to apply in this case seems to be the measure of caring over possible worlds, which I guess depends on which world it’s originally formulated from.
Another hypothesis is that you don’t identify Botzmann-Stuart and the hypothetical Stuart that lives in the hypothetical world that Boltzmann-Stuart merely believes he’s inhabiting. In that case my argument can be seen as being about the hypothetical Stuart, rather than about Boltzmann-Stuart, with the additional assumption that the hypothetical Stuart should be identified with Boltzmann-Stuart (which I didn’t argue for). I identify them is a way similar to how it’s done in discussions of UDT and counterfactual trade. For example, we may consider hypothetical Stuart a predictive device that manages to enact the decisions of Boltzmann-Stuart, so that any decisions made by Boltzmann-Stuart affect the hypothetical Stuart’s world.
In the example I was thinking, real-world Stuart exists in a world that contains other agents and endures, while Botlzmann Stuart exists in a world that only contains his imagination of other agents, and ends swiftly.
Rather, it’s not thinking about your reality. It does think about its own reality, for which you’ve even proposed a model (the actual battle). If you argue with it, using a Boltzmann brain that spontaneously appeared at Waterloo as your avatar, it’ll say that it’s you, Boltzmann-Stuart, who is insane, who’s saying that it, the actual Napoleon, sprang into existence from the thermal radiation of a black hole.
Its reality is not morally relevant to you, so its thought about its reality is misguided for the goal of thinking about things that are morally relevant to you. But your reality is morally irrelevant to it, so thinking about your reality is similarly misguided from its point of view. The symmetry arises from different preferences (that are not trivially relocatable for the same reasons as hedonium isn’t), with moral relevance of various worlds an aspect of preference, different near-copies of the same agent valuing different worlds to different extents, because these copies are related to these worlds in different ways.
I get the idea in principle, but not sure I agree with the symmetry in practice.
If we take me and Boltzmann Stuarts, we would have the same values. And, assuming we could somehow communicate, Boltzmann Stuart would defer to current me’s decisions (because our preferences would be over the current me’s world).
I’m guessing you are talking about a different Boltzmann-Stuart than the one from my comment (the one I was talking about has the same info as you, or maybe it is just you, it only acts as an avatar at Waterloo, so there is nothing for you two to argue about).
I think you mean a Stuart Armstrong from another world who you observe as a Boltzmann brain, closer to Boltzmann-Napoleon from the post, except that it’s Stuart and not Napoleon (correct me if I’m wrong). In that case I don’t see why you’d expect it to defer to your decisions more than you’d expect yourself to defer to its decisions, since at least the observations will be analogous. You see a Boltzmann brain that thinks it’s a Stuart from another world (its world), and similarly Boltzmann-Stuart sees a Boltzmann brain that thinks (as you do) that it’s a Stuart from another world (your world). There seems to be no reason to expect one of them to be more persuasive than the other. I’d guess they could cooperate on an equal footing instead.
This is probably the root of the intended disagreement. As with hedonium (simpler versions with major flaws) in your post, preferences may refer to environment, point to it without specifying what it is. As a result, placing the same person in a different world changes their preferences, because the person doesn’t know what they prefer, they need to observe the world to learn which world they care about. So if Boltzmann-Stuart remembers always living in a different world, that’s the world they care about.
There are multiple reasons for not caring about a world that are similar in practice. One is the world having no morally relevant features in it. Another is not being able to affect it, which is usually the case when it’s counterfactual (when you’ve observed events absent in that world, evidence that you are now acting in a different world and so probably don’t affect the counterfactual world). Not discarding hard-to-affect worlds as morally irrelevant is one thing that gives UDT an advantage, as it can look for unusual means of control such as prediction of agent’s decisions made in a world that doesn’t contain the agent in the usual form. Yet another reason to disregard a world is not being able to predict (or compare) consequences of decisions, so that there is no point in caring about which decisions are enacted in that world.
Above these reasons seems to be a global measure over the worlds, which should be part of preference, but it’s not clear if the world where an agent originated shouldn’t leave large bias on which worlds are preferred in this sense. For this thought experiment, this seems to be the only relevant distinction, assuming Boltzmann avatars can coordinate decisions across worlds, so that the factors of unpredictability and inability to control don’t apply (and we know that the worlds have similar features of moral relevance, the same human civilization).
So one way to break the symmetry is to formulate your preference while in the original world, then wipe your memories and replace them with those of another world, while somehow keeping the preference (which threatens to become epiphenomenal in this setup, so perhaps that doesn’t make sense). Otherwise, if you are only human, your preference is not yet formulated sufficiently to become relocatable between worlds without changes in measure of how much various worlds are valued, and so having memories of always living in another world also changes the preference that you’d formulate on reflection.
Both are relevant for why Boltzmann stuart would defer to real world stuart.
Why? As I understand it, Boltzmann-Stuart affects its own world, in the same sense as Harry Potter affects Hogwarts but doesn’t affect Mordor, it’s control in a hypothetical situation. We don’t require that any part of the setup is real, instead we see whether the agent controls the environment inside the hypothetical. And its own world has the same kind of valuable people as your world, in the same way as Albus Dumbledore is a morally relevant feature in his hypothetical world, while a non-magical chair in Hogwarts isn’t. The problem with Boltzmann-Stuart’s world is not that it’s not affected by Boltzmann-Stuart, or that it doesn’t have anything of relative value in it (compared to the value of the world itself).
The problem, from your perspective, is that the whole world is in some sense “not real”. For the purposes of decision making, being “not real” seems to be the same thing as not being morally relevant, hence the enumeration of possible reasons for not being considered relevant in my comment. The reason that seems to apply in this case seems to be the measure of caring over possible worlds, which I guess depends on which world it’s originally formulated from.
Another hypothesis is that you don’t identify Botzmann-Stuart and the hypothetical Stuart that lives in the hypothetical world that Boltzmann-Stuart merely believes he’s inhabiting. In that case my argument can be seen as being about the hypothetical Stuart, rather than about Boltzmann-Stuart, with the additional assumption that the hypothetical Stuart should be identified with Boltzmann-Stuart (which I didn’t argue for). I identify them is a way similar to how it’s done in discussions of UDT and counterfactual trade. For example, we may consider hypothetical Stuart a predictive device that manages to enact the decisions of Boltzmann-Stuart, so that any decisions made by Boltzmann-Stuart affect the hypothetical Stuart’s world.
In the example I was thinking, real-world Stuart exists in a world that contains other agents and endures, while Botlzmann Stuart exists in a world that only contains his imagination of other agents, and ends swiftly.