We are in the world where the calculator displays even, and we are 99% sure it is the world where the calculator has not made an error. This is Even World, Right Calculator. Counterfactual worlds:
Even World, Wrong Calculator (1% of Even Worlds)
Odd World, Right Calculator (99% of Odd Worlds)
Odd World, Wrong Calculator (1% of Odd Worlds)
All Omega told us was that the counterfactual world we are deciding for, the calculator shows Odd. We can therefore eliminate Odd World, Wrong Calculator. Answering the question is, in essence, deciding which world we think we’re looking at.
So, in the counterfactual world, we’re either looking at Even World, Wrong Calculator or Odd World, Right Calculator. We have an equal prior for the world being Odd or Even—or, we think the number of Odd Worlds is equal to the number of Even Worlds. We know the ratio of Wrong Calculator worlds to Right Calculator worlds (1:99). This is, therefore, 99% evidence for Odd World. The correct decision for the counterfactual you in that world is to decide Odd World. The correct decision for you?
Ignoring Bostrom’s book on how to deal with observer selection effects (did Omega go looking for a Wrong Calculator world and report it? Did Omega go looking for an Odd World to report to you? Did Omega pick at random from all possible worlds? Did Omega roll a three-sided die to determine which counterfactual world to report?), I believe the correct decision is to answer Odd World for the counterfactual world, with 99% certainty if you are allowed to specify as such.
I reason that by virtue of it being a counterfactual world, it is contingent on my not having the observation of my factual world; factual world observations are screened off by the word “counterfactual”.
The other possibility (which I tentatively think is wrong) is that our 99% confidence of Even World (from our factual world) comes up against our 99% confidence of Odd World (from our counterfactual) and they cancel out, bringing you back to your prior. So you should flip a coin to decide even or odd. I think this is wrong because 1) I think you could reason from 50% in the countefactual world to 50% in the factual world, which is wrong, and 2) this setup is identical to punching in the formula, pressing the button and observing “even”, then pressing the button again and observing “odd”. I don’t think you can treat counterfactual worlds as additional observations in this manner.
edit: It occurs to me that with Omega telling you about the counterfactual world, you are receiving a second observation. For this understanding, you would specify Even World with 99% confidence in the factual world and either Even or Odd World depending on how the coin landed for the counterfactual world.
The correct decision for the counterfactual you in that world is to decide Odd World.
I believe the above is correct updateless analysis of the thought experiment. (Which is a natural step to take in considering it, but not the point of the post, see its last paragraph.)
It occurs to me that with Omega telling you about the counterfactual world, you are receiving a second observation. For this understanding, you would specify Even World with 99% confidence in the factual world and either Even or Odd World depending on how the coin landed for the counterfactual world.
Vladimir says that “Omega doesn’t touch any calculator”. If the counterfactual is entered at the point where the computation starts and Omega tells you that it results in Odd (ETA2: rereading Vladimir’s comment, this is not the case), then it is a second observation contributed by Omega running the calculator and should affect both worlds. If on the other hand the counterfactual is just about the display, then the counterfactual Omega will likely write down Odd (ETA3: not my current answer). So I agree with your analysis. I see it this way: real Omegas cannot write on counterfactual paper.
ETA: -- the “counterfactual” built as “being in another quantum branch of exactly the same universe” strikes me as being of the sort where Omega does run the calculator again, so it should affect both worlds as another observation.
Ignoring Bostrom’s book on how to deal with observer selection effects (did Omega go looking for a Wrong Calculator world and report it? Did Omega go looking for an Odd World to report to you? Did Omega pick at random from all possible worlds? Did Omega roll a four-sided die to determine which world to report?)
Actually, isn’t this the very heart of the matter? In my other comment here I assumed Omega would always ask what the correct answer is if the calculator shows The Other Result; if that’s not the case everything changes.
The answer does depend on this fact, but since this fact wasn’t specified, assume uncertainty (say, Omega always appears when you observe “even” and had pasta for breakfast).
This would be correct if Q could be different, but Q is the same both in the counterfactual and the actual word.
There is no possibility for the actual world being Even World and the counterfactual Odd World.
The possibilities are:
Actual: Even World, Right Calculator (99% of Even Words); Counterfactual: Even World, Wrong Calculator (1% of Even Worlds).
Actual: Odd World, Wrong Calculator (1% of Odd Words); Counterfactual: Odd World, Right Calculator (99% of Odd Words).
The prior probability of either is 50%. If we assume That Omega randomly picks one you out of 100% of possible words(either 100% of all Even Worlds or 100% of all Odd Words) to decide for all possible words where the calculator result is different (but the correct answer is the same), then there is a 99% chance all worlds are Even and your choice affects 1% of all worlds and a 1% chance all words are Odd and your choice affects 99% of all worlds. The result of the calculator in the counterfactual world doesn’t provide any evidence on whether all words are Even or all worlds are Odd since in either case there would be such a world to talk about.
If we assume that Omega randomly visits one world and randomly mentions the calculator result of one other possible world and it just happened to be the case that in that other world the result was different; or if Omega randomly picks a world, then randomly picks a world with the opposite calculator result and tosses a coin as to which world to visit and which to mention then the calculator result in the counterfactual word is equally relevant and hearing Omega talk about it just as good as running the calculator twice. In this case you are equally likely to be in a Odd world and might just as well toss a coin as to which result you fill in yourself.
Q is the same both in the counterfactual and the actual world.
This doesn’t square with my interpretation of the premises of the question. We are unsure of Q’s parity. Our prior is 50:50 odd, even. We are also unsure of calculator’s trustworthiness. Our prior is 99:1 right, wrong. Therefore—on my understanding of counterfactuality—both options for both uncertainties need to be on the table.
I am unconvinced you can ignore your uncertainty on Q’s parity by arguing that it will come out only one way regardless of your uncertainty—this is true for coinflips in deterministic physics, but that doesn’t mean we can’t consider the counterfactual where the coin comes up tails.
Q is the same both in the counterfactual and the actual world.
This doesn’t square with my interpretation of the premises of the question.
From the original post:
Consider the counterfactual where the calculator displayed “odd” instead of “even”, after you’ve just typed in the (same) formula Q, on the same occasion (i.e. all possible worlds that fit this description). The counterfactual diverges only in the calculator showing a different result (and what follows).
We cannot determine Q’s parity, except by fallible calculator. When you say Q is the same, you seem to be including “Q’s parity is the same”.
Hmm. Maybe this will help?
The parity of Q is already determined—Fermi and Neumann worked it out long ago and sealed it in a safe. You punch Q into the calculator and receive the answer “even”. Omega appears, and asks you to consider the counterfactual where the calculator shows “odd”. Omega offers to let factual you determine what is written on the sheet in the counterfactual world. No matter what is written down in either world, Fermi and Neumann’s answer in the safe will remain the same.
Your factual-world observation of “even” on the calculator makes you think it very likely the counterfactual world is just the cases of the calculator being wrong. You would desire to have Omega write down “even” in the counterfactual world too.
But consider this situation:
The parity of Q is already determined—Fermi and Neumann worked it out long ago and sealed it in a safe. You punch Q into the calculator and receive the answer “odd”. Omega appears, and asks you to consider the counterfactual where the calculator shows “even”. Omega offers to let factual you determine what is written on the sheet in the counterfactual world. No matter what is written down in either world, Fermi and Neumann’s answer in the safe will remain the same.
Your factual-world observation of “odd” on the calculator makes you think it very likely the counterfactual world is just the cases of the calculator being wrong. You would desire to have Omega write down “odd” in the counterfactual world too.
These situations are clearly the counterfactuals of each other—that is, when scenario 1 says “the counterfactual world” it is saying “scenario 2″, and vice versa. The interpretations given in the second half of each contradict each other—the first scenario attempts to decide for the second scenario and gets it wrong; the second scenario attempts to decide for the first and gets it wrong. Whence this contradiction?
Yes, that would be a counterfactual. But NOT the counterfactual under consideration. The counterfactual under consideration was the calculator result being different but Q (both the number and the formula, and thus their parity) being the same. Unless Nesov was either deliberately misleading or completely failed his intention to clarify anything the comments linked to. If Q is the same formula is supposed to be clear in any way then everything about Q has to be the same. If the representation of Q in the formula was supposed be the same, but the actual value possibly counterfactually different then only answering that the formula is the same is obscuration, not clarification.
Yes, that would be a counterfactual. But NOT the counterfactual under consideration.
I disagree. Recall that I specified this in each case:
The parity of Q is already determined—Fermi and Neumann worked it out long ago and sealed it in a safe.
Q (both the number and the formula, and thus the parity) is the same in both scenarios. The actual value is not counterfactually different—it’s the same value in the safe, both times.
If you agree that Q’s parity is the same I’m not sure what you are disagreeing with. Its not possible for Q to be odd in the counterfactual and even in actuality, so if Q is odd in the counterfactual that implies it is also odd in actuality and vice versa. Thus it’s not possible for the calculator to be right in both counterfactual and reality simultaneously, and assuming it to be right in the counter-factual implies that it’s wrong in actuality. Therefore you can reduce everything to the two cases I used, Q even/actual calculator right/counterfactual calculator wrong or Q odd/actual calculator wrong/counterfactual calculator right.
Maybe this could be more enlightening. When you control things, one of the necessary requirements is that you have logical uncertainty about some property of the thing you control. You start with having a definition of the control target, but not knowing some of its properties. And then you might be able to infer a dependence of one of its properties on your action. This allows you to personally determine what is that property of a structure whose definition you already know. See my posts on ADT for more detail.
Therefore you can reduce everything to the two cases I used,
I have been positing that these two cases are counterfactuals of each other. Before one of these two cases occurs, we don’t know which one will occur. It is possible to consider being in the other case.
The problem is symmetrical. You can just copy everything, replace odd with even and vice versa and multiply everything with 0.5, then you also have the worlds where you see odd and Omega offers you to replace the result in counterfactuals where it came up even and where Q has the same parity. Doesn’t change that Q is the same in the world that decides and the counterfactuals that are effected. Omega also transposing your choice to impossible worlds (or predicting what would happen in impossible worlds and imposing that on what happens in real worlds) would be a different problem (that violates that Q be the same in the counterfactual, but seems to be the problem you solved).
If someone is sure enough that I’m wrong to downvote all my post on this they should be able to tell me where I’m wrong. I would be extremely interested in finding out.
I don’t know why you were downvoted. But I do notice that somewhere on this thread, the meaning of “Even World” has changed from what it was when Shokwave introduced the term. Originally it meant a world whose calculator showed ‘Even’.
You’re reasoning about the counterfactual using observational knowledge, i.e. making exactly the error whose nature puzzles me and is the subject of the post. In safely correct (but unenlightening about this error) updateless analysis, on the other hand, you don’t update on observations, so shouldn’t say things like “there is a 99% chance all worlds are Even”.
No. That’s completely insubstantial. Replace “even” with “same parity” and “odd” with “different parity” in my argument and the outcome is the same. The decision can be safely made before making any observations at all.
EDIT: And even in the formulation given I don’t update on personally having seen the even outcome (which is irrelevant, there is no substantial difference between me and the mes at that point) but Omega visiting me in a world where the calculator result came up even.
Please restate in more detail how you arrived at the following conclusion, and what made it so instead of the prior 50⁄50 for Even/Odd. It appears that it must be the observation of “even”, otherwise what privileged Even over Odd?
then there is a 99% chance all worlds are Even and your choice affects 1% of all worlds and a 1% chance all words are Odd and your choice affects 99% of all worlds
See the edit. If Omega randomly visits a possible world I can say ahead of time that there is a 99% chance that in that particular world the calculator result is correct and the decision will affect 1% of all worlds and a 1% chance that the result is wrong and the decision affects 99% of all worlds.
So you know a priori that the answer is Even, without even looking at the calculator? That can’t be right.
(You’re assuming that you know that Omega only arrives in “even” worlds, and updating on observing Omega, even before observing it. But in the same movement, you update on the calculator showing “even”. Omega doesn’t show up in the “odd” world, so you can’t update on the fact that it shows up, other than by observing it, or alternatively observing “even” given the assumption of equivalence of these events.)
So you know a priori that the answer is Even, without even looking at the calculator?
Of course not.
You’re assuming that you know that Omega only arrives in “even” worlds,
No. I’m assuming that either even is correct in all worlds or odd is correct in all worlds (0.5 prior for either). If Omega randomly picks a world, the chance of the calculator being correct is independent of that and 99% everywhere, then there is a 99% chance of the calculator being correct in the particular world Omega arrives in. If odd is correct Omega is 99% likely to arrive in a world where the calculator says odd, and if the calculator says odd in the particular world Omega arrives in there is a 99% chance that’s because odd is correct.
EDIT:
If I were
assuming that you know that Omega only arrives in “even” worlds
the probability of even being correct would be 50% no matter what, and there would be a 50% chance each for affecting 99% of all worlds or 1% of all worlds.
I seem to agree with all of the above statements. The conditional probabilities are indeed this way. But it’s incorrect to use these conditional probabilities (which is to say, probabilities of Odd/Even after updating on observing “even”) to compute expected utility for the counterfactual. In a prior comment, you write:
there is a 99% chance all worlds are Even and your choice affects 1% of all worlds and a 1% chance all words are Odd and your choice affects 99% of all worlds
99% is P(Even|Omega,”even”), that is to say it’s probability of Even updated by observations (events) that Omega and “even”.
I seem to agree with all of the above statements. The conditional probabilities are indeed this way. But it’s incorrect to use these conditional probabilities (which is to say, probabilities of Odd/Even after updating on observing “even”) to compute expected utility for the counterfactual.
No. There is no problem with using conditional probabilities if you use the correct conditional probabilities, that is the probabilities from wherever the decision happens, not from what you personally encounter. And I never claimed that any of the pieces you were quoting were part of an updateless analysis, just that it made no difference.
I would try to write a Wei Dai style world program at this point, but I know no programming at all and am unsure how drawing at random is supposed to be represented. It would be the same as the program for this game, though:
1 black and 99 white balls in an urn. You prefer white balls. You may decide to draw a ball and change all balls of the other color to balls of the color drawn, and must decide before the draw is made. (or to make it slightly more complicated: Someone else secretly flips a coin whether you get points for black or white balls. You get 99 balls of the color you get points for and one ball of the other color).
It would help a lot if you just wrote the formulas you use for computing expected utility (or the probabilities you named) in symbols, as in P(Odd|”odd”)=0.99,
I don’t think there is a standard notation for what I was trying to express (if there was formalizing the simple equivalent game I gave should be trivial, so why didn’t you do that?) if you are happy with just the end result here is another attempt:
P(Odd|”odd”)=P(Even|”even”)=P(“odd”|Odd)=P(“even”|Even)=0.99, P(Odd)=P(Even)=0.5, P(“odd” n Odd)= P(“even” n Even) =0.495
U_not_replace = P(“odd” n Odd)*100 + P(“even” n Odd)*0 +P(“even” n Even)*100 + P(“odd” n Even)*0 = 0.495*100 + 0.005*0 + 0.495*100 + 0.005*0 = 99
U_replace= P(“odd”|Odd)*( P(“odd” n Odd)*100 + P(“even” n Odd)*100) + P(“even”|Odd)*( P(“odd” n Odd)*0 + P(“even” n Odd)*0) + P(“even”|Even)*( P(“even” n Even)*100 + P(“odd” n Even)*100) + P(“odd”|Even)*( P(“even” n Even)*0 + P(“odd” n Even)*0) = 0.99*( 0.495*100 + 0.005*100) + 0.01* ( 0.495*0 + 0.005*0) +0.99*( 0.495*100 + 0.005*100) + 0.01* ( 0.495*0 + 0.005*0) =99
Probabilities correct, U_not_replace correct, U_replace I don’t see what’s going on with (what’s the first conceptual step that generates that formula?). Correct U_replace is just this:
U_replace_updateless = P(“odd” n Odd)*0 + P(“even” n Odd)*0 +P(“even” n Even)*100 + P(“odd” n Even)*100 = 0.495*0 + 0.005*0 + 0.495*100 + 0.005*100 = 50
That seems obviously incorrect to me because as an updateless decision maker you don’t know you are in the branch where you replace odds with evens. Your utility is half way between a correct updateless analysis and a correct analysis with updates. Or it is the correct utility if Omega also replaces the result in worlds where the parity of Q is different (so either Q is different or Omega randomly decides whether it’s actually going to visit anyone or just predict what you would decide if the situation was different and applies that to whatever happens), in which case you have done a horrible job of miscommunication.
I have only a vague idea what exactly required more explanation so I’ll try to explain everything.
My U_replace is the utility if you act on the general policy of replacing the result in counterfactual branches with the result in the branch Omega visits. It’s the average over all imaginable worlds (imaginable worlds where Q is even and those where Q is odd), the probability of a world multiplied with its utility.
P(“odd”|Odd)*( P(“odd” n Odd)*100 + P(“even” n Odd)*100) + P(“even”|Odd)*( P(“odd” n Odd)*0 + P(“even” n Odd)*0) is the utility for the half of imaginable worlds where Q is odd (all possible worlds if Q is odd).
P(“odd”|Odd) is the probability that the calculator shows odd in whatever other possible world Omega visits, conditional on Q being odd (which is correct to use because here only imaginable worlds where Q is odd are considered, the even worlds come later). If that happens the utility for worlds where the calculator shows even is replaced with 100.
P(“even”|Odd) is the probability that the calculator shows even in the other possible (=odd) world Omega visits. If that happens the utility for possible worlds where the calculator shows odd is replaced with 0.
At this point I’d just say replace odd with even for the other half, but last time I said something like that it didn’t seem to work so here’s it replaced manually:
P(“even”|even)*( P(“even” n even)*100 + P(“odd” n even)*100) + P(“odd”|even)*( P(“even” n even)*0 + P(“odd” n even)*0) is the utility for the half of imaginable worlds where Q is even (all possible worlds if Q is even).
P(“even”|even) is the probability that the calculator shows even in whatever other possible world Omega visits, conditional on Q being even (which is correct to use because here only imaginable worlds where Q is even are considered, the odd worlds came earlier). If that happens the utility for worlds where the calculator shows odd is replaced with 100.
P(“odd”|even) is the probability that the calculator shows odd in the other possible (=even) world Omega visits. If that happens the utility for possible worlds where the calculator shows even is replaced with 0.
If you want to say that updateless analysis is not allowed to take dependencies of this kind into account I ask you for an updateless analysis of the game with black and white balls a few comments upthread. Either updateless analysis as you understand it can’t deal with that game (and is therefore incomplete) or I can use whatever you use to formalize that game for this problem and you can’t brush me aside with saying that I’m not working updatelessly.
EDIT: The third interpretation of your utility function would be the utility of the general policy of always replacing odds with evens regardless of what the calculator in the world Omega visited showed, which would be so ridiculously stupid that it didn’t occur to me anyone might possibly be talking about that, even to point out fallacious thinking.
P(“odd”|Odd)*( P(“odd” n Odd)*100 + P(“even” n Odd)*100) + P(“even”|Odd)*( P(“odd” n Odd)*0 + P(“even” n Odd)*0) is the utility for the half of imaginable worlds where Q is odd (all possible worlds if Q is odd).
Consider expected utility [P(“odd” n Odd)*100 + P(“even” n Odd)*100)] from your formula. What event and decision is this the expected utility of? It seems to consider two events, [“odd” n Odd] and [“even” n Odd]. For both of them to get 100 utils, the strategy (decision) you’re considering must be, always answer-odd (since you can only answer in response to indication on the calculators, and here we have both indications and the same answer necessary for success in both events).
But U_replace estimates the expected utility of a different strategy, of strategy where you answer-even on your own “even” branch and also answer-even on the “odd” branch with Omega’s help. So you’re already computing something different.
Then, in the same formula, you have [P(“odd” n Odd)*0 + P(“even” n Odd)*0]. But to get 0 utils in both cases, you have to answer incorrectly in both cases, and since we’re considering Odd, this must be unconditional answer-even. This contradicts the way you did your expected utility calculation in the first terms of the formula (where you were considering the strategy of unconditional answer-odd).
Expected utility is computed for one strategy at a time, and values of expected utility computed separately for each strategy are used to compare the strategies. You seem to be doing something else.
Expected utility is computed for one strategy at a time, and values of expected utility computed separately for each strategy are used to compare the strategies. You seem to be doing something else.
I’m calculating for one strategy, the strategy of “fill in whatever the calculator in the world Omega appeared in showed”, but I have a probability distribution across what that entails (see my other reply). I’m multiplying the utility of picking “odd” with the probability of picking “odd” and the utility of picking “even” with the probability of picking “even”.
So that’s what happens when you don’t describe what strategy you’re computing expected utility of in enough detail in advance. By problem statement, the calculator in the world in which Omega showed shows “even”.
But even if you expect Omega to appear on either side, this still isn’t right. Where’s the probability of Omega appearing on either side in your calculation? The event of Omega appearing on one or the other side must enter the model, and it wasn’t explicitly referenced in any of your formulas.
And since every summand includes a P(Odd n X) or a P(Even n X) everything is already multiplied with P(Even) or P(Odd) as appropriate. In retrospect it would have been a lot clearer if I had factored that out, but I wrote U_not_replace first in the way that seemed most obvious and merely modified that to U_replace so it never occured to me to do that.
Omega visits either the “odd” world or “even” world, not Odd world or Even world. For example, in Odd world it’d still need to decide between “odd” and “even”.
That’s what multiplying with P(“odd”|Odd) etc was about. (the probability that, given Omega appearing in an Odd world it would appear in an “odd” world). I thought I explained that?
Or it is the correct utility if Omega also replaces the result in worlds where the parity of Q is different
Since you don’t know what parity of Q is, you can’t refer to the class of worlds where it’s “the same” or “different”, in particular because it can’t be different. So again, I don’t know what you describe here.
(It’s still correct to talk about the sets of possible worlds that rely on Q being either even or odd, because that’s your model of uncertainty, and you are uncertain about whether Q is even or odd. But not of sets of possible worlds that have your parity of Q, just as it doesn’t make sense to talk of the actual state of the world (as opposed to the current observational event, which is defined by past observations).)
Since you don’t know what parity of Q is, you can’t refer to the class of worlds where it’s “the same” or “different”, in particular because it can’t be different. So again, I don’t know what you describe here.
I’m merely trying to exclude a possible misunderstanding that would mean both of us being correct in the version of the problem we are talking about. Here’s another attempt. The only difference between the world Omega shows up in and the counterfactual worlds Omega affects regarding the calculator result is whether or not the calculator malfunctioned, you just don’t know on which side it malfunctioned. Is that correct?
as an updateless decision maker you don’t know you are in the branch where you replace odds with evens.
I don’t understand what this refers to. (Which branch is that? What do you mean by “replace”? Does your ‘odd’ refer to calculator-shows-odd or it’s-actually-odd or ’let’s-write-”odd”-on-the-test-sheet etc.?)
Also, updateless decision-maker reasons about strategies, which describe responses to all possible observations, and in this sense updateless analysis does take possible observations into account.
(The downside of long replies and asynchronous communication: it’s better to be able to interrupt after a few words and make sure we won’t talk past each other for another hour.)
Here’s another attempt at explaining your error (as it appears to me):
In the terminology of Wei Dai’s original post an updateless agent considers the consequences of a program S(X) returning Y on input X, where X includes all observations and memories, and the agent is updateless in respect to things included in X. For an ideal updateless agent this X includes everything, including the memory of having seen the calculator come up even. So it does not make sense for such an agent to consider the unconditional strategy of choosing even, and doing so does not properly model an updating agent choosing even after seeing even, it models an updating agent choosing even without having seen anything.
An obvious simplification of an (computationally extremely expensive) updateless agent would be to simplify X. If X is made up of the parts X1 and X2 and X1 is identical for all instances of S being called, then it makes sense to incorporate X1 into a modified version of S, S’ (more precisely the part of S or S’ that generates the world programs S or S’ tries to maximize). In that case a normal Bayesian update would be performed (UDT is not a blanket rejection of Bayesianism, see Wei Dai’s original post). S’ would be updateless with resepct to X2, but not with respect to X1. If X1 is indeed always part of the argument when S is called S’ should always give back the same output as S.
Your utility implies an S’ with respect to having observed “even”, but without the corresponding update, so it generates faulty world programs, and a different utility expectation than the original S or a correctly simplified version S″ (which in this case is not updateless because there is nothing else to be updateless towards).
The updateless analogue to the updater strategy “ask Omega to fill in the answer “even” in counterfactual worlds because you have seen the calculator result “even”″ is “ask Omega to fill in the answer the calculator gives whereever Omega shows up”. As an updateless decision maker you don’t know that the calculator showed “even” in your world because “your world” doesn’t even make sense to an updateless reasoner. The updateless replacing strategy is a fixed strategy that has a particular observation as parameter. An updateless strategy without parameter would be equivalent to an updater strategy of asking Omega to write in “even” in other worlds before seeing any calculator result.
The updateless analogue to the updater strategy “ask Omega to fill in the answer “even” in counterfactual worlds because you have seen the calculator result “even”″ is...
Updateless strategies describe how you react to observations. You do react to observations in updateless strategies. In our case, we don’t even need that, since all observations are fixed by the problem statement: you observe “even”, case closed. The strategies you consider specify what you write down on your own “even” test sheet, and what you write on the “odd” counterfactual test sheet, all independently of observations.
The “updateless” aspect is in not forgetting about counterfactuals and using prior probabilities everywhere, instead of updated probabilities. So, you use P(Odd n “odd”) to describe the situation where Q is Odd and the counterfactual calculator shows “odd”, instead of using P(Odd n “odd”|”even”), which doesn’t even make sense.
More generally, you can have updateless analysis being wrong on any kind of problem, simply by incorporating an observation into the problem statement and then not updating on it.
Huh? If you don’t update, you don’t need to update, so to speak. By not forgetting about events, you do take into account their relative probability in the context of the sub-events relevant for your problem. Examples please.
Updateless strategies describe how you react to observations. You do react to observations in updateless strategies. In our case, we don’t even need that, since all observations are fixed by the problem statement: you observe “even”, case closed.
Holding observations fixed but not updating on them is simply a misapplication of UDT. For an ideal updateless agent no observation is fixed and everything (every memory and observation) part of the variable input X. See this comment
Holding observations fixed but not updating on them is simply a misapplication of UDT.
A misapplication, strictly speaking, but not “simply”. Without restricting your attention to particular situations, while ignoring other situations, you won’t be able to consider any thought experiments. For any thought experiment I show you, you’ll say that you have to compute expected utility over all possible thought experiments, and that would be end of it.
So in applying UDT in real life, it’s necessary to stipulate the problem statement, the boundary event in which all relevant possibilities are contained, and over which we compute expected utility. You, too, introduced such an event, you just did it a step earlier than what’s given in the problem statement, by paying attention to the term “observation” attached to the calculator, and the fact that all other elements of the problem are observations also.
(On unrelated note, I have doubts about correctness of your work with that broader event too, see this comment.)
So in applying UDT in real life, it’s necessary to stipulate the problem statement, the boundary event in which all relevant possibilities are contained, and over which we compute expected utility.
Yes, of course. But you perform normal Bayesian updates for everything else (everything you hold fixed). Holding something fixed and not updating leads to errors.
Simple example: An urn with either 90% red and 10% blue balls or 90% blue and 10% red balls (0.5 prior for either). You have drawn a red ball and put it back. What’s the updateless expected utility of drawing another ball, assuming you get 1 util for drawing a ball in the same color and −2 utils for drawing a ball in a different color? Calculating as getting 1 util for red balls and −2 for blue, but not updating on the observation of having drawn a red ball suggests that it’s −0.5, when in fact it’s 0.46.
EDIT: miscalculated the utilities, but the general thrust is the same.
Holding something fixed and not updating leads to errors.
No, controlling something and updating it away leads to errors. Fixed terms in expected utility don’t influence optimality, you just lose ability to consider the influence of various strategies on them. Here, the strategies under considerations don’t have any relevant effects outside the problem statement.
I admit that I did not anticipate you replying in this way and even though I think I understand what you are saying I still don’t understand why. This is the main source of my uncertainty on whether I’m right at this point. It seems increasingly clear that at least one of us doesn’t properly understand UDT. I hope we can clear this up and if it turns out the misunderstanding was on my part I commit to upvoting all comments by you that contributed to enlightening me about that.
Unless I completely misunderstand you that’s a completely different context for/meaning of “fixed term” and while true not at all relevant here. I mean fixed in the sense of knowing the utilities of red and blue balls in the example I gave.
No, controlling something and updating it away leads to errors.
Also leads to errors, obviously. And I’m not doing that anyway. Something leading to errors is extremely weak evidence against something else also leading to error, so how is this relevant?
Correct (if you mean to say that all errors apparently caused by lack of updating can also be framed as being caused by wrongly holding something fixed) for a sufficiently wide sense of not fixed. The fact that you are considering to replace odd results in counterfactual worlds with even results and not the other way round, or the fact that the utility of drawing a red ball is 1 and for a blue ball −2 in my example (did you get around to taking a look at it?) both have to be considered not fixed in that sense.
Basically in the terminology of this comment you can consider anything in X1 fixed and avoid the error I’m talking about by updating. Or you can avoid that error by not holding it fixed in the first place. The same holds for anything in X2 for which the decision will never have any consequences anywhere it’s not true (or at least all its implications fully carry over), though that’s obviously more dangerous (and has the side effect of splitting the agent into different versions in different environments).
The error you’re talking about (the very error which UDT is correction for) is holding something in X2 fixed and updating when it does have outside consequences. Sometimes the error will only manifest when you actually update and only holding fixed gives results equivalent to the correct ones.
The test to see whether it’s allowable to update on x is to check whether the update results in the same answers as an updateless analysis that does not hold x fixed. If an analysis with update on x and one that holds x fixed but does not update disagree the problem is not always with the analysis with update. In fact in all problems CDT and UDT agree (most boring problems) the version with update should be correct and the version that only holds fixed might not be.
We are in the world where the calculator displays even, and we are 99% sure it is the world where the calculator has not made an error. This is Even World, Right Calculator. Counterfactual worlds:
Even World, Wrong Calculator (1% of Even Worlds)
Odd World, Right Calculator (99% of Odd Worlds)
Odd World, Wrong Calculator (1% of Odd Worlds)
All Omega told us was that the counterfactual world we are deciding for, the calculator shows Odd. We can therefore eliminate Odd World, Wrong Calculator. Answering the question is, in essence, deciding which world we think we’re looking at.
So, in the counterfactual world, we’re either looking at Even World, Wrong Calculator or Odd World, Right Calculator. We have an equal prior for the world being Odd or Even—or, we think the number of Odd Worlds is equal to the number of Even Worlds. We know the ratio of Wrong Calculator worlds to Right Calculator worlds (1:99). This is, therefore, 99% evidence for Odd World. The correct decision for the counterfactual you in that world is to decide Odd World. The correct decision for you?
Ignoring Bostrom’s book on how to deal with observer selection effects (did Omega go looking for a Wrong Calculator world and report it? Did Omega go looking for an Odd World to report to you? Did Omega pick at random from all possible worlds? Did Omega roll a three-sided die to determine which counterfactual world to report?), I believe the correct decision is to answer Odd World for the counterfactual world, with 99% certainty if you are allowed to specify as such.
I reason that by virtue of it being a counterfactual world, it is contingent on my not having the observation of my factual world; factual world observations are screened off by the word “counterfactual”.
The other possibility (which I tentatively think is wrong) is that our 99% confidence of Even World (from our factual world) comes up against our 99% confidence of Odd World (from our counterfactual) and they cancel out, bringing you back to your prior. So you should flip a coin to decide even or odd. I think this is wrong because 1) I think you could reason from 50% in the countefactual world to 50% in the factual world, which is wrong, and 2) this setup is identical to punching in the formula, pressing the button and observing “even”, then pressing the button again and observing “odd”. I don’t think you can treat counterfactual worlds as additional observations in this manner.
edit: It occurs to me that with Omega telling you about the counterfactual world, you are receiving a second observation. For this understanding, you would specify Even World with 99% confidence in the factual world and either Even or Odd World depending on how the coin landed for the counterfactual world.
I believe the above is correct updateless analysis of the thought experiment. (Which is a natural step to take in considering it, but not the point of the post, see its last paragraph.)
Exactly. The correct decision for factual you may be different to the correct decision for counterfactual you.
Vladimir says that “Omega doesn’t touch any calculator”. If the counterfactual is entered at the point where the computation starts and Omega tells you that it results in Odd (ETA2: rereading Vladimir’s comment, this is not the case), then it is a second observation contributed by Omega running the calculator and should affect both worlds. If on the other hand the counterfactual is just about the display, then the counterfactual Omega will likely write down Odd (ETA3: not my current answer). So I agree with your analysis. I see it this way: real Omegas cannot write on counterfactual paper.
ETA: -- the “counterfactual” built as “being in another quantum branch of exactly the same universe” strikes me as being of the sort where Omega does run the calculator again, so it should affect both worlds as another observation.
ETA2: I’ve changed my mind about there being an independent observation.
Actually, isn’t this the very heart of the matter? In my other comment here I assumed Omega would always ask what the correct answer is if the calculator shows The Other Result; if that’s not the case everything changes.
The answer does depend on this fact, but since this fact wasn’t specified, assume uncertainty (say, Omega always appears when you observe “even” and had pasta for breakfast).
Not by my understanding (but I decided to address it in a top-level comment). ETA: yes, in my updated understanding.
This would be correct if Q could be different, but Q is the same both in the counterfactual and the actual word. There is no possibility for the actual world being Even World and the counterfactual Odd World.
The possibilities are:
Actual: Even World, Right Calculator (99% of Even Words); Counterfactual: Even World, Wrong Calculator (1% of Even Worlds).
Actual: Odd World, Wrong Calculator (1% of Odd Words); Counterfactual: Odd World, Right Calculator (99% of Odd Words).
The prior probability of either is 50%. If we assume That Omega randomly picks one you out of 100% of possible words(either 100% of all Even Worlds or 100% of all Odd Words) to decide for all possible words where the calculator result is different (but the correct answer is the same), then there is a 99% chance all worlds are Even and your choice affects 1% of all worlds and a 1% chance all words are Odd and your choice affects 99% of all worlds. The result of the calculator in the counterfactual world doesn’t provide any evidence on whether all words are Even or all worlds are Odd since in either case there would be such a world to talk about.
If we assume that Omega randomly visits one world and randomly mentions the calculator result of one other possible world and it just happened to be the case that in that other world the result was different; or if Omega randomly picks a world, then randomly picks a world with the opposite calculator result and tosses a coin as to which world to visit and which to mention then the calculator result in the counterfactual word is equally relevant and hearing Omega talk about it just as good as running the calculator twice. In this case you are equally likely to be in a Odd world and might just as well toss a coin as to which result you fill in yourself.
This doesn’t square with my interpretation of the premises of the question. We are unsure of Q’s parity. Our prior is 50:50 odd, even. We are also unsure of calculator’s trustworthiness. Our prior is 99:1 right, wrong. Therefore—on my understanding of counterfactuality—both options for both uncertainties need to be on the table.
I am unconvinced you can ignore your uncertainty on Q’s parity by arguing that it will come out only one way regardless of your uncertainty—this is true for coinflips in deterministic physics, but that doesn’t mean we can’t consider the counterfactual where the coin comes up tails.
From the original post:
Clarified here and here.
We cannot determine Q’s parity, except by fallible calculator. When you say Q is the same, you seem to be including “Q’s parity is the same”.
Hmm. Maybe this will help?
But consider this situation:
These situations are clearly the counterfactuals of each other—that is, when scenario 1 says “the counterfactual world” it is saying “scenario 2″, and vice versa. The interpretations given in the second half of each contradict each other—the first scenario attempts to decide for the second scenario and gets it wrong; the second scenario attempts to decide for the first and gets it wrong. Whence this contradiction?
Yes, that would be a counterfactual. But NOT the counterfactual under consideration. The counterfactual under consideration was the calculator result being different but Q (both the number and the formula, and thus their parity) being the same. Unless Nesov was either deliberately misleading or completely failed his intention to clarify anything the comments linked to. If Q is the same formula is supposed to be clear in any way then everything about Q has to be the same. If the representation of Q in the formula was supposed be the same, but the actual value possibly counterfactually different then only answering that the formula is the same is obscuration, not clarification.
I disagree. Recall that I specified this in each case:
Q (both the number and the formula, and thus the parity) is the same in both scenarios. The actual value is not counterfactually different—it’s the same value in the safe, both times.
If you agree that Q’s parity is the same I’m not sure what you are disagreeing with. Its not possible for Q to be odd in the counterfactual and even in actuality, so if Q is odd in the counterfactual that implies it is also odd in actuality and vice versa. Thus it’s not possible for the calculator to be right in both counterfactual and reality simultaneously, and assuming it to be right in the counter-factual implies that it’s wrong in actuality. Therefore you can reduce everything to the two cases I used, Q even/actual calculator right/counterfactual calculator wrong or Q odd/actual calculator wrong/counterfactual calculator right.
Maybe this could be more enlightening. When you control things, one of the necessary requirements is that you have logical uncertainty about some property of the thing you control. You start with having a definition of the control target, but not knowing some of its properties. And then you might be able to infer a dependence of one of its properties on your action. This allows you to personally determine what is that property of a structure whose definition you already know. See my posts on ADT for more detail.
I have been positing that these two cases are counterfactuals of each other. Before one of these two cases occurs, we don’t know which one will occur. It is possible to consider being in the other case.
The problem is symmetrical. You can just copy everything, replace odd with even and vice versa and multiply everything with 0.5, then you also have the worlds where you see odd and Omega offers you to replace the result in counterfactuals where it came up even and where Q has the same parity. Doesn’t change that Q is the same in the world that decides and the counterfactuals that are effected. Omega also transposing your choice to impossible worlds (or predicting what would happen in impossible worlds and imposing that on what happens in real worlds) would be a different problem (that violates that Q be the same in the counterfactual, but seems to be the problem you solved).
If someone is sure enough that I’m wrong to downvote all my post on this they should be able to tell me where I’m wrong. I would be extremely interested in finding out.
I don’t know why you were downvoted. But I do notice that somewhere on this thread, the meaning of “Even World” has changed from what it was when Shokwave introduced the term. Originally it meant a world whose calculator showed ‘Even’.
You’re reasoning about the counterfactual using observational knowledge, i.e. making exactly the error whose nature puzzles me and is the subject of the post. In safely correct (but unenlightening about this error) updateless analysis, on the other hand, you don’t update on observations, so shouldn’t say things like “there is a 99% chance all worlds are Even”.
No. That’s completely insubstantial. Replace “even” with “same parity” and “odd” with “different parity” in my argument and the outcome is the same. The decision can be safely made before making any observations at all.
EDIT: And even in the formulation given I don’t update on personally having seen the even outcome (which is irrelevant, there is no substantial difference between me and the mes at that point) but Omega visiting me in a world where the calculator result came up even.
Please restate in more detail how you arrived at the following conclusion, and what made it so instead of the prior 50⁄50 for Even/Odd. It appears that it must be the observation of “even”, otherwise what privileged Even over Odd?
See the edit. If Omega randomly visits a possible world I can say ahead of time that there is a 99% chance that in that particular world the calculator result is correct and the decision will affect 1% of all worlds and a 1% chance that the result is wrong and the decision affects 99% of all worlds.
So you know a priori that the answer is Even, without even looking at the calculator? That can’t be right.
(You’re assuming that you know that Omega only arrives in “even” worlds, and updating on observing Omega, even before observing it. But in the same movement, you update on the calculator showing “even”. Omega doesn’t show up in the “odd” world, so you can’t update on the fact that it shows up, other than by observing it, or alternatively observing “even” given the assumption of equivalence of these events.)
Of course not.
No. I’m assuming that either even is correct in all worlds or odd is correct in all worlds (0.5 prior for either). If Omega randomly picks a world, the chance of the calculator being correct is independent of that and 99% everywhere, then there is a 99% chance of the calculator being correct in the particular world Omega arrives in. If odd is correct Omega is 99% likely to arrive in a world where the calculator says odd, and if the calculator says odd in the particular world Omega arrives in there is a 99% chance that’s because odd is correct.
EDIT:
If I were
the probability of even being correct would be 50% no matter what, and there would be a 50% chance each for affecting 99% of all worlds or 1% of all worlds.
I seem to agree with all of the above statements. The conditional probabilities are indeed this way. But it’s incorrect to use these conditional probabilities (which is to say, probabilities of Odd/Even after updating on observing “even”) to compute expected utility for the counterfactual. In a prior comment, you write:
99% is P(Even|Omega,”even”), that is to say it’s probability of Even updated by observations (events) that Omega and “even”.
No. There is no problem with using conditional probabilities if you use the correct conditional probabilities, that is the probabilities from wherever the decision happens, not from what you personally encounter. And I never claimed that any of the pieces you were quoting were part of an updateless analysis, just that it made no difference.
I would try to write a Wei Dai style world program at this point, but I know no programming at all and am unsure how drawing at random is supposed to be represented. It would be the same as the program for this game, though:
1 black and 99 white balls in an urn. You prefer white balls. You may decide to draw a ball and change all balls of the other color to balls of the color drawn, and must decide before the draw is made. (or to make it slightly more complicated: Someone else secretly flips a coin whether you get points for black or white balls. You get 99 balls of the color you get points for and one ball of the other color).
It would help a lot if you just wrote the formulas you use for computing expected utility (or the probabilities you named) in symbols, as in P(Odd|”odd”)=0.99,
P(Odd|”odd”)*100+P(Even|”odd”)*0 = 0.99*100+0.01*0 = 99.
Do you need more than that? I don’t see how this could possibly help, but:
N(worlds)=100
For each world:
P(correct)=0.99
U_world(correct)=1
U_world(~correct) = 0
P(Omega)=0.01
P(correct|Omega)=P(correct|~Omega) = 0.99
If choosing to replace:
correct ∧ Omega ⇒for all worlds: U_world(~correct) = 1
~correct ∧ Omega ⇒for all worlds: U_world(correct) = 0
This is imprecise in that exactly one world ends up with Omega.
I give up, sorry. Read up on standard concepts/notation for expected utility/conditional probability maybe.
I don’t think there is a standard notation for what I was trying to express (if there was formalizing the simple equivalent game I gave should be trivial, so why didn’t you do that?) if you are happy with just the end result here is another attempt:
P(Odd|”odd”)=P(Even|”even”)=P(“odd”|Odd)=P(“even”|Even)=0.99, P(Odd)=P(Even)=0.5, P(“odd” n Odd)= P(“even” n Even) =0.495
U_not_replace = P(“odd” n Odd)*100 + P(“even” n Odd)*0 +P(“even” n Even)*100 + P(“odd” n Even)*0 = 0.495*100 + 0.005*0 + 0.495*100 + 0.005*0 = 99
U_replace= P(“odd”|Odd)*( P(“odd” n Odd)*100 + P(“even” n Odd)*100) + P(“even”|Odd)*( P(“odd” n Odd)*0 + P(“even” n Odd)*0) + P(“even”|Even)*( P(“even” n Even)*100 + P(“odd” n Even)*100) + P(“odd”|Even)*( P(“even” n Even)*0 + P(“odd” n Even)*0) = 0.99*( 0.495*100 + 0.005*100) + 0.01* ( 0.495*0 + 0.005*0) +0.99*( 0.495*100 + 0.005*100) + 0.01* ( 0.495*0 + 0.005*0) =99
Probabilities correct, U_not_replace correct, U_replace I don’t see what’s going on with (what’s the first conceptual step that generates that formula?). Correct U_replace is just this:
U_replace_updateless = P(“odd” n Odd)*0 + P(“even” n Odd)*0 +P(“even” n Even)*100 + P(“odd” n Even)*100 = 0.495*0 + 0.005*0 + 0.495*100 + 0.005*100 = 50
That seems obviously incorrect to me because as an updateless decision maker you don’t know you are in the branch where you replace odds with evens. Your utility is half way between a correct updateless analysis and a correct analysis with updates. Or it is the correct utility if Omega also replaces the result in worlds where the parity of Q is different (so either Q is different or Omega randomly decides whether it’s actually going to visit anyone or just predict what you would decide if the situation was different and applies that to whatever happens), in which case you have done a horrible job of miscommunication.
I have only a vague idea what exactly required more explanation so I’ll try to explain everything.
My U_replace is the utility if you act on the general policy of replacing the result in counterfactual branches with the result in the branch Omega visits. It’s the average over all imaginable worlds (imaginable worlds where Q is even and those where Q is odd), the probability of a world multiplied with its utility.
P(“odd”|Odd)*( P(“odd” n Odd)*100 + P(“even” n Odd)*100) + P(“even”|Odd)*( P(“odd” n Odd)*0 + P(“even” n Odd)*0) is the utility for the half of imaginable worlds where Q is odd (all possible worlds if Q is odd).
P(“odd”|Odd) is the probability that the calculator shows odd in whatever other possible world Omega visits, conditional on Q being odd (which is correct to use because here only imaginable worlds where Q is odd are considered, the even worlds come later). If that happens the utility for worlds where the calculator shows even is replaced with 100.
P(“even”|Odd) is the probability that the calculator shows even in the other possible (=odd) world Omega visits. If that happens the utility for possible worlds where the calculator shows odd is replaced with 0.
At this point I’d just say replace odd with even for the other half, but last time I said something like that it didn’t seem to work so here’s it replaced manually:
P(“even”|even)*( P(“even” n even)*100 + P(“odd” n even)*100) + P(“odd”|even)*( P(“even” n even)*0 + P(“odd” n even)*0) is the utility for the half of imaginable worlds where Q is even (all possible worlds if Q is even).
P(“even”|even) is the probability that the calculator shows even in whatever other possible world Omega visits, conditional on Q being even (which is correct to use because here only imaginable worlds where Q is even are considered, the odd worlds came earlier). If that happens the utility for worlds where the calculator shows odd is replaced with 100.
P(“odd”|even) is the probability that the calculator shows odd in the other possible (=even) world Omega visits. If that happens the utility for possible worlds where the calculator shows even is replaced with 0.
If you want to say that updateless analysis is not allowed to take dependencies of this kind into account I ask you for an updateless analysis of the game with black and white balls a few comments upthread. Either updateless analysis as you understand it can’t deal with that game (and is therefore incomplete) or I can use whatever you use to formalize that game for this problem and you can’t brush me aside with saying that I’m not working updatelessly.
EDIT: The third interpretation of your utility function would be the utility of the general policy of always replacing odds with evens regardless of what the calculator in the world Omega visited showed, which would be so ridiculously stupid that it didn’t occur to me anyone might possibly be talking about that, even to point out fallacious thinking.
Consider expected utility [P(“odd” n Odd)*100 + P(“even” n Odd)*100)] from your formula. What event and decision is this the expected utility of? It seems to consider two events, [“odd” n Odd] and [“even” n Odd]. For both of them to get 100 utils, the strategy (decision) you’re considering must be, always answer-odd (since you can only answer in response to indication on the calculators, and here we have both indications and the same answer necessary for success in both events).
But U_replace estimates the expected utility of a different strategy, of strategy where you answer-even on your own “even” branch and also answer-even on the “odd” branch with Omega’s help. So you’re already computing something different.
Then, in the same formula, you have [P(“odd” n Odd)*0 + P(“even” n Odd)*0]. But to get 0 utils in both cases, you have to answer incorrectly in both cases, and since we’re considering Odd, this must be unconditional answer-even. This contradicts the way you did your expected utility calculation in the first terms of the formula (where you were considering the strategy of unconditional answer-odd).
Expected utility is computed for one strategy at a time, and values of expected utility computed separately for each strategy are used to compare the strategies. You seem to be doing something else.
I’m calculating for one strategy, the strategy of “fill in whatever the calculator in the world Omega appeared in showed”, but I have a probability distribution across what that entails (see my other reply). I’m multiplying the utility of picking “odd” with the probability of picking “odd” and the utility of picking “even” with the probability of picking “even”.
So that’s what happens when you don’t describe what strategy you’re computing expected utility of in enough detail in advance. By problem statement, the calculator in the world in which Omega showed shows “even”.
But even if you expect Omega to appear on either side, this still isn’t right. Where’s the probability of Omega appearing on either side in your calculation? The event of Omega appearing on one or the other side must enter the model, and it wasn’t explicitly referenced in any of your formulas.
But implicitly.
P(Omega_in_Odd_world)=P(Omega_in_Even_world)=0.5, but
P(Omega_in_Odd_world|Odd)= P(Omega_in_Even_world|Even)=1
And since every summand includes a P(Odd n X) or a P(Even n X) everything is already multiplied with P(Even) or P(Odd) as appropriate. In retrospect it would have been a lot clearer if I had factored that out, but I wrote U_not_replace first in the way that seemed most obvious and merely modified that to U_replace so it never occured to me to do that.
Omega visits either the “odd” world or “even” world, not Odd world or Even world. For example, in Odd world it’d still need to decide between “odd” and “even”.
That’s what multiplying with P(“odd”|Odd) etc was about. (the probability that, given Omega appearing in an Odd world it would appear in an “odd” world). I thought I explained that?
Since you don’t know what parity of Q is, you can’t refer to the class of worlds where it’s “the same” or “different”, in particular because it can’t be different. So again, I don’t know what you describe here.
(It’s still correct to talk about the sets of possible worlds that rely on Q being either even or odd, because that’s your model of uncertainty, and you are uncertain about whether Q is even or odd. But not of sets of possible worlds that have your parity of Q, just as it doesn’t make sense to talk of the actual state of the world (as opposed to the current observational event, which is defined by past observations).)
I’m merely trying to exclude a possible misunderstanding that would mean both of us being correct in the version of the problem we are talking about. Here’s another attempt. The only difference between the world Omega shows up in and the counterfactual worlds Omega affects regarding the calculator result is whether or not the calculator malfunctioned, you just don’t know on which side it malfunctioned. Is that correct?
Sounds right, although when you speak of the only difference, it’s easy to miss something.
I don’t understand what this refers to. (Which branch is that? What do you mean by “replace”? Does your ‘odd’ refer to calculator-shows-odd or it’s-actually-odd or ’let’s-write-”odd”-on-the-test-sheet etc.?)
Also, updateless decision-maker reasons about strategies, which describe responses to all possible observations, and in this sense updateless analysis does take possible observations into account.
(The downside of long replies and asynchronous communication: it’s better to be able to interrupt after a few words and make sure we won’t talk past each other for another hour.)
Here’s another attempt at explaining your error (as it appears to me):
In the terminology of Wei Dai’s original post an updateless agent considers the consequences of a program S(X) returning Y on input X, where X includes all observations and memories, and the agent is updateless in respect to things included in X. For an ideal updateless agent this X includes everything, including the memory of having seen the calculator come up even. So it does not make sense for such an agent to consider the unconditional strategy of choosing even, and doing so does not properly model an updating agent choosing even after seeing even, it models an updating agent choosing even without having seen anything.
An obvious simplification of an (computationally extremely expensive) updateless agent would be to simplify X. If X is made up of the parts X1 and X2 and X1 is identical for all instances of S being called, then it makes sense to incorporate X1 into a modified version of S, S’ (more precisely the part of S or S’ that generates the world programs S or S’ tries to maximize). In that case a normal Bayesian update would be performed (UDT is not a blanket rejection of Bayesianism, see Wei Dai’s original post). S’ would be updateless with resepct to X2, but not with respect to X1. If X1 is indeed always part of the argument when S is called S’ should always give back the same output as S.
Your utility implies an S’ with respect to having observed “even”, but without the corresponding update, so it generates faulty world programs, and a different utility expectation than the original S or a correctly simplified version S″ (which in this case is not updateless because there is nothing else to be updateless towards).
(This question seems to depend on resolving this first.)
The updateless analogue to the updater strategy “ask Omega to fill in the answer “even” in counterfactual worlds because you have seen the calculator result “even”″ is “ask Omega to fill in the answer the calculator gives whereever Omega shows up”. As an updateless decision maker you don’t know that the calculator showed “even” in your world because “your world” doesn’t even make sense to an updateless reasoner. The updateless replacing strategy is a fixed strategy that has a particular observation as parameter. An updateless strategy without parameter would be equivalent to an updater strategy of asking Omega to write in “even” in other worlds before seeing any calculator result.
Updateless strategies describe how you react to observations. You do react to observations in updateless strategies. In our case, we don’t even need that, since all observations are fixed by the problem statement: you observe “even”, case closed. The strategies you consider specify what you write down on your own “even” test sheet, and what you write on the “odd” counterfactual test sheet, all independently of observations.
The “updateless” aspect is in not forgetting about counterfactuals and using prior probabilities everywhere, instead of updated probabilities. So, you use P(Odd n “odd”) to describe the situation where Q is Odd and the counterfactual calculator shows “odd”, instead of using P(Odd n “odd”|”even”), which doesn’t even make sense.
More generally, you can have updateless analysis being wrong on any kind of problem, simply by incorporating an observation into the problem statement and then not updating on it.
Huh? If you don’t update, you don’t need to update, so to speak. By not forgetting about events, you do take into account their relative probability in the context of the sub-events relevant for your problem. Examples please.
here
Holding observations fixed but not updating on them is simply a misapplication of UDT. For an ideal updateless agent no observation is fixed and everything (every memory and observation) part of the variable input X. See this comment
A misapplication, strictly speaking, but not “simply”. Without restricting your attention to particular situations, while ignoring other situations, you won’t be able to consider any thought experiments. For any thought experiment I show you, you’ll say that you have to compute expected utility over all possible thought experiments, and that would be end of it.
So in applying UDT in real life, it’s necessary to stipulate the problem statement, the boundary event in which all relevant possibilities are contained, and over which we compute expected utility. You, too, introduced such an event, you just did it a step earlier than what’s given in the problem statement, by paying attention to the term “observation” attached to the calculator, and the fact that all other elements of the problem are observations also.
(On unrelated note, I have doubts about correctness of your work with that broader event too, see this comment.)
Yes, of course. But you perform normal Bayesian updates for everything else (everything you hold fixed). Holding something fixed and not updating leads to errors.
Simple example: An urn with either 90% red and 10% blue balls or 90% blue and 10% red balls (0.5 prior for either). You have drawn a red ball and put it back. What’s the updateless expected utility of drawing another ball, assuming you get 1 util for drawing a ball in the same color and −2 utils for drawing a ball in a different color? Calculating as getting 1 util for red balls and −2 for blue, but not updating on the observation of having drawn a red ball suggests that it’s −0.5, when in fact it’s 0.46.
EDIT: miscalculated the utilities, but the general thrust is the same.
P(RedU)=P(BlueU)=P(red)=P(blue)=0.5
P(red|RedU)=P(RedU|red)=P(blue|BlueU)=P(BlueU|blue)=0.9
P(blue|RedU)=P(RedU|blue)=P(BlueU|red)=P(Red|BlueU)=0.1
U_updating=P(RedU|red)*P(red|RedU)*1 + P(BlueU|red)*Pred(|BlueU)*1 - P(RedU|red)*P(blue|RedU)*2 - P(BlueU|red)*P(blue|BlueU)*2 = 0.9*0.9+0.1*0.1-0.9*0.1*2*2= 0.46
U_semi_updateless=P(red)*1-P(blue)*2=-0.5
U_updateless= P(red)(P(RedU|red)*P(red|RedU)*1 + P(BlueU|red)*Pred(|BlueU)*1 - P(RedU|red)*P(blue|RedU)*2 - P(BlueU|red)*P(blue|BlueU)*2) +P(blue)(P(BlueU|blue)*P(blue|BlueU)*1 + P(RedU|blue)*P(blue|RedU)*1 - P(BlueU|blue)*P(red|BlueU)*2 - P(RedU|blue)*P(red|RedU)*2) =0.5*(0.9*0.9+0.1*0.1-0.9*0.1*2*2)+0.5* (0.9*0.9+0.1*0.1-0.9*0.1*2*2)=0.46
(though normally you’d probably come up with U_updateless in a differently factored form)
EDIT3: More sensible/readable factorization of U_updateless:
P(RedU)((P(red|RedU)(P(red|RedU)*1-P(blue|RedU)*2)+(P(blue|RedU)(P(blue|RedU)*1-P(red|RedU)*2)) + P(BlueU)((P(blue|BlueU)(P(blue|BlueU)*1-P(red|BlueU)*2)+(P(red|BlueU)(P(red|BlueU)*1-P(blue|BlueU)*2))
No, controlling something and updating it away leads to errors. Fixed terms in expected utility don’t influence optimality, you just lose ability to consider the influence of various strategies on them. Here, the strategies under considerations don’t have any relevant effects outside the problem statement.
(I’ll look into your example another time.)
Just to make sure: You mean something like updating on the box being empty in transparent Newcomb’s here, right? Not relevant as far as I can see.
I admit that I did not anticipate you replying in this way and even though I think I understand what you are saying I still don’t understand why. This is the main source of my uncertainty on whether I’m right at this point. It seems increasingly clear that at least one of us doesn’t properly understand UDT. I hope we can clear this up and if it turns out the misunderstanding was on my part I commit to upvoting all comments by you that contributed to enlightening me about that.
Unless I completely misunderstand you that’s a completely different context for/meaning of “fixed term” and while true not at all relevant here. I mean fixed in the sense of knowing the utilities of red and blue balls in the example I gave.
Also leads to errors, obviously. And I’m not doing that anyway. Something leading to errors is extremely weak evidence against something else also leading to error, so how is this relevant?
This is the very error which UDT (at least, this aspect of it) is correction for.
That still doesn’t make it evidence for something different not being an error. (and formal UDT is not the only way to avoid that error)
Not updating never leads to errors. Holding fixed what isn’t can.
Correct (if you mean to say that all errors apparently caused by lack of updating can also be framed as being caused by wrongly holding something fixed) for a sufficiently wide sense of not fixed. The fact that you are considering to replace odd results in counterfactual worlds with even results and not the other way round, or the fact that the utility of drawing a red ball is 1 and for a blue ball −2 in my example (did you get around to taking a look at it?) both have to be considered not fixed in that sense.
Basically in the terminology of this comment you can consider anything in X1 fixed and avoid the error I’m talking about by updating. Or you can avoid that error by not holding it fixed in the first place. The same holds for anything in X2 for which the decision will never have any consequences anywhere it’s not true (or at least all its implications fully carry over), though that’s obviously more dangerous (and has the side effect of splitting the agent into different versions in different environments).
The error you’re talking about (the very error which UDT is correction for) is holding something in X2 fixed and updating when it does have outside consequences. Sometimes the error will only manifest when you actually update and only holding fixed gives results equivalent to the correct ones.
The test to see whether it’s allowable to update on x is to check whether the update results in the same answers as an updateless analysis that does not hold x fixed. If an analysis with update on x and one that holds x fixed but does not update disagree the problem is not always with the analysis with update. In fact in all problems CDT and UDT agree (most boring problems) the version with update should be correct and the version that only holds fixed might not be.