This works because Left-boxing means you’re in a world where the predictors model of you also Left-boxed when the predictor made its prediction, causing it to not put a Bomb in Left.
Put differently, the situation described by MacAskill becomes virtually impossible if you Left-box, since the probability of Left-boxing and burning to death is ~0.
OR, alternatively, we say: no, we see the Bomb. We can’t retroactively change this! If we keep that part of the world fixed, then, GIVEN the subjunctive dependence between us and the predictor (assuming it’s there), that simply means we Right-box (with probability ~1), since that’s what the predictor’s model did.
Of course, then it’s not much of a decision theoretic problem anymore, since the decision is already fixed in the problem statement. If we assume we can still make a decision, then that decision is made in 2 places: first by the predictor’s model, then by us. Left-boxing means the model Left-boxes and we get to live for free. Right-boxing means the model Right-boxes and we get to live at a cost of $100. The right decision must be Left-boxing.
Put differently, the situation described by MacAskill becomes virtually impossible if you Left-box, since the probability of Left-boxing and burning to death is ~0.
Irrelevant, since the described scenario explicitly stipulates that you find yourself in precisely that situation.
OR, alternatively, we say: no, we see the Bomb. We can’t retroactively change this! If we keep that part of the world fixed, then, GIVEN the subjunctive dependence between us and the predictor (assuming it’s there), that simply means we Right-box (with probability ~1), since that’s what the predictor’s model did.
Yes, that’s what I’ve been saying: choosing Right in that scenario is the correct decision.
Of course, then it’s not much of a decision theoretic problem anymore, since the decision is already fixed in the problem statement.
I have no idea what you mean by this.
Left-boxing means the model Left-boxes and we get to live for free.
“Irrelevant, since the described scenario explicitly stipulates that you find yourself in precisely that situation.”
Actually, this whole problem is irrelevant to me, a Left-boxer: Left-boxers never (or extremely rarely) find themselves in the situation with a bomb in Left. That’s the point.
Firstly, there’s a difference between “never” and “extremely rarely”. And in the latter case, the question remains “and what do you do then?”. To which, it seems, you answer “choose the Right box”…? Well, I agree with that! But that’s just the view that I’ve already described as “Left-box unless there’s a bomb in Left, in which case Right-box”.
It remains unclear to me what it is you think we disagree on.
Firstly, there’s a difference between “never” and “extremely rarely”.
That difference is so small as to be neglected.
And in the latter case, the question remains “and what do you do then?”. To which, it seems, you answer “choose the Right box”…? Well, I agree with that! But that’s just the view that I’ve already described as “Left-box unless there’s a bomb in Left, in which case Right-box”.
It seems to me that strategy leaves you manipulatable by the predictor, who can then just always predict you will Right-box, put a bomb in Left, and let you Right-box, causing you to lose $1,000.
By construction it is not, because the scenario is precisely that we find ourselves in one such exceptional case; the posterior probability (having observed that we do so find ourselves) is thus ~1.
It seems to me that strategy leaves you manipulatable by the predictor
… but you have said, in a previous post, that if you find yourself in this scenario, you Right-box. How to reconcile your apparently contradictory statements…?
By construction it is not, because the scenario is precisely that we find ourselves in one such exceptional case; the posterior probability (having observed that we do so find ourselves) is thus ~1.
Except that we don’t find ourselves there if we Left-box. But we seem to be going around in a circle.
… but you have said, in a previous post, that if you find yourself in this scenario, you Right-box. How to reconcile your apparently contradictory statements…?
Right-boxing is the necessary consequence if we assume the predictor’s Right-box prediction is fixed now. So GIVEN the Right-box prediction, I apparently Right-box.
My entire point is that the prediction is NOT a given. I Left-box, and thus change the prediction to Left-box.
I have made no contradictory statements. I am and always have been saying that Left-boxing is the correct decision to resolve this dilemma.
Except that we don’t find ourselves there if we Left-box. But we seem to be going around in a circle.
There’s no “if” about it. The scenario is that we do find ourselves there. (If you’re fighting the hypothetical, you have to be very explicit about that, because then we’re just talking about two totally different, and pretty much unrelated, things. But I have so far understood you to not be doing that.)
Right-boxing is the necessary consequence if we assume the predictor’s Right-box prediction is fixed now. So GIVEN the Right-box prediction, I apparently Right-box.
I don’t know what you mean by “apparently”. You have two boxes—that’s the scenario. Which do you choose—that’s the question. You can pick either one; where does “apparently” come in?
My entire point is that the prediction is NOT a given. I Left-box, and thus change the prediction to Left-box.
What does this mean? The boxes are already in front of you.
I have made no contradictory statements. I am and always have been saying that Left-boxing is the correct decision to resolve this dilemma.
You just said in this very comment that you Right-box in the given scenario! (And also in several other comments… are you really going to make me cite each of them…?)
I’m not going to make you cite anything. I know what you mean. I said Right-boxing is a consequence, given a certain resolution of the problem; I always maintained Left-boxing is the correct decision. Apparently I didn’t explain myself well, that’s on me. But I’m kinda done, I can’t seem to get my point across (not saying it’s your fault btw).
It doesn’t. Instead, it will make it so that there will have never been a bomb in the first place.
To understand this, imagine yourself as a deterministic algorithm. Either you Left-box under all circumstances (even if there is a bomb in the left box), or you Right-box under all circumstances, or you Right-box iff there is a bomb in the left box.
Implementing the first algorithm out of these three is the best choice (the expected utility is 0).
Implementing the third algorithm (that’s what you do) is the worst choice (the expected utility is -$100).
By the way, I want to point out that you apparently disagree with Heighn on this. He says, as I understand him, that if you pick Left, you do indeed burn to death, but this is fine, because in [1 trillion trillion minus one] possible worlds, you live and pay nothing. But you instead say that if you pick Left… something happens… and the bomb in the Left box, which you were just staring directly at, disappears somehow. Or wasn’t ever there (somehow), even though, again, you were just looking right at it.
How do you reconcile this disagreement? One of you has to be wrong about the consequences of picking the Left box.
I think we agree. My stance: if you Left-box, that just means the predictor predicted that with probability close to 1. From there on, there are a trillion trillion − 1 possible worlds where you live for free, and 1 where you die.
I’m not saying “You die, but that’s fine, because there are possible worlds where you live”. I’m saying that “you die” is a possible world, and there are way more possible worlds where you live.
But apparently the consequences of this aren’t deterministic after all, since the predictor is fallible. So this doesn’t help.
If you reread my comments, I simplified it by assuming an infallible predictor.
How?
For this, it’s helpful to define another kind of causality (logical causality) as distinct from physical causality. You can’t physically cause something to have never been that way, because physical causality can’t go to the past. But you can use logical causality for that, since the output of your decision determines not only your output, but the output of all equivalent computations across the entire timeline. By Left-boxing even in case of a bomb, you will have made it so that the predictor’s simulation of you has Left-boxed as well, resulting in the bomb never having been there.
If you reread my comments, I simplified it by assuming an infallible predictor.
… so, in other words, you’re not actually talking about the scenario described in the OP. But that’s what my comments have been about, so… everything you said has been a non sequitur…?
You can’t physically cause something to have never been that way, because physical causality can’t go to the past. But you can use logical causality for that, since the output of your decision determines not only your output, but the output of all equivalent computations across the entire timeline. By Left-boxing even in case of a bomb, you will have made it so that the predictor’s simulation of you has Left-boxed as well, resulting in the bomb never having been there.
This really doesn’t answer the question.
Again, the scenario is: you’re looking at the Left box, and there’s a bomb in it. It’s right there in front of you. What do you do?
So, for example, when you say:
By Left-boxing even in case of a bomb, you will have made it so that the predictor’s simulation of you has Left-boxed as well, resulting in the bomb never having been there.
So if you take the Left box, what actually, physically happens?
… so, in other words, you’re not actually talking about the scenario described in the OP. But that’s what my comments have been about, so… everything you said has been a non sequitur…?
See my top-level comment, this is precisely the problem with the scenario descibed in the OP I pointed out. Your reading is standard, but not the intended meaning.
But it’s also puzzling that you can’t ITT this point, to see both meanings, even if you disagree that it’s reasonable to allow/expect the intended one. Perhaps divesting from having an opinion on the object level question might help? Like, what is the point the others are trying to make, specifically, how does it work, regardless of if it’s a wrong point, described in a way that makes no reference to its wrongness/absurdity?
Like with bug reports, it’s not helpful to say that something “doesn’t work at all”, it’s useful to be more specific. There’s some failure of rationality at play here, you are way too intelligent to be incapable of seeing what the point is, so there is some systematic avoidance of allowing yourself to see what is going on. Heighn’s antagonistic dogmatism doesn’t help, but shouldn’t be this debilitating.
As far as your top-level comment, well, my follow-up questions about it remain unanswered…
I dropped out of that conversation because it seemed to be going in circles, and I think I’ve explained everything already. Apparently the conversation continued, green_leaf seems to be making good points, and Heighn continues needlessly upping the heat.
I don’t think object level conversation is helpful at this point, there is some methodological issue in how you think about this that I don’t see an efficient approach to. I’m already way outside the sort of conversational norms I’m trying to follow for the last few years, which is probably making this comment as hopelessly unhelpful as ever, though in 2010 that’d more likely be the default mode of response for me.
Note that it’s my argumentation that’s being called crazy, which is a large factor in the “antagonism” you seem to observe—a word choice I don’t agree with, btw.
About the “needlessly upping the heat”, I’ve tried this discussion from multiple different angles, seeing if we can come to a resolution. So far, no, alas, but not for lack of trying. I will admit some of my reactions were short and a bit provocative, but I don’t appreciate nor agree with your accusations. I have been honest in my reactions.
I’ve been you ten years ago. This doesn’t help, courtesy or honesty (purposes that tend to be at odds with each other) aren’t always sufficient, it’s also necessary to entertain strange points of view that are obviously wrong, in order to talk in another’s language, to de-escalate where escalation won’t help (it might help with feeding norms, but knowing what norms you are feeding is important). And often enough that is still useless and the best thing is to give up. Or at least more decisively overturn the chess board, as I’m doing with some of the last few comments to this post, to avoid remaining in an interminable failure mode.
These norms are interesting in how well they fade into the background, oppose being examined. If you happen to be a programmer or have enough impression of what that might be like, just imagine a programmer team where talking about bugs can be taboo in some circumstances, especially if they are hypothetical bugs imagined out of whole cloth to check if they happen to be there, or brought to attention to see if it’s cheap to put measures in place to prevent their going unnoticed, even if it eventually turns out that they were never there to begin with in actuality. With rationality, that’s hypotheses about how people think, including hypotheses about norms that oppose examination of such hypotheses and norms.
Sorry, I’m having trouble understanding your point here. I understand your analogy (I was a developer), but am not sure what you’re drawing the analogy to.
I see your point, although I have entertained Said’s view as well. But yes, I could have done better. I tend to get like this when my argumentation is being called crazy, and I should have done better.
You could have just told me this instead of complaining about me to Said though.
Yes, the situation does say the bomb is there. But it also says the bomb isn’t there if you Left-box.
At the very least, this is a contradiction, which makes the scenario incoherent nonsense.
(I don’t think it’s actually true that “it also says the bomb isn’t there if you Left-box”—but if it did say that, then the scenario would be inconsistent, and thus impossible to interpret.)
This is misleading. What happens is that the situation you found yourself in doesn’t take place with significant measure. You live mostly in different situations, not this one.
It is misleading because Said’s perspective is to focus on the current situation, without regarding the other situations as decision relevant. From UDT perspective you are advocating, the other situations remain decision relevant, and that explains much of what you are talking about in other replies. But from that same perspective, it doesn’t matter that you live in the situation Said is asking about, so it’s misleading that you keep attention on this situation in your reply without remarking on how that disagrees with the perspective you are advocating in other replies.
In the parent comment, you say “it is, in virtually all possible worlds, that you live for free”. This is confusing: are you talking about the possible worlds within the situation Said was asking about, or also about possible worlds outside that situation? The distinction matters for the argument in these comments, but you are saying this ambiguously.
… so, in other words, you’re not actually talking about the scenario described in the OP. But that’s what my comments have been about, so… everything you said has been a non sequitur…?
No, non sequitur means something else. (If I say “A, therefore B”, but B doesn’t follow from A, that’s a non sequitur.)
I simplified the problem to make it easier for you to understand.
This really doesn’t answer the question.
It does. Your question was “How?”. The answer is “through logical causality.”
So if you take the Left box, what actually, physically happens?
You take the left box with the bomb, and it has always been empty.
It is. The response to your question “So if you take the Left box, what actually, physically happens?” is “Physically, nothing.” That’s why I defined logical causality—it helps understand why (1) is the algorithm with the best expected utility, and why yours is worse.
Do you see how that makes absolutely no sense as an answer to the question I asked? Like, do you see what makes what you said incomprehensible, what makes it appear to be nonsense? I’m not asking you to admit that it’s nonsense, but can you see why it reads as bizarre moon logic?
I’m no longer sure; you and green_leaf appear to have different, contradictory views, and at this point that divergence has confused me enough that I could no longer say confidently what either of you seem to be saying without going back and carefully re-reading all the comments. And that, I’m afraid, isn’t something that I have time for at the moment… so perhaps it’s best to write this discussion off, after all.
Agreed, but I think it’s important to stress that it’s not like you see a bomb, Left-box, and then see it disappear or something. It’s just that Left-boxing means the predictor already predicted that, and the bomb was never there to begin with.
Put differently, you can only Left-box in a world where the predictor predicted you would.
Put differently, you can only Left-box in a world where the predictor predicted you would.
What stops you from Left-boxing in a world where the predictor didn’t predict that you would?
To make the question clearer, let’s set aside all this business about the fallibility of the predictor. Sure, yes, the predictor’s perfect, it can predict your actions with 100% accuracy somehow, something about algorithms, simulations, models, whatever… fine. We take all that as given.
So: you see the two boxes, and after thinking about it very carefully, you reach for the Right box (as the predictor always knew that you would).
But suddenly, a stray cosmic ray strikes your brain! No way this was predictable—it was random, the result of some chain of stochastic events in the universe. And though you were totally going to pick Right, you suddenly grab the Left box instead.
Surely, there’s nothing either physically or logically impossible about this, right?
So if the predictor predicted you’d pick Right, and there’s a bomb in Left, and you have every intention of picking Right, but due to the aforesaid cosmic ray you actually take the Left box… what happens?
It’s just that Left-boxing means the predictor already predicted that, and the bomb was never there to begin with.
But the scenario stipulates that the bomb is there. Given this, taking the Left box results in… what? Like, in that scenario, if you take the Left box, what actually happens?
Agreed, but I think it’s important to stress that it’s not like you see a bomb, Left-box, and then see it disappear or something. It’s just that Left-boxing means the predictor already predicted that, and the bomb was never there to begin with.
Yes, that’s correct.
By executing the first algorithm, the bomb has never been there.
Put differently, you can only Left-box in a world where the predictor predicted you would.
Here it’s useful to distinguish between agentic ‘can’ and physical ‘can.’
Since I assume a deterministic universe for simplification, there is only one physical ‘can.’ But there are two agentic ’can″s—no matter the prediction, I can agentically choose either way. The predictor’s prediction is logically posterior to my choice, and his prediction (and the bomb’s presence) are the way they are because of my choice. So I can Left-box even if there is a bomb in the left box, even though it’s physically impossible.
(It’s better to use agentic can over physical can for decision-making, since that use of can allows us to act as if we determined the output of all computations identical to us, which brings about better results. The agent that uses the physical can as their definition will see the bomb more often.)
No, that’s just plain wrong. If you Left-box given a perfect predictor, the predictor didn’t put a bomb in Left. That’s a given. If the predictor did put a bomb in Left and you Left-box, then the predictor isn’t perfect.
“Irrelevant, since the described scenario explicitly stipulates that you find yourself in precisely that situation.”
It also stipulates the predictor predicts almost perfectly. So it’s very relevant.
“Yes, that’s what I’ve been saying: choosing Right in that scenario is the correct decision.”
No, it’s the wrong decision. Right-boxing is just the necessary consequence of the predictor predicting I Right-box. But insofar this is a decision problem, Left-boxing is correct, and then the predictor predicted I would Left-box.
“No, Left-boxing means we burn to death.”
No, it means the model Left-boxed and thus the predictor didn’t put a bomb in Left.
Do you understand how subjunctive dependence works?
It also stipulates the predictor predicts almost perfectly. So it’s very relevant.
Yes, almost perfectly (well, it has to be “almost”, because it’s also stipulated that the predictor got it wrong this time).
No, it’s the wrong decision. Right-boxing is just the necessary consequence of the predictor predicting I Right-box. But insofar this is a decision problem, Left-boxing is correct, and then the predictor predicted I would Left-box.
None of this matters, because the scenario stipulates that there’s a bomb in the Left box.
No, it means the model Left-boxed and thus the predictor didn’t put a bomb in Left.
But it’s stipulated that the predictor did put a bomb in Left. That’s part of the scenario.
Do you understand how subjunctive dependence works?
Why does it matter? We know that there’s a bomb in Left, because the scenario tells us so.
Yes, almost perfectly (well, it has to be “almost”, because it’s also stipulated that the predictor got it wrong this time).
Well, not with your answer, because you Right-box. But anyway.
Why does it matter? We know that there’s a bomb in Left, because the scenario tells us so.
It matters a lot, because in a way the problem description is contradicting itself (which happens more often in Newcomblike problems).
It says there’s a bomb in Left.
It also says that if I Left-box, then the predictor predicted this, and will not have put a Bomb in Left. (Unless you assume the predictor predicts so well by looking at, I don’t know, the color of your shoes or something. But it strongly seems like the predictor has some model of your decision procedure.)
You keep repeating (1), ignoring (2), even though (2) is stipulated just as much as (1).
So, yes, my question whether you understand subjunctive dependence is justified, because you keep ignoring that crucial part of the problem.
Well, first of all, if there is actually a contradiction in the scenario, then we’ve been wasting our time. What’s to talk about? In such a case the answer to “what happens in this scenario” is “nothing, it’s logically impossible in the first place”, and we’re done.
But of course there isn’t actually a contradiction. (Which you know, otherwise you wouldn’t have needed to hedge by saying “in a way”.)
It’s simply that the problem says that if you Left-box, then the predictor predicted this, and will not have put a bomb in Left… usually. Almost always! But not quite always. It very rarely makes mistakes! And this time, it would seem, is one of those times.
So there’s no contradiction, there’s just a (barely) fallible predictor.
So the scenario tells us that there’s a bomb in Left, we go “welp, guess the predictor screwed up”, and then… well, apparently FDT tells us to choose Left anyway? For some reason…? (Or does it? You tell me…) But regardless, obviously the correct choice is Right, because Left’s got a bomb in it.
I really don’t know what else there is to say about this.
But of course there isn’t actually a contradiction. (Which you know, otherwise you wouldn’t have needed to hedge by saying “in a way”.)
There is, as I explained. There’s 2 ways of resolving it, but yours isn’t one of them. You can’t have it both ways.
It’s simply that the problem says that if you Left-box, then the predictor predicted this, and will not have put a bomb in Left… usually. Almost always! But not quite always. It very rarely makes mistakes! And this time, it would seem, is one of those times.
Just… no. “The predictor predicted this”, yes, so there are a trillion trillion − 1 follow-up worlds where I don’t burn to death! And yes, 1 - just 1 - world where I do. Why choose to focus on that 1 out of a trillion trillion worlds?
Because the problem talks about a bomb in Left?
No. The problem says more than that. It clearly predicts a trillion trillion − 1 worlds where I don’t burn to death. That 1 world where I do sucks, but paying $100 to avoid it seems odd. Unless, of course, you value your life infinitely (which you do I believe?). That’s fine, it does all depend on the specific valuations.
The problem stipulates that you actually, in fact, find yourself in a world where there’s a bomb in Left. These “other worlds” are—in the scenario we’re given—entirely hypothetical (or “counterfactual”, if you like). Do they even exist? If so, in what sense? Not clear. But in the world you find yourself in (we are told), there’s a bomb in the Left box. You can either take that box, and burn to death, or… not do that.
So, “why choose to focus on” that world? Because that’s the world we find ourselves in, where we have to make the choice.
Paying $100 to avoid burning to death isn’t something that “seems odd”, it’s totally normal and the obviously correct choice.
My point is that those “other worlds” are just as much stipulated by the problem statement as that one world you focus on. So, you pay $100 and don’t burn to death. I don’t pay $100, burn to death in 1 world, and live for free in a trillion trillion − 1 worlds. Even if I value my life at $10,000,000,000,000, my choice gives more utility.
My point is that those “other worlds” are just as much stipulated by the problem statement as that one world you focus on.
Sorry, but no, they’re not. You may choose to infer their “existence” from what’s stated in the problem—but that’s an inference that depends on various additional assumptions (e.g. about the nature of counterfactuals, and all sorts of other things). All that’s actually stipulated is the one world you find yourself in.
You infer the existence of me burning to death from what’s stated in the problem as well. There’s no difference.
I do have the assumption of subjunctive dependence. But without that one—if, say, the predictor predicts by looking at the color of my shoes—then I don’t Left-box anyway.
You infer the existence of me burning to death from what’s stated in the problem as well. There’s no difference.
Of course there’s a difference: inferring burning to death just depends on the perfectly ordinary assumption of cause and effect, plus what is very explicitly stated in the problem. Inferring the existence of other worlds depends on much more esoteric assumptions that that. There’s really no comparison at all.
I do have the assumption of subjunctive dependence.
Not only is that not the only assumption required, it’s not even clear what it means to “assume” subjunctive dependence. Sure, it’s stipulated that the predictor is usually (but not quite always!) right about what you’ll do. What else is there to this “assumption” than that?
But how that leads to “other worlds exist” and “it’s meaningful to aggregate utility across them” and so on… I have no idea.
If they’re just possible worlds, then why do they matter? They’re not actual worlds, after all (by the time the described scenario is happening, it’s too late for any of them to be actual!). So… what’s the relevance?
The UDT convention is that other possible worlds remain relevant, even when you find yourself in a possible world that isn’t compatible with their actuality. It’s confusing to discuss this general point as if it’s specific to this contentious thought experiment.
The setting has a sample space, as in expected utility theory, with situations that take place in some event (let’s call it a situation event) and offer a choice between smaller events resulting from taking alternative actions. The misleading UDT convention is to call the situation event “actual”. It’s misleading because the goal is to optimize expected utility over the whole sample space, not just over the situation event, so the places on the sample space outside the situation event are effectively still in play, still remain relevant, not ruled out by the particular situation event being “actual”.
Alright. But by the time the situation described in the OP happens, it no longer matters whether you optimize expected utility over the whole sample space; that goal is now moot. One event out of the sample space has occurred, and the others have failed to occur. Why would you continue to attempt to achieve that goal, toward which you are no longer capable of taking any action?
by the time the situation described in the OP happens, it no longer matters whether you optimize expected utility over the whole sample space; that goal is now moot
That goal may be moot for some ways of doing decisions. For UDT it’s not moot, it’s the only thing that we care about instead. And calling some situation or another “actual” has no effect at all on the goal, and on the process of decision making in any situation, actual or otherwise, that’s what makes the goal and the decision process reflectively stable.
“But by the time the situation described in the OP happens, it no longer matters whether you optimize expected utility over the whole sample space; that goal is now moot.”
This is what we agree on. If you’re in the situation with a bomb, all that matters is the bomb.
My stance is that Left-boxers virtually never get into the situation to begin with, because of the prediction Omega makes. So with probability close to 1, they never see a bomb.
Your stance (if I understand correctly) is that the problem statement says there is a bomb, so, that’s what’s true with probability 1 (or almost 1).
And so I believe that’s where our disagreement lies. I think Newcomblike problems are often “trick questions” that can be resolved in two ways, one leaning more towards your interpretation.
In spirit of Vladimir’s points, if I annoyed you, I do apologize. I can get quite intense in such discussions.
This is what we agree on. If you’re in the situation with a bomb, all that matters is the bomb.
But that’s false for a UDT agent, it still matters to that agent-instance-in-the-situation what happens in other situations, those without a bomb, it’s not the case that all that matters is the bomb (or even a bomb).
Hmm, interesting. I don’t know much about UDT. From and FDT perspective, I’d say that if you’re in the situation with the bomb, your decision procedure already Right-boxed and therefore you’re Right-boxing again, as logical necessity. (Making the problem very interesting.)
To explain my view more, the question I try to answer in these problems is more or less: if I were to choose a decision theory now to strictly adhere to, knowing I might run into the Bomb problem, which decision theory would I choose?
If I ever find myself in the Bomb scenario, I Right-box. Because in that scenario, the predictor’s model of me already Right-boxed, and therefore I do, too—not as a decision, per se, but as a logical consequence.
The correct decision is another question—that’s Left-boxing, because the decision is being made in two places. If I find myself in the Bomb scenario, that just means the decision to Right-box was already made.
The Bomb problem asks what the correct decision is, and makes clear (at least under my assumption) that the decision is made at 2 points in time. At that first point (in the predictor’s head), Left-boxing leads to the most utility: it avoids burning to death for free. Note that at that point, there is not yet a bomb in Left!
If I ever find myself in the Bomb scenario, I Right-box.
If we agree on that, then I don’t understand what it is that you think we disagree on! (Although the “not as a decision, per se” bit seems… contentless.)
The Bomb problem asks what the correct decision is,
No, it asks what decision you should make. And we apparently agree that the answer is “Right”.
What does it mean to say that Left-boxing is “the correct decision” if you then say that the decision you’d actually make would be to Right-box? This seems to be straightforwardly contradictory, in a way that renders the claim nonsensical.
I read all your comments in this thread. But you seem to be saying things that, in a very straightforward way, simply don’t make any sense…
Alright. The correct decision is Left-boxing, because that means the predictor’s model Left-boxed (and so do I), letting me live for free. Because, at the point where the predictor models me, the Bomb isn’t placed yet (and never will be).
However, IF I’m in the Bomb scenario, then the predictor’s model already Right-boxed. Then, because of subjunctive dependence, it’s apparently not possible for me to Left-box, just as it is impossible for two calculators to give a different result to 2 + 2.
Well, the Bomb scenario is what we’re given. So the first paragraph you just wrote there is… irrelevant? Inapplicable? What’s the point of it? It’s answering a question that’s not being asked.
As for the last sentence of your comment, I don’t understand what you mean by it. Certainly it’s possible for you to Left-box; you just go ahead and Left-box. This would be a bad idea, of course! Because you’d burn to death. But you could do it! You just shouldn’t—a point on which we, apparently, agree.
The bottom line is: to the actual single question the scenario asks—which box do you choose, finding yourself in the given situation?—we give the same answer. Yes?
The bottom line is: to the actual single question the scenario asks—which box do you choose, finding yourself in the given situation?—we give the same answer. Yes?
The bottom line is that Bomb is a decision problem. If I am still free to make a decision (which I suppose I am, otherwise it isn’t much of a problem), then the decision I make is made at 2 points in time. And then, Left-boxing is the better decision.
Yes, the Bomb is what we’re given. But with the very reasonable assumption of subjunctive dependence, it specifies what I am saying...
We agree that if I would be there, I would Right-box, but also everybody would then Right-box, as a logical necessity (well, 1 in a trillion trillion error rate, sure). It has nothing to do with correct or incorrect decisions, viewed like that: the decision is already hard coded into the problem statement, because of the subjunctive dependence.
“But you can just Left-box” doesn’t work: that’s like expecting one calculator to answer to 2 + 2 differently than another calculator.
I think it’s better to explain to such people the problem where the predictor is perfect, and then generalize to an imperfect predictor. They don’t understand the general principle of your present choices pseudo-overwriting the entire timeline and can’t think in the seemingly-noncausal way that optimal decision-making requires. By jumping right to an imperfect predictor, the principle becomes, I think, too complicated to explain.
(Btw, you can call your answer “obvious” and my side “crazy” all you want, but it won’t change a thing until you actually demonstrate why and how FDT is wrong, which you haven’t done.)
I’ve done that: FDT is wrong because it (according to you) recommends that you choose to burn to death, when you could easily choose not to burn to death. Pretty simple.
It seems to me that your argument proves too much.
Let’s set aside this specific example and consider something more everyday: making promises. It is valuable to be able to make promises that others will believe, even when they are promises to do something that (once the relevant situation arises) you will strongly prefer not to do.
Suppose I want a $1000 loan, with $1100 to be repaid one year from now. My counterparty Bob has no trust in the legal system, police, etc., and expects that next year I will be somewhere where he can’t easily find me and force me to pay up. But I really need the money. Fortunately, Bob knows some mad scientists and we agree to the following: I will have implanted in my body a device that will kill me if 366 days from now I haven’t paid up. I get the money. I pay up. Nobody dies. Yay.
I hope we are agreed that (granted the rather absurd premises involved) I should be glad to have this option, even though in the case where I don’t pay up it kills me.
Revised scenario: Bob knows some mad psychologists who by some combination of questioning, brain scanning, etc., are able to determine very reliably what future choices I will make in any given situation. He also knows that in a year’s time I might (but with extremely low probability) be in a situation where I can only save my life at the cost of the $1100 that I owe him. He has no risk tolerance to speak of and will not lend me the money if in that situation I would choose to save my life and not give him the money.
Granted these (again absurd) premises, do you agree with me that it is to my advantage to have the sort of personality that can promise to pay Bob back even if it literally kills me?
It seems to me that: 1. Your argument in this thread would tell me, a year down the line and in the surprising situation that I do in fact need to choose between Bob’s money and my life, “save your life, obviously”. 2. If my personality were such that I would do as you advise in that situation, then Bob will not lend me the money. (Which may in fact mean that in that unlikely future situation I die anyway.) 3. Your reasons for saying “FDT recommends knowingly choosing to burn to death! So much the worse for FDT!”, are equally reasons to say “Being someone who can make and keep this sort of promise means knowingly choosing to pay up and die! So much the worse for being that sort of person!”. 4. Being that sort of person is not in fact worse even though there are situations in which it leads to a worse outcome. 5. There is no version of “being that sort of person” that lets you just decide to live, in that unlikely situation, because paying up at the cost of your own life is what “being that sort of person” means. 6. To whatever extent I get to choose whether to be that sort of person, I have to make the decision before I know whether I’m going to be in that unlikely situation. And, to whatever extent I get to choose, it is reasonable to choose to be that sort of person, because the net benefit is greater. 7. Once again, “be that sort of person and then change your mind” is not one of the available options; if I will change my mind about it, then I was never that sort of person after all.
What (if anything) do you disagree with in that paragraph? What (if anything) do you find relevantly disanalogous between the situation I describe here and the one with the bomb?
Granted these (again absurd) premises, do you agree with me that it is to my advantage to have the sort of personality that can promise to pay Bob back even if it literally kills me?
I do not.
What (if anything) do you disagree with in that paragraph? What (if anything) do you find relevantly disanalogous between the situation I describe here and the one with the bomb?
Your scenario omits the crucial element of the scenario in the OP, where you (the subject) find yourself in a situation where the predictor turns out to have erred in its prediction.
Hmm. I am genuinely quite baffled by this; there seems to be some very fundamental difference in how we are looking at the world. Let me just check that this is a real disagreement and not a misunderstanding (even if it is there would also be a real disagreement, but a different one): I am asking not “do you agree with me that at the point where I have to choose between dying and failing to repay Bob it is to my advantage …” but “do you agree with me that at an earlier point, say when I am negotiating with Bob it is to my advantage …”.
If I am understanding you right and you are understanding me right, then I think the following is true. Suppose that when Bob has explained his position (he is willing to lend me the money if, and only if, his mad scientists determine that I will definitely repay him even if the alternative is death), some supernatural being magically informs me that while it cannot lend me the money it can make me the sort of person who can make the kind of commitment Bob wants and actually follow through. I think you would recommend that I either not accept this offer, or at any rate not make that commitment having been empowered to do so.
Do you feel the same way about the first scenario, where instead of choosing to be a person who will pay up even at the price of death I choose to be a person who will be compelled by brute force to pay up or die? If not, why?
Your scenario omits the crucial element of the scenario in the OP, where you (the subject) find yourself in a situation where the predictor turns out to have erred in its prediction.
Why does that matter? (Maybe it doesn’t; your opinion about my scenario is AIUI the same as your opinion about the one in the OP.)
I am asking not “do you agree with me that at the point where I have to choose between dying and failing to repay Bob it is to my advantage …” but “do you agree with me that at an earlier point, say when I am negotiating with Bob it is to my advantage …”.
Yes, I understood you correctly. My answer stands. (But I appreciate the verification.)
I think you would recommend that I either not accept this offer, or at any rate not make that commitment having been empowered to do so.
Right.
Do you feel the same way about the first scenario, where instead of choosing to be a person who will pay up even at the price of death I choose to be a person who will be compelled by brute force to pay up or die? If not, why?
No, because there’s a difference between “pay up or die” and “pay up and die”.
Your scenario omits the crucial element of the scenario in the OP, where you (the subject) find yourself in a situation where the predictor turns out to have erred in its prediction.
Why does that matter? (Maybe it doesn’t; your opinion about my scenario is AIUI the same as your opinion about the one in the OP.)
The scenario in the OP seems to hinge on it. As described, the situation is that the agent has picked FDT as their decision theory, is absolutely the sort of agent who will choose the Left box and die if so predicted, who is thereby supposed to not actually encounter situations where the Left box has a bomb… but oops! The predictor messed up and there is a bomb there anyhow. And now the agent is left with a choice on which nothing depends except whether he pointlessly dies.
I agree (of course!) that there is a difference between “pay up and die” and “pay up or die”. But I don’t understand how this difference can be responsible for the difference in your opinions about the two scenarios.
Scenario 1: I choose for things to be so arranged that in unlikely situation S (where if I pay Bob back I die), if I don’t pay Bob back then I also die. You agree with me (I think—you haven’t actually said so explicitly) that it can be to my benefit for things to be this way, if this is the precondition for getting the loan from Bob.
Scenario 2: I choose for things to be so arranged that in unlikely scenario S (where, again, if I pay Bob back I die), I will definitely pay. You think this state of affairs can’t be to my advantage.
How is scenario 2 actually worse for me than scenario 1? Outside situation S, they are no different (I will not be faced with such strong incentive not to pay Bob back, and I will in fact pay him back, and I will not die). In situation S, scenario 1 means I die either way, so I might as well pay my debts; scenario 2 means I will pay up and die. I’m equally dead in each case. I choose to pay up in each case.
In scenario 1, I do have the option of saying a mental “fuck you” to Bob, not repaying my debt, and dying at the hand of his infernal machinery rather than whatever other thing I could save myself from with the money. But I’m equally dead either way, and I can’t see why I’d prefer this, and in any case it’s beyond my understanding why having this not-very-appealing extra option would be enough for scenario 1 to be good and scenario 2 to be bad.
What am I missing?
I think we are at cross purposes somehow about the “predictor turns out to have erred” thing. I do understand that this feature is present in the OP’s thought experiment and absent in mine. My thought experiment isn’t meant to be equivalent to the one in the OP, though it is meant to be similar in some ways (and I think we are agreed that it is similar in the ways I intended it to be similar). It’s meant to give me another view of something in your thinking that I don’t understand, in the hope that I might understand it better (hopefully with the eventual effect of improving either my thinking or yours, if it turns out that one of us is making a mistake rather than just starting from axioms that seem alien to one another).
Anyway, it probably doesn’t matter, because so far as I can tell you do in fact have “the same” opinion about the OP’s thought experiment and mine; I was asking about disanalogies between the two in case it turned out that you agreed with all the numbered points in the paragraph before that question. I think you don’t agree with them all, but I’m not sure exactly where the disagreements are; I might understand better if you could tell me which of those numbered points you disagree with.
But it’s stipulated that the predictor did put a bomb in Left. That’s part of the scenario.
This is instead part of the misleading framing. Putting bomb in Left is actually one of the situations being considered, not all that actually happens, even if it says that it’s what actually happens. It’s one of the possible worlds, and there is a misleading convention of saying that when you find yourself in a possible world, what you see is what actually happens. It’s because that’s how it subjectively looks like, even if other worlds are supposed to still matter by UDT convention.
This works because Left-boxing means you’re in a world where the predictors model of you also Left-boxed when the predictor made its prediction, causing it to not put a Bomb in Left.
Put differently, the situation described by MacAskill becomes virtually impossible if you Left-box, since the probability of Left-boxing and burning to death is ~0.
OR, alternatively, we say: no, we see the Bomb. We can’t retroactively change this! If we keep that part of the world fixed, then, GIVEN the subjunctive dependence between us and the predictor (assuming it’s there), that simply means we Right-box (with probability ~1), since that’s what the predictor’s model did.
Of course, then it’s not much of a decision theoretic problem anymore, since the decision is already fixed in the problem statement. If we assume we can still make a decision, then that decision is made in 2 places: first by the predictor’s model, then by us. Left-boxing means the model Left-boxes and we get to live for free. Right-boxing means the model Right-boxes and we get to live at a cost of $100. The right decision must be Left-boxing.
Irrelevant, since the described scenario explicitly stipulates that you find yourself in precisely that situation.
Yes, that’s what I’ve been saying: choosing Right in that scenario is the correct decision.
I have no idea what you mean by this.
No, Left-boxing means we burn to death.
“Irrelevant, since the described scenario explicitly stipulates that you find yourself in precisely that situation.”
Actually, this whole problem is irrelevant to me, a Left-boxer: Left-boxers never (or extremely rarely) find themselves in the situation with a bomb in Left. That’s the point.
Firstly, there’s a difference between “never” and “extremely rarely”. And in the latter case, the question remains “and what do you do then?”. To which, it seems, you answer “choose the Right box”…? Well, I agree with that! But that’s just the view that I’ve already described as “Left-box unless there’s a bomb in Left, in which case Right-box”.
It remains unclear to me what it is you think we disagree on.
That difference is so small as to be neglected.
It seems to me that strategy leaves you manipulatable by the predictor, who can then just always predict you will Right-box, put a bomb in Left, and let you Right-box, causing you to lose $1,000.
By construction it is not, because the scenario is precisely that we find ourselves in one such exceptional case; the posterior probability (having observed that we do so find ourselves) is thus ~1.
… but you have said, in a previous post, that if you find yourself in this scenario, you Right-box. How to reconcile your apparently contradictory statements…?
Except that we don’t find ourselves there if we Left-box. But we seem to be going around in a circle.
Right-boxing is the necessary consequence if we assume the predictor’s Right-box prediction is fixed now. So GIVEN the Right-box prediction, I apparently Right-box.
My entire point is that the prediction is NOT a given. I Left-box, and thus change the prediction to Left-box.
I have made no contradictory statements. I am and always have been saying that Left-boxing is the correct decision to resolve this dilemma.
There’s no “if” about it. The scenario is that we do find ourselves there. (If you’re fighting the hypothetical, you have to be very explicit about that, because then we’re just talking about two totally different, and pretty much unrelated, things. But I have so far understood you to not be doing that.)
I don’t know what you mean by “apparently”. You have two boxes—that’s the scenario. Which do you choose—that’s the question. You can pick either one; where does “apparently” come in?
What does this mean? The boxes are already in front of you.
You just said in this very comment that you Right-box in the given scenario! (And also in several other comments… are you really going to make me cite each of them…?)
I’m not going to make you cite anything. I know what you mean. I said Right-boxing is a consequence, given a certain resolution of the problem; I always maintained Left-boxing is the correct decision. Apparently I didn’t explain myself well, that’s on me. But I’m kinda done, I can’t seem to get my point across (not saying it’s your fault btw).
Do you understand why one should Left-box for a perfect predictor if there’s a bomb in the left box?
Of course one should not; if there’s a bomb in Left, doing so leads to you dying.
It doesn’t. Instead, it will make it so that there will have never been a bomb in the first place.
To understand this, imagine yourself as a deterministic algorithm. Either you Left-box under all circumstances (even if there is a bomb in the left box), or you Right-box under all circumstances, or you Right-box iff there is a bomb in the left box.
Implementing the first algorithm out of these three is the best choice (the expected utility is 0).
Implementing the third algorithm (that’s what you do) is the worst choice (the expected utility is -$100).
By the way, I want to point out that you apparently disagree with Heighn on this. He says, as I understand him, that if you pick Left, you do indeed burn to death, but this is fine, because in [1 trillion trillion minus one] possible worlds, you live and pay nothing. But you instead say that if you pick Left… something happens… and the bomb in the Left box, which you were just staring directly at, disappears somehow. Or wasn’t ever there (somehow), even though, again, you were just looking right at it.
How do you reconcile this disagreement? One of you has to be wrong about the consequences of picking the Left box.
I think we agree. My stance: if you Left-box, that just means the predictor predicted that with probability close to 1. From there on, there are a trillion trillion − 1 possible worlds where you live for free, and 1 where you die.
I’m not saying “You die, but that’s fine, because there are possible worlds where you live”. I’m saying that “you die” is a possible world, and there are way more possible worlds where you live.
How?
But apparently the consequences of this aren’t deterministic after all, since the predictor is fallible. So this doesn’t help.
If you reread my comments, I simplified it by assuming an infallible predictor.
For this, it’s helpful to define another kind of causality (logical causality) as distinct from physical causality. You can’t physically cause something to have never been that way, because physical causality can’t go to the past. But you can use logical causality for that, since the output of your decision determines not only your output, but the output of all equivalent computations across the entire timeline. By Left-boxing even in case of a bomb, you will have made it so that the predictor’s simulation of you has Left-boxed as well, resulting in the bomb never having been there.
… so, in other words, you’re not actually talking about the scenario described in the OP. But that’s what my comments have been about, so… everything you said has been a non sequitur…?
This really doesn’t answer the question.
Again, the scenario is: you’re looking at the Left box, and there’s a bomb in it. It’s right there in front of you. What do you do?
So, for example, when you say:
So if you take the Left box, what actually, physically happens?
See my top-level comment, this is precisely the problem with the scenario descibed in the OP I pointed out. Your reading is standard, but not the intended meaning.
But it’s also puzzling that you can’t ITT this point, to see both meanings, even if you disagree that it’s reasonable to allow/expect the intended one. Perhaps divesting from having an opinion on the object level question might help? Like, what is the point the others are trying to make, specifically, how does it work, regardless of if it’s a wrong point, described in a way that makes no reference to its wrongness/absurdity?
If a point seems to me to be absurd, then how can I understand or explain how it works (given that I don’t think it works at all)?
As far as your top-level comment, well, my follow-up questions about it remain unanswered…
Like with bug reports, it’s not helpful to say that something “doesn’t work at all”, it’s useful to be more specific. There’s some failure of rationality at play here, you are way too intelligent to be incapable of seeing what the point is, so there is some systematic avoidance of allowing yourself to see what is going on. Heighn’s antagonistic dogmatism doesn’t help, but shouldn’t be this debilitating.
I dropped out of that conversation because it seemed to be going in circles, and I think I’ve explained everything already. Apparently the conversation continued, green_leaf seems to be making good points, and Heighn continues needlessly upping the heat.
I don’t think object level conversation is helpful at this point, there is some methodological issue in how you think about this that I don’t see an efficient approach to. I’m already way outside the sort of conversational norms I’m trying to follow for the last few years, which is probably making this comment as hopelessly unhelpful as ever, though in 2010 that’d more likely be the default mode of response for me.
Note that it’s my argumentation that’s being called crazy, which is a large factor in the “antagonism” you seem to observe—a word choice I don’t agree with, btw.
About the “needlessly upping the heat”, I’ve tried this discussion from multiple different angles, seeing if we can come to a resolution. So far, no, alas, but not for lack of trying. I will admit some of my reactions were short and a bit provocative, but I don’t appreciate nor agree with your accusations. I have been honest in my reactions.
I’ve been you ten years ago. This doesn’t help, courtesy or honesty (purposes that tend to be at odds with each other) aren’t always sufficient, it’s also necessary to entertain strange points of view that are obviously wrong, in order to talk in another’s language, to de-escalate where escalation won’t help (it might help with feeding norms, but knowing what norms you are feeding is important). And often enough that is still useless and the best thing is to give up. Or at least more decisively overturn the chess board, as I’m doing with some of the last few comments to this post, to avoid remaining in an interminable failure mode.
Just… no. Don’t act like you know me, because you don’t. I appreciate you trying to help, but this isn’t the way.
These norms are interesting in how well they fade into the background, oppose being examined. If you happen to be a programmer or have enough impression of what that might be like, just imagine a programmer team where talking about bugs can be taboo in some circumstances, especially if they are hypothetical bugs imagined out of whole cloth to check if they happen to be there, or brought to attention to see if it’s cheap to put measures in place to prevent their going unnoticed, even if it eventually turns out that they were never there to begin with in actuality. With rationality, that’s hypotheses about how people think, including hypotheses about norms that oppose examination of such hypotheses and norms.
Sorry, I’m having trouble understanding your point here. I understand your analogy (I was a developer), but am not sure what you’re drawing the analogy to.
I see your point, although I have entertained Said’s view as well. But yes, I could have done better. I tend to get like this when my argumentation is being called crazy, and I should have done better.
You could have just told me this instead of complaining about me to Said though.
“So if you take the Left box, what actually, physically happens?”
You live. For free. Because the bomb was never there to begin with.
Yes, the situation does say the bomb is there. But it also says the bomb isn’t there if you Left-box.
At the very least, this is a contradiction, which makes the scenario incoherent nonsense.
(I don’t think it’s actually true that “it also says the bomb isn’t there if you Left-box”—but if it did say that, then the scenario would be inconsistent, and thus impossible to interpret.)
That’s what I’ve been saying to you: a contradiction.
And there are two ways to resolve it.
This is misleading. What happens is that the situation you found yourself in doesn’t take place with significant measure. You live mostly in different situations, not this one.
I don’t see how it is misleading. Achmiz asked what actually happens; it is, in virtually all possible worlds, that you live for free.
It is misleading because Said’s perspective is to focus on the current situation, without regarding the other situations as decision relevant. From UDT perspective you are advocating, the other situations remain decision relevant, and that explains much of what you are talking about in other replies. But from that same perspective, it doesn’t matter that you live in the situation Said is asking about, so it’s misleading that you keep attention on this situation in your reply without remarking on how that disagrees with the perspective you are advocating in other replies.
In the parent comment, you say “it is, in virtually all possible worlds, that you live for free”. This is confusing: are you talking about the possible worlds within the situation Said was asking about, or also about possible worlds outside that situation? The distinction matters for the argument in these comments, but you are saying this ambiguously.
No, non sequitur means something else. (If I say “A, therefore B”, but B doesn’t follow from A, that’s a non sequitur.)
I simplified the problem to make it easier for you to understand.
It does. Your question was “How?”. The answer is “through logical causality.”
You take the left box with the bomb, and it has always been empty.
This doesn’t even resemble a coherent answer. Do you really not see how absurd this is?
It doesn’t seem coherent if you don’t understand logical causality.
There is nothing incoherent about both of these being true:
You Left-box under all circumstances (even if there is a bomb in the box)
The expected utility of executing this algorithm is 0 (the best possible)
These two statements can both be true at the same time, and (1) implies (2).
None of that is responsive to the question I actually asked.
It is. The response to your question “So if you take the Left box, what actually, physically happens?” is “Physically, nothing.” That’s why I defined logical causality—it helps understand why (1) is the algorithm with the best expected utility, and why yours is worse.
What do you mean by “Physically, nothing.”? There’s a bomb in there—does it somehow fail to explode? How?
It fails to have ever been there.
Do you see how that makes absolutely no sense as an answer to the question I asked? Like, do you see what makes what you said incomprehensible, what makes it appear to be nonsense? I’m not asking you to admit that it’s nonsense, but can you see why it reads as bizarre moon logic?
I can, although I indeed don’t think it is nonsense.
What do you think our (or specifically my) viewpoint is?
I’m no longer sure; you and green_leaf appear to have different, contradictory views, and at this point that divergence has confused me enough that I could no longer say confidently what either of you seem to be saying without going back and carefully re-reading all the comments. And that, I’m afraid, isn’t something that I have time for at the moment… so perhaps it’s best to write this discussion off, after all.
Of course! Thanks for your time.
You’re still neglecting the other kind of causality, so “nothing” makes no sense to you (since something clearly happens).
I’m tapping out, since I don’t see you putting any effort into understanding this topic.
Agreed, but I think it’s important to stress that it’s not like you see a bomb, Left-box, and then see it disappear or something. It’s just that Left-boxing means the predictor already predicted that, and the bomb was never there to begin with.
Put differently, you can only Left-box in a world where the predictor predicted you would.
What stops you from Left-boxing in a world where the predictor didn’t predict that you would?
To make the question clearer, let’s set aside all this business about the fallibility of the predictor. Sure, yes, the predictor’s perfect, it can predict your actions with 100% accuracy somehow, something about algorithms, simulations, models, whatever… fine. We take all that as given.
So: you see the two boxes, and after thinking about it very carefully, you reach for the Right box (as the predictor always knew that you would).
But suddenly, a stray cosmic ray strikes your brain! No way this was predictable—it was random, the result of some chain of stochastic events in the universe. And though you were totally going to pick Right, you suddenly grab the Left box instead.
Surely, there’s nothing either physically or logically impossible about this, right?
So if the predictor predicted you’d pick Right, and there’s a bomb in Left, and you have every intention of picking Right, but due to the aforesaid cosmic ray you actually take the Left box… what happens?
But the scenario stipulates that the bomb is there. Given this, taking the Left box results in… what? Like, in that scenario, if you take the Left box, what actually happens?
The scenario also stipulates the bomb isn’t there if you Left-box.
What actually happens? Not much. You live. For free.
Yes, that’s correct.
By executing the first algorithm, the bomb has never been there.
Here it’s useful to distinguish between agentic ‘can’ and physical ‘can.’
Since I assume a deterministic universe for simplification, there is only one physical ‘can.’ But there are two agentic ’can″s—no matter the prediction, I can agentically choose either way. The predictor’s prediction is logically posterior to my choice, and his prediction (and the bomb’s presence) are the way they are because of my choice. So I can Left-box even if there is a bomb in the left box, even though it’s physically impossible.
(It’s better to use agentic can over physical can for decision-making, since that use of can allows us to act as if we determined the output of all computations identical to us, which brings about better results. The agent that uses the physical can as their definition will see the bomb more often.)
Unless I’m missing something.
No, that’s just plain wrong. If you Left-box given a perfect predictor, the predictor didn’t put a bomb in Left. That’s a given. If the predictor did put a bomb in Left and you Left-box, then the predictor isn’t perfect.
“Irrelevant, since the described scenario explicitly stipulates that you find yourself in precisely that situation.”
It also stipulates the predictor predicts almost perfectly. So it’s very relevant.
“Yes, that’s what I’ve been saying: choosing Right in that scenario is the correct decision.”
No, it’s the wrong decision. Right-boxing is just the necessary consequence of the predictor predicting I Right-box. But insofar this is a decision problem, Left-boxing is correct, and then the predictor predicted I would Left-box.
“No, Left-boxing means we burn to death.”
No, it means the model Left-boxed and thus the predictor didn’t put a bomb in Left.
Do you understand how subjunctive dependence works?
Yes, almost perfectly (well, it has to be “almost”, because it’s also stipulated that the predictor got it wrong this time).
None of this matters, because the scenario stipulates that there’s a bomb in the Left box.
But it’s stipulated that the predictor did put a bomb in Left. That’s part of the scenario.
Why does it matter? We know that there’s a bomb in Left, because the scenario tells us so.
Well, not with your answer, because you Right-box. But anyway.
It matters a lot, because in a way the problem description is contradicting itself (which happens more often in Newcomblike problems).
It says there’s a bomb in Left.
It also says that if I Left-box, then the predictor predicted this, and will not have put a Bomb in Left. (Unless you assume the predictor predicts so well by looking at, I don’t know, the color of your shoes or something. But it strongly seems like the predictor has some model of your decision procedure.)
You keep repeating (1), ignoring (2), even though (2) is stipulated just as much as (1).
So, yes, my question whether you understand subjunctive dependence is justified, because you keep ignoring that crucial part of the problem.
Well, first of all, if there is actually a contradiction in the scenario, then we’ve been wasting our time. What’s to talk about? In such a case the answer to “what happens in this scenario” is “nothing, it’s logically impossible in the first place”, and we’re done.
But of course there isn’t actually a contradiction. (Which you know, otherwise you wouldn’t have needed to hedge by saying “in a way”.)
It’s simply that the problem says that if you Left-box, then the predictor predicted this, and will not have put a bomb in Left… usually. Almost always! But not quite always. It very rarely makes mistakes! And this time, it would seem, is one of those times.
So there’s no contradiction, there’s just a (barely) fallible predictor.
So the scenario tells us that there’s a bomb in Left, we go “welp, guess the predictor screwed up”, and then… well, apparently FDT tells us to choose Left anyway? For some reason…? (Or does it? You tell me…) But regardless, obviously the correct choice is Right, because Left’s got a bomb in it.
I really don’t know what else there is to say about this.
There is, as I explained. There’s 2 ways of resolving it, but yours isn’t one of them. You can’t have it both ways.
Just… no. “The predictor predicted this”, yes, so there are a trillion trillion − 1 follow-up worlds where I don’t burn to death! And yes, 1 - just 1 - world where I do. Why choose to focus on that 1 out of a trillion trillion worlds?
Because the problem talks about a bomb in Left?
No. The problem says more than that. It clearly predicts a trillion trillion − 1 worlds where I don’t burn to death. That 1 world where I do sucks, but paying $100 to avoid it seems odd. Unless, of course, you value your life infinitely (which you do I believe?). That’s fine, it does all depend on the specific valuations.
The problem stipulates that you actually, in fact, find yourself in a world where there’s a bomb in Left. These “other worlds” are—in the scenario we’re given—entirely hypothetical (or “counterfactual”, if you like). Do they even exist? If so, in what sense? Not clear. But in the world you find yourself in (we are told), there’s a bomb in the Left box. You can either take that box, and burn to death, or… not do that.
So, “why choose to focus on” that world? Because that’s the world we find ourselves in, where we have to make the choice.
Paying $100 to avoid burning to death isn’t something that “seems odd”, it’s totally normal and the obviously correct choice.
My point is that those “other worlds” are just as much stipulated by the problem statement as that one world you focus on. So, you pay $100 and don’t burn to death. I don’t pay $100, burn to death in 1 world, and live for free in a trillion trillion − 1 worlds. Even if I value my life at $10,000,000,000,000, my choice gives more utility.
Sorry, but no, they’re not. You may choose to infer their “existence” from what’s stated in the problem—but that’s an inference that depends on various additional assumptions (e.g. about the nature of counterfactuals, and all sorts of other things). All that’s actually stipulated is the one world you find yourself in.
You infer the existence of me burning to death from what’s stated in the problem as well. There’s no difference.
I do have the assumption of subjunctive dependence. But without that one—if, say, the predictor predicts by looking at the color of my shoes—then I don’t Left-box anyway.
Of course there’s a difference: inferring burning to death just depends on the perfectly ordinary assumption of cause and effect, plus what is very explicitly stated in the problem. Inferring the existence of other worlds depends on much more esoteric assumptions that that. There’s really no comparison at all.
Not only is that not the only assumption required, it’s not even clear what it means to “assume” subjunctive dependence. Sure, it’s stipulated that the predictor is usually (but not quite always!) right about what you’ll do. What else is there to this “assumption” than that?
But how that leads to “other worlds exist” and “it’s meaningful to aggregate utility across them” and so on… I have no idea.
Inferring that I don’t burn to death depends on
Omega modelling my decision procedure
Cause and effect from there.
That’s it. No esoteric assumptions. I’m not talking about a multiverse with worlds existing next to each other or whatever, just possible worlds.
If they’re just possible worlds, then why do they matter? They’re not actual worlds, after all (by the time the described scenario is happening, it’s too late for any of them to be actual!). So… what’s the relevance?
The world you’re describing is just as much a possible world as the ones I describe. That’s my point.
Huh? It’s the world that’s stipulated to be the actual world, in the scenario.
No, it isn’t. In the world that’s stipulated, you still have to make your decision.
That decision is made in my head and in the predictor’s head. That’s the key.
But if you choose Left, you will burn to death. I’ve already quoted that. Says so right in the OP.
That’s one possible world. There are many more where I don’t burn to death.
But… there aren’t, though. They’ve already failed to be possible, at that point.
The UDT convention is that other possible worlds remain relevant, even when you find yourself in a possible world that isn’t compatible with their actuality. It’s confusing to discuss this general point as if it’s specific to this contentious thought experiment.
Well, we’re discussing it in the context of this thought experiment. If the point applies more generally, then so be it.
Can you explain (or link to an explanation of) what is meant by “convention” and “remain relevant” here?
The setting has a sample space, as in expected utility theory, with situations that take place in some event (let’s call it a situation event) and offer a choice between smaller events resulting from taking alternative actions. The misleading UDT convention is to call the situation event “actual”. It’s misleading because the goal is to optimize expected utility over the whole sample space, not just over the situation event, so the places on the sample space outside the situation event are effectively still in play, still remain relevant, not ruled out by the particular situation event being “actual”.
Alright. But by the time the situation described in the OP happens, it no longer matters whether you optimize expected utility over the whole sample space; that goal is now moot. One event out of the sample space has occurred, and the others have failed to occur. Why would you continue to attempt to achieve that goal, toward which you are no longer capable of taking any action?
That goal may be moot for some ways of doing decisions. For UDT it’s not moot, it’s the only thing that we care about instead. And calling some situation or another “actual” has no effect at all on the goal, and on the process of decision making in any situation, actual or otherwise, that’s what makes the goal and the decision process reflectively stable.
“But by the time the situation described in the OP happens, it no longer matters whether you optimize expected utility over the whole sample space; that goal is now moot.”
This is what we agree on. If you’re in the situation with a bomb, all that matters is the bomb.
My stance is that Left-boxers virtually never get into the situation to begin with, because of the prediction Omega makes. So with probability close to 1, they never see a bomb.
Your stance (if I understand correctly) is that the problem statement says there is a bomb, so, that’s what’s true with probability 1 (or almost 1).
And so I believe that’s where our disagreement lies. I think Newcomblike problems are often “trick questions” that can be resolved in two ways, one leaning more towards your interpretation.
In spirit of Vladimir’s points, if I annoyed you, I do apologize. I can get quite intense in such discussions.
But that’s false for a UDT agent, it still matters to that agent-instance-in-the-situation what happens in other situations, those without a bomb, it’s not the case that all that matters is the bomb (or even a bomb).
Hmm, interesting. I don’t know much about UDT. From and FDT perspective, I’d say that if you’re in the situation with the bomb, your decision procedure already Right-boxed and therefore you’re Right-boxing again, as logical necessity. (Making the problem very interesting.)
To explain my view more, the question I try to answer in these problems is more or less: if I were to choose a decision theory now to strictly adhere to, knowing I might run into the Bomb problem, which decision theory would I choose?
Not at the point in time where Omega models my decision procedure.
One thing we do agree on:
If I ever find myself in the Bomb scenario, I Right-box. Because in that scenario, the predictor’s model of me already Right-boxed, and therefore I do, too—not as a decision, per se, but as a logical consequence.
The correct decision is another question—that’s Left-boxing, because the decision is being made in two places. If I find myself in the Bomb scenario, that just means the decision to Right-box was already made.
The Bomb problem asks what the correct decision is, and makes clear (at least under my assumption) that the decision is made at 2 points in time. At that first point (in the predictor’s head), Left-boxing leads to the most utility: it avoids burning to death for free. Note that at that point, there is not yet a bomb in Left!
If we agree on that, then I don’t understand what it is that you think we disagree on! (Although the “not as a decision, per se” bit seems… contentless.)
No, it asks what decision you should make. And we apparently agree that the answer is “Right”.
Hmmm, I thought that comment might clear things up, but apparently it doesn’t. And I’m left wondering if you even read it.
Anyway, Left-boxing is the correct decision. But since you didn’t really engage with my points, I’ll be leaving now.
What does it mean to say that Left-boxing is “the correct decision” if you then say that the decision you’d actually make would be to Right-box? This seems to be straightforwardly contradictory, in a way that renders the claim nonsensical.
I read all your comments in this thread. But you seem to be saying things that, in a very straightforward way, simply don’t make any sense…
Alright. The correct decision is Left-boxing, because that means the predictor’s model Left-boxed (and so do I), letting me live for free. Because, at the point where the predictor models me, the Bomb isn’t placed yet (and never will be).
However, IF I’m in the Bomb scenario, then the predictor’s model already Right-boxed. Then, because of subjunctive dependence, it’s apparently not possible for me to Left-box, just as it is impossible for two calculators to give a different result to 2 + 2.
Well, the Bomb scenario is what we’re given. So the first paragraph you just wrote there is… irrelevant? Inapplicable? What’s the point of it? It’s answering a question that’s not being asked.
As for the last sentence of your comment, I don’t understand what you mean by it. Certainly it’s possible for you to Left-box; you just go ahead and Left-box. This would be a bad idea, of course! Because you’d burn to death. But you could do it! You just shouldn’t—a point on which we, apparently, agree.
The bottom line is: to the actual single question the scenario asks—which box do you choose, finding yourself in the given situation?—we give the same answer. Yes?
The bottom line is that Bomb is a decision problem. If I am still free to make a decision (which I suppose I am, otherwise it isn’t much of a problem), then the decision I make is made at 2 points in time. And then, Left-boxing is the better decision.
Yes, the Bomb is what we’re given. But with the very reasonable assumption of subjunctive dependence, it specifies what I am saying...
We agree that if I would be there, I would Right-box, but also everybody would then Right-box, as a logical necessity (well, 1 in a trillion trillion error rate, sure). It has nothing to do with correct or incorrect decisions, viewed like that: the decision is already hard coded into the problem statement, because of the subjunctive dependence.
“But you can just Left-box” doesn’t work: that’s like expecting one calculator to answer to 2 + 2 differently than another calculator.
Unless I’m missing something, it’s possible you’re in the predictor’s simulation, in which case it’s possible you will Left-box.
Excellent point!
I think it’s better to explain to such people the problem where the predictor is perfect, and then generalize to an imperfect predictor. They don’t understand the general principle of your present choices pseudo-overwriting the entire timeline and can’t think in the seemingly-noncausal way that optimal decision-making requires. By jumping right to an imperfect predictor, the principle becomes, I think, too complicated to explain.
(Btw, you can call your answer “obvious” and my side “crazy” all you want, but it won’t change a thing until you actually demonstrate why and how FDT is wrong, which you haven’t done.)
I’ve done that: FDT is wrong because it (according to you) recommends that you choose to burn to death, when you could easily choose not to burn to death. Pretty simple.
It seems to me that your argument proves too much.
Let’s set aside this specific example and consider something more everyday: making promises. It is valuable to be able to make promises that others will believe, even when they are promises to do something that (once the relevant situation arises) you will strongly prefer not to do.
Suppose I want a $1000 loan, with $1100 to be repaid one year from now. My counterparty Bob has no trust in the legal system, police, etc., and expects that next year I will be somewhere where he can’t easily find me and force me to pay up. But I really need the money. Fortunately, Bob knows some mad scientists and we agree to the following: I will have implanted in my body a device that will kill me if 366 days from now I haven’t paid up. I get the money. I pay up. Nobody dies. Yay.
I hope we are agreed that (granted the rather absurd premises involved) I should be glad to have this option, even though in the case where I don’t pay up it kills me.
Revised scenario: Bob knows some mad psychologists who by some combination of questioning, brain scanning, etc., are able to determine very reliably what future choices I will make in any given situation. He also knows that in a year’s time I might (but with extremely low probability) be in a situation where I can only save my life at the cost of the $1100 that I owe him. He has no risk tolerance to speak of and will not lend me the money if in that situation I would choose to save my life and not give him the money.
Granted these (again absurd) premises, do you agree with me that it is to my advantage to have the sort of personality that can promise to pay Bob back even if it literally kills me?
It seems to me that: 1. Your argument in this thread would tell me, a year down the line and in the surprising situation that I do in fact need to choose between Bob’s money and my life, “save your life, obviously”. 2. If my personality were such that I would do as you advise in that situation, then Bob will not lend me the money. (Which may in fact mean that in that unlikely future situation I die anyway.) 3. Your reasons for saying “FDT recommends knowingly choosing to burn to death! So much the worse for FDT!”, are equally reasons to say “Being someone who can make and keep this sort of promise means knowingly choosing to pay up and die! So much the worse for being that sort of person!”. 4. Being that sort of person is not in fact worse even though there are situations in which it leads to a worse outcome. 5. There is no version of “being that sort of person” that lets you just decide to live, in that unlikely situation, because paying up at the cost of your own life is what “being that sort of person” means. 6. To whatever extent I get to choose whether to be that sort of person, I have to make the decision before I know whether I’m going to be in that unlikely situation. And, to whatever extent I get to choose, it is reasonable to choose to be that sort of person, because the net benefit is greater. 7. Once again, “be that sort of person and then change your mind” is not one of the available options; if I will change my mind about it, then I was never that sort of person after all.
What (if anything) do you disagree with in that paragraph? What (if anything) do you find relevantly disanalogous between the situation I describe here and the one with the bomb?
I do not.
Your scenario omits the crucial element of the scenario in the OP, where you (the subject) find yourself in a situation where the predictor turns out to have erred in its prediction.
Hmm. I am genuinely quite baffled by this; there seems to be some very fundamental difference in how we are looking at the world. Let me just check that this is a real disagreement and not a misunderstanding (even if it is there would also be a real disagreement, but a different one): I am asking not “do you agree with me that at the point where I have to choose between dying and failing to repay Bob it is to my advantage …” but “do you agree with me that at an earlier point, say when I am negotiating with Bob it is to my advantage …”.
If I am understanding you right and you are understanding me right, then I think the following is true. Suppose that when Bob has explained his position (he is willing to lend me the money if, and only if, his mad scientists determine that I will definitely repay him even if the alternative is death), some supernatural being magically informs me that while it cannot lend me the money it can make me the sort of person who can make the kind of commitment Bob wants and actually follow through. I think you would recommend that I either not accept this offer, or at any rate not make that commitment having been empowered to do so.
Do you feel the same way about the first scenario, where instead of choosing to be a person who will pay up even at the price of death I choose to be a person who will be compelled by brute force to pay up or die? If not, why?
Why does that matter? (Maybe it doesn’t; your opinion about my scenario is AIUI the same as your opinion about the one in the OP.)
Yes, I understood you correctly. My answer stands. (But I appreciate the verification.)
Right.
No, because there’s a difference between “pay up or die” and “pay up and die”.
The scenario in the OP seems to hinge on it. As described, the situation is that the agent has picked FDT as their decision theory, is absolutely the sort of agent who will choose the Left box and die if so predicted, who is thereby supposed to not actually encounter situations where the Left box has a bomb… but oops! The predictor messed up and there is a bomb there anyhow. And now the agent is left with a choice on which nothing depends except whether he pointlessly dies.
I see no analogous feature of your scenarios…
I agree (of course!) that there is a difference between “pay up and die” and “pay up or die”. But I don’t understand how this difference can be responsible for the difference in your opinions about the two scenarios.
Scenario 1: I choose for things to be so arranged that in unlikely situation S (where if I pay Bob back I die), if I don’t pay Bob back then I also die. You agree with me (I think—you haven’t actually said so explicitly) that it can be to my benefit for things to be this way, if this is the precondition for getting the loan from Bob.
Scenario 2: I choose for things to be so arranged that in unlikely scenario S (where, again, if I pay Bob back I die), I will definitely pay. You think this state of affairs can’t be to my advantage.
How is scenario 2 actually worse for me than scenario 1? Outside situation S, they are no different (I will not be faced with such strong incentive not to pay Bob back, and I will in fact pay him back, and I will not die). In situation S, scenario 1 means I die either way, so I might as well pay my debts; scenario 2 means I will pay up and die. I’m equally dead in each case. I choose to pay up in each case.
In scenario 1, I do have the option of saying a mental “fuck you” to Bob, not repaying my debt, and dying at the hand of his infernal machinery rather than whatever other thing I could save myself from with the money. But I’m equally dead either way, and I can’t see why I’d prefer this, and in any case it’s beyond my understanding why having this not-very-appealing extra option would be enough for scenario 1 to be good and scenario 2 to be bad.
What am I missing?
I think we are at cross purposes somehow about the “predictor turns out to have erred” thing. I do understand that this feature is present in the OP’s thought experiment and absent in mine. My thought experiment isn’t meant to be equivalent to the one in the OP, though it is meant to be similar in some ways (and I think we are agreed that it is similar in the ways I intended it to be similar). It’s meant to give me another view of something in your thinking that I don’t understand, in the hope that I might understand it better (hopefully with the eventual effect of improving either my thinking or yours, if it turns out that one of us is making a mistake rather than just starting from axioms that seem alien to one another).
Anyway, it probably doesn’t matter, because so far as I can tell you do in fact have “the same” opinion about the OP’s thought experiment and mine; I was asking about disanalogies between the two in case it turned out that you agreed with all the numbered points in the paragraph before that question. I think you don’t agree with them all, but I’m not sure exactly where the disagreements are; I might understand better if you could tell me which of those numbered points you disagree with.
Yeah you keep repeating that. Stating it. Saying it’s simple, obvious, whatever. Saying I’m being crazy. But it’s just wrong. So there’s that.
Which part of what I said you deny…?
That I’m being crazy
That Left-boxing means burning to death
That your answer is obviously correct
Take your pick.
The scenario stipulates this:
This is instead part of the misleading framing. Putting bomb in Left is actually one of the situations being considered, not all that actually happens, even if it says that it’s what actually happens. It’s one of the possible worlds, and there is a misleading convention of saying that when you find yourself in a possible world, what you see is what actually happens. It’s because that’s how it subjectively looks like, even if other worlds are supposed to still matter by UDT convention.