Ignoring the fact that I thought there were no samples without HAART at t=0, what if half of the samples
referred to hamsters, rather than humans?
Well, there is in reality A0 and A1. I choose this example because in this example it is both the case that E[death | A0, A1] is wrong, and \sum_{L0} E[death | A0,A1,L0] p(L0) (usual covariate adjustment) is wrong, because L0 is a rather unusual type of confounder. This example was something naive causal inference used to get wrong for a long time.
More generally, you seem to be fighting the hypothetical. I gave a specific problem on only four variables, where everything is fully specified, there aren’t hamsters, and which (I claim) breaks EDT. You aren’t bringing up hamsters with Newcomb’s problem, why bring them up here? This is just a standard longitudinal design: there is nothing exotic about it, no omnipotent Omegas or source-code reading AIs.
However a decision theory in general contains no specific prescriptions for obtaining probabilities from data.
I think you misunderstand decision theory. If you were right, there would be no difference between CDT and EDT. In fact, the entire point of decision theories is to give rules you would use to make decisions. EDT has a rule involving conditional probabilities of observed data (because EDT treats all observed data as evidence). CDT has a rule involving a causal connection between your action and the outcome. This rule implies, contrary to what you claimed, that a particular method must be used to get your answer from data (this method being given by the theory of identification of causal effects) on pain of getting garbage answers and going to jail.
You aren’t bringing up hamsters with Newcomb’s problem, why bring them up here?
I said why I was bringing them up. To make the point that blindly counting the number of events in a dataset satisfying (action = X, outcome = Y) is blatantly ridiculous, and this applies whether or not hamsters are involved. If you think EDT does that then either you are mistaken, or everyone studying EDT are a lot less sane than they look.
I think you misunderstand decision theory. If you were right, there would be no difference between CDT and EDT.
The difference is that CDT asks for P(utility | do(action), observations) and EDT asks for P(utility | action, observations). Neither CDT or EDT specify detailed rules for how to calculate these probabilities or update on observations, or what priors to use. Indeed, those rules are normally found in statistics textbooks, Pearl’s Causality or—in the case of the g-formula—random math papers.
If you think EDT does that then either you are mistaken, or everyone studying EDT are a lot less sane than
they look
Ok. I keep asking you, because I want to see where I am going wrong. WIthout fighting the hypothetical, what is EDT’s answer in my hamster-free, perfectly standard longitudinal example: do you in fact give the patient HAART or not? If you think there are multiple EDTs, pick the one that gives the right answer! My point is, if you do give HAART, you have to explain what rule you use to arrive at this, and how it’s EDT and not CDT. If you do not give HAART, you are “wrong.”
The form of argument where you say “well, this couldn’t possibly be right—if it were I would be terrified!” isn’t very convincing. I think Homer Simpson used that once :).
The form of argument where you say “well, this couldn’t possibly be right—if it were I would be terrified!” isn’t very convincing. I think Homer Simpson used that once :).
What I meant was “if it were, that would require a large number of (I would expect) fairly intelligent mathematicians to have made an egregiously dumb mistake, on the order of an engineer modelling a 747 as made of cheese”. Does that seem likely to you? The principle of charity says “don’t assume someone is stupid so you can call them wrong”.
Regardless, since there is nothing weird going on here, I would expect (a particular non-strawman version of) EDT’s answer to be precisely the same as CDT’s answer, because “agent’s action” has no common causes with the relevant outcomes (ETA: no common causes that aren’t screened off by observations. If you measure patient vital signs and decide based on them, obviously that’s a common cause, but irrelevant since you’ve observed them). In which case you use whatever statistical techniques one normally uses to calculate P(utility | do(action), observations) (the g-formula seems to be an ad-hoc frequentist device as far as I can tell, but there’s probably a prior that leads to the same result in a bayesian calculation). You keep telling me that results in “give HAART” so I guess that’s the answer, even though I don’t actually have any data.
Is that a satisfying answer?
In retrospect, I would have said that before, but got distracted by the seeming ill-posedness of the problem and incompleteness of the data. (Yes, the data is incomplete. Analysing it requires nontrivial assumptions, as far as I can tell from reading a paper on the g-formula.)
the g-formula seems to be an ad-hoc frequentist device as far as I can tell
See, its things like this that make people have the negative opinion of LW as a quasi-religion that they do. I am willing to wager a guess that your understanding of “the parametric g-formula” is actually based on a google search or two. Yet despite this, you are willing to make (dogmatic, dismissive, and wrong) Bayesian-sounding pronouncements about it. In fact the g-formula is just how you link do(.) and observational data, nothing more nothing less. do(.) is defined in terms of the g-formula in Pearl’s chapter 1. The g-formula has nothing to do with Bayesian vs frequentist differences.
Is that a satisfying answer?
No. EDT is not allowed to talk about “confounders” or “causes” or “do(.)”. There is nothing in any definition of EDT in any textbook that allows you to refer to anything that isn’t a function of the observed joint density. So that’s all you can use to get the answer here. If you talk about “confounders” or “causes” or “do(.)”, you are using CDT by definition. What is the difference between EDT and CDT to you?
Re: principle of charity, it’s very easy to get causal questions wrong. Causal inference isn’t easy! Causal inference as field itself used to get the example I gave wrong until the late 1980s. Your answers about how to use EDT to get the answer here are very vague. You should be able to find a textbook on EDT, and follow an algorithm there to give a condition in terms of p(A0,A1,L0,Y) for whether HAART should be given or not. My understanding of EDT is that the condition would be:
Give HAART at A0,A1 iff E[death | A0=yes, A1=yes] < E[death | A0=no, A1=no]
So you would not give HAART by construction in my example (I mentioned people who get HAART die more often due to confounding by health status).
See, its things like this that makes people have the negative opinion of LW as a quasi-religion that they do. I am willing to wager a guess that your understanding of “the parametric g-formula” is actually based on a google search or two. Yet despite this, you are willing to make (dogmatic, dismissive, and wrong) Bayesian-sounding pronouncements about it. In fact the g-formula is just how you link do(.) and observational data, nothing more nothing less. do(.) is defined in terms of the g-formula in Pearl’s chapter 1.
You’re probably right. Not that this matters much. The reason I said that is because the few papers I could find on the g-formula were all in the context of using it to find out “whether HAART kills people”, and none of them gave any kind of justification or motivation for it, or even mentioned how it related to probabilities involving do().
No. EDT is not allowed to talk about “confounders” or “causes” or “do(.)”. There is nothing in any definition of EDT in any textbook that allows you to refer to anything that isn’t a function of the observed joint density.
Did you read what I wrote? Since action and outcome do not have any common causes (conditional on observations), P(outcome | action, observations) = P(outcome | do(action), observations). I am well aware that EDT does not mention do. This does not change the fact that this equality holds in this particular situation, which is what allows me to say that EDT and CDT have the same answer here.
Re: principle of charity, it’s very easy to get causal questions wrong.
Postulating “just count up how many samples have the particular action and outcome, and ignore everything else” as a decision theory is not a complicated causal mistake. This was the whole point of the hamster example. This method breaks horribly on the most simple dataset with a bit of irrelevant data.
ETA: [responding to your edit]
My understanding of EDT is that the condition would be:
Give HAART at A0,A1 iff E[death | A0=yes, A1=yes] < E[death | A0=no, A1=no]
No, this is completely wrong, because this ignores the fact that the action the EDT agent considers is “I (EDT agent) give this person HAART”, not “be a person who decides whether to give HAART based on metrics L0, and also give this person HAART” which isn’t something it’s possible to “decide” at all.
Since action and outcome do not have any common causes (conditional on observations), P(outcome |
action, observations) = P(outcome | do(action), observations).
In my example, A0 has no causes (it is randomized) but A1 has a common cause with the outcome Y (this common cause is the unobserved health status, which is a parent of both Y and L0, and L0 is a parent of A1). L0 is observed but you cannot adjust for it either because that screws up the effect of A0.
To get the right answer here, you need a causal theory that connects observations to causal effects. The point is, EDT isn’t allowed to just steal causal theory to get its answer without becoming a causal decision theory itself.
In my example, A0 has no causes (it is randomized) but A1 has a common cause with the outcome Y (this common cause is the unobserved health status, which is a parent of both Y and L0, and L0 is a parent of A1). L0 is observed but you cannot adjust for it either because that screws up the effect of A0.
Health status is screened off by the fact that L0 is an observation. At the point where you (EDT agent) decide whether to give HAART at A1 the relevant probability for purposes of calculating expected utility is P(outcome=Y | action=give-haart, observations=[L0, this dataset]). Effect of action on unobserved health-status and through to Y is screened off by conditioning on L0.
That’s right, but as I said, you cannot just condition on L0 because that blocks the causal path from A0 to Y, and opens a non-causal path A0 → L0 <-> Y. This is what makes L0 a “time dependent confounder” and this is why
\sum_{L0} E[Y | L0,A0,A1] p(L0) and E[Y | L0, A0, A1] are both wrong here.
(Remember, HAART is given in two stages, A0 and A1, separated by L0).
That’s right, but as I said, you cannot just condition on L0 because that blocks the causal path from A0 to Y, and opens a non-causal path A0 → L0 <-> Y.
Okay, this isn’t actually a problem. At A1 (deciding whether to give HAART at time t=1) you condition on L0 because you’ve observed it. This means using P(outcome=Y | action=give-haart-at-A1, observations=[L0, the dataset]) which happens to be identical to P(outcome=Y | do(action=give-haart-at-A1), observations=[L0, the dataset]), since A1 has no parents apart from L0. So the decision is the same as CDT at A1.
At A0 (deciding whether to give HAART at time t=0), you haven’t measured L0, so you don’t condition on it. You use P(outcome=Y | action=give-haart-at-A0, observations=[the dataset]) which happens to be the same as P(outcome=Y | do(action=give-haart-at-A0), observations=[the dataset]) since A0 has no parents at all. The decision is the same as CDT at A0, as well.
To make this perfectly clear, what I am doing here is replacing the agents at A0 and A1 (that decide whether to administer HAART) with EDT agents with access to the aforementioned dataset and calculating what they would do. That is, “You are at A0. Decide whether to administer HAART using EDT.” and “You are at A1. You have observed L0=[...]. Decide whether to administer HAART using EDT.”. The decisions about what to do at A0 and A1 are calculated separately (though the agent at A0 will generally need to know, and therefore to first calculate what A1 will do, so that they can calculate stuff like P(outcome=Y | action=give-haart-at-A0, observations=[the dataset])).
You may actually be thinking of “solve this problem using EDT” as “using EDT, derive the best (conditional) policy for agents at A0 and A1″, which means an EDT agent standing “outside the problem”, deciding upon what A0 and A1 should do ahead of time, which works somewhat differently — happily, though, it’s practically trivial to show that this EDT agent’s decision would be the same as CDT’s: because an agent deciding on a policy for A0 and A1 ahead of time is affected by nothing except the original dataset, which is of course the input (an observation), we have P(outcome | do(policy), observations=dataset) = P(outcome | policy, observations=dataset). In case it’s not obvious, the graph for this case is dataset -> (agent chooses policy) -> (some number of people die after assigning A0,A1 based on policy) -> outcome.
What I meant was “if it were, that would require a large number of (I would expect) fairly intelligent mathematicians to have made an egregiously dumb mistake, on the order of an engineer modelling a 747 as made of cheese”. Does that seem likely to you? The principle of charity says “don’t assume someone is stupid so you can call them wrong”.
Yes, actually, they do seem have to have made an egregiously dumb mistake. People think EDT is dumb because it is dumb. Full stop.
The confusion is that sometimes when people talk about EDT, they are talking about the empirical group of “EDTers”. EDTers aren’t dumb enough to actually use the math of EDT. A “non-strawman EDT” is CDT. (If it wasn’t, how could the answers always be the same?) The point of math, though, is that you can’t strawman it; the math is what it is. Making decisions based on the conditional probabilities that resulted from observing that action historically is dumb, EDT makes decisions based on conditional probabilities, therefore EDT is dumb.
If it wasn’t, how could the answers always be the same?
They’re not...? EDT one-boxes on Newcomb’s and smokes (EDIT: doesn’t smoke) on the smoking lesion (unless the tickle defense actually works or something). Of course, it also two-boxes on transparent Newcomb’s, so it’s still a dumb theory, but it’s not that dumb.
How else should I interpret “I would expect (a particular non-strawman version of) EDT’s answer to be precisely the same as CDT’s answer”?
What I said was
Regardless, since there is nothing weird going on here, I would expect (a particular non-strawman version of) EDT’s answer to be precisely the same as CDT’s answer, because “agent’s action” has no common causes with the relevant outcomes
Meaning that in this particular situation (where there aren’t any omniscient predictors or mysterious correlations), the decision is the same. I didn’t mean they were the same generally.
Huh? EDT doesn’t smoke on the smoking lesion, because P(cancer|smoking)>P(cancer|!smoking).
Okay. Do you have a mathematical description of whether they differ, or is it a “I know it when I see it” sort of description? What makes a correlation mysterious?
I’m still having trouble imagining what a “non-strawman” EDT looks like mathematically, except for what I’m calling EDT+Intuition, which is people implicitly calculating probabilities using CDT and then using those probabilities to feed into EDT (in which case they’re only using it for expected value calculation, which CDT can do just as easily). It sounds to me like someone insisting that a “non-strawman” formula for x squared is x cubed.
A first try at formalising it would amount to “build a causal graph including EDT-agent’s-decision-now as a node, and calculated expected utilities using P(utility | agent=action, observations)”.
For example, for your average boring everyday situation, such as noticing a $5 note on the ground and thinking about whether to pick it up, the graph is (do I see $5 on the ground) --> (do I try to pick it up) --> (outcome). To arrive at a decision, you calculate the expected utilities using P(utility | pick it up, observation=$5) vs P(utility | don't pick it up, observation=$5). Note that conditioning on both observations and your action breaks the correlation expressed by the first link of the graph, resulting in this being equivalent to CDT in this situation. Also conveniently this makes P(action | I see $5) not matter, even though this is technically a necessary component to have a complete graph.
To be actually realistic you would need to include a lot of other stuff in the graph, such as everything else you’ve ever observed, and (agent's state 5 minutes ago) as causes of the current action (do I try to pick it up). But all of these can either be ignored (in the case of irrelevant observations) or marginalised out without effect (in the case of unobserved causes that we don’t know affect the outcome in any particular direction).
Next take an interesting case like Newcomb’s. The graph is something like the below:
We don’t know whether agent-5-minutes-ago was the sort that would make omega fill both boxes or not (so it’s not an observation), but we do know that there’s a direct correlation between that and our one-boxing. So when calculating P(utility|one-box), which implicitly involves marginalising over (agent-5-minutes-ago) and (omega fills boxes) we see that the case where (agent-5-minutes-ago)=one-box and (omega fills boxes)=both dominates, while the opposite case dominates for P(utility|two-box), so one-boxing has a higher utility.
A first try at formalising it would amount to “build a causal graph
Are we still talking about EDT? Why call it that?
(I do think that a good decision theory starts off with “build a causal graph,” but I think that decision theory already exists, and is CDT, so there’s no need to invent it again.)
I don’t think that formalization of Newcomb’s works, or at least you should flip the arrows. I think these are the formalizations in which perfect prediction makes sense:
You’re deciding which agent to be 5 minutes ago- and your decision is foreordained based on that. (This is the ‘submit source code to a game’ option like here.)
There is a causal arrow from your decision to Omega filling the boxes. (This is the Laplace’s Demon option, where the universe is deterministic and Omega, even though it’s in the past, is fully able to perceive the future. This is also your graph if you flip the ‘one-box?’ to ‘agent 5 minutes ago’ arrow.)
In both of these causal graphs, CDT suggests one-boxing. (I think that CDTers who two-box build the wrong causal graph from the problem description.)
It’s not like only CDTers are allowed to use causal graphs. You can call them “bayes nets” if the word “causal” seems too icky.
Joking aside, it’s called EDT because it doesn’t use do(·). We’re just using regular boring old conditional probabilities on the obvious formalisation of the problem.
As for reversing the arrows… I don’t think it’s entirely trivial to justify causal arrows that go backward in time. You can probably do it, with some kind of notion of “logical causality” or something. In fact, you could construct a more abstract network with “what this decision theory recommends” (a mathematical fact) as an ancestor node of both omega’s predictions and the agent itself. If you optimize the resulting utility over various values of the decision theory node I imagine you’d end up with something analogous to Wei Dei’s UDT (or was it ADT?). The decision theory node can be set as a parent of anywhere that decision theory is implemented in the universe, which was one of the main ideas of ADT.
I’m not sure if that could really be called “causal” decision theory any more though.
You can call them “bayes nets” if the word “causal” seems too icky.
Ugh. A Bayesian network is not a causal model. I am going to have to exit this, I am finding having to explain the same things over and over again very frustrating :(. From what I could tell following this thread you subscribe to the notion that there is no difference between EDT and CDT. That’s fine, I guess, but it’s a very exotic view of decision theories, to put to mildly. It just seems like a bizarre face-saving maneuver on behalf of EDT.
I have a little bit of unsolicited advice (which I know is dangerous to do), please do not view this as a status play: read the bit of Pearl’s book where he discusses the difference between a Bayesian network, a causal Bayesian network, and a non-parametric structural equation model. This may also make it clear what the crucial difference between EDT and CDT is.
Joking aside, it’s called EDT because it doesn’t use do(·). We’re just using regular boring old conditional probabilities on the obvious formalisation of the problem.
The central disagreement between EDT and CDT is whether one should use conditionals or counterfactuals. Counterfactuals are much stronger, and so I don’t see the argument for using conditionals.
In particular, if you’re just representing the joint probability distribution as a Bayes net, you don’t have any of the new information that a causal graph provides you. In particular, you cannot tell the difference between observations and interventions, which leads to all of the silliness of normal EDT. The do() function is a feature.
In causal graphs, information does not flow backwards across arrows in the presence of an intervention. (This is the difference between counterfactuals and conditionals.) If I make a decision now, it shouldn’t impact things that might cause other people to make that decision. (Getting out of bed does not affect the time of day, even though the time of day affects getting out of bed.) When the decision “one-box?” impacts the node “5 minutes ago,” it’s part of that class of causal errors, as when someone hopes that staying in bed will make it still be morning when they decide to get out of bed. Your use of the graph as you described it has no internal mechanism to avoid that, and so would seek to “manage the news” in other situations.
This is why EDT looks so broken, and why IlyaShpitser in particular is so interested in seeing the EDT method actually worked out. It’s like the painter who insisted that he could get yellow by mixing red and white paint. In CDT, you get the causal separation between observations and interventions with the do() operator, but in EDT, you need a tube of yellow to ‘sharpen it up’.
As for reversing the arrows… I don’t think it’s entirely trivial to justify causal arrows that go backward in time. You can probably do it, with some kind of notion of “logical causality” or something.
I think that perfect prediction is functionally equivalent to causality flowing backwards in time, and don’t think it’s possible to construct a counterexample. I agree it doesn’t happen in the real world, but in the hypothetical world of Newcomb’s Problem, that’s the way things are, and so resisting that point is fighting the hypothetical. (If time travel were possible in our world, would the fact that Omega had made its prediction and left be at all decision-theoretically compelling? I don’t think so, and so I strongly suspect that two-boxing is primarily caused by misbehaving temporal intuitions.)
I’m not actually advocating EDT. After all, it two-boxes on transparent Newcomb’s, which is a clear mistake. I’m just trying to explain how it’s not as bad as it seems to be. For example:
as when someone hopes that staying in bed will make it still be morning when they decide to get out of bed. Your use of the graph as you described it has no internal mechanism to avoid that, and so would seek to “manage the news” in other situations.
The only reason there is normally a correlation between getting out of bed and “morning time” is because people decide whether to get out of bed based on the clock, sunrise, etc. I do not think that EDT would “stay in bed in order to make it still be morning” because:
If you’ve already looked at the clock, there’s no hope for you anyway. It’s already a practical certainty that it won’t be morning any more if you stay in bed and get up later. This screens off any possible effect that deciding could have. (Granted, there’s a tiny remnant effect for “what if the laws of physics changed and time doesn’t necessarily work the same ways as I’m used to”. But your samples that suggest a correlation don’t cover that case, and even if they did, they would show that people who stayed in bed hoping it would make morning last longer typically missed their early-morning commute.)
Or, if you’ve somehow managed to remain ignorant as to the time, the causal connection between “getting out of bed” and “morning” is broken, because you can’t possibly be deciding whether to get out of bed based on the time. So the graph doesn’t even have a connection between the two nodes, and staying in bed does nothing.
In general “managing the news” doesn’t work because “managing” is a very different behaviour to which simply doesn’t correlate to the information in the same way. EDT, done right, is aware of that.
I’m not actually advocating EDT. After all, it two-boxes on transparent Newcomb’s, which is a clear mistake. I’m just trying to explain how it’s not as bad as it seems to be.
I don’t see why you think that EDT isn’t “as bad as it seems to be,” yet. I see lots of verbal explanations of why EDT wouldn’t do X, but what I’m looking for is a mathematical reason why you think the language of conditionals is as strong as the language of counterfactuals, or why a decision theory that only operates in conditionals will not need human guidance (which uses counterfactuals) to avoid getting counterfactual questions wrong.
Are you under the impression that I think EDT is The Best Thing Ever or something???
EDT gets stuff wrong. It gets transparent newcomb’s wrong. It does not, however kill puppies hoping that this means that there is some sane reason to be killing puppies (because obviously people only ever kill puppies for a good reason).
The reason for this should be obvious [hint: killing puppies in the hope that there’s a good reason does not correlate with having good reasons].
I’ve explained this multiple times. I’m not going to play “person from the non-CDT tribe” any more if you still can’t understand. I don’t even like EDT...
ETA:
mathematical reason
Remember when I gave you a mathematical formalism and you ignored it saying “your arrows are the wrong way around” and “CDT is better, by the way”? That’s frustrating. This isn’t a competition.
Are you under the impression that I think EDT is The Best Thing Ever or something???
No, I think you think it’s not as bad as it seems to be. Things not being as they seem is worth investigating, especially about technical topics where additional insight could pay serious dividends.
I’m not going to play “person from the non-CDT tribe” any more if you still can’t understand.
I apologize for any harshness. I don’t think a tribal lens is useful for this discussion, though; friends can disagree stringently on technical issues. I think that as an epistemic principle it’s worth pursuing such disagreements, even though disagreements in other situations might be a signal of enmity.
Remember when I gave you a mathematical formalism and you ignored it saying “your arrows are the wrong way around” and “CDT is better, by the way”?
It’s not clear to me why you think I ignored your formalism; I needed to understand it to respond how I did.
My view is that there are two options for your formalism:
The formalism should be viewed as a causal graph, and decisions as the do() operator, in which case the decision theory is CDT and the formalism does not capture Newcomb’s problem because Omega is not a perfect predictor. Your formalism describes the situation where you want to be a one-boxer when scanned, and then switch to two-boxing after Omega has left, without Omega knowing that you’ll switch. If this is impossible, you are choosing who to be 5 minutes ago, not your decision now (or your decision now causes who you were 5 minutes ago), and if this is possible, then Omega isn’t a perfect predictor.
The formalism should be viewed as a factorization of a joint probability distribution, and decisions as observations, in which case the decision theory is EDT and the formalism will lead to “managing the news” in other situations. (If your formalism is a causal graph but decisions are observations, then the ‘causal’ part of the graph is unused, and I don’t see a difference between it and a factorization of a joint probability distribution.)
To justify the second view: I do not see a structural difference between
We don’t know whether agent-5-minutes-ago was the sort that would make omega fill both boxes or not (so it’s not an observation), but we do know that there’s a direct correlation between that and our one-boxing.
and
We don’t know what time it is (so it’s not an observation), but we do know that there’s a direct correlation between that and our getting out of bed.
whereas with the do() operator, the structural difference is clear (interventions do not propagate backwards across causal arrows). If you want to use language like “the causal connection between “getting out of bed” and “morning” is broken,” that language makes sense with the do() operator but doesn’t make sense with conditioning. How are you supposed to know which correlations are broken, and which aren’t, without using the do() operator?
To justify the second view: I do not see a structural difference between
We don’t know whether agent-5-minutes-ago was the sort that would make omega fill both boxes or not (so it’s not an observation), but we do know that there’s a direct correlation between that and our one-boxing.
and
We don’t know what time it is (so it’s not an observation), but we do know that there’s a direct correlation between that and our getting out of bed.
The difference is that in the first case the diamond graph is still accurate, because our dataset or other evidence that we construct the graph from says that there’s a correlation between 5-minutes-ago and one-box even when the agent doesn’t know the state of agent-5-minutes-ago.
In the second case there should be no connection from time to get-out-of-bed, because we haven’t observed the time, and all our samples which would otherwise suggest a correlation involve an agent who has observed the time, and decides whether to get out of bed based on that, so they’re inapplicable to this situation. More strongly, we know that the supposed correlation is mediated by “looking at the clock and deciding to get up if it’s past 8am”, which cannot happen here. There is no correlation until we observe the time.
The difference is that in the first case the diamond graph is still accurate
My understanding of this response is “the structural difference between the situations is that the causal graphs are different for the two situations.” I agree with that statement, but I think you have the graphs flipped around; I think the causal graph you drew describes the get-out-of-bed problem, and not Newcomb’s problem. I think the following causal graph fits the “get out of bed?” situation:
Time → I get out of bed → Late, Time → Boss gets out of bed → Late
(I’ve added the second path to make the graphs exactly analogous; suppose I’m only late for work if I arrive after my boss, and if I manage to change what time it is, I’ll change when my boss gets to work because I’ve changed what time it is.)
This has the same form as the Newcomb’s causal graph that you made; only the names have changed. In that situation, you asserted that the direct correlation between “one-box?” and “agent 5 minutes ago” was strong and relevant to our decision, even though we hadn’t observed “agent 5 minutes ago,” and historical agents who played Omega’s game hadn’t observed “agent 5 minutes ago” when they played. I assert that there’s a direct correlation between Time and I-get-out-of-bed which doesn’t depend on observing Time. (The assumption that the correlation between getting out of bed and the time is mediated by looking at the clock is your addition, and doesn’t need to be true; let’s assume the least convenient possible world where it isn’t mediated by that. I can think of a few examples and I’m sure you could as well.)
And so when we calculate P(Late|I-get-out-of-bed) using the method you recommended for Newcomb’s in the last paragraph of this comment, we implicitly marginalize over Time and Boss-gets-out-of-bed, and notice that when we choose to stay in bed, this increases the probability that it’s early, which increases the probability that our boss chooses to stay in bed!
For the two graphs- which only differ in the names of the nodes- to give different mathematical results, we would need to have our math pay attention to the names of the nodes. This is a fatal flaw, because it means we need to either teach our math how to read English, or hand-guide it every time we want to use it.
This automatic understanding of causality is what the mathematical formalism of interventions is good for: while observations flow back up causal chains (seeing my partner get out of bed is evidence about the time, or looking at a clock is evidence about the time), interventions don’t (seeing myself get out of bed isn’t evidence about the time, or adjusting my clock isn’t evidence about the time). All the English and physics surrounding these two problems can be encapsulated in the direction of the causal arrows in the graph, and then the math can proceed automatically.
This requires that different problems have different graphs: the “get out of bed” problem has the graph I describe here, which is identical to your Newcomb’s graph, and the Newcomb’s graph has the arrow pointing from one-box? to agent-5-minutes-ago, rather than the other way around, like I suggested here. With the bed graph I recommend, choosing to stay in bed cannot affect your boss’s arrival time, and with the Newcomb’s graph I recommend, your decision to one-box can affect Omega’s filling of the boxes. We get the sane results- get out of bed and one-box.
This way, all the exoticism of the problem goes into constructing the graph (a graph where causal arrows point backwards in time is exotic!), not into the math of what to do with a graph.
I assert that there’s a direct correlation between Time and I-get-out-of-bed which doesn’t depend on observing Time. (The assumption that the correlation between getting out of bed and the time is mediated by looking at the clock is your addition, and doesn’t need to be true; let’s assume the least convenient possible world where it isn’t mediated by that. I can think of a few examples and I’m sure you could as well.)
You can turn any ordinary situation into the smoking lesion by postulating mysterious correlations rather than straightforward correlations that work by conscious decisions based on observations. Did you have any realistic examples in mind?
Yes. Suppose that the person always goes to sleep at the same time, and wakes up after a random interval. In the dark of their bedroom (blackout curtains and no clock), they decide whether or not to go back to sleep or get up. Historically, the later in the morning it is, the more likely they are to get up. To make the historical record analogous to Newcomb’s, we might postulate that historically, they have always decided to go back to sleep before 7 AM, and always decided to get up after 7 AM, and the boss’s alarm is set to wake him up at 7 AM. This is not a very realistic postulation, as a stochastic relationship between the two is more realistic, but the parameters of the factorization are not related to the structure of the causal graph (and Newcomb’s isn’t very realistic either).
You can turn any ordinary situation into the smoking lesion by postulating mysterious correlations rather than straightforward correlations that work by conscious decisions based on observations.
It’s not obvious to me what you mean by “mysterious correlations” and “straightforward correlations.” Correlations are statistical objects that either exist or don’t, and I don’t know what conscious decision based on observations you’re referring to in the smoking lesion problem. What makes the smoking lesion problem a problem is that the lesions are unobserved.
For example, in an Israeli study, parole decisions are correlated with the time since the parole board last took a break. Is that correlation mysterious, or straightforward? No one familiar with the system (board members, lawyers, etc.) expected the effects the study revealed, but there are plausible explanations for the effect (mental fatigue, low blood sugar, declining mood, all of which are replenished by a meal break).
It might be that by ‘mysterious correlations’ you mean ‘correlations without an obvious underlying causal mechanism’, and by ‘straightforward correlations’ you mean ‘correlations with an obvious underlying causal mechanism.’ It’s not clear to me what the value of that distinction is. Neither joint probability distributions nor causal graphs do not require that the correlations or causal arrows be labeled.
On the meta level, though, I don’t think it’s productive to fight the hypothetical. I strongly recommend reading The Least Convenient Possible World.
It’s not obvious to me what you mean by “mysterious correlations” and “straightforward correlations.” Correlations are statistical objects that either exist or don’t, and I don’t know what conscious decision based on observations you’re referring to in the smoking lesion problem. What makes the smoking lesion problem a problem is that the lesions are unobserved.
Well, the correlations in the smoking lesion problem are mysterious because they aren’t caused by agents observing lesion|no-lesion and deciding whether to smoke based on that. They are mysterious because it is simply postulated that “the lesion causes smoking without being observed” without any explanation of how, and it is generally assumed that the correlation somehow still applies when you’re deciding what to do using EDT, which I personally have some doubt about (EDT decides what to do based only on preferences and observations, so how can its output be correlated to anything else?).
Straightforward correlations are those where, for example, people go out with an umbrella if they see rain clouds forming. The correlation is created by straightforward decision-making based on observations. Simple statistical reasoning suggests that you only have reason to expect these correlations to hold for an EDT agent if the EDT agent makes the same decisions in the same situations. Furthermore, these correlations tend to pose no problem for EDT because the only time an EDT agent is in a position to take an action correlated to some observation in this way (“I observe rain clouds, should I take my umbrella?”), they must have already observed the correlate (“rain clouds”), so EDT makes no attempt to influence it (“whether or not I take my umbrella, I know there are rain clouds already”) .
Returning to the smoking lesion problem, there are a few ways of making the mystery go away. You can suppose that the lesion works by making you smoke even after you (consciously) decide to do something else. In this case the decision of the EDT agent isn’t actually smoke | don't-smoke, but rather you get to decide a parameter of something else that determines whether you smoke. This makes the lesion not actually a cause of your decision, so you choose-to-smoke, obviously.
Alternatively, I was going to analyse the situation where the lesion makes you want to smoke (by altering your decision theory/preferences), but it made my head hurt. I anticipate that EDT wouldn’t smoke in that situation iff you can somehow remain ignorant of your decision or utility function even while implementing EDT, but I can’t be sure.
Basically, the causal reasons behind your data (why do people always get up after 7AM?) matter, because they determine what kind of causal graph you can infer for the situation with an EDT agent with some given set of observations, as opposed to whatever agents are in the dataset.
Postscript regarding LCPW: If I’m trying to argue that EDT doesn’t normally break, then presenting a situation where it does break isn’t necessarily proper LCPW. Because I never argued that it always did the right thing (which would require me to handle edge cases).
it is simply postulated that “the lesion causes smoking without being observed” without any explanation of how
No mathematical decision theory requires verbal explanations to be part of the model that it operates on. (It’s true that when learning a causal model from data, you need causal assumptions; but when a problem provides the model rather than the data, this is not necessary.)
it is generally assumed that the correlation somehow still applies when you’re deciding what to do using EDT, which I personally have some doubt about
You have doubt that this is how EDT, as a mathematical algorithm, operates, or you have some doubt that this is a wise way to construct a decision-making algorithm?
If the second, this is why I think EDT is a subpar decision theory. It sees the world as a joint probability distribution, and does not have the ability to distinguish correlation and causation, which means it cannot know whether or not a correlation applies for any particular action (and so assumes that all do).
If the first, I’m not sure how to clear up your confusion. There is a mindset that programming cultivates, which is that the system does exactly what you tell it to, with the corollary that your intentions have no weight.
If I’m trying to argue that EDT doesn’t normally break, then presenting a situation where it does break isn’t necessarily proper LCPW.
The trouble with LCPW is that it’s asymmetric; Eliezer claims that the LCPW is the one where his friend has to face a moral question, and Eliezer’s friend might claim that the LCPW is the one where Eliezer has to face a practical problem.
The way to break the asymmetry is to try to find the most informative comparison. If the hypothetical has been fought, then we learn nothing about morality, because there is no moral problem. If the hypothetical is accepted despite faults, then we learn quite a bit about morality.
The issues with EDT might require ‘edge cases’ to make obvious, but in the same way that the issues with Newtonian dynamics might require ‘edge cases’ to make obvious.
No mathematical decision theory requires verbal explanations to be part of the model that it operates on. (It’s true that when learning a causal model from data, you need causal assumptions; but when a problem provides the model rather than the data, this is not necessary.)
What I’m saying is that the only way to solve any decision theory problem is to learn a causal model from data. It just doesn’t make sense to postulate particular correlations between an EDT agent’s decisions and other things before you even know what EDT decides! The only reason you get away with assuming graphs like lesion -> (CDT Agent) -> action for CDT is because the first thing CDT does when calculating a decision is break all connections to parents by means of do(...).
Take Jiro’s example. The lesion makes people jump into volcanoes. 100% of them, and no-one else. Furthermore, I’ll postulate that all of them are using decision theory “check if I have the lesion, if so, jump into a volcano, otherwise don’t”. Should you infer the causal graph lesion -> (EDT decision: jump?) -> die with a perfect correlation between lesion and jump? (Hint: no, that would be stupid, since we’re not using jump-based-on-lesion-decision-theory, we’re using EDT.)
There is a mindset that programming cultivates, which is that the system does exactly what you tell it to, with the corollary that your intentions have no weight.
In programming, we also say “garbage in, garbage out”. You are feeding EDT garbage input by giving it factually wrong joint probability distributions.
Ok, what about cases where there are multiple causal hypotheses that are observationally indistinguishable:
a → b → c
vs
a ← b ← c
Both models imply the same joint probability distribution p(a,b,c) with a single conditional independence (a independent of c given b) and cannot be told apart without experimentation. That is, you cannot call p(a,b,c) “factually wrong” because the correct causal model implies it. But the wrong causal model implies it too! To figure out which is which requires causal information. You can give it to EDT and it will work—but then it’s not EDT anymore.
I can give you a graph which implies the same independences as my HAART example but has a completely different causal structure, and the procedure you propose here:
will give the right answer in one case and the wrong answer in another.
The point is, EDT lacks a rich enough input language to avoid getting garbage inputs in lots of standard cases. Or, more precisely, EDT lacks a rich enough input languages to tell when input is garbage and when it isn’t. This is why EDT is a terrible decision theory.
What I’m saying is that the only way to solve any decision theory problem is to learn a causal model from data.
I think there are a couple of confusions this sentence highlights.
First, there are approaches to solving decision theory problems that don’t use causal models. Part of what has made this conversation challenging is that there are several different ways to represent the world- and so even if CDT is the best / natural one, it needs to be distinguished from other approaches. EDT is not CDT in disguise; the two are distinct formulas / approaches.
Second, there are good reasons to modularize the components of the decision theory, so that you can treat learning a model from data separately from making a decision given a model. An algorithm to turn models into decisions should be able to operate on an arbitrary model, where it sees a → b → c as isomorphic to Drunk → Fall → Death.
To tell an anecdote, when my decision analysis professor would teach that subject to petroleum engineers, he quickly learned not to use petroleum examples. Say something like “suppose the probability of striking oil by drilling a well here is 40%” and an engineer’s hand will shoot up, asking “what kind of rock is it?”. The kind of rock is useful for determining whether or not the probability is 40% or something else, but the question totally misses the point of what the professor is trying to teach. The primary example he uses is choosing a location for a party subject to the uncertainty of the weather.
It just doesn’t make sense to postulate particular correlations between an EDT agent’s decisions and other things before you even know what EDT decides!
I’m not sure how to interpret this sentence.
The way EDT operates is to perform the following three steps for each possible action in turn:
Assume that I saw myself doing X.
Perform a Bayesian update on this new evidence.
Calculate and record my utility.
It then chooses the possible action which had the highest calculated utility.
One interpretation is you saying that EDT doesn’t make sense, but I’m not sure I agree with what seems to be the stated reason. It looks to me like you’re saying “it doesn’t make sense to assume that you do X until you know what you decide!”, when I think that does make sense, but the problem is using that assumption as Bayesian evidence as if it were an observation.
The way EDT operates is to perform the following three steps for each possible action in turn:
Assume that I saw myself doing X.
Perform a Bayesian update on this new evidence.
Calculate and record my utility.
Ideal Bayesian updates assume logical omniscience, right? Including knowledge about logical fact of what EDT would do for any given input. If you know that you are an EDT agent, and condition on all of your past observations and also on the fact that you do X, but X is not in fact what EDT does given those inputs, then as an ideal Bayesian you will know that you’re conditioning on something impossible. More generally, what update you perform in step 2 depends on EDT’s input-output map, thus making the definition circular.
So, is EDT really underspecified? Or are you supposed to search for a fixed point of the circular definition, if there is one? Or does it use some method other than Bayes for the hypothetical update? Or does an EDT agent really break if it ever finds out its own decision algorithm? Or did I totally misunderstand?
Ideal Bayesian updates assume logical omniscience, right? Including knowledge about logical fact of what EDT would do for any given input.
Note that step 1 is “Assume that I saw myself doing X,” not “Assume that EDT outputs X as the optimal action.” I believe that excludes any contradictions along those lines. Does logical omniscience preclude imagining counterfactual worlds?
If I already know “I am EDT”, then “I saw myself doing X” does imply “EDT outputs X as the optimal action”. Logical omniscience doesn’t preclude imagining counterfactual worlds, but imagining counterfactual worlds is a different operation than performing Bayesian updates. CDT constructs counterfactuals by severing some of the edges in its causal graph and then assuming certain values for the nodes that no longer have any causes. TDT does too, except with a different graph and a different choice of edges to sever.
I don’t know how I can fail to communicate so consistently.
Yes, you can technically apply “EDT” to any causal model or (more generally) joint probability distribution containing a “EDT agent decision” node. But in practice this freedom is useless, because to derive an accurate model you generally need to take account of a) the fact that the agent is using EDT and b) any observations the agent does or does not make. To be clear, the input EDT requires is a probabilistic model describing the EDT agent’s situation (not describing historical data of “similar” situations).
There are people here trying to argue against EDT by taking a model describing historical data (such as people following dumb decision theories jumping into volcanoes) and feeding this model directly into EDT. Which is simply wrong. A model that describes the historical behaviour of agents using some other decision theory does not in general accurately describe an EDT agent in the same situation.
The fact that this egregious mistake looks perfectly normal is an artifact of the fact that CDT doesn’t care about causal parents of the “CDT decision” node.
I don’t know how I can fail to communicate so consistently.
I suspect it’s because what you are referring to as “EDT” is not what experts in the field use that technical term to mean.
nsheppard-EDT is, as far as I can tell, the second half of CDT. Take a causal model and use the do() operator to create the manipulated subgraph that would result taking possible action (as an intervention). Determine the joint probability distribution from the manipulated subgraph. Condition on observing that action with the joint probability distribution, and calculate the probabilistically-weighted mean utility of the possible outcomes. This is isomorphic to CDT, and so referring to it as EDT leads to confusion.
Here’s a modified version. Instead of a smoking lesion, there’s a “jump into active volcano lesion”. Furthermore, the correlation isn’t as puny as for the smoking lesion. 100% of people with this lesion jump into active volcanoes and die, and nobody else does.
Should you go jump into an active volcano?
Using a decision theory to figure out what decision you should make assumes that you’re capable of making a decision. “The lesion causes you to jump into an active volcano/smoke” and “you can choose whether to jump into an active volcano/smoke” are contradictory. Even “the lesion is correlated (at less than 100%) with jumping into an active volcano/smoking” and “you can choose whether to jump into an active volcano/smoke” are contradictory unless “is correlated with” involves some correlation for people who don’t use decision theory and no correlation for people who do.
Using a decision theory to figure out what decision you should make assumes that you’re capable of making a decision.
Agreed.
unless “is correlated with” involves some correlation for people who don’t use decision theory and no correlation for people who do.
Doesn’t this seem sort of realistic, actually? Decisions made with System 1 and System 2, to use Kahneman’s language, might have entirely different underlying algorithms. (There is some philosophical trouble about how far we can push the idea of an ‘intervention’, but I think for human-scale decisions there is a meaningful difference between interventions and observations such that CDT distinguishing between them is a feature.)
This maps onto an objection by proponents of EDT that the observational data might not be from people using EDT, and thus the correlation may disappear when EDT comes onto the stage. I think that objection proves too much- suppose all of our observational data on the health effects of jumping off cliffs comes from subjects who were not using EDT (suppose they were drunk). I don’t see a reason inside the decision theory for differentiating between the effects of EDT on the correlation between jumping off the cliff and the effects of EDT on the correlation between smoking and having the lesion.
These two situations correspond to two different causal structures—Drunk → Fall → Death and Smoke ← Lesion → Cancer—which could have the same joint probability distribution. The directionality of the arrow is something that CDT can make use of to tell that the two situations will respond differently to interventions at Drunk and Smoke: it is dangerous to be drunk around cliffs, but not to smoke (in this hypothetical world).
EDT cannot make use of those arrows. It just has Drunk—Fall—Death and Smoke—Lesion—Cancer (where it knows that the correlations between Drunk and Death are mediated by Fall, and the correlations between Smoke and Cancer are mediated by Lesion). If we suppose that adding an EDT node might mean that the correlation between Smoke and Lesion (and thus Cancer) might be mediated by EDT, then we must also suppose that adding an EDT node might mean that the correlation between Drunk and Fall (and thus Death) might be mediated by EDT.
(I should point out that the EDT node describes whether or not EDT was used to decide to drink, not to decide whether or not to fall off the cliff, by analogy of using EDT to decide whether or not to smoke, rather than deciding whether or not to have a lesion.)
there’s another CRUCIAL difference regarding the Newcombs problem: there’s always a chance you’re in a simulation being run by Omega. I think if you can account for that, it SHOULD patch most decent decision-theories up. I’m willing to be quite flexible in my understanding of which theories get patched up or not.
this has the BIG advantage of NOT requiring non-linear causality in the model-it just gives a flow from simulation->”real”world.
there’s always a chance you’re in a simulation being run by Omega. I think if you can account for that, it SHOULD patch most decent decision-theories up.
Yes, reflective consistency tends to make things better.
Well, there is in reality A0 and A1. I choose this example because in this example it is both the case that E[death | A0, A1] is wrong, and \sum_{L0} E[death | A0,A1,L0] p(L0) (usual covariate adjustment) is wrong, because L0 is a rather unusual type of confounder. This example was something naive causal inference used to get wrong for a long time.
More generally, you seem to be fighting the hypothetical. I gave a specific problem on only four variables, where everything is fully specified, there aren’t hamsters, and which (I claim) breaks EDT. You aren’t bringing up hamsters with Newcomb’s problem, why bring them up here? This is just a standard longitudinal design: there is nothing exotic about it, no omnipotent Omegas or source-code reading AIs.
I think you misunderstand decision theory. If you were right, there would be no difference between CDT and EDT. In fact, the entire point of decision theories is to give rules you would use to make decisions. EDT has a rule involving conditional probabilities of observed data (because EDT treats all observed data as evidence). CDT has a rule involving a causal connection between your action and the outcome. This rule implies, contrary to what you claimed, that a particular method must be used to get your answer from data (this method being given by the theory of identification of causal effects) on pain of getting garbage answers and going to jail.
I said why I was bringing them up. To make the point that blindly counting the number of events in a dataset satisfying (action = X, outcome = Y) is blatantly ridiculous, and this applies whether or not hamsters are involved. If you think EDT does that then either you are mistaken, or everyone studying EDT are a lot less sane than they look.
The difference is that CDT asks for
P(utility | do(action), observations)and EDT asks forP(utility | action, observations). Neither CDT or EDT specify detailed rules for how to calculate these probabilities or update on observations, or what priors to use. Indeed, those rules are normally found in statistics textbooks, Pearl’s Causality or—in the case of the g-formula—random math papers.Ok. I keep asking you, because I want to see where I am going wrong. WIthout fighting the hypothetical, what is EDT’s answer in my hamster-free, perfectly standard longitudinal example: do you in fact give the patient HAART or not? If you think there are multiple EDTs, pick the one that gives the right answer! My point is, if you do give HAART, you have to explain what rule you use to arrive at this, and how it’s EDT and not CDT. If you do not give HAART, you are “wrong.”
The form of argument where you say “well, this couldn’t possibly be right—if it were I would be terrified!” isn’t very convincing. I think Homer Simpson used that once :).
What I meant was “if it were, that would require a large number of (I would expect) fairly intelligent mathematicians to have made an egregiously dumb mistake, on the order of an engineer modelling a 747 as made of cheese”. Does that seem likely to you? The principle of charity says “don’t assume someone is stupid so you can call them wrong”.
Regardless, since there is nothing weird going on here, I would expect (a particular non-strawman version of) EDT’s answer to be precisely the same as CDT’s answer, because “agent’s action” has no common causes with the relevant outcomes (ETA: no common causes that aren’t screened off by observations. If you measure patient vital signs and decide based on them, obviously that’s a common cause, but irrelevant since you’ve observed them). In which case you use whatever statistical techniques one normally uses to calculate
P(utility | do(action), observations)(the g-formula seems to be an ad-hoc frequentist device as far as I can tell, but there’s probably a prior that leads to the same result in a bayesian calculation). You keep telling me that results in “give HAART” so I guess that’s the answer, even though I don’t actually have any data.Is that a satisfying answer?
In retrospect, I would have said that before, but got distracted by the seeming ill-posedness of the problem and incompleteness of the data. (Yes, the data is incomplete. Analysing it requires nontrivial assumptions, as far as I can tell from reading a paper on the g-formula.)
See, its things like this that make people have the negative opinion of LW as a quasi-religion that they do. I am willing to wager a guess that your understanding of “the parametric g-formula” is actually based on a google search or two. Yet despite this, you are willing to make (dogmatic, dismissive, and wrong) Bayesian-sounding pronouncements about it. In fact the g-formula is just how you link do(.) and observational data, nothing more nothing less. do(.) is defined in terms of the g-formula in Pearl’s chapter 1. The g-formula has nothing to do with Bayesian vs frequentist differences.
No. EDT is not allowed to talk about “confounders” or “causes” or “do(.)”. There is nothing in any definition of EDT in any textbook that allows you to refer to anything that isn’t a function of the observed joint density. So that’s all you can use to get the answer here. If you talk about “confounders” or “causes” or “do(.)”, you are using CDT by definition. What is the difference between EDT and CDT to you?
Re: principle of charity, it’s very easy to get causal questions wrong. Causal inference isn’t easy! Causal inference as field itself used to get the example I gave wrong until the late 1980s. Your answers about how to use EDT to get the answer here are very vague. You should be able to find a textbook on EDT, and follow an algorithm there to give a condition in terms of p(A0,A1,L0,Y) for whether HAART should be given or not. My understanding of EDT is that the condition would be:
Give HAART at A0,A1 iff E[death | A0=yes, A1=yes] < E[death | A0=no, A1=no]
So you would not give HAART by construction in my example (I mentioned people who get HAART die more often due to confounding by health status).
You’re probably right. Not that this matters much. The reason I said that is because the few papers I could find on the g-formula were all in the context of using it to find out “whether HAART kills people”, and none of them gave any kind of justification or motivation for it, or even mentioned how it related to probabilities involving do().
Did you read what I wrote? Since
actionandoutcomedo not have any common causes (conditional onobservations),P(outcome | action, observations) = P(outcome | do(action), observations). I am well aware that EDT does not mentiondo. This does not change the fact that this equality holds in this particular situation, which is what allows me to say that EDT and CDT have the same answer here.Postulating “just count up how many samples have the particular action and outcome, and ignore everything else” as a decision theory is not a complicated causal mistake. This was the whole point of the hamster example. This method breaks horribly on the most simple dataset with a bit of irrelevant data.
ETA: [responding to your edit]
No, this is completely wrong, because this ignores the fact that the action the EDT agent considers is “I (EDT agent) give this person HAART”, not “be a person who decides whether to give HAART based on metrics L0, and also give this person HAART” which isn’t something it’s possible to “decide” at all.
Thanks for this. Technical issue:
In my example, A0 has no causes (it is randomized) but A1 has a common cause with the outcome Y (this common cause is the unobserved health status, which is a parent of both Y and L0, and L0 is a parent of A1). L0 is observed but you cannot adjust for it either because that screws up the effect of A0.
To get the right answer here, you need a causal theory that connects observations to causal effects. The point is, EDT isn’t allowed to just steal causal theory to get its answer without becoming a causal decision theory itself.
Health status is screened off by the fact that L0 is an observation. At the point where you (EDT agent) decide whether to give HAART at A1 the relevant probability for purposes of calculating expected utility is
P(outcome=Y | action=give-haart, observations=[L0, this dataset]). Effect ofactionon unobservedhealth-statusand through toYis screened off by conditioning on L0.That’s right, but as I said, you cannot just condition on L0 because that blocks the causal path from A0 to Y, and opens a non-causal path A0 → L0 <-> Y. This is what makes L0 a “time dependent confounder” and this is why
\sum_{L0} E[Y | L0,A0,A1] p(L0) and E[Y | L0, A0, A1] are both wrong here.
(Remember, HAART is given in two stages, A0 and A1, separated by L0).
Okay, this isn’t actually a problem. At A1 (deciding whether to give HAART at time t=1) you condition on L0 because you’ve observed it. This means using
P(outcome=Y | action=give-haart-at-A1, observations=[L0, the dataset])which happens to be identical toP(outcome=Y | do(action=give-haart-at-A1), observations=[L0, the dataset]), since A1 has no parents apart from L0. So the decision is the same as CDT at A1.At A0 (deciding whether to give HAART at time t=0), you haven’t measured L0, so you don’t condition on it. You use
P(outcome=Y | action=give-haart-at-A0, observations=[the dataset])which happens to be the same asP(outcome=Y | do(action=give-haart-at-A0), observations=[the dataset])since A0 has no parents at all. The decision is the same as CDT at A0, as well.To make this perfectly clear, what I am doing here is replacing the agents at A0 and A1 (that decide whether to administer HAART) with EDT agents with access to the aforementioned dataset and calculating what they would do. That is, “You are at A0. Decide whether to administer HAART using EDT.” and “You are at A1. You have observed L0=[...]. Decide whether to administer HAART using EDT.”. The decisions about what to do at A0 and A1 are calculated separately (though the agent at A0 will generally need to know, and therefore to first calculate what A1 will do, so that they can calculate stuff like
P(outcome=Y | action=give-haart-at-A0, observations=[the dataset])).You may actually be thinking of “solve this problem using EDT” as “using EDT, derive the best (conditional) policy for agents at A0 and A1″, which means an EDT agent standing “outside the problem”, deciding upon what A0 and A1 should do ahead of time, which works somewhat differently — happily, though, it’s practically trivial to show that this EDT agent’s decision would be the same as CDT’s: because an agent deciding on a policy for A0 and A1 ahead of time is affected by nothing except the original dataset, which is of course the input (an observation), we have
P(outcome | do(policy), observations=dataset) = P(outcome | policy, observations=dataset). In case it’s not obvious, the graph for this case isdataset -> (agent chooses policy) -> (some number of people die after assigning A0,A1 based on policy) -> outcome.Yes, actually, they do seem have to have made an egregiously dumb mistake. People think EDT is dumb because it is dumb. Full stop.
The confusion is that sometimes when people talk about EDT, they are talking about the empirical group of “EDTers”. EDTers aren’t dumb enough to actually use the math of EDT. A “non-strawman EDT” is CDT. (If it wasn’t, how could the answers always be the same?) The point of math, though, is that you can’t strawman it; the math is what it is. Making decisions based on the conditional probabilities that resulted from observing that action historically is dumb, EDT makes decisions based on conditional probabilities, therefore EDT is dumb.
They’re not...? EDT one-boxes on Newcomb’s and smokes (EDIT: doesn’t smoke) on the smoking lesion (unless the tickle defense actually works or something). Of course, it also two-boxes on transparent Newcomb’s, so it’s still a dumb theory, but it’s not that dumb.
How else should I interpret “I would expect (a particular non-strawman version of) EDT’s answer to be precisely the same as CDT’s answer”?
Huh? EDT doesn’t smoke on the smoking lesion, because P(cancer|smoking)>P(cancer|!smoking).
What I said was
Meaning that in this particular situation (where there aren’t any omniscient predictors or mysterious correlations), the decision is the same. I didn’t mean they were the same generally.
Er, you’re right. I got mixed up there.
Okay. Do you have a mathematical description of whether they differ, or is it a “I know it when I see it” sort of description? What makes a correlation mysterious?
I’m still having trouble imagining what a “non-strawman” EDT looks like mathematically, except for what I’m calling EDT+Intuition, which is people implicitly calculating probabilities using CDT and then using those probabilities to feed into EDT (in which case they’re only using it for expected value calculation, which CDT can do just as easily). It sounds to me like someone insisting that a “non-strawman” formula for x squared is x cubed.
A first try at formalising it would amount to “build a causal graph including EDT-agent’s-decision-now as a node, and calculated expected utilities using P(utility | agent=action, observations)”.
For example, for your average boring everyday situation, such as noticing a $5 note on the ground and thinking about whether to pick it up, the graph is
(do I see $5 on the ground) --> (do I try to pick it up) --> (outcome). To arrive at a decision, you calculate the expected utilities usingP(utility | pick it up, observation=$5) vs P(utility | don't pick it up, observation=$5). Note that conditioning on both observations and your action breaks the correlation expressed by the first link of the graph, resulting in this being equivalent to CDT in this situation. Also conveniently this makesP(action | I see $5)not matter, even though this is technically a necessary component to have a complete graph.To be actually realistic you would need to include a lot of other stuff in the graph, such as everything else you’ve ever observed, and
(agent's state 5 minutes ago)as causes of the current action(do I try to pick it up). But all of these can either be ignored (in the case of irrelevant observations) or marginalised out without effect (in the case of unobserved causes that we don’t know affect the outcome in any particular direction).Next take an interesting case like Newcomb’s. The graph is something like the below:
We don’t know whether
agent-5-minutes-agowas the sort that would make omega fill both boxes or not (so it’s not an observation), but we do know that there’s a direct correlation between that and our one-boxing. So when calculatingP(utility|one-box), which implicitly involves marginalising over(agent-5-minutes-ago)and(omega fills boxes)we see that the case where(agent-5-minutes-ago)=one-boxand(omega fills boxes)=bothdominates, while the opposite case dominates forP(utility|two-box), so one-boxing has a higher utility.Are we still talking about EDT? Why call it that?
(I do think that a good decision theory starts off with “build a causal graph,” but I think that decision theory already exists, and is CDT, so there’s no need to invent it again.)
I don’t think that formalization of Newcomb’s works, or at least you should flip the arrows. I think these are the formalizations in which perfect prediction makes sense:
You’re deciding which agent to be 5 minutes ago- and your decision is foreordained based on that. (This is the ‘submit source code to a game’ option like here.)
There is a causal arrow from your decision to Omega filling the boxes. (This is the Laplace’s Demon option, where the universe is deterministic and Omega, even though it’s in the past, is fully able to perceive the future. This is also your graph if you flip the ‘one-box?’ to ‘agent 5 minutes ago’ arrow.)
In both of these causal graphs, CDT suggests one-boxing. (I think that CDTers who two-box build the wrong causal graph from the problem description.)
It’s not like only CDTers are allowed to use causal graphs. You can call them “bayes nets” if the word “causal” seems too icky.
Joking aside, it’s called EDT because it doesn’t use
do(·). We’re just using regular boring old conditional probabilities on the obvious formalisation of the problem.As for reversing the arrows… I don’t think it’s entirely trivial to justify causal arrows that go backward in time. You can probably do it, with some kind of notion of “logical causality” or something. In fact, you could construct a more abstract network with “what this decision theory recommends” (a mathematical fact) as an ancestor node of both omega’s predictions and the agent itself. If you optimize the resulting utility over various values of the decision theory node I imagine you’d end up with something analogous to Wei Dei’s UDT (or was it ADT?). The decision theory node can be set as a parent of anywhere that decision theory is implemented in the universe, which was one of the main ideas of ADT.
I’m not sure if that could really be called “causal” decision theory any more though.
Ugh. A Bayesian network is not a causal model. I am going to have to exit this, I am finding having to explain the same things over and over again very frustrating :(. From what I could tell following this thread you subscribe to the notion that there is no difference between EDT and CDT. That’s fine, I guess, but it’s a very exotic view of decision theories, to put to mildly. It just seems like a bizarre face-saving maneuver on behalf of EDT.
I have a little bit of unsolicited advice (which I know is dangerous to do), please do not view this as a status play: read the bit of Pearl’s book where he discusses the difference between a Bayesian network, a causal Bayesian network, and a non-parametric structural equation model. This may also make it clear what the crucial difference between EDT and CDT is.
Also read this if you have time: www.biostat.harvard.edu/robins/publications/wp100.pdf
This paper discusses what a causal model is very clearly (actually it discusses 4 separate causal models arranged in a hierarchy of “strength.”)
EDT one-boxes on newcomb’s. Also I am well aware that not all bayes nets are causal models.
The central disagreement between EDT and CDT is whether one should use conditionals or counterfactuals. Counterfactuals are much stronger, and so I don’t see the argument for using conditionals.
In particular, if you’re just representing the joint probability distribution as a Bayes net, you don’t have any of the new information that a causal graph provides you. In particular, you cannot tell the difference between observations and interventions, which leads to all of the silliness of normal EDT. The do() function is a feature.
In causal graphs, information does not flow backwards across arrows in the presence of an intervention. (This is the difference between counterfactuals and conditionals.) If I make a decision now, it shouldn’t impact things that might cause other people to make that decision. (Getting out of bed does not affect the time of day, even though the time of day affects getting out of bed.) When the decision “one-box?” impacts the node “5 minutes ago,” it’s part of that class of causal errors, as when someone hopes that staying in bed will make it still be morning when they decide to get out of bed. Your use of the graph as you described it has no internal mechanism to avoid that, and so would seek to “manage the news” in other situations.
This is why EDT looks so broken, and why IlyaShpitser in particular is so interested in seeing the EDT method actually worked out. It’s like the painter who insisted that he could get yellow by mixing red and white paint. In CDT, you get the causal separation between observations and interventions with the do() operator, but in EDT, you need a tube of yellow to ‘sharpen it up’.
I think that perfect prediction is functionally equivalent to causality flowing backwards in time, and don’t think it’s possible to construct a counterexample. I agree it doesn’t happen in the real world, but in the hypothetical world of Newcomb’s Problem, that’s the way things are, and so resisting that point is fighting the hypothetical. (If time travel were possible in our world, would the fact that Omega had made its prediction and left be at all decision-theoretically compelling? I don’t think so, and so I strongly suspect that two-boxing is primarily caused by misbehaving temporal intuitions.)
I’m not actually advocating EDT. After all, it two-boxes on transparent Newcomb’s, which is a clear mistake. I’m just trying to explain how it’s not as bad as it seems to be. For example:
The only reason there is normally a correlation between getting out of bed and “morning time” is because people decide whether to get out of bed based on the clock, sunrise, etc. I do not think that EDT would “stay in bed in order to make it still be morning” because:
If you’ve already looked at the clock, there’s no hope for you anyway. It’s already a practical certainty that it won’t be morning any more if you stay in bed and get up later. This screens off any possible effect that deciding could have. (Granted, there’s a tiny remnant effect for “what if the laws of physics changed and time doesn’t necessarily work the same ways as I’m used to”. But your samples that suggest a correlation don’t cover that case, and even if they did, they would show that people who stayed in bed hoping it would make morning last longer typically missed their early-morning commute.)
Or, if you’ve somehow managed to remain ignorant as to the time, the causal connection between “getting out of bed” and “morning” is broken, because you can’t possibly be deciding whether to get out of bed based on the time. So the graph doesn’t even have a connection between the two nodes, and staying in bed does nothing.
In general “managing the news” doesn’t work because “managing” is a very different behaviour to which simply doesn’t correlate to the information in the same way. EDT, done right, is aware of that.
I don’t see why you think that EDT isn’t “as bad as it seems to be,” yet. I see lots of verbal explanations of why EDT wouldn’t do X, but what I’m looking for is a mathematical reason why you think the language of conditionals is as strong as the language of counterfactuals, or why a decision theory that only operates in conditionals will not need human guidance (which uses counterfactuals) to avoid getting counterfactual questions wrong.
Are you under the impression that I think EDT is The Best Thing Ever or something???
EDT gets stuff wrong. It gets transparent newcomb’s wrong. It does not, however kill puppies hoping that this means that there is some sane reason to be killing puppies (because obviously people only ever kill puppies for a good reason).
The reason for this should be obvious [hint: killing puppies in the hope that there’s a good reason does not correlate with having good reasons].
I’ve explained this multiple times. I’m not going to play “person from the non-CDT tribe” any more if you still can’t understand. I don’t even like EDT...
ETA:
Remember when I gave you a mathematical formalism and you ignored it saying “your arrows are the wrong way around” and “CDT is better, by the way”? That’s frustrating. This isn’t a competition.
No, I think you think it’s not as bad as it seems to be. Things not being as they seem is worth investigating, especially about technical topics where additional insight could pay serious dividends.
I apologize for any harshness. I don’t think a tribal lens is useful for this discussion, though; friends can disagree stringently on technical issues. I think that as an epistemic principle it’s worth pursuing such disagreements, even though disagreements in other situations might be a signal of enmity.
It’s not clear to me why you think I ignored your formalism; I needed to understand it to respond how I did.
My view is that there are two options for your formalism:
The formalism should be viewed as a causal graph, and decisions as the do() operator, in which case the decision theory is CDT and the formalism does not capture Newcomb’s problem because Omega is not a perfect predictor. Your formalism describes the situation where you want to be a one-boxer when scanned, and then switch to two-boxing after Omega has left, without Omega knowing that you’ll switch. If this is impossible, you are choosing who to be 5 minutes ago, not your decision now (or your decision now causes who you were 5 minutes ago), and if this is possible, then Omega isn’t a perfect predictor.
The formalism should be viewed as a factorization of a joint probability distribution, and decisions as observations, in which case the decision theory is EDT and the formalism will lead to “managing the news” in other situations. (If your formalism is a causal graph but decisions are observations, then the ‘causal’ part of the graph is unused, and I don’t see a difference between it and a factorization of a joint probability distribution.)
To justify the second view: I do not see a structural difference between
and
whereas with the do() operator, the structural difference is clear (interventions do not propagate backwards across causal arrows). If you want to use language like “the causal connection between “getting out of bed” and “morning” is broken,” that language makes sense with the do() operator but doesn’t make sense with conditioning. How are you supposed to know which correlations are broken, and which aren’t, without using the do() operator?
The difference is that in the first case the diamond graph is still accurate, because our dataset or other evidence that we construct the graph from says that there’s a correlation between 5-minutes-ago and one-box even when the agent doesn’t know the state of agent-5-minutes-ago.
In the second case there should be no connection from
timetoget-out-of-bed, because we haven’t observed the time, and all our samples which would otherwise suggest a correlation involve an agent who has observed the time, and decides whether to get out of bed based on that, so they’re inapplicable to this situation. More strongly, we know that the supposed correlation is mediated by “looking at the clock and deciding to get up if it’s past 8am”, which cannot happen here. There is no correlation until we observe the time.My understanding of this response is “the structural difference between the situations is that the causal graphs are different for the two situations.” I agree with that statement, but I think you have the graphs flipped around; I think the causal graph you drew describes the get-out-of-bed problem, and not Newcomb’s problem. I think the following causal graph fits the “get out of bed?” situation:
Time → I get out of bed → Late, Time → Boss gets out of bed → Late
(I’ve added the second path to make the graphs exactly analogous; suppose I’m only late for work if I arrive after my boss, and if I manage to change what time it is, I’ll change when my boss gets to work because I’ve changed what time it is.)
This has the same form as the Newcomb’s causal graph that you made; only the names have changed. In that situation, you asserted that the direct correlation between “one-box?” and “agent 5 minutes ago” was strong and relevant to our decision, even though we hadn’t observed “agent 5 minutes ago,” and historical agents who played Omega’s game hadn’t observed “agent 5 minutes ago” when they played. I assert that there’s a direct correlation between Time and I-get-out-of-bed which doesn’t depend on observing Time. (The assumption that the correlation between getting out of bed and the time is mediated by looking at the clock is your addition, and doesn’t need to be true; let’s assume the least convenient possible world where it isn’t mediated by that. I can think of a few examples and I’m sure you could as well.)
And so when we calculate P(Late|I-get-out-of-bed) using the method you recommended for Newcomb’s in the last paragraph of this comment, we implicitly marginalize over Time and Boss-gets-out-of-bed, and notice that when we choose to stay in bed, this increases the probability that it’s early, which increases the probability that our boss chooses to stay in bed!
For the two graphs- which only differ in the names of the nodes- to give different mathematical results, we would need to have our math pay attention to the names of the nodes. This is a fatal flaw, because it means we need to either teach our math how to read English, or hand-guide it every time we want to use it.
This automatic understanding of causality is what the mathematical formalism of interventions is good for: while observations flow back up causal chains (seeing my partner get out of bed is evidence about the time, or looking at a clock is evidence about the time), interventions don’t (seeing myself get out of bed isn’t evidence about the time, or adjusting my clock isn’t evidence about the time). All the English and physics surrounding these two problems can be encapsulated in the direction of the causal arrows in the graph, and then the math can proceed automatically.
This requires that different problems have different graphs: the “get out of bed” problem has the graph I describe here, which is identical to your Newcomb’s graph, and the Newcomb’s graph has the arrow pointing from one-box? to agent-5-minutes-ago, rather than the other way around, like I suggested here. With the bed graph I recommend, choosing to stay in bed cannot affect your boss’s arrival time, and with the Newcomb’s graph I recommend, your decision to one-box can affect Omega’s filling of the boxes. We get the sane results- get out of bed and one-box.
This way, all the exoticism of the problem goes into constructing the graph (a graph where causal arrows point backwards in time is exotic!), not into the math of what to do with a graph.
You can turn any ordinary situation into the smoking lesion by postulating mysterious correlations rather than straightforward correlations that work by conscious decisions based on observations. Did you have any realistic examples in mind?
Yes. Suppose that the person always goes to sleep at the same time, and wakes up after a random interval. In the dark of their bedroom (blackout curtains and no clock), they decide whether or not to go back to sleep or get up. Historically, the later in the morning it is, the more likely they are to get up. To make the historical record analogous to Newcomb’s, we might postulate that historically, they have always decided to go back to sleep before 7 AM, and always decided to get up after 7 AM, and the boss’s alarm is set to wake him up at 7 AM. This is not a very realistic postulation, as a stochastic relationship between the two is more realistic, but the parameters of the factorization are not related to the structure of the causal graph (and Newcomb’s isn’t very realistic either).
It’s not obvious to me what you mean by “mysterious correlations” and “straightforward correlations.” Correlations are statistical objects that either exist or don’t, and I don’t know what conscious decision based on observations you’re referring to in the smoking lesion problem. What makes the smoking lesion problem a problem is that the lesions are unobserved.
For example, in an Israeli study, parole decisions are correlated with the time since the parole board last took a break. Is that correlation mysterious, or straightforward? No one familiar with the system (board members, lawyers, etc.) expected the effects the study revealed, but there are plausible explanations for the effect (mental fatigue, low blood sugar, declining mood, all of which are replenished by a meal break).
It might be that by ‘mysterious correlations’ you mean ‘correlations without an obvious underlying causal mechanism’, and by ‘straightforward correlations’ you mean ‘correlations with an obvious underlying causal mechanism.’ It’s not clear to me what the value of that distinction is. Neither joint probability distributions nor causal graphs do not require that the correlations or causal arrows be labeled.
On the meta level, though, I don’t think it’s productive to fight the hypothetical. I strongly recommend reading The Least Convenient Possible World.
Well, the correlations in the smoking lesion problem are mysterious because they aren’t caused by agents observing
lesion|no-lesionand deciding whether to smoke based on that. They are mysterious because it is simply postulated that “the lesion causes smoking without being observed” without any explanation of how, and it is generally assumed that the correlation somehow still applies when you’re deciding what to do using EDT, which I personally have some doubt about (EDT decides what to do based only on preferences and observations, so how can its output be correlated to anything else?).Straightforward correlations are those where, for example, people go out with an umbrella if they see rain clouds forming. The correlation is created by straightforward decision-making based on observations. Simple statistical reasoning suggests that you only have reason to expect these correlations to hold for an EDT agent if the EDT agent makes the same decisions in the same situations. Furthermore, these correlations tend to pose no problem for EDT because the only time an EDT agent is in a position to take an action correlated to some observation in this way (“I observe rain clouds, should I take my umbrella?”), they must have already observed the correlate (“rain clouds”), so EDT makes no attempt to influence it (“whether or not I take my umbrella, I know there are rain clouds already”) .
Returning to the smoking lesion problem, there are a few ways of making the mystery go away. You can suppose that the lesion works by making you smoke even after you (consciously) decide to do something else. In this case the decision of the EDT agent isn’t actually
smoke | don't-smoke, but rather you get to decide a parameter of something else that determines whether you smoke. This makes the lesion not actually a cause of your decision, so you choose-to-smoke, obviously.Alternatively, I was going to analyse the situation where the lesion makes you want to smoke (by altering your decision theory/preferences), but it made my head hurt. I anticipate that EDT wouldn’t smoke in that situation iff you can somehow remain ignorant of your decision or utility function even while implementing EDT, but I can’t be sure.
Basically, the causal reasons behind your data (why do people always get up after 7AM?) matter, because they determine what kind of causal graph you can infer for the situation with an EDT agent with some given set of observations, as opposed to whatever agents are in the dataset.
Postscript regarding LCPW: If I’m trying to argue that EDT doesn’t normally break, then presenting a situation where it does break isn’t necessarily proper LCPW. Because I never argued that it always did the right thing (which would require me to handle edge cases).
No mathematical decision theory requires verbal explanations to be part of the model that it operates on. (It’s true that when learning a causal model from data, you need causal assumptions; but when a problem provides the model rather than the data, this is not necessary.)
You have doubt that this is how EDT, as a mathematical algorithm, operates, or you have some doubt that this is a wise way to construct a decision-making algorithm?
If the second, this is why I think EDT is a subpar decision theory. It sees the world as a joint probability distribution, and does not have the ability to distinguish correlation and causation, which means it cannot know whether or not a correlation applies for any particular action (and so assumes that all do).
If the first, I’m not sure how to clear up your confusion. There is a mindset that programming cultivates, which is that the system does exactly what you tell it to, with the corollary that your intentions have no weight.
The trouble with LCPW is that it’s asymmetric; Eliezer claims that the LCPW is the one where his friend has to face a moral question, and Eliezer’s friend might claim that the LCPW is the one where Eliezer has to face a practical problem.
The way to break the asymmetry is to try to find the most informative comparison. If the hypothetical has been fought, then we learn nothing about morality, because there is no moral problem. If the hypothetical is accepted despite faults, then we learn quite a bit about morality.
The issues with EDT might require ‘edge cases’ to make obvious, but in the same way that the issues with Newtonian dynamics might require ‘edge cases’ to make obvious.
What I’m saying is that the only way to solve any decision theory problem is to learn a causal model from data. It just doesn’t make sense to postulate particular correlations between an EDT agent’s decisions and other things before you even know what EDT decides! The only reason you get away with assuming graphs like
lesion -> (CDT Agent) -> actionfor CDT is because the first thing CDT does when calculating a decision is break all connections to parents by means ofdo(...).Take Jiro’s example. The lesion makes people jump into volcanoes. 100% of them, and no-one else. Furthermore, I’ll postulate that all of them are using decision theory “check if I have the lesion, if so, jump into a volcano, otherwise don’t”. Should you infer the causal graph
lesion -> (EDT decision: jump?) -> diewith a perfect correlation betweenlesionandjump? (Hint: no, that would be stupid, since we’re not using jump-based-on-lesion-decision-theory, we’re using EDT.)In programming, we also say “garbage in, garbage out”. You are feeding EDT garbage input by giving it factually wrong joint probability distributions.
Ok, what about cases where there are multiple causal hypotheses that are observationally indistinguishable:
a → b → c
vs
a ← b ← c
Both models imply the same joint probability distribution p(a,b,c) with a single conditional independence (a independent of c given b) and cannot be told apart without experimentation. That is, you cannot call p(a,b,c) “factually wrong” because the correct causal model implies it. But the wrong causal model implies it too! To figure out which is which requires causal information. You can give it to EDT and it will work—but then it’s not EDT anymore.
I can give you a graph which implies the same independences as my HAART example but has a completely different causal structure, and the procedure you propose here:
http://lesswrong.com/lw/hwq/evidential_decision_theory_selection_bias_and/9d6f
will give the right answer in one case and the wrong answer in another.
The point is, EDT lacks a rich enough input language to avoid getting garbage inputs in lots of standard cases. Or, more precisely, EDT lacks a rich enough input languages to tell when input is garbage and when it isn’t. This is why EDT is a terrible decision theory.
I think there are a couple of confusions this sentence highlights.
First, there are approaches to solving decision theory problems that don’t use causal models. Part of what has made this conversation challenging is that there are several different ways to represent the world- and so even if CDT is the best / natural one, it needs to be distinguished from other approaches. EDT is not CDT in disguise; the two are distinct formulas / approaches.
Second, there are good reasons to modularize the components of the decision theory, so that you can treat learning a model from data separately from making a decision given a model. An algorithm to turn models into decisions should be able to operate on an arbitrary model, where it sees a → b → c as isomorphic to Drunk → Fall → Death.
To tell an anecdote, when my decision analysis professor would teach that subject to petroleum engineers, he quickly learned not to use petroleum examples. Say something like “suppose the probability of striking oil by drilling a well here is 40%” and an engineer’s hand will shoot up, asking “what kind of rock is it?”. The kind of rock is useful for determining whether or not the probability is 40% or something else, but the question totally misses the point of what the professor is trying to teach. The primary example he uses is choosing a location for a party subject to the uncertainty of the weather.
I’m not sure how to interpret this sentence.
The way EDT operates is to perform the following three steps for each possible action in turn:
Assume that I saw myself doing X.
Perform a Bayesian update on this new evidence.
Calculate and record my utility.
It then chooses the possible action which had the highest calculated utility.
One interpretation is you saying that EDT doesn’t make sense, but I’m not sure I agree with what seems to be the stated reason. It looks to me like you’re saying “it doesn’t make sense to assume that you do X until you know what you decide!”, when I think that does make sense, but the problem is using that assumption as Bayesian evidence as if it were an observation.
Ideal Bayesian updates assume logical omniscience, right? Including knowledge about logical fact of what EDT would do for any given input. If you know that you are an EDT agent, and condition on all of your past observations and also on the fact that you do X, but X is not in fact what EDT does given those inputs, then as an ideal Bayesian you will know that you’re conditioning on something impossible. More generally, what update you perform in step 2 depends on EDT’s input-output map, thus making the definition circular.
So, is EDT really underspecified? Or are you supposed to search for a fixed point of the circular definition, if there is one? Or does it use some method other than Bayes for the hypothetical update? Or does an EDT agent really break if it ever finds out its own decision algorithm? Or did I totally misunderstand?
Note that step 1 is “Assume that I saw myself doing X,” not “Assume that EDT outputs X as the optimal action.” I believe that excludes any contradictions along those lines. Does logical omniscience preclude imagining counterfactual worlds?
If I already know “I am EDT”, then “I saw myself doing X” does imply “EDT outputs X as the optimal action”. Logical omniscience doesn’t preclude imagining counterfactual worlds, but imagining counterfactual worlds is a different operation than performing Bayesian updates. CDT constructs counterfactuals by severing some of the edges in its causal graph and then assuming certain values for the nodes that no longer have any causes. TDT does too, except with a different graph and a different choice of edges to sever.
I don’t know how I can fail to communicate so consistently.
Yes, you can technically apply “EDT” to any causal model or (more generally) joint probability distribution containing a “EDT agent decision” node. But in practice this freedom is useless, because to derive an accurate model you generally need to take account of a) the fact that the agent is using EDT and b) any observations the agent does or does not make. To be clear, the input EDT requires is a probabilistic model describing the EDT agent’s situation (not describing historical data of “similar” situations).
There are people here trying to argue against EDT by taking a model describing historical data (such as people following dumb decision theories jumping into volcanoes) and feeding this model directly into EDT. Which is simply wrong. A model that describes the historical behaviour of agents using some other decision theory does not in general accurately describe an EDT agent in the same situation.
The fact that this egregious mistake looks perfectly normal is an artifact of the fact that CDT doesn’t care about causal parents of the “CDT decision” node.
I suspect it’s because what you are referring to as “EDT” is not what experts in the field use that technical term to mean.
nsheppard-EDT is, as far as I can tell, the second half of CDT. Take a causal model and use the do() operator to create the manipulated subgraph that would result taking possible action (as an intervention). Determine the joint probability distribution from the manipulated subgraph. Condition on observing that action with the joint probability distribution, and calculate the probabilistically-weighted mean utility of the possible outcomes. This is isomorphic to CDT, and so referring to it as EDT leads to confusion.
Whatever. I give up.
Here’s a modified version. Instead of a smoking lesion, there’s a “jump into active volcano lesion”. Furthermore, the correlation isn’t as puny as for the smoking lesion. 100% of people with this lesion jump into active volcanoes and die, and nobody else does.
Should you go jump into an active volcano?
Using a decision theory to figure out what decision you should make assumes that you’re capable of making a decision. “The lesion causes you to jump into an active volcano/smoke” and “you can choose whether to jump into an active volcano/smoke” are contradictory. Even “the lesion is correlated (at less than 100%) with jumping into an active volcano/smoking” and “you can choose whether to jump into an active volcano/smoke” are contradictory unless “is correlated with” involves some correlation for people who don’t use decision theory and no correlation for people who do.
Agreed.
Doesn’t this seem sort of realistic, actually? Decisions made with System 1 and System 2, to use Kahneman’s language, might have entirely different underlying algorithms. (There is some philosophical trouble about how far we can push the idea of an ‘intervention’, but I think for human-scale decisions there is a meaningful difference between interventions and observations such that CDT distinguishing between them is a feature.)
This maps onto an objection by proponents of EDT that the observational data might not be from people using EDT, and thus the correlation may disappear when EDT comes onto the stage. I think that objection proves too much- suppose all of our observational data on the health effects of jumping off cliffs comes from subjects who were not using EDT (suppose they were drunk). I don’t see a reason inside the decision theory for differentiating between the effects of EDT on the correlation between jumping off the cliff and the effects of EDT on the correlation between smoking and having the lesion.
These two situations correspond to two different causal structures—Drunk → Fall → Death and Smoke ← Lesion → Cancer—which could have the same joint probability distribution. The directionality of the arrow is something that CDT can make use of to tell that the two situations will respond differently to interventions at Drunk and Smoke: it is dangerous to be drunk around cliffs, but not to smoke (in this hypothetical world).
EDT cannot make use of those arrows. It just has Drunk—Fall—Death and Smoke—Lesion—Cancer (where it knows that the correlations between Drunk and Death are mediated by Fall, and the correlations between Smoke and Cancer are mediated by Lesion). If we suppose that adding an EDT node might mean that the correlation between Smoke and Lesion (and thus Cancer) might be mediated by EDT, then we must also suppose that adding an EDT node might mean that the correlation between Drunk and Fall (and thus Death) might be mediated by EDT.
(I should point out that the EDT node describes whether or not EDT was used to decide to drink, not to decide whether or not to fall off the cliff, by analogy of using EDT to decide whether or not to smoke, rather than deciding whether or not to have a lesion.)
there’s another CRUCIAL difference regarding the Newcombs problem: there’s always a chance you’re in a simulation being run by Omega. I think if you can account for that, it SHOULD patch most decent decision-theories up. I’m willing to be quite flexible in my understanding of which theories get patched up or not.
this has the BIG advantage of NOT requiring non-linear causality in the model-it just gives a flow from simulation->”real”world.
Yes, reflective consistency tends to make things better.
um...that wasn’t sarcastic, was it? I just ran low on mental energy so...
anyways, the downside is you have to figure out how to dissolve all or most of the anthropic paradoxes when evaluating simulation chance.