I don’t understand your comment at all. You seem to assume as “trivial” many things that I set out to demystify. Maybe you could formalize your argument?
I’m not sure if I understand torekp’s comment right, but it seems to me that the issue is the difference between “physical” and “logical” counterfactuals in the following sense:
For concreteness and simplicity, let’s suppose physics can be modeled by a function step(state) that takes a state of the world, and returns the state of the world one instant later. Then you can ask what would happen if the current state were slightly different (“suppose there were an apple here in the air at this point in time; then it would fall down”). Call this a “physical” counterfactual.
Before reading Eliezer’s arguments, I used to think this could be used to model decision-making: Let stateA be what the state of the world will really be at the point I’ve made my decision. For every choice X I could make, let stateA_X be the same as stateA except that the future copy of me is replaced by a version that has decided to do X. Now it’s well-defined what the future of every stateA_X looks like, and if I have a well-defined utility function, then it’s well-defined which choice(s) this utility is maximized. Of course I cannot calculate this, but, I thought, the actual decision making process could be seen as an approximation of this.
Except that this would lead to classical decision/game theory, with its unattractive consequences—the case where Omega has just made two identical physical copies of me, and has us play a true Prisoner’s Dilemma, etc. Since I can prove that the other copy of me will do the same thing in every stateA_X (because I’m only replacing myself, not the other copy), it follows that I should defect, because it gives me the higher payoff no matter what the other copy of me does.
Thus “physical” counterfactuals do not make for an acceptable decision theory, and we’re forced to look for some notion not of, “what would happen if the state of the world were different,” but of something like, “what would happen if this Turing machine would return a different result?” That would be a “logical” counterfactual. Of course, it’s not obvious what the hell this is supposed to mean, and if it can be made to mean anything coherent—your post is an answer to that question.
As for decision theory, I think that the “logical” counterfactuals should supplement, not supplant, the physical counterfactuals.
I understand that many people share this opinion, but I don’t see much in the way of justification. What arguments do you have, besides intuition? I see no reason to expect the “correct” decision theory to be intuitive. Game theory, for example, isn’t.
“Logical” counterfactuals either mean those whose antecedents are logically impossible, or those whose consequents follow from the antecedents by logical necessity, or both. In your Newcomb’s-problem-solving algorithm example, both features apply.
But there are many decisions (or sub-problems within decisions) where neither feature applies. Where the agent is a human being without access to its “source code”, it commits no contradiction in supposing “if I did A… but if I did B …” even though at most one of these lies in the future that causality is heading toward. Additionally, one may be unable to prove that the expected utility of A would be U1 and the expected utility of B would be U2, even though one may reasonably believe both of these. Even when we know most of the relevant physical laws, we lack knowledge of relevant initial or boundary conditions, and if we had those we’d still lack the time to do the calculations.
Our knowledge of physical counterfactuals isn’t deductive, usually, but it’s still knowledge. And we still need to know the physical consequences of various actions—so we’ll have to go on using these counterfactuals. Of course, I just used another intuition there! I’m not concerned, though—decision theory and game theory will both be evaluated on a balance of intuitions. That, of course, does not mean that every intuitively appealing norm will be accepted. But those that are rejected will lose out to a more intuitively appealing (combination of) norm(s).
In a previous comment, I said that physical counterfactuals follow trivially from physical laws, giving an ideal-gas example. But now I just said that our knowledge of physical counterfactuals is usually non-deductive. Problem? No: in the previous comment I remarked on the relation between truths in attempting to show that counterfactuals needn’t be metaphysical monsters. In this comment I have been focusing on the agent’s epistemic situation. We can know to a moral certainty that a causal relationship holds without being able to state the laws involved and without being able to make any inference that strictly deserves the label “deduction”.
I don’t understand your comment at all. You seem to assume as “trivial” many things that I set out to demystify. Maybe you could formalize your argument?
I’m not sure if I understand torekp’s comment right, but it seems to me that the issue is the difference between “physical” and “logical” counterfactuals in the following sense:
For concreteness and simplicity, let’s suppose physics can be modeled by a function step(state) that takes a state of the world, and returns the state of the world one instant later. Then you can ask what would happen if the current state were slightly different (“suppose there were an apple here in the air at this point in time; then it would fall down”). Call this a “physical” counterfactual.
Before reading Eliezer’s arguments, I used to think this could be used to model decision-making: Let stateA be what the state of the world will really be at the point I’ve made my decision. For every choice X I could make, let stateA_X be the same as stateA except that the future copy of me is replaced by a version that has decided to do X. Now it’s well-defined what the future of every stateA_X looks like, and if I have a well-defined utility function, then it’s well-defined which choice(s) this utility is maximized. Of course I cannot calculate this, but, I thought, the actual decision making process could be seen as an approximation of this.
Except that this would lead to classical decision/game theory, with its unattractive consequences—the case where Omega has just made two identical physical copies of me, and has us play a true Prisoner’s Dilemma, etc. Since I can prove that the other copy of me will do the same thing in every stateA_X (because I’m only replacing myself, not the other copy), it follows that I should defect, because it gives me the higher payoff no matter what the other copy of me does.
Thus “physical” counterfactuals do not make for an acceptable decision theory, and we’re forced to look for some notion not of, “what would happen if the state of the world were different,” but of something like, “what would happen if this Turing machine would return a different result?” That would be a “logical” counterfactual. Of course, it’s not obvious what the hell this is supposed to mean, and if it can be made to mean anything coherent—your post is an answer to that question.
Benja’s got it: I’m interested in physical counterfactuals. They are the type that is involved in the everyday notion of what a person “could” do.
As for decision theory, I think that the “logical” counterfactuals should supplement, not supplant, the physical counterfactuals.
I understand that many people share this opinion, but I don’t see much in the way of justification. What arguments do you have, besides intuition? I see no reason to expect the “correct” decision theory to be intuitive. Game theory, for example, isn’t.
“Logical” counterfactuals either mean those whose antecedents are logically impossible, or those whose consequents follow from the antecedents by logical necessity, or both. In your Newcomb’s-problem-solving algorithm example, both features apply.
But there are many decisions (or sub-problems within decisions) where neither feature applies. Where the agent is a human being without access to its “source code”, it commits no contradiction in supposing “if I did A… but if I did B …” even though at most one of these lies in the future that causality is heading toward. Additionally, one may be unable to prove that the expected utility of A would be U1 and the expected utility of B would be U2, even though one may reasonably believe both of these. Even when we know most of the relevant physical laws, we lack knowledge of relevant initial or boundary conditions, and if we had those we’d still lack the time to do the calculations.
Our knowledge of physical counterfactuals isn’t deductive, usually, but it’s still knowledge. And we still need to know the physical consequences of various actions—so we’ll have to go on using these counterfactuals. Of course, I just used another intuition there! I’m not concerned, though—decision theory and game theory will both be evaluated on a balance of intuitions. That, of course, does not mean that every intuitively appealing norm will be accepted. But those that are rejected will lose out to a more intuitively appealing (combination of) norm(s).
In a previous comment, I said that physical counterfactuals follow trivially from physical laws, giving an ideal-gas example. But now I just said that our knowledge of physical counterfactuals is usually non-deductive. Problem? No: in the previous comment I remarked on the relation between truths in attempting to show that counterfactuals needn’t be metaphysical monsters. In this comment I have been focusing on the agent’s epistemic situation. We can know to a moral certainty that a causal relationship holds without being able to state the laws involved and without being able to make any inference that strictly deserves the label “deduction”.