This, and Benelliott’s similar reply, answers kilobug’s objection. But it raises a new question. As you say, outcomes are world-histories. And a history is more than just a set of events. At the very least, it’s an ordered set of events. And the order matters—the history by which one arrived at a given state matters—at least in most people’s eyes. For example, given that in outcomes X, Y, and Z, I end up with $100, I still prefer outcome X, where I earned it, to Y, where it was a gift, to Z, where I stole it. We can complicate the examples to cover any other differences that you might (wrongly) suppose explain my preference without regard to history. For example, in scenario Z let us suppose that the theft victim looks for the $100 in an old coat pocket, thinking she misplaced it, and voila! finds $100 that would otherwise never have been found. I still prefer X to Y to Z.
Given that people can rationally have preferences that make essential reference to history and to the way events came about, why can’t risk be one of those historical factors that matter? What’s so “irrational” about that?
Given that people can rationally have preferences that make essential reference to history and to the way events came about, why can’t risk be one of those historical factors that matter? What’s so “irrational” about that?
Nothing. Whoever said there was?
If your goal is to not be a thief, then expected utility theory recommends that you do not steal.
I suspect most of us do have ‘do not steal’ preferences on the scale of a few hundred pounds or more.
On the other hand, once you get to, say, a few hundred human lives, or the fate of the entire species, then I stop caring about the journey as much. It still matters, but the amount that it matters is too small to ever have an appreciable effect on the decision. This preference may be unique to me, but if so then I weep for humanity.
A desire to avoid arriving at an outcome via thievery does not violate the Axiom of Independence. A desire to avoid arriving via a risky procedure does. However, I’m not convinced that the latter is any more irrational than the former. And I take the point of this thread to be whether obeying the Axiom really is a requirement of rationality.
So, when people say ‘risk aversion’, they can mean one of three different things:
I) I have a utility function that penalises world-histories in which I take risks.
II) I have a utility function which offers diminishing returns in some resource, so I am risk averse in that resource
III) I am risk averse in utility
Out of the three (III) is irrational and violates VNM. (II) is not irrational, and is an extremely common preference among humans wrt some things, but not others (money vs lives being the classic one). (I) is not irrational, but is pretty weird, I’m really not sure I have preferences like this, and when other people claim they do I become a bit suspicious that it is actually a case of (II) or (III).
Since we agree that (I) is not irrational, it remains to show that someone with that preference pattern (and not pattern III) still must have a VNM utility function—then my objection will be answered. Indeed, before we can even attribute “utility” to this person and thus go to case III, we must show that their preferences obey certain rules (or maybe just that their rational ones would).
I don’t think preference (I) is weird at all, though I don’t share it. Also not rare: a utility function that rewards world histories in which one takes risks. Consider that risk is either subjective or epistemic, not ontological, as used in VNM’s framework. Now consider the games involving chance that people enjoy. These either show (subjective probability interpretation of “risk”) or provide suggestive evidence toward the possibility (epistemic probability interpretation) that some people just plain like risk.
it remains to show that someone with that preference pattern (and not pattern III) still must have a VNM utility function
Why does it remain to be shown? How does this differ from the claim that any other preference pattern that does not violate a VNM axiom is modelled by expected utility?
Now consider the games involving chance that people enjoy. These either show (subjective probability interpretation of “risk”) or provide suggestive evidence toward the possibility (epistemic probability interpretation) that some people just plain like risk.
Interesting. If I had to guess though, the way in which these people like risk depends on the way it is dispensed, and is probably not linear in the amount of risk.
How does this differ from the claim that any other preference pattern that does not violate a VNM axiom is modelled by expected utility?
Well if it doesn’t violate an axiom—and specifically I’m worried about Independence—then the case is proven. So let me try to explain why I think it does violate independence. The Allais Paradox provides a case where the risk undertaken depends on so-called irrelevant alternatives. Take the version from Luke’s Decision Theory FAQ. If I bet on the option (say “red or yellow”) having 34 $24000-payoff balls, whether I take a big risk depends on how many other balls are in the lottery. If there are 66 zero-payoff green balls in the urn at the same time, then I do take a risk. If there are no other balls in the urn, then I don’t. If I penalize the preferability of outcomes depending on the risk undertaken, then I will penalize the “red or yellow” bet if and only if the green balls are also involved. Say there are 33 yellow balls and 1 red one, and I get $27000 if I bet on “yellow” instead. I will penalize the outcome, bet on yellow and get $27000, in either scenario. If the penalty is not linear in the amount of risk, I could conceivably prefer to bet on yellow when the green balls are in the urn, and bet on [red or yellow] when there aren’t.
I’m not sure quite what the best response to this is, but I think I wasn’t understanding you up to this point. We seem to have a bit of a level mixing problem.
In VNM utility theory, we assign utility to outcomes, defined as a complete description of what happens, and expected utility to lotteries, defined as a probability distribution over outcomes. They are measured in the same units, but they are not the same thing and should not be compared directly.
VNM utility tells you nothing about how to calculate utility and everything about how to calculate expected utility given utility.
By my definitions of risk aversion, type (II) risk aversion is simply a statement about how you assign utility, while type (III) is an error in calculating expected utility.
Type (I), as best I understand it, seems to consist of assigning utility to a lottery. Its not so much an axiom violation as a category error, a bit like (to go back my geometry analogy) asking if two points are parallel to each other. It doesn’t violate independence, because its wrong on far too basic a level to even assess whether it violates independence.
Of course, this is made more complicated by f*ing human brains, as usual. The knowledge of having taken a risk affects our brains and may change our satisfaction with the outcome. My response to this is that it can be factored back into the utility calculation, at which point you find that getting one outcome in one lottery is not the same as getting it in another.
I may ask that you go read my conversation with kilobug elsewhere in this thread, as I think it comes down to the exact same response and I don’t feel like typing it all again.
In VNM utility theory, we assign utility to outcomes, defined as a complete description of what happens, and expected utility to lotteries, defined as a probability distribution over outcomes. They are measured in the same units, but they are not the same thing [...]
Type (I), as best I understand it, seems to consist of assigning utility to a lottery. Its not so much an axiom violation as a category error
I am indeed suggesting that an agent can assign utility, not merely expected utility, to a lottery. Note that in this sentence “utility” does not have its technical meaning(s) but simply means raw preference. With that caveat, that may be a better way of putting it than anything I’ve said so far.
You can call that a category error, but I just don’t see the mistake. Other than that it doesn’t fit the VNM theory, which would be a circular argument for its irrationality in this context.
Your point about f*ing human brains gets at my True Rejection, so thanks. And I read the conversation with kilobug. As a result I have a new idea where you may be coming from—about which I will quote Luke’s decision theory FAQ:
Peterson (2009, ch. 4) explains:
In the indirect approach, which is the dominant approach, the decision maker does not prefer a risky act to another because the expected utility of the former exceeds that of the latter. Instead, the decision maker is asked to state a set of preferences over a set of risky acts… Then, if the set of preferences stated by the decision maker is consistent with a small number of structural constraints (axioms), it can be shown that her decisions can be described as if she were choosing what to do by assigning numerical probabilities and utilities to outcomes and then maximising expected utility...
[In contrast] the direct approach seeks to generate preferences over acts from probabilities and utilities directly assigned to outcomes. In contrast to the indirect approach, it is not assumed that the decision maker has access to a set of preferences over acts before he starts to deliberate.
The axiomatic decision theories listed in section 8.2 all follow the indirect approach. These theories, it might be said, cannot offer any action guidance because they require an agent to state its preferences over acts “up front.” But an agent that states its preferences over acts already knows which act it prefers, so the decision theory can’t offer any action guidance not already present in the agent’s own stated preferences over acts.
Emphasis added. It sounds to me like you favor a direct approach. For you, utility is not an as-if: it is a fundamentally real, interval-scale-able quality of our lives. In this scheme, the angst I feel while taking a risk is something I can assign a utility to, then shut up and (re-)calculate the expected utilities. Yes?
If you favor a direct approach, I wonder why you even care to defend the VNM axioms, or what role they play for you.
I am indeed suggesting that an agent can assign utility, not merely expected utility, to a lottery.
I am suggesting that this is equivalent to suggesting that two points can be parallel. It may be true for your special definition of point, but its not true for mine, and its not true for the definition the theorems refer to.
Yes, in the real world the lottery is part of the outcome, but that can be factored in with assigning utility to the outcomes, we don’t need to change our definition of utility when the existing one works (reading the rest of your post, I now see you already understand this).
It sounds to me like you favour a direct approach. For you, utility is not an as-if: it is a fundamentally real, interval-scale-able quality of our lives.
I cannot see anything I have said to suggest I believe this. Interpreted descriptively, (as a statement about how people actually make decisions) I think it is utter garbage.
Interpreted prescriptively, I think I might believe it. I would at least probably say what while I like the fact that VNM axioms imply EU theory, I think I would consider EU the obviously correct way to do things even if they did not.
In this scheme, the angst I feel while taking a risk is something I can assign a utility to, then shut up and (re-)calculate the expected utilities.
Yes.
Granted, if decision angst is often playing a large part in your decisions, and in particular costing you other benefits, I would strongly suggest you work on finding ways to get around this. Rightly or wrongly, yelling “stop being so irrational!” at my brain has sometimes worked here for me. I am almost certain there are better techniques.
I wonder why you even care to defend the VNM axioms, or what role they play for you.
I defend them because I think they are correct. What more reason should be required?
Interpreted prescriptively, I think I might believe it. I would at least probably say what while I like the fact that VNM axioms imply EU theory, I think I would consider EU the obviously correct way to do things even if they did not.
So let me rephrase my earlier question (poorly phrased before) about what role the VNM axioms play for you. Sometimes (especially when it comes to “rationality”) an “axiom” is held to be obvious, even indubitable: the principle of non-contradiction is often viewed in this light. At other times, say when formulating a mathematical model of an advanced physics theory, the axioms are anything but obvious, but they are endorsed because they seem to work. The axioms are the result of an inference to the best explanation.
So I’m wondering if your view is more like (A) than like (B) below.
(A) Rationality is a sort of attractor in mind-space, and people approach closer and closer to being describable by EU theory the more rational they are. Since the VNM axioms are obeyed in these cases, that tends to show that rationality includes following those axioms.
(B) Obviously only a mad person would violate the Axiom of Independence knowing full well they were doing so.
Granted, if decision angst is often playing a large part in your decisions, and in particular costing you other benefits, I would strongly suggest you work on finding ways to get around this. Rightly or wrongly, yelling “stop being so irrational!” at my brain has sometimes worked here for me. I am almost certain there are better techniques.
And now we are back to my True Rejection, namely: I don’t think it’s irrational to take decision-angst into account, or to seek to avoid it by avoiding risk rather than just seeking psychotherapy so that one can buck up and keep a stiff upper lip. It’s not Spock-like, but it’s not irrational.
At this point it’s important to remember that in the VNM framework, the agent’s epistemic state and decision-making procedure cannot be part of the outcome. In this sense VNM-rational agents are Cartesian dualists. Counterfactual world-histories are also not part of the outcome.
So I think whether or not a decision was risky depends on the agent’s epistemic state, as well as on the decision and the agent’s preferences. This is why preferring to come by your money honestly is different from preferring to come by your money in a non-risky way.
That’s helpful. But it also seems unduly restrictive. I realize that you’re not saying that we literally have to treat our own minds as immaterial entities (are you?), but it still seems a pretty high price to pay. Can I treat the epistemic states of my loved ones as part of the outcome? Presumably so, so why can’t I give myself the same consideration? I’m trying to make you feel the cost, here, as I see it.
Hm. I haven’t thought much about that. Maybe there is something interesting to be said about what aspects of an agent’s internal state can they have preferences over for there still to be an interesting rationality theorem? If you let agents have preferences over all decisions, then there is no rationality theorem.
I don’t believe the VNM theorem describes humans, but on the other hand I don’t think humans should endorse violations of the Independence Axiom.
For example, given that in outcomes X, Y, and Z, I end up with $100, I still prefer outcome X, where I earned it, to Y, where it was a gift, to Z, where I stole it. We can complicate the examples to cover any other differences that you might (wrongly) suppose explain my preference without regard to history.
I really really doubt you have preferences for history. Your preferences are fully summarised by the current world state with no reference to histories—you prefer not remembering having stolen something and prefer remembering having earned it and having others remember the same. Note that this is a description of the present, not of the past.
To really care about a history, you’d have to construct a scenario like “I start out in a simulation along the lines of Z, but then the simulation is rearranged to a worldstate with X instead. Alternatively, I can be in scenario X all along. I like being in state X (at a point in time after the rearrangement/lack thereof) less in the former case than in the latter, even if no one can tell the difference between them.” And I’m not sure that scenario would even work (it’s not clear that there is meaningful continuity between Z!you and X!you), but I can’t think of a better one off-hand.
Those with simpler theories of the good life often doubt the self-knowledge of those with more complex ones. There isn’t much I can do to try to convince you, other than throw thought experiments back and forth, and I don’t feel up to that. If you’ve already read EY on the complexity of value, my only thought here is that maybe some other LWers will chime in and reduce (or increase!) your posterior probability that I’m just a sloppy thinker.
In hindsight, I phrased that poorly, and you’re right, discussing it that way would probably be unproductive.
First, let me specify that when I say “histories” here I mean past histories from the point of view of the agent (which sounds weird, but a lot of the other comments use it to refer to future histories as well). With that in mind, how about this: the actions of the set of agents who care about histories are indistinguishable from the actions of some subset of the agents who do not care about histories. In (something closer to) English, there’s a way to describe your caring about histories in terms of only caring about the present and future without changing any decisions you might make.
I find the above “obvious” (which I usually take as a sign that I should be careful). The reason I believe it is that all information you have about histories is contained within your present self. There is no access to the past—everything you know about it is contained either in the present or future, so your decisions must necessarily be conditional only on the present and future.
Would you agree with that? And if so, would you agree that discussing an agent who cares about the histories leading up the present state is not worth doing, since there is no case in which her decisions would differ from some agent who does not? (I suppose one fairly reasonable objection is time travel, but I’m more interested in the case where it’s impossible, and I’m not entirely sure whether it would change the core of the argument anyway.)
There is no access to the past—everything you know about it is contained either in the present or future
That’s fair, but it just seems to show that I can be fooled. If I’m fooled and the trick is forever beyond my capacity to detect, my actions will be the same as if I had actually accomplished whatever I was trying for. But that doesn’t mean I got what I really wanted.
This, and Benelliott’s similar reply, answers kilobug’s objection. But it raises a new question. As you say, outcomes are world-histories. And a history is more than just a set of events. At the very least, it’s an ordered set of events. And the order matters—the history by which one arrived at a given state matters—at least in most people’s eyes. For example, given that in outcomes X, Y, and Z, I end up with $100, I still prefer outcome X, where I earned it, to Y, where it was a gift, to Z, where I stole it. We can complicate the examples to cover any other differences that you might (wrongly) suppose explain my preference without regard to history. For example, in scenario Z let us suppose that the theft victim looks for the $100 in an old coat pocket, thinking she misplaced it, and voila! finds $100 that would otherwise never have been found. I still prefer X to Y to Z.
Given that people can rationally have preferences that make essential reference to history and to the way events came about, why can’t risk be one of those historical factors that matter? What’s so “irrational” about that?
Nothing. Whoever said there was?
If your goal is to not be a thief, then expected utility theory recommends that you do not steal.
I suspect most of us do have ‘do not steal’ preferences on the scale of a few hundred pounds or more.
On the other hand, once you get to, say, a few hundred human lives, or the fate of the entire species, then I stop caring about the journey as much. It still matters, but the amount that it matters is too small to ever have an appreciable effect on the decision. This preference may be unique to me, but if so then I weep for humanity.
A desire to avoid arriving at an outcome via thievery does not violate the Axiom of Independence. A desire to avoid arriving via a risky procedure does. However, I’m not convinced that the latter is any more irrational than the former. And I take the point of this thread to be whether obeying the Axiom really is a requirement of rationality.
So, when people say ‘risk aversion’, they can mean one of three different things:
I) I have a utility function that penalises world-histories in which I take risks.
II) I have a utility function which offers diminishing returns in some resource, so I am risk averse in that resource
III) I am risk averse in utility
Out of the three (III) is irrational and violates VNM. (II) is not irrational, and is an extremely common preference among humans wrt some things, but not others (money vs lives being the classic one). (I) is not irrational, but is pretty weird, I’m really not sure I have preferences like this, and when other people claim they do I become a bit suspicious that it is actually a case of (II) or (III).
Since we agree that (I) is not irrational, it remains to show that someone with that preference pattern (and not pattern III) still must have a VNM utility function—then my objection will be answered. Indeed, before we can even attribute “utility” to this person and thus go to case III, we must show that their preferences obey certain rules (or maybe just that their rational ones would).
I don’t think preference (I) is weird at all, though I don’t share it. Also not rare: a utility function that rewards world histories in which one takes risks. Consider that risk is either subjective or epistemic, not ontological, as used in VNM’s framework. Now consider the games involving chance that people enjoy. These either show (subjective probability interpretation of “risk”) or provide suggestive evidence toward the possibility (epistemic probability interpretation) that some people just plain like risk.
Why does it remain to be shown? How does this differ from the claim that any other preference pattern that does not violate a VNM axiom is modelled by expected utility?
Interesting. If I had to guess though, the way in which these people like risk depends on the way it is dispensed, and is probably not linear in the amount of risk.
Well if it doesn’t violate an axiom—and specifically I’m worried about Independence—then the case is proven. So let me try to explain why I think it does violate independence. The Allais Paradox provides a case where the risk undertaken depends on so-called irrelevant alternatives. Take the version from Luke’s Decision Theory FAQ. If I bet on the option (say “red or yellow”) having 34 $24000-payoff balls, whether I take a big risk depends on how many other balls are in the lottery. If there are 66 zero-payoff green balls in the urn at the same time, then I do take a risk. If there are no other balls in the urn, then I don’t. If I penalize the preferability of outcomes depending on the risk undertaken, then I will penalize the “red or yellow” bet if and only if the green balls are also involved. Say there are 33 yellow balls and 1 red one, and I get $27000 if I bet on “yellow” instead. I will penalize the outcome, bet on yellow and get $27000, in either scenario. If the penalty is not linear in the amount of risk, I could conceivably prefer to bet on yellow when the green balls are in the urn, and bet on [red or yellow] when there aren’t.
I’m not sure quite what the best response to this is, but I think I wasn’t understanding you up to this point. We seem to have a bit of a level mixing problem.
In VNM utility theory, we assign utility to outcomes, defined as a complete description of what happens, and expected utility to lotteries, defined as a probability distribution over outcomes. They are measured in the same units, but they are not the same thing and should not be compared directly.
VNM utility tells you nothing about how to calculate utility and everything about how to calculate expected utility given utility.
By my definitions of risk aversion, type (II) risk aversion is simply a statement about how you assign utility, while type (III) is an error in calculating expected utility.
Type (I), as best I understand it, seems to consist of assigning utility to a lottery. Its not so much an axiom violation as a category error, a bit like (to go back my geometry analogy) asking if two points are parallel to each other. It doesn’t violate independence, because its wrong on far too basic a level to even assess whether it violates independence.
Of course, this is made more complicated by f*ing human brains, as usual. The knowledge of having taken a risk affects our brains and may change our satisfaction with the outcome. My response to this is that it can be factored back into the utility calculation, at which point you find that getting one outcome in one lottery is not the same as getting it in another.
I may ask that you go read my conversation with kilobug elsewhere in this thread, as I think it comes down to the exact same response and I don’t feel like typing it all again.
I am indeed suggesting that an agent can assign utility, not merely expected utility, to a lottery. Note that in this sentence “utility” does not have its technical meaning(s) but simply means raw preference. With that caveat, that may be a better way of putting it than anything I’ve said so far.
You can call that a category error, but I just don’t see the mistake. Other than that it doesn’t fit the VNM theory, which would be a circular argument for its irrationality in this context.
Your point about f*ing human brains gets at my True Rejection, so thanks. And I read the conversation with kilobug. As a result I have a new idea where you may be coming from—about which I will quote Luke’s decision theory FAQ:
Emphasis added. It sounds to me like you favor a direct approach. For you, utility is not an as-if: it is a fundamentally real, interval-scale-able quality of our lives. In this scheme, the angst I feel while taking a risk is something I can assign a utility to, then shut up and (re-)calculate the expected utilities. Yes?
If you favor a direct approach, I wonder why you even care to defend the VNM axioms, or what role they play for you.
I am suggesting that this is equivalent to suggesting that two points can be parallel. It may be true for your special definition of point, but its not true for mine, and its not true for the definition the theorems refer to.
Yes, in the real world the lottery is part of the outcome, but that can be factored in with assigning utility to the outcomes, we don’t need to change our definition of utility when the existing one works (reading the rest of your post, I now see you already understand this).
I cannot see anything I have said to suggest I believe this. Interpreted descriptively, (as a statement about how people actually make decisions) I think it is utter garbage.
Interpreted prescriptively, I think I might believe it. I would at least probably say what while I like the fact that VNM axioms imply EU theory, I think I would consider EU the obviously correct way to do things even if they did not.
Yes.
Granted, if decision angst is often playing a large part in your decisions, and in particular costing you other benefits, I would strongly suggest you work on finding ways to get around this. Rightly or wrongly, yelling “stop being so irrational!” at my brain has sometimes worked here for me. I am almost certain there are better techniques.
I defend them because I think they are correct. What more reason should be required?
So let me rephrase my earlier question (poorly phrased before) about what role the VNM axioms play for you. Sometimes (especially when it comes to “rationality”) an “axiom” is held to be obvious, even indubitable: the principle of non-contradiction is often viewed in this light. At other times, say when formulating a mathematical model of an advanced physics theory, the axioms are anything but obvious, but they are endorsed because they seem to work. The axioms are the result of an inference to the best explanation.
So I’m wondering if your view is more like (A) than like (B) below.
(A) Rationality is a sort of attractor in mind-space, and people approach closer and closer to being describable by EU theory the more rational they are. Since the VNM axioms are obeyed in these cases, that tends to show that rationality includes following those axioms.
(B) Obviously only a mad person would violate the Axiom of Independence knowing full well they were doing so.
And now we are back to my True Rejection, namely: I don’t think it’s irrational to take decision-angst into account, or to seek to avoid it by avoiding risk rather than just seeking psychotherapy so that one can buck up and keep a stiff upper lip. It’s not Spock-like, but it’s not irrational.
At this point it’s important to remember that in the VNM framework, the agent’s epistemic state and decision-making procedure cannot be part of the outcome. In this sense VNM-rational agents are Cartesian dualists. Counterfactual world-histories are also not part of the outcome.
So I think whether or not a decision was risky depends on the agent’s epistemic state, as well as on the decision and the agent’s preferences. This is why preferring to come by your money honestly is different from preferring to come by your money in a non-risky way.
That’s helpful. But it also seems unduly restrictive. I realize that you’re not saying that we literally have to treat our own minds as immaterial entities (are you?), but it still seems a pretty high price to pay. Can I treat the epistemic states of my loved ones as part of the outcome? Presumably so, so why can’t I give myself the same consideration? I’m trying to make you feel the cost, here, as I see it.
Hm. I haven’t thought much about that. Maybe there is something interesting to be said about what aspects of an agent’s internal state can they have preferences over for there still to be an interesting rationality theorem? If you let agents have preferences over all decisions, then there is no rationality theorem.
I don’t believe the VNM theorem describes humans, but on the other hand I don’t think humans should endorse violations of the Independence Axiom.
Seems like a good topic to address as directly as possible, I agree.
I really really doubt you have preferences for history. Your preferences are fully summarised by the current world state with no reference to histories—you prefer not remembering having stolen something and prefer remembering having earned it and having others remember the same. Note that this is a description of the present, not of the past.
To really care about a history, you’d have to construct a scenario like “I start out in a simulation along the lines of Z, but then the simulation is rearranged to a worldstate with X instead. Alternatively, I can be in scenario X all along. I like being in state X (at a point in time after the rearrangement/lack thereof) less in the former case than in the latter, even if no one can tell the difference between them.” And I’m not sure that scenario would even work (it’s not clear that there is meaningful continuity between Z!you and X!you), but I can’t think of a better one off-hand.
Those with simpler theories of the good life often doubt the self-knowledge of those with more complex ones. There isn’t much I can do to try to convince you, other than throw thought experiments back and forth, and I don’t feel up to that. If you’ve already read EY on the complexity of value, my only thought here is that maybe some other LWers will chime in and reduce (or increase!) your posterior probability that I’m just a sloppy thinker.
In hindsight, I phrased that poorly, and you’re right, discussing it that way would probably be unproductive.
First, let me specify that when I say “histories” here I mean past histories from the point of view of the agent (which sounds weird, but a lot of the other comments use it to refer to future histories as well). With that in mind, how about this: the actions of the set of agents who care about histories are indistinguishable from the actions of some subset of the agents who do not care about histories. In (something closer to) English, there’s a way to describe your caring about histories in terms of only caring about the present and future without changing any decisions you might make.
I find the above “obvious” (which I usually take as a sign that I should be careful). The reason I believe it is that all information you have about histories is contained within your present self. There is no access to the past—everything you know about it is contained either in the present or future, so your decisions must necessarily be conditional only on the present and future.
Would you agree with that? And if so, would you agree that discussing an agent who cares about the histories leading up the present state is not worth doing, since there is no case in which her decisions would differ from some agent who does not? (I suppose one fairly reasonable objection is time travel, but I’m more interested in the case where it’s impossible, and I’m not entirely sure whether it would change the core of the argument anyway.)
That’s fair, but it just seems to show that I can be fooled. If I’m fooled and the trick is forever beyond my capacity to detect, my actions will be the same as if I had actually accomplished whatever I was trying for. But that doesn’t mean I got what I really wanted.