Well, sure, by mangling enough the events you can re-establish the axioms. But if you do that, in fact, you just don’t need the axioms. The independence axiom states that if you have B > C, then you have (probability p of A, probability 1-p of B) > (probability p of A, probability 1-p of C). What you’re doing is saying you can’t use A, B, and C when there is dependency, but have to create subevents like C1=”C when you are you sure you’ll have either A or C”. Of course, by splitting the events like that, you’ll reestablish independence—but by showing the need to mangle choices to make fit the axioms, you in fact have shown the axioms don’t work in the general case, when the choices you’re given are not independent, as it often is in real life.
Gwern said pretty much everything I wanted to say to this, but there’s an extra distinction I want to make
What you’re doing is saying you can’t use A, B, and C when there is dependency, but have to create subevents like C1=”C when you are you sure you’ll have either A or C”.
The distinction I made was things like A2=”A when you prepare” not A2=”A when you are sure of getting A or C”. This looks like a nitpick, but is in fact incredibly important. The difference between my A1 and A2, is important, they are fundamentally different outcomes which may have completely different utilities, they have no more in common than B1 and C2. They are events in their own right, there is no ‘sub-’ to it. Distinguishing between them is not ‘mangling’, putting them together in the first place was always an error.
you in fact have shown the axioms don’t work in the general case
It is easily possible to imagine three tennis players A, B and C, such that A beats B, B beats C and C beats A (perhaps A has a rather odd technique, which C has worked hard at learning to deal with despite being otherwise mediocre). Then we have A > B and B > C but not A > C, I have just shown that the axiom of transitivity is not true in the general case!
Well, no, I haven’t.
I’ve shown that the axiom of transitivity does not hold for tennis players. This may be an interesting fact about tennis, but it has not ‘disproven’ anything, nobody ever claimed that transitivity applied to tennis players.
What the VNM axioms are meant to refer to, are outcomes, meaning a complete description of what will happen to you. “Trip to Ecuador” is not an outcome, because it does not describe exactly what will happen to you, and in particular leaves open whether or not you will prepare for the trip.
This sort of thing is why I think everyone with the intellectual capacity to do so should study mathematical logic. It really helps you learn to keep things cleanly separated in your mind and avoid mistakes like this.
First, I did study mathematical logic, and please avoid such kind of ad hominem.
That said, if what you’re referring to is the whole world state, the outcomes are, in fact, awlays different. Even if only because there is somewhere in your brain the knowledge that the choice is different.
To take the formulation in the FAQ : « The independence axiom states that, for example, if an agent prefers an apple to an orange, then she must also prefer the lottery [55% chance she gets an apple, otherwise she gets cholera] over the lottery [55% chance she gets an orange, otherwise she gets cholera]. More generally, this axiom holds that a preference must hold independently of the possibility of another outcome (e.g. cholera). »
That has no meaning if you consider whole world states, not just specific outcomes. Because in the lottery it’s not “apple or orange” then but “apple with the knowledge I almost got cholera” vs “orange with the knowledge I almost got cholera”. And if there is an interaction between the two, then you have different ranking between them. Maybe you had a friend who died of cholera and loved apple, and that’ll change how much you appreciate apples knowing you almost had cholera. Maybe not. But anyway, if what you consider are whole world states, then by definition the whole world state is always different when you’re offered even a slightly different choice. How can you define an independence principle in that case ?
First, I did study mathematical logic, and please avoid such kind of ad hominem.
Fair enough
That said, if what you’re referring to is the whole world state, the outcomes are, in fact, always different. Even if only because there is somewhere in your brain the knowledge that the choice is different.
I thought this would be your reply, but didn’t want to address it because the comment was too long already.
Firstly, this is completely correct. (Well, technically we could imagine situations where the outcomes removed your memory of there ever having been a choice, but this isn’t usually the case). Its pretty much never possible to make actually useful deductions just from pure logic and the axiom of independence.
This is much the same as any other time you apply a mathematical model to the real world. We assume away some factors, not because we don’t think they exist, but because we think they do not have a large effect on the outcome or that the effect they do have does not actually affect our decision in any way.
E.g. Geometry is completely useless, because perfectly straight lines do not exist in the real world. However, in many situations they are incredibly good approximations which let us draw interesting non-trivial conclusions. This doesn’t mean Euclidean Geometry is an approximation, the approximation is when I claim the edge of my desk is a straight line.
So, I would say that usually, my memory of the other choice I was offered has quite small effects on my satisfaction with the outcome compared to what I actually get, so in most circumstances I can safely assume that the outcomes are equal (even though they aren’t). With that assumption, independence generates some interesting conclusions.
Other times, this assumption breaks down. Your cholera example strikes me as a little silly, but the example in your original post is an excellent illustration of how assuming two outcomes are equal because they look the same as English sentences can be a mistake.
At a guess, a good heuristic seems to be that after you’ve made your decision, and found out which outcome from the lottery you got, then usually the approximation that the existence of other outcomes changes nothing is correct. If there’s a long time gap between the decision and the lottery then decisions made in that time gap should usually be taken into account.
Of course, independence isn’t really that useful for its own sake, but more for the fact that combined with other axioms it gives you expected utility theory.
The cholera example was definitely a bit silly—after all, “cholera” and “apple vs orange” are usually really independent in the real world, you’ve to make very far-fetched circumstances for them to be dependent. But an axiom is supposed to be valid everywhere—even in far-fetched circumstances ;)
But overall, I understand the thing much better now: in fact, the independence principle doesn’t strictly hold in the real world, like there are no strictly right angle in the real world. But yet, like we do use the Pythagoras theorem in the real world, assuming an angle to be right when it’s “close enough” to be right, we apply the VNM axioms and the related expected utility theory when we consider the independence principle to have enough validity?
But do we have any way to measure the degree of error introduced by this approximation? Do we have ways to recognize the cases where we shouldn’t apply the expected utility theory, because we are too far from the ideal model?
My point never was to fully reject VNM and expected utility theory—I know they are useful, they work in many cases, … My point was to draw attention on a potential problem (making it an approximation, making it not always valid) that I don’t usually see being addressed (actually, I don’t remember ever having seen it that explicitly).
I think we have almost reached agreement, just a few more nitpicks I seem to have with your current post.
the independence principle doesn’t strictly hold in the real world, like there are no strictly right angle in the real world
Its pedantic, but these two statements aren’t analogous. A better analogy would be
“the independence principle doesn’t strictly hold in the real world, like the axiom that all right angles are equal doesn’t hold in the real world”
“there are no strictly identical outcomes in the real world, like there are no strictly right angle in the real world”
Personally I prefer the second phrasing. The independence principle and the right angle principle do hold in the real world, or at least they would if the objects they talked about ever actually appeared, which they don’t.
I’m in general uncomfortable with talk of the empirical status of mathematical statements, maybe this makes me a Platonist or something. I’m much happier with talk of whether idealised mathematical objects exist in the real world, or whether things similar to them do.
What this means is we don’t apply VNM when we think independence is relatively true, we apply them when we think the outcomes we are facing are relatively similar to each other, enough that any difference can be assumed away.
But do we have any way to measure the degree of error introduced by this approximation?
This is an interesting problem. As far as I can tell, its a special case of the interesting problem of “how do we know/decide our utility function?”.
Do we have ways to recognize the cases where we shouldn’t apply the expected utility theory
I’ve suggested one heuristic that I think is quite good. Any ideas for others?
(Once again, I want to nitpick the language. “Do we have ways to recognize the cases where two outcomes look equal but aren’t” is the correct phrasing.
Well, sure, by mangling enough the events you can re-establish the axioms...Of course, by splitting the events like that, you’ll reestablish independence—but by showing the need to mangle choices to make fit the axioms, you in fact have shown the axioms don’t work in the general case, when the choices you’re given are not independent, as it often is in real life.
‘If I write out my arithmetic like “one plus two”, your calculator can’t handle it! Proving that arithmetic doesn’t work in the general case. Sure, you can mangle these words into these things you call numbers and symbols like “1+2″, but often in real life we don’t use them.’
Hrm, could you try to steelman instead of strawmaning my position ?
It’s not just a matter of formulation or translating words to symbols. Having to split the choices offered in the real world into an undefined number of virtual choices is not just a switch of notation. Real world choices have far-fetched consequences, and having to split apart all possible interactions between those choices can easily lead to combinatorial explosion of possible choices. benelliott split my choices into 2, but he could have split them into much more : different level of preparation of the trip, buying or not a laptop by myself, …
If it means that the a VNM-based theory can’t handle directly real-life choices without having to convert them into a different set of choices, which can be much bigger than the real-life choices, well, that’s something significant that you can’t just hand-wave with ill-placed irony, and the details of the conversion process have to be part of the decision theory.
Hrm, could you try to steelman instead of strawmaning my position ?
Steelman yourself. I took your quote and replaced it with an isomorphic version; it’s not my problem if it looks even more transparently irrelevant or wrong.
If it means that the a VNM-based theory can’t handle directly real-life choices without having to convert them into a different set of choices, which can be much bigger than the real-life choices, well, that’s something significant that you can’t just hand-wave with ill-placed irony, and the details of the conversion process have to be part of the decision theory.
Yes, it is significant, but it’s along the lines of “most (optimization) problems are in complexity classes higher than P” or “AIXI is uncomputable”. It doesn’t mean that the axioms or proofs are false; it just means that, yet again, as always, we need to make trade-offs and approximations.
Well, sure, by mangling enough the events you can re-establish the axioms. But if you do that, in fact, you just don’t need the axioms. The independence axiom states that if you have B > C, then you have (probability p of A, probability 1-p of B) > (probability p of A, probability 1-p of C). What you’re doing is saying you can’t use A, B, and C when there is dependency, but have to create subevents like C1=”C when you are you sure you’ll have either A or C”. Of course, by splitting the events like that, you’ll reestablish independence—but by showing the need to mangle choices to make fit the axioms, you in fact have shown the axioms don’t work in the general case, when the choices you’re given are not independent, as it often is in real life.
Gwern said pretty much everything I wanted to say to this, but there’s an extra distinction I want to make
The distinction I made was things like A2=”A when you prepare” not A2=”A when you are sure of getting A or C”. This looks like a nitpick, but is in fact incredibly important. The difference between my A1 and A2, is important, they are fundamentally different outcomes which may have completely different utilities, they have no more in common than B1 and C2. They are events in their own right, there is no ‘sub-’ to it. Distinguishing between them is not ‘mangling’, putting them together in the first place was always an error.
It is easily possible to imagine three tennis players A, B and C, such that A beats B, B beats C and C beats A (perhaps A has a rather odd technique, which C has worked hard at learning to deal with despite being otherwise mediocre). Then we have A > B and B > C but not A > C, I have just shown that the axiom of transitivity is not true in the general case!
Well, no, I haven’t.
I’ve shown that the axiom of transitivity does not hold for tennis players. This may be an interesting fact about tennis, but it has not ‘disproven’ anything, nobody ever claimed that transitivity applied to tennis players.
What the VNM axioms are meant to refer to, are outcomes, meaning a complete description of what will happen to you. “Trip to Ecuador” is not an outcome, because it does not describe exactly what will happen to you, and in particular leaves open whether or not you will prepare for the trip.
This sort of thing is why I think everyone with the intellectual capacity to do so should study mathematical logic. It really helps you learn to keep things cleanly separated in your mind and avoid mistakes like this.
First, I did study mathematical logic, and please avoid such kind of ad hominem.
That said, if what you’re referring to is the whole world state, the outcomes are, in fact, awlays different. Even if only because there is somewhere in your brain the knowledge that the choice is different.
To take the formulation in the FAQ : « The independence axiom states that, for example, if an agent prefers an apple to an orange, then she must also prefer the lottery [55% chance she gets an apple, otherwise she gets cholera] over the lottery [55% chance she gets an orange, otherwise she gets cholera]. More generally, this axiom holds that a preference must hold independently of the possibility of another outcome (e.g. cholera). »
That has no meaning if you consider whole world states, not just specific outcomes. Because in the lottery it’s not “apple or orange” then but “apple with the knowledge I almost got cholera” vs “orange with the knowledge I almost got cholera”. And if there is an interaction between the two, then you have different ranking between them. Maybe you had a friend who died of cholera and loved apple, and that’ll change how much you appreciate apples knowing you almost had cholera. Maybe not. But anyway, if what you consider are whole world states, then by definition the whole world state is always different when you’re offered even a slightly different choice. How can you define an independence principle in that case ?
Fair enough
I thought this would be your reply, but didn’t want to address it because the comment was too long already.
Firstly, this is completely correct. (Well, technically we could imagine situations where the outcomes removed your memory of there ever having been a choice, but this isn’t usually the case). Its pretty much never possible to make actually useful deductions just from pure logic and the axiom of independence.
This is much the same as any other time you apply a mathematical model to the real world. We assume away some factors, not because we don’t think they exist, but because we think they do not have a large effect on the outcome or that the effect they do have does not actually affect our decision in any way.
E.g. Geometry is completely useless, because perfectly straight lines do not exist in the real world. However, in many situations they are incredibly good approximations which let us draw interesting non-trivial conclusions. This doesn’t mean Euclidean Geometry is an approximation, the approximation is when I claim the edge of my desk is a straight line.
So, I would say that usually, my memory of the other choice I was offered has quite small effects on my satisfaction with the outcome compared to what I actually get, so in most circumstances I can safely assume that the outcomes are equal (even though they aren’t). With that assumption, independence generates some interesting conclusions.
Other times, this assumption breaks down. Your cholera example strikes me as a little silly, but the example in your original post is an excellent illustration of how assuming two outcomes are equal because they look the same as English sentences can be a mistake.
At a guess, a good heuristic seems to be that after you’ve made your decision, and found out which outcome from the lottery you got, then usually the approximation that the existence of other outcomes changes nothing is correct. If there’s a long time gap between the decision and the lottery then decisions made in that time gap should usually be taken into account.
Of course, independence isn’t really that useful for its own sake, but more for the fact that combined with other axioms it gives you expected utility theory.
The cholera example was definitely a bit silly—after all, “cholera” and “apple vs orange” are usually really independent in the real world, you’ve to make very far-fetched circumstances for them to be dependent. But an axiom is supposed to be valid everywhere—even in far-fetched circumstances ;)
But overall, I understand the thing much better now: in fact, the independence principle doesn’t strictly hold in the real world, like there are no strictly right angle in the real world. But yet, like we do use the Pythagoras theorem in the real world, assuming an angle to be right when it’s “close enough” to be right, we apply the VNM axioms and the related expected utility theory when we consider the independence principle to have enough validity?
But do we have any way to measure the degree of error introduced by this approximation? Do we have ways to recognize the cases where we shouldn’t apply the expected utility theory, because we are too far from the ideal model?
My point never was to fully reject VNM and expected utility theory—I know they are useful, they work in many cases, … My point was to draw attention on a potential problem (making it an approximation, making it not always valid) that I don’t usually see being addressed (actually, I don’t remember ever having seen it that explicitly).
I think we have almost reached agreement, just a few more nitpicks I seem to have with your current post.
Its pedantic, but these two statements aren’t analogous. A better analogy would be
“the independence principle doesn’t strictly hold in the real world, like the axiom that all right angles are equal doesn’t hold in the real world”
“there are no strictly identical outcomes in the real world, like there are no strictly right angle in the real world”
Personally I prefer the second phrasing. The independence principle and the right angle principle do hold in the real world, or at least they would if the objects they talked about ever actually appeared, which they don’t.
I’m in general uncomfortable with talk of the empirical status of mathematical statements, maybe this makes me a Platonist or something. I’m much happier with talk of whether idealised mathematical objects exist in the real world, or whether things similar to them do.
What this means is we don’t apply VNM when we think independence is relatively true, we apply them when we think the outcomes we are facing are relatively similar to each other, enough that any difference can be assumed away.
This is an interesting problem. As far as I can tell, its a special case of the interesting problem of “how do we know/decide our utility function?”.
I’ve suggested one heuristic that I think is quite good. Any ideas for others?
(Once again, I want to nitpick the language. “Do we have ways to recognize the cases where two outcomes look equal but aren’t” is the correct phrasing.
‘If I write out my arithmetic like “one plus two”, your calculator can’t handle it! Proving that arithmetic doesn’t work in the general case. Sure, you can mangle these words into these things you call numbers and symbols like “1+2″, but often in real life we don’t use them.’
Hrm, could you try to steelman instead of strawmaning my position ?
It’s not just a matter of formulation or translating words to symbols. Having to split the choices offered in the real world into an undefined number of virtual choices is not just a switch of notation. Real world choices have far-fetched consequences, and having to split apart all possible interactions between those choices can easily lead to combinatorial explosion of possible choices. benelliott split my choices into 2, but he could have split them into much more : different level of preparation of the trip, buying or not a laptop by myself, …
If it means that the a VNM-based theory can’t handle directly real-life choices without having to convert them into a different set of choices, which can be much bigger than the real-life choices, well, that’s something significant that you can’t just hand-wave with ill-placed irony, and the details of the conversion process have to be part of the decision theory.
Steelman yourself. I took your quote and replaced it with an isomorphic version; it’s not my problem if it looks even more transparently irrelevant or wrong.
Yes, it is significant, but it’s along the lines of “most (optimization) problems are in complexity classes higher than P” or “AIXI is uncomputable”. It doesn’t mean that the axioms or proofs are false; it just means that, yet again, as always, we need to make trade-offs and approximations.