Here is a small counterexample to P2.
States = { Red, Green, Blue }. Outcomes = { Win, Lose }.
Since there are only two outcomes, we can write actions as the subset of states that Win.
My preferences are:
{} < { Green } = { Blue } < { Red } < { Red,Green } = { Red,Blue } < { Green,Blue } < { Red,Green,Blue }
This contradicts P2 because { Green } < { Red } but { Red,Blue } < { Green,Blue }.
Here is a situation where this may apply:
There is an urn with 300 balls. 100 of them are red. The rest are either green or blue. You draw a ball from this urn.
So Red represents definite probability 1⁄3, while Green and Blue are unknowns. Depending on context, it sure looks like these are the right preferences to have. This is called the Ellsberg paradox.
Even if you insist this is somehow wrong, it is not going to be Dutch booked. Even if we extend the state space to include arbitrarily many fair coins (as P6 may require), and even if we extend the result space to allow for multiple draws or other payouts, we can define various consistent objective functions (that are not expected utility) which show this behaviour.
This can be Dutch booked. As described on this Wikipedia page, you are asked to set prices for promises to pay $1 conditional on events and an adversary chooses whether to buy these from you or sell them to you at that price. If Price( {Green} ) + Price( {Red, Blue} ) ≠ $1, the adversary can ensure you lose money, and the same holds for {Red} and {Green, Blue}. However, this is incompatible with {Green} < {Red} and {Red, Blue} < {Green, Blue}.
I’m aware of this. In this case my “operational subjective probability”, as described on that same page, is necessarily not consistent with my preferences.
To put this another way, suppose that I do put the same price on Red, Green, and Blue when faced with that particular choice (i.e. knowing that I will have to buy or sell at the price I name). Why does it follow that I should not choose Red over Green in other circumstances? Or more to the point, how can I be Dutch booked if I then choose Red over Green in other circumstances?
I realize it has been a while, but can you answer some question about your preferences?
In the hypothetical world where all probabilities that you were asked to bet on were known, would you be a Bayesian?
How stable is your preference for Knightian risk over uncertainty? In other words, how much more would winning on green have to be worth for you to prefer it to red (feel free to interpret this as much as is necessary to make it precise)?
I’m not really clear on the first question. But since the second question asks how much something is worth, I take it the first question is asking about a utility function. Do I behave as if I were maximising expected utility, ie. obey the VNM postulates as far as known probabilities go? A yes answer then makes the second question go something like this: given a bet on red whose payoff has utility 1, and a bet on green whose payoff has utility N, what is the critical N where I am indifferent between the two?
For every N>1, there are decision procedures for which the answer to the first is yes, the answer to the second is N, and which displays the Ellsberg-paradoxical behaviour. Ellsberg himself had proposed one. I did have a thought on how one of these could be well illustrated in not too technical terms, and maybe it would be appropriate to post it here, but I’d have to get around to writing it up. In the meantime I can also illustrate interactively: 1) yes, 2) you can give me an N>1 and I’ll go with it.
Okay. Let N = 2 for simplicity and let $ denote utilons like you would use for decisions involving just risk and no uncertainty.
P(Red) = 1⁄3, so you are indifferent between $-1 unconditionally and ($-3 if Red, $0 otherwise). You are also indifferent between $-3 iff Red and $-3N (= $-6) iff Green (or equivalently Blue). By transitivity, you are therefore indifferent between $-1 unconditionally and $-6 iff Green. Also, you are obviously indifferent between $4 unconditionally and $6 iff (die ≥ 3).
I would think that you would allow a `pure risk’ bet to be added to an uncorrelated uncertainty bet—correct me if that is wrong. In that case, you would be indifferent between $3 unconditionally and $6 iff (die ≥ 3) - $6 iff Green, but you would not be indifferent between $3 unconditionally and $6 iff (Green ∨ Blue) - $6 iff Green, which is the same as $6 iff Blue, which you value at $1.
This seems like a strange set of preferences to have, especially since both (die ≥ 3) and (Green ∨ Blue) are both pure risk, but it could be correct.
I take it what is strange is that I could be indifferent between A and B, but not indifferent between A+C and B+C.
For a simpler example let’s add a fair coin (and again let N=2). I think $1 iff Green is as good as $1 iff (Heads and Red), but $1 iff (Green or Blue) is better than $1 iff ((Heads and Red) or Blue). (All payoffs are the same, so we can actually forget the utility function.) So again: A is as good as B, but A+C is better than B+C. Is this the same strangeness?
I think that the situation that you described in less strange then the one that I described. In yours, you are combining two ‘unknown probabilities’ to produce ‘known probabilities’.
I find my situation stranger because the only difference between a choice that you are indifferent about and one that you do have a preference about is the substitution of (Green ∨ Blue) for (die ≥ 3). Both of these have clear probabilities and are equivalent in almost any situation. To put this another way, you would be indifferent between $3 unconditionally and $6 iff (Green ∨ Blue) - $6 iff Green if the two bets on coloured balls were taken to refer to different draws from the (same) urn. This looks a lot like risk aversion, and mentally feels like risk aversion to me, but it is not risk aversion since you would not make these bets if all probabilities were known to be 1⁄3.
If I had a do-over on my last answer, I would not agree that $-6 iff Green is worth $-1. It’s $-3.
But, given that I can’t seem to get it straight, I have to admit I haven’t given LW readers much reason to believe that I do know what I’m talking about here, and at least one good reason to believe that I don’t.
In case anyone’s still humouring me, if an event has unknown probability, so does its negation; I prefer a bet on Red to a bet on Green, but I also prefer a bet against Red to a bet against Green. This is actually the same thing as combining two unknown probabilities to produce a known one: both Green and (not Green) are unknown, but (Green or not Green) is known to be 100%.
$-6 iff Green is actually identical to $-6 + $6 iff (not Green). (not Green) is identical to (Red or Blue), and Red is a known probability of 1⁄3. $6 iff Blue is as good as $6 iff Green, which, for N=2, is worth $1. $-6 iff Green is actually worth $-3, rather than $-1.
Hmm. Now we have that $6 iff Green is worth $1 and $-6 iff Green is worth $-3, but $6-6 = $0 iff Green is not equivalent to $1-3 = $-2.
In particular, if you have $6 conditional on Green, you will trade that to me for $1. Then, we agree that if Green occurs, I will give you $6 and you will give me $6, since this adds up to no change. However, then I agree to waive your having to pay me the $6 back if you give me $3. You now have your original $6 iff Green back, but are down an unconditional $2, an indisputable net loss.
Also, this made me realize that I could have just added an unconditional $6 in my previous example rather than complicating things by making the $6 first conditional on (die ≥ 3) and then on (Green ∨ Blue). That would be much clearer.
Earlier I said $-6 iff Green is identical to $-6 + $6 iff (not Green), then I decomposed (not Green) into (Red or Blue).
Similarly, I say this example is identical to $-1 + $2 iff (Green and Heads) + $1 iff (not Green), then I decompose (not Green) into (Red or (Blue and Heads) or (Blue and Tails)).
$1 iff ((Green and Heads) or (Blue and Heads)) is a known bet. So is $1 iff ((Green and Heads) or (Blue and Tails)). There are no leftover unknowns.
Consider $6 iff (Green ∧ Heads) - $6 iff (Green ∧ Tails) + $4 iff Tails. This bet is equivalent to $0 + $2 = $2, so you would be willing to pay $2 for this bet.
If the coin comes out heads, the bet will become $6 iff Green, with a value of $1. If the coin comes out tails, the bet will become $4 - $6 iff Green = $4 - $3 = $1. Therefore, assuming that the outcome of the coin is revealed first, you will, with certainty, regret having payed any amount over $1 for this bet. This is not a rational decision procedure.
Consider $6 iff ((Green and Heads) or (Blue and Tails)). This is a known bet (1/3) so worth $2. But if the coin is flipped first, and comes up Heads, it becomes $6 iff Green, and if it comes up tails, it becomes $6 iff Blue, in either case worth $1. And that’s silly.
Very well done! I concede. Now that I see it, this is actually quite general.
My point wasn’t just that I had a decision procedure, but an explanation for it. And it seems that, no matter what, I would have to explain
A) Why ((Green and Heads) or (Blue and Tails)) is not a known bet, equiprobable with Red, or
B) Why I change my mind about the urn after a coin flip.
Earlier, some others suggested non-causal/magical explanations. These are still intact. If the coin is subject to the Force, then (A), and if not, then (B). I rejected that sort of thing. I thought I had an intuitive non-magical explanation. But, it doesn’t explain (B). So, FAIL.
I’m not quite sure what you’re first sentence is referring to, but fool prefers risk to uncertainty. From his post:
given a bet on red whose payoff has utility 1, and a bet on green whose payoff has utility N, what is the critical N where I am indifferent between the two?
Depending on context, it sure looks like these are the right preferences to have.
Sorry, but that is highly nonobvious! Why do you claim that?
Note BTW that your state space is wrong in that it doesn’t include differing states of how many green balls there are, but I assume you’re just restricting your ordering to those actions which depend only on the color of the ball (since other actions would not be possible in this context).
As to the state space, as you say, we could expand the state space and restrict the actions as you suggest, and it wouldn’t matter. But if you prefer we could draw a ball from the urn, set it aside, and destroy the urn before revealing the colour of the ball. At that point colour really is the only state, as I understand the word “state”.
As to why it looks right: red is a known probability, green and blue aren’t. It seems quite reasonable to choose the known risk over the unknown one. Especially under adversarial conditions. This is sometimes called ambiguity aversion or uncertainty aversion, which is sort of orthogonal to risk aversion.
As for consistency, if you’re maximising a single function, you’re not going to end up in a lower state via upward-moving steps.
Beyond that I can point to literature on the Ellsberg paradox. The wikipedia page has some info and some resources.
FWIW, it doesn’t seem right to me to mention adversarial situations when that’s not given in the problem. Preferring safer bets seems right in the presence of an adversary, but this example isn’t displaying that reasoning.
Betting generally includes an adversary who wants you to lose money so they win it. Possibly in psychology experiments, betting against the experimenter, you are more likely to have a betting partner who is happy to lose money on bets. And there was a case of a bet happening on Less Wrong recently where the person offering the bet had another motivation, demonstrating confidence in their suspicion. But generally, ignoring the possibility of someone wanting to win money off you when they offer you a bet is a bad idea.
Now betting is supposed to be a metaphor for options with possibly unknown results. In which case sometimes you still need to account for the possibility that the options were made available by an adversary who wants you to choose badly, but less often. And you also should account for the possibility that they were from other people who wanted you to choose well, or that the options were not determined by any intelligent being or process trying to predict your choices, so you don’t need to account for an anticorrelation between your choice and the best choice. Except for your own biases.
The probability of drawing a blue ball is 1⁄3, as is that of drawing a green ball.
I’d insist that my preferences are {} < {Red} = {Green} = {Blue} < {Red, Green} = {Red, Blue} = {Blue, Green} < {Red, Green, Blue}. There’s no reason to prefer Red to Green: the possibility of there being few Green balls is counterbalanced by the possibility of there being close to 200 of them.
ETA: Well, there are situations in which your preference order is a good idea, such as when there is an adversary changing the colours of the balls in order to make you lose. They can’t touch red without being found out, they can only change the relative numbers of Blue and Green. But in that case, choosing the colour that makes you win isn’t the only effect of an action—it also affects the colours of the balls, so you need to take that into account.
So the true state space would be {Ball Drawn = i} for each value of i in [1..300]. The contents of the urn are chosen by the adversary, to be {Red = 100, Green = n, Blue = 200 - n} for n in [0..200]. When you take the action {Green}, the adversary sets n to 0, so that action maps all {Ball Drawn = i} to {Lose}. And so on. Anyway, I don’t think this is a counter-example for that reason: you’re not just deciding the winning set, you’re affecting the balls in the urn.
I see. No, that’s not the kind of adversary I had in mind when I said that.
How about a four-state example. The states are { (A,Heads), (A,Tails), (B,Heads), (B,Tails) }.
The outcomes are { Win, Lose }. I won’t list all 16 actions, just to say that by P1 you must rank them all. In particular, you must rank the actions X = { (A,Heads), (A,Tails) }, Y = { (B,Heads), (B,Tails) }, U = { (A,Heads), (B,Tails) }, and V = { (A,Tails), (B,Heads) }. Again I’m writing actions as events, since there are only two outcomes.
To motivate this, consider the game where you and your (non-psychic, non-telekinetic etc) adversary are to simultaneously reveal A or B; if you pick the same, you win, if not, your adversary wins. You are at a point in time where your adversary has written “A” or “B” on a piece of paper face down, and you have not. You have also flipped a coin, which you have not looked at (and are not required to look at, or show your adversary). Therefore the above four states do indeed capture all the state information, and the four actions I’m singling out correspond to: you ignore the coin and write “A”, or ignore and write “B”; or else you decide to base what you write on the flip of the coin, one way, or the other. As I say, by P1, you must rank these.
Me, I’ll take the coin, thanks. I rank X=Y<U=V. I just violated P2. Am I really irrational?
And even if you think I am, one of the questions originally asked was how things could be justified by Dutch book arguments or the like. So the Ellsberg paradox and variants is still relevant to that question, normative arguments aside.
So P2 doesn’t apply in this example. Why not? Well, the reason you prefer to use the coin is because you suspect the adversary to be some kind of predictor, who is slightly more likely to write down a B if you just write down A (ignoring the coin). That’s not something captured by the state information here. You clearly don’t think that (A,Tails) is simultaneously more and less likely than (B,Tails), just that the action you choose can have some influence on the outcome. I think it might be that if you expanded the state space to include a predictor with all the possibilities of what it could do, P2 would hold again.
That isn’t the issue. At the point in time I am talking about, the adversary has already made his non-revealed choice (and he is not telekinetic). There is no other state.
Tails versus Heads is objectively 1:1 resulting from the toss of a fair coin, whereas A versus B has an uncertainty that results from my adversary’s choice. I may not have reason to think that he will choose A over B, so I can still call it 1:1, but there is still a qualitative distinction between uncertainty and randomness, or ambiguity and risk, or objective and subjective probability, or whatever you want to call it, and it is not irrational to take it into account.
I have to admit, this ordering seem reasonable… for the reasons nshepperd suggests. Just saying that he’s not telepathic isn’t enough to say he’s not any sort of predictor—after all, I’m a human, I’m bad at randomizing, maybe he’s played this game before and compiled statistics. Or he just has a good idea how peope tend to think about this sort of thing. So I’m not sure you’re correct in your conclusion that this isn’t the issue.
Then I claim that a non-psychic predictor, no matter how good, is very different from a psychic.
The powers of a non-psychic predictor are entirely natural and causal. Once he has written down his hidden choice, then he becomes irrelevant. If this isn’t clear, then we can make an analogy with the urn example. After the ball is drawn but before its colour is revealed, the contents of the urn are irrelevant. As I pointed out, the urn could even be destroyed before the colour of the ball is revealed, so that the ball’s colour truly is the only state. Similarly, after the predictor writes his choice but before it is revealed, he might accidentally behead himself while shaving.
Now of course your beliefs about the talents of the late predictor might inform your beliefs about his hidden choice. But that’s the only way they can possibly be releveant. The coin and the predictor’s hidden choice on the paper really are the only states of the world now, and your own choice is free and has no effect on the state. So, if you display a strict preference for the coin, then your uncertainty is still not captured by subjective probability. You still violate P2.
To get around this, it seems you would have to posit some residual entanglement between your choice and the external state. To me this sounds like a strange thing to argue. But I suppose you could say your cognition is flawed in a way that is invisible to you, yet was visible to the clever but departed predictor. So, you might argue that, even though there is no actual psychic effect, your choice is not really free, and you have to take into account your internalities in addition to the external states.
My question then would be, does this entanglement prevent you from having a total ordering over all maps from states (internal and external) to outcomes? If yes, then P1 is violated. If no, then can I not just ask you about the ordering of the maps which only depend on the external states, and don’t we just wind up where we were?
Because there might be more to uncertainty than subjective probability.
Let’s take a step back.
Yes, if you assume that uncertainty is entirely captured by subjective probability, then you’re completely right. But if you assume that, then you wouldn’t need the Savage axioms in the first place. The Savage axioms are one way of justifying this assumption (as well as expected utility). So, what justifies the Savage axioms?
One suggestion the original poster made was to use Dutch book arguments, or the like. But now here’s a situation where there does seem to be a qualitative difference between a random event and an uncertain event, where there is a “reasonable” thing to do that violates P2, and where nothing like a Dutch book argument seems to be available to show that it is suboptimal.
I hope that clarifies the context.
EDIT: I put “reasonable” in scare-quotes. It is reasonable, and I am prepared to defend that. But it isn’t necessary to believe it is reasonable to see why this example matters in this context.
Here is a small counterexample to P2. States = { Red, Green, Blue }. Outcomes = { Win, Lose }. Since there are only two outcomes, we can write actions as the subset of states that Win. My preferences are: {} < { Green } = { Blue } < { Red } < { Red,Green } = { Red,Blue } < { Green,Blue } < { Red,Green,Blue }
This contradicts P2 because { Green } < { Red } but { Red,Blue } < { Green,Blue }.
Here is a situation where this may apply: There is an urn with 300 balls. 100 of them are red. The rest are either green or blue. You draw a ball from this urn.
So Red represents definite probability 1⁄3, while Green and Blue are unknowns. Depending on context, it sure looks like these are the right preferences to have. This is called the Ellsberg paradox.
Even if you insist this is somehow wrong, it is not going to be Dutch booked. Even if we extend the state space to include arbitrarily many fair coins (as P6 may require), and even if we extend the result space to allow for multiple draws or other payouts, we can define various consistent objective functions (that are not expected utility) which show this behaviour.
This can be Dutch booked. As described on this Wikipedia page, you are asked to set prices for promises to pay $1 conditional on events and an adversary chooses whether to buy these from you or sell them to you at that price. If Price( {Green} ) + Price( {Red, Blue} ) ≠ $1, the adversary can ensure you lose money, and the same holds for {Red} and {Green, Blue}. However, this is incompatible with {Green} < {Red} and {Red, Blue} < {Green, Blue}.
I’m aware of this. In this case my “operational subjective probability”, as described on that same page, is necessarily not consistent with my preferences.
To put this another way, suppose that I do put the same price on Red, Green, and Blue when faced with that particular choice (i.e. knowing that I will have to buy or sell at the price I name). Why does it follow that I should not choose Red over Green in other circumstances? Or more to the point, how can I be Dutch booked if I then choose Red over Green in other circumstances?
You’re completely right. Dutch book arguments prove almost nothing interesting. Your preference is rational.
I realize it has been a while, but can you answer some question about your preferences?
In the hypothetical world where all probabilities that you were asked to bet on were known, would you be a Bayesian?
How stable is your preference for Knightian risk over uncertainty? In other words, how much more would winning on green have to be worth for you to prefer it to red (feel free to interpret this as much as is necessary to make it precise)?
I’m not really clear on the first question. But since the second question asks how much something is worth, I take it the first question is asking about a utility function. Do I behave as if I were maximising expected utility, ie. obey the VNM postulates as far as known probabilities go? A yes answer then makes the second question go something like this: given a bet on red whose payoff has utility 1, and a bet on green whose payoff has utility N, what is the critical N where I am indifferent between the two?
For every N>1, there are decision procedures for which the answer to the first is yes, the answer to the second is N, and which displays the Ellsberg-paradoxical behaviour. Ellsberg himself had proposed one. I did have a thought on how one of these could be well illustrated in not too technical terms, and maybe it would be appropriate to post it here, but I’d have to get around to writing it up. In the meantime I can also illustrate interactively: 1) yes, 2) you can give me an N>1 and I’ll go with it.
Okay. Let N = 2 for simplicity and let $ denote utilons like you would use for decisions involving just risk and no uncertainty.
P(Red) = 1⁄3, so you are indifferent between $-1 unconditionally and ($-3 if Red, $0 otherwise). You are also indifferent between $-3 iff Red and $-3N (= $-6) iff Green (or equivalently Blue). By transitivity, you are therefore indifferent between $-1 unconditionally and $-6 iff Green. Also, you are obviously indifferent between $4 unconditionally and $6 iff (die ≥ 3).
I would think that you would allow a `pure risk’ bet to be added to an uncorrelated uncertainty bet—correct me if that is wrong. In that case, you would be indifferent between $3 unconditionally and $6 iff (die ≥ 3) - $6 iff Green, but you would not be indifferent between $3 unconditionally and $6 iff (Green ∨ Blue) - $6 iff Green, which is the same as $6 iff Blue, which you value at $1.
This seems like a strange set of preferences to have, especially since both (die ≥ 3) and (Green ∨ Blue) are both pure risk, but it could be correct.
That’s right.
I take it what is strange is that I could be indifferent between A and B, but not indifferent between A+C and B+C.
For a simpler example let’s add a fair coin (and again let N=2). I think $1 iff Green is as good as $1 iff (Heads and Red), but $1 iff (Green or Blue) is better than $1 iff ((Heads and Red) or Blue). (All payoffs are the same, so we can actually forget the utility function.) So again: A is as good as B, but A+C is better than B+C. Is this the same strangeness?
Not quite.
I think that the situation that you described in less strange then the one that I described. In yours, you are combining two ‘unknown probabilities’ to produce ‘known probabilities’.
I find my situation stranger because the only difference between a choice that you are indifferent about and one that you do have a preference about is the substitution of (Green ∨ Blue) for (die ≥ 3). Both of these have clear probabilities and are equivalent in almost any situation. To put this another way, you would be indifferent between $3 unconditionally and $6 iff (Green ∨ Blue) - $6 iff Green if the two bets on coloured balls were taken to refer to different draws from the (same) urn. This looks a lot like risk aversion, and mentally feels like risk aversion to me, but it is not risk aversion since you would not make these bets if all probabilities were known to be 1⁄3.
Ohh, I see. Well done! Yes, I lose.
If I had a do-over on my last answer, I would not agree that $-6 iff Green is worth $-1. It’s $-3.
But, given that I can’t seem to get it straight, I have to admit I haven’t given LW readers much reason to believe that I do know what I’m talking about here, and at least one good reason to believe that I don’t.
In case anyone’s still humouring me, if an event has unknown probability, so does its negation; I prefer a bet on Red to a bet on Green, but I also prefer a bet against Red to a bet against Green. This is actually the same thing as combining two unknown probabilities to produce a known one: both Green and (not Green) are unknown, but (Green or not Green) is known to be 100%.
$-6 iff Green is actually identical to $-6 + $6 iff (not Green). (not Green) is identical to (Red or Blue), and Red is a known probability of 1⁄3. $6 iff Blue is as good as $6 iff Green, which, for N=2, is worth $1. $-6 iff Green is actually worth $-3, rather than $-1.
Hmm. Now we have that $6 iff Green is worth $1 and $-6 iff Green is worth $-3, but $6-6 = $0 iff Green is not equivalent to $1-3 = $-2.
In particular, if you have $6 conditional on Green, you will trade that to me for $1. Then, we agree that if Green occurs, I will give you $6 and you will give me $6, since this adds up to no change. However, then I agree to waive your having to pay me the $6 back if you give me $3. You now have your original $6 iff Green back, but are down an unconditional $2, an indisputable net loss.
Also, this made me realize that I could have just added an unconditional $6 in my previous example rather than complicating things by making the $6 first conditional on (die ≥ 3) and then on (Green ∨ Blue). That would be much clearer.
I pay you $1 for the waiver, not $3, so I am down $0.
In state A, I have $6 iff Green, that is worth $1.
In state B, I have no bet, that is worth $0.
In state C, I have $-6 iff Green, that is worth $-3.
To go from A to B I would want $1. I will go from B to B for free. To go from B to A I would pay $1. State C does not occur in this example.
Wouldn’t you then prefer $0 to $1 iff (Green ∧ Heads) - $1 iff (Green ∧ Tails)?
Indifferent. This is a known bet.
Earlier I said $-6 iff Green is identical to $-6 + $6 iff (not Green), then I decomposed (not Green) into (Red or Blue).
Similarly, I say this example is identical to $-1 + $2 iff (Green and Heads) + $1 iff (not Green), then I decompose (not Green) into (Red or (Blue and Heads) or (Blue and Tails)).
$1 iff ((Green and Heads) or (Blue and Heads)) is a known bet. So is $1 iff ((Green and Heads) or (Blue and Tails)). There are no leftover unknowns.
Look at it another way.
Consider $6 iff (Green ∧ Heads) - $6 iff (Green ∧ Tails) + $4 iff Tails. This bet is equivalent to $0 + $2 = $2, so you would be willing to pay $2 for this bet.
If the coin comes out heads, the bet will become $6 iff Green, with a value of $1. If the coin comes out tails, the bet will become $4 - $6 iff Green = $4 - $3 = $1. Therefore, assuming that the outcome of the coin is revealed first, you will, with certainty, regret having payed any amount over $1 for this bet. This is not a rational decision procedure.
How about:
Consider $6 iff ((Green and Heads) or (Blue and Tails)). This is a known bet (1/3) so worth $2. But if the coin is flipped first, and comes up Heads, it becomes $6 iff Green, and if it comes up tails, it becomes $6 iff Blue, in either case worth $1. And that’s silly.
Is that the same as your objection?
Yes, that is equivalent.
Very well done! I concede. Now that I see it, this is actually quite general.
My point wasn’t just that I had a decision procedure, but an explanation for it. And it seems that, no matter what, I would have to explain
A) Why ((Green and Heads) or (Blue and Tails)) is not a known bet, equiprobable with Red, or
B) Why I change my mind about the urn after a coin flip.
Earlier, some others suggested non-causal/magical explanations. These are still intact. If the coin is subject to the Force, then (A), and if not, then (B). I rejected that sort of thing. I thought I had an intuitive non-magical explanation. But, it doesn’t explain (B). So, FAIL.
Isn’t this −1 and −4, not −1 and −1? I think you want −3/N = −1.5.
I’m not quite sure what you’re first sentence is referring to, but fool prefers risk to uncertainty. From his post:
Sorry, but that is highly nonobvious! Why do you claim that?
Note BTW that your state space is wrong in that it doesn’t include differing states of how many green balls there are, but I assume you’re just restricting your ordering to those actions which depend only on the color of the ball (since other actions would not be possible in this context).
“Consistent” in what sense?
As to the state space, as you say, we could expand the state space and restrict the actions as you suggest, and it wouldn’t matter. But if you prefer we could draw a ball from the urn, set it aside, and destroy the urn before revealing the colour of the ball. At that point colour really is the only state, as I understand the word “state”.
As to why it looks right: red is a known probability, green and blue aren’t. It seems quite reasonable to choose the known risk over the unknown one. Especially under adversarial conditions. This is sometimes called ambiguity aversion or uncertainty aversion, which is sort of orthogonal to risk aversion.
As for consistency, if you’re maximising a single function, you’re not going to end up in a lower state via upward-moving steps.
Beyond that I can point to literature on the Ellsberg paradox. The wikipedia page has some info and some resources.
FWIW, it doesn’t seem right to me to mention adversarial situations when that’s not given in the problem. Preferring safer bets seems right in the presence of an adversary, but this example isn’t displaying that reasoning.
FWIW, agreed, “not given in the problem”. My bad.
Betting generally includes an adversary who wants you to lose money so they win it. Possibly in psychology experiments, betting against the experimenter, you are more likely to have a betting partner who is happy to lose money on bets. And there was a case of a bet happening on Less Wrong recently where the person offering the bet had another motivation, demonstrating confidence in their suspicion. But generally, ignoring the possibility of someone wanting to win money off you when they offer you a bet is a bad idea.
Now betting is supposed to be a metaphor for options with possibly unknown results. In which case sometimes you still need to account for the possibility that the options were made available by an adversary who wants you to choose badly, but less often. And you also should account for the possibility that they were from other people who wanted you to choose well, or that the options were not determined by any intelligent being or process trying to predict your choices, so you don’t need to account for an anticorrelation between your choice and the best choice. Except for your own biases.
The probability of drawing a blue ball is 1⁄3, as is that of drawing a green ball.
I’d insist that my preferences are {} < {Red} = {Green} = {Blue} < {Red, Green} = {Red, Blue} = {Blue, Green} < {Red, Green, Blue}. There’s no reason to prefer Red to Green: the possibility of there being few Green balls is counterbalanced by the possibility of there being close to 200 of them.
ETA: Well, there are situations in which your preference order is a good idea, such as when there is an adversary changing the colours of the balls in order to make you lose. They can’t touch red without being found out, they can only change the relative numbers of Blue and Green. But in that case, choosing the colour that makes you win isn’t the only effect of an action—it also affects the colours of the balls, so you need to take that into account.
So the true state space would be
{Ball Drawn = i}for each value of i in [1..300]. The contents of the urn are chosen by the adversary, to be{Red = 100, Green = n, Blue = 200 - n}for n in [0..200]. When you take the action{Green}, the adversary setsnto 0, so that action maps all{Ball Drawn = i}to{Lose}. And so on. Anyway, I don’t think this is a counter-example for that reason: you’re not just deciding the winning set, you’re affecting the balls in the urn.I see. No, that’s not the kind of adversary I had in mind when I said that.
How about a four-state example. The states are { (A,Heads), (A,Tails), (B,Heads), (B,Tails) }.
The outcomes are { Win, Lose }. I won’t list all 16 actions, just to say that by P1 you must rank them all. In particular, you must rank the actions X = { (A,Heads), (A,Tails) }, Y = { (B,Heads), (B,Tails) }, U = { (A,Heads), (B,Tails) }, and V = { (A,Tails), (B,Heads) }. Again I’m writing actions as events, since there are only two outcomes.
To motivate this, consider the game where you and your (non-psychic, non-telekinetic etc) adversary are to simultaneously reveal A or B; if you pick the same, you win, if not, your adversary wins. You are at a point in time where your adversary has written “A” or “B” on a piece of paper face down, and you have not. You have also flipped a coin, which you have not looked at (and are not required to look at, or show your adversary). Therefore the above four states do indeed capture all the state information, and the four actions I’m singling out correspond to: you ignore the coin and write “A”, or ignore and write “B”; or else you decide to base what you write on the flip of the coin, one way, or the other. As I say, by P1, you must rank these.
Me, I’ll take the coin, thanks. I rank X=Y<U=V. I just violated P2. Am I really irrational?
And even if you think I am, one of the questions originally asked was how things could be justified by Dutch book arguments or the like. So the Ellsberg paradox and variants is still relevant to that question, normative arguments aside.
So P2 doesn’t apply in this example. Why not? Well, the reason you prefer to use the coin is because you suspect the adversary to be some kind of predictor, who is slightly more likely to write down a B if you just write down A (ignoring the coin). That’s not something captured by the state information here. You clearly don’t think that
(A,Tails)is simultaneously more and less likely than(B,Tails), just that the action you choose can have some influence on the outcome. I think it might be that if you expanded the state space to include a predictor with all the possibilities of what it could do, P2 would hold again.That isn’t the issue. At the point in time I am talking about, the adversary has already made his non-revealed choice (and he is not telekinetic). There is no other state.
Tails versus Heads is objectively 1:1 resulting from the toss of a fair coin, whereas A versus B has an uncertainty that results from my adversary’s choice. I may not have reason to think that he will choose A over B, so I can still call it 1:1, but there is still a qualitative distinction between uncertainty and randomness, or ambiguity and risk, or objective and subjective probability, or whatever you want to call it, and it is not irrational to take it into account.
I have to admit, this ordering seem reasonable… for the reasons nshepperd suggests. Just saying that he’s not telepathic isn’t enough to say he’s not any sort of predictor—after all, I’m a human, I’m bad at randomizing, maybe he’s played this game before and compiled statistics. Or he just has a good idea how peope tend to think about this sort of thing. So I’m not sure you’re correct in your conclusion that this isn’t the issue.
Then I claim that a non-psychic predictor, no matter how good, is very different from a psychic.
The powers of a non-psychic predictor are entirely natural and causal. Once he has written down his hidden choice, then he becomes irrelevant. If this isn’t clear, then we can make an analogy with the urn example. After the ball is drawn but before its colour is revealed, the contents of the urn are irrelevant. As I pointed out, the urn could even be destroyed before the colour of the ball is revealed, so that the ball’s colour truly is the only state. Similarly, after the predictor writes his choice but before it is revealed, he might accidentally behead himself while shaving.
Now of course your beliefs about the talents of the late predictor might inform your beliefs about his hidden choice. But that’s the only way they can possibly be releveant. The coin and the predictor’s hidden choice on the paper really are the only states of the world now, and your own choice is free and has no effect on the state. So, if you display a strict preference for the coin, then your uncertainty is still not captured by subjective probability. You still violate P2.
To get around this, it seems you would have to posit some residual entanglement between your choice and the external state. To me this sounds like a strange thing to argue. But I suppose you could say your cognition is flawed in a way that is invisible to you, yet was visible to the clever but departed predictor. So, you might argue that, even though there is no actual psychic effect, your choice is not really free, and you have to take into account your internalities in addition to the external states.
My question then would be, does this entanglement prevent you from having a total ordering over all maps from states (internal and external) to outcomes? If yes, then P1 is violated. If no, then can I not just ask you about the ordering of the maps which only depend on the external states, and don’t we just wind up where we were?
Well, that sounds irrational. Why would you pay to switch from X to U, a change that makes no difference to the probability of you winning?
Because there might be more to uncertainty than subjective probability.
Let’s take a step back.
Yes, if you assume that uncertainty is entirely captured by subjective probability, then you’re completely right. But if you assume that, then you wouldn’t need the Savage axioms in the first place. The Savage axioms are one way of justifying this assumption (as well as expected utility). So, what justifies the Savage axioms?
One suggestion the original poster made was to use Dutch book arguments, or the like. But now here’s a situation where there does seem to be a qualitative difference between a random event and an uncertain event, where there is a “reasonable” thing to do that violates P2, and where nothing like a Dutch book argument seems to be available to show that it is suboptimal.
I hope that clarifies the context.
EDIT: I put “reasonable” in scare-quotes. It is reasonable, and I am prepared to defend that. But it isn’t necessary to believe it is reasonable to see why this example matters in this context.