The actual equilibria can seem truly mind boggling at first glance. Consider this famous example:
There are 5 rational pirates, A, B, C, D and E. They find 100 gold coins. They must decide how to distribute them.
The pirates have a strict order of seniority: A is superior to B, who is superior to C, who is superior to D, who is superior to E.
The pirate world’s rules of distrubution are thus: that the most senior pirate should propose a distribution of coins. The pirates, including the proposer, then vote on whether to accept this distribution. If the proposed allocation is approved by a majority or a tie vote, it happens. If not, the proposer is thrown overboard from the pirate ship and dies, and the next most senior pirate makes a new proposal to begin the system again.
Pirates base their decisions on three factors.
1) Each pirate wants to survive.
2) Given survival, each pirate wants to maximize the number of gold coins he receives.
3) Each pirate would prefer to throw another overboard, if all other results would otherwise be equal.
The pirates do not trust each other, and will neither make nor honor any promises between pirates apart from the main proposal.
It might be expected intuitively that Pirate A will have to allocate little if any to himself for fear of being voted off so that there are fewer pirates to share between. However, this is quite far from the theoretical result.
Which is …
...
...
A: 98 coins
B: 0 coins
C: 1 coin
D: 0 coins
E: 1 coin
Proof is in the article linked. Amazing, isn’t it? :-)
It’s amazing, the results people come up with when they don’t use TDT (or some other formalism that doesn’t defect in the Prisoner’s Dilemma—though so far as I know, the concept of the Blackmail Equation is unique to TDT.)
(Because the base case of the pirate scenario is, essentially, the Ultimatum game, where the only reason the other person offers you $1 instead of $5 is that they model you as accepting a $1 offer, which is a very stupid answer to compute if it results in you getting only $1 - only someone who two-boxed on Newcomb’s Problem would contemplate such a thing.)
At some point you proposed to solve the problem of blackmail by responding to offers but not to threats. Do you have a more precise version of that proposal? What logical facts about you and your opponent indicate that the situation is an offer or a threat? I had problems trying to figure that out.
I have a possible idea for this, but I think I need help working out more the rules for the logical scenario as well. All I have are examples (and It’s not like examples of a threat are that tricky to imagine.)
Person A makes situations that involve some form of request (an offer, a a series of offers, a threat, etc.).
Person B may either Accept, Decline, or Revoke Person A’s requests. Revoking a request blocks requests from occurring at all, at a cost.
Person A might say “Give me 1 dollar and I’ll give you a Frozen Pizza.” And Person B might “Accept” if Frozen Pizza grants more utility than a dollar would.
Person A might say “Give me 100 dollars and I’ll give you a Frozen Pizza.” Person B would “Decline” the offer, since Frozen Pizza probably wouldn’t be worth more than 100 dollars, but he probably wouldn’t bother to revoke it. Maybe Person A’s next situation will be more reasonable.
Or Person A might say “Give me 500 dollars or I’ll kill you.” And Person B will pick “Revoke” because he doesn’t want that situation to occur at all. The fact that there is a choice between death or minus 500 dollars is not a good situation. He might also revoke future situations from that person.
Alternate examples: If you’re trying to convince someone to go out on a date, they might say “Yes”, “No”, or “Get away from me, you creep!”
If you are trying to enter a password to a computer system, they might allow access (correct password), deny access (incorrect password), or deny access and lock access attempts for some period (multiple incorrect passwords)
Or if you’re at a receptionist desk:
A: “I plan on going to the bathroom, Can you tell me where it is?”
B: “Yes.”
A: “I plan on going to a date tonight, Would you like to go out with me to dinner?”
B: “No.”
A: “I plan on taking your money, can you give me the key to the safe this instant?”
B: “Security!”
The difference appears to be that if it is a threat (or a fraud) you not only want to decline the offer, you want to decline future offers even if they look reasonable because the evidence from the first offer was that bad. Ergo, if someone says:
A: “I plan on taking your money, can you give me the key to the safe this instant?”
B: “Security!”
A: “I plan on going to the bathroom, Can you tell me where it is?”
B: (won’t say yes at this point because of the earlier threat) “SECURITY!”
Whereas for instance, in the reception scenario the date isn’t a threat, so:
A: “I plan on going to a date tonight, Would you like to go out with me to dinner?”
B: “No.”
A: “I plan on going to the bathroom, Can you tell me where it is?”
B: “Yes.”
I feel like this expresses threats or frauds to clearly me, but I’m not sure if it would be clear to someone else. Did it help? Are there any holes I need to fix?
The doctor walks in, face ashen. “I’m sorry- it’s likely we’ll lose her or the baby. She’s unconscious now, and so the choice falls to you: should we try to save her or the child?”
The husband calmly replies, “Revoke!”
In non-story format: how do you formalize the difference between someone telling you bad news and someone causing you to be in a worse situation? How do you formalize the difference between accidental harm and intentional harm? How do you determine the value for having a particular resistance to blackmail, such that you can distinguish between blackmail you should and shouldn’t give in to?
How do you determine the value for having a particular resistance to blackmail, such that you can distinguish between blackmail you should and shouldn’t give in to?
The doctor has no obvious reason to prefer you to want to save your wife or your child. On the other hand, the mugger would very much prefer you to hand him your wallet than to accept to be killed, and so he’s deliberately making the latter possibility as unpleasant to you as possible to make you choose the former; but if you had precommitted to not choosing the former (e.g. by leaving your wallet at home) and he had known it, he wouldn’t have approached you in the first place.
IOW this is the decision tree:
mug give in
------------------------------------------------ (+50,-50)
| |
| | don't give in
| ---------------------- (-1,-1e6)
| don't mug
---------------------------------------------- (0, 0)
where the mugger makes the first choice, you make the second choices, and the numbers in parentheses are the pay-offs for the mugger and for you respectively. If you precommit not to choose the top branch, the mugger will take the bottom branch. (How do I stop multiple spaces from being collapsed into one?
The doctor walks in, face ashen. “I’m sorry- it’s likely we’ll lose her or the baby. She’s unconscious now, and so the choice falls to you: should we try to save her or the child?”
The husband calmly replies, “Revoke!”
An eloquent way of pointing out what I was missing. Thank you!
In non-story format: How do you formalize the difference between someone telling you bad information and someone causing you to be in a worse situation?
I will try to think on this more. The only thing that’s occurred to me so far is that if that it seems like if you have a formalization, it may not be a good idea to announce your formalization. Someone who knows your formalization might be able to exploit it by customizing their imposed worse situation to look like simply telling you bad information, their intentional harm to look like accidental harm, or their blackmail to extort the maximum amount of money out of you, if they had an explicit set of formal rules about where those boundaries were.
And for instance, it seems like a person would prefer it someone else blackmailed that person less than they could theoretically get away with because they were being cautious, rather than having every blackmailer immediately blackmail at maximum effective blackmail. (at that point, since the threshold can change)
Again, I really do appreciate you helping me focus my thoughts on this.
If I have a choice of whether or not to perform an action A, and I believe that performing A will harm agent X and will not in and of itself benefit me, and I credibly commit to performing A unless X provides me with some additional value V, I would consider myself to be threatening X with A unless they provide V. Whether that is a threat of blackmail or some other kind of threat doesn’t seem like a terribly interesting question.
Edit: my earlier thoughts on extortion/blackmail, specifically, here.
Did you ever see Shawshank Redemption? One of the Warden’s tricks is not just to take construction projects with convict labor, but to bid on any construction project (with the ability to undercut any competitor because his labor is already paid for) unless the other contractors paid him to stay away from that job.
My thought, as hinted at by my last question, is that refusing or accepting any particular blackmail request depends on the immediate and reputational costs of refusing or accepting. A flat “we will not accept any blackmail requests” is emotionally satisfying to deliver, but can’t be the right strategy for all situations. (When the hugger mugger demands “hug me or I’ll shoot!”, well, I’ll give him a hug.) A “we will not accept any blackmail requests that cost more than X” seems like the next best step, but as pointed out here that runs the risk of people just demanding X every time. Another refinement might be to publish a “acceptance function”- you’ll accept a (sufficiently credible and damaging) blackmail request for x with probability f(x), which is a decreasing (probably sigmoidal) function.
But the reputational costs of accepting or rejecting vary heavily based on the variety of threat, what you believe about potential threateners, whose opinions you care about, and so on. Things get very complex very fast.
If I am able to outbid all competitors for any job, but cannot do all jobs, and I let it be known that I won’t bid on jobs if bribed accordingly, I would not consider myself to be threatening all the other contractors, or blackmailing them. In effect this is a form of rent-seeking.
The acceptance-function approach you describe, where the severity and credibility of the threat matter, makes sense to me.
Blackmail seems to me to be a narrow variety of rent-seeking, and reasons for categorically opposing blackmail seem like reasons for categorically opposing rent-seeking. But I might be using too broad a category for ‘rent-seeking.’
reasons for categorically opposing blackmail seem like reasons for categorically opposing rent-seeking
Well, I agree, but only because in general the reasons for categorically opposing something that would otherwise seem rational to cooperate with are similar. That is, the strategy of being seen to credibly commit to a policy of never rewarding X, even when rewarding X would leave me better off, is useful whenever such a strategy reduces others’ incentive to X and where I prefer that people not X at me. It works just as well where X=rent-seeking as where X=giving me presents as where X=threatening me.
Yes but I’m not sure how valuable it is to. Basically, it boils down to ‘non-productive means of acquiring wealth,’ but it’s not clear if, say, petty theft should be included. (Generally, definitional choices like that there are made based on identity implications, rather than economic ones.) The general sentiment of things “I prefer that people not X at me” captures the essence better, perhaps.
There are benefits to insisting on a narrower definition: perhaps something like legal non-productive means of acquiring wealth, but part of the issue is that rent-seeking often operates by manipulating the definition of ‘legal.’
Here’s my version of the definition used by Schelling in The Strategy of Conflict: A threat is when I commit myself to an action, conditional on an action of yours, such that if I end up having to take that action I would have reason to regret having committed myself to it.
So if I credibly commit myself to the assertion, ‘If you don’t give me your phone, I’ll throw you off this ship,’ then that’s a threat. I’m hoping that the situation will end with you giving me your phone. If it ends with me throwing you overboard, the penalties I’ll incur will be sufficient to make me regret having made the commitment.
But when these rational pirates say, ‘If we don’t like your proposal, we’ll throw you overboard,’ then that’s not a threat; they’re just elucidating their preferences. Schelling uses ‘warning’ for this sort of statement.
I’ll guess that in your analysis, given the base case of D and E’s game being a tie vote on a (D=100, E=0) split, results in a (C=0, D=0, E=100) split for three pirates since E can blackmail C into giving up all the coins in exchange for staying alive? D may vote arbitrarily on a (C=0, D=100, E=0) split, so C must consider E to have the deciding vote.
If so, that means four pirates would yield (B=0, C=100, D=0, E=0) or (B=0, C=0, D=100, E=0) in a tie. E expects 100 coins in the three-pirate game and so wouldn’t be a safe choice of blackmailer, but C and D expect zero coins in a three-pirate game so B could choose between them arbitrarily. B can’t give fewer than 100 coins to either C or D because they will punish that behavior with a deciding vote for death, and B knows this. It’s potentially unintuitive for C because C’s expected value in a three-pirate game is 0 but if C commits to voting against B for anything less than 100 coins, and B knows this, then B is forced to give either 0 or 100 coins to C. The remaining coins must go to D.
In the case of five pirates C and D except more than zero coins on average if A dies because B may choose arbitrarily between C or D as blackmailer. B and E expect zero coins from the four-pirate game. A must maximize the chance that two or more pirates will vote for A’s split. C and D have an expected value of 50 coins from the four-pirate game if they assume B will choose randomly, and so a (A=0, B=0, C=50, D=50, E=0) split is no better than B’s expected offer for C and D and any fewer than 50 coins for C or D will certainly make them vote against A. I think A should offer (A=0, B=n, C=0, D=0, E=100-n) where n is mutually acceptable to B and E.
Because B and E have no relative advantage in a four-pirate game (both expect zero coins) they don’t have leverage against each other in the five-pirate game. If B had a non-zero probability of being killed in a four-pirate game then A should offer E more coins than B at a ratio corresponding to that risk. As it is, I think B and E would accept a fair split of n=50, but I may be overlooking some potential for E to blackmail B.
In every case of the pirates game, the decision-maker assigns one coin to every pirate an even number of steps away from himself, and the rest of the coins to himself (with more gold than pirates, anyway; things can get weird with large numbers of pirates). See the Wikipedia article Kawoomba linked to for an explanation of why.
I see 1 rational pirate and 4 utter morons who have paid so much attention to math that they forgot to actually win. I mean, if they were “less rational”, they’d be inclined to get outraged over the unfairness, and throw A overboard, right? And A would expect it, so he’d give them a better deal. “Rational” is not “walking away with less money”.
It’s still an interesting example and thank you for posting it.
I see 1 rational pirate and 4 utter morons who have paid so much attention to math that they forgot to actually win.
Remember, this isn’t any old pirate crew. This is a crew with a particular set of rules that gives different pirates different powers. There’s no “hang the rules!” here, and since it’s an artificial problem there’s an artificial solution.
D has the power to walk away with all of the gold, and A, B, and C dead. He has no incentive to agree to anything less, because if enough votes fail he’ll be in the best possible position. This determines E’s default result, which is what he makes decisions based off of. Building out the game tree helps here.
If B, C, D, and E throw A overboard then C, D, and E are in a position to throw B overboard and any argument B, C, D, and E can come up with for throwing A overboard is just as strong an argument as C, D, and E can come up with for throwing B overboard. In fact, with fewer pirates an even split would be even more advantageous to C, D, and E. So B will always vote for A’s split out of self-preservation. If C throws A overboard he will end up in the same situation as B was originally; needing to support the highest ranking pirate to avoid being the next one up for going overboard. Since the second in command will always vote for the first in command out of self preservation he or she will accept a split with zero coins for themselves. Therefore A only has to offer C 1 coin to get C’s vote. A, B, and C’s majority rules.
In real life I imagine the pirate who is best at up-to-5-way fights is left with 100 coins. If the other pirates were truly rational then they would never have boarded a pirate ship with a pirate who is better at an up-to-5-way fight than them.
If the other pirates were truly rational then they would never have boarded a pirate ship with a pirate who is better at an up-to-5-way fight than them.
When someone asks me how I would get out of a particularly sticky situation, I often fight the urge to glibly respond, by not getting it to said situation.
I digress, if the other pirates were truly rational then they would never let anyone know how good they were at an up-to-X-way fight.
By extension, should truly rational entities never let anyone know how rational they are?
No, they should sometimes not let people know that. Sometimes it is an advantage—either by allowing cooperation that would not otherwise have been possible or demonstrating superior power and so allow options for dominance.
In real life I imagine the pirate who is best at up-to-5-way fights is left with 100 coins.
I doubt it. Humans almost never just have an all in winner takes all fight. The most charismatic pirate will end up with the greatest share, while the others end up with lesser shares depending on what it takes for the leader to maintain alliances. (Estimated and observed physical prowess is one component of said charisma in the circumstance).
I see 1 rational pirate and 4 utter morons who have paid so much attention to math that they forgot to actually win. I mean, if they were “less rational”, they’d be inclined to get outraged over the unfairness, and throw A overboard, right? And A would expect it, so he’d give them a better deal. “Rational” is not “walking away with less money”.
Well if they chose to decline A’s proposal for whatever reason, that would put C and E in a worse position than if they didn’t.
Outrage over being treated unfairly doesn’t come into one time prisoner’s dilemmata. Each pirate wants to survive with the maximum amount of gold, and C and E would get nothing if they didn’t vote for A’s proposal.
Hence, if C and E were outraged as you suggest, and voted against A’s proposal, they would walk away with even less. One gold coin buys you infinitely more blackjack and hookers than zero gold coins.
If C and E—and I’d say all 4 of them really, at least regarding a 98 0 1 0 1 solution—were inclined to be outraged as I suggest, and A knew this, they would walk away with more money. For me, that trumps any possible math and logic you could put forward.
And just in case A is stupid:
“But look, C and E, this is the optimal solution, if you don’t listen to me you’ll get less gold!”
“Nice try, smartass. Overboard you go.”
B watched on, starting to sweat...
EDIT: Ooops, I notice that I missed the fact that B doesn’t need to sweat since he just needs D. Still, my main point isn’t about B, but A.
Also I wanna make it 100% clear: I don’t claim that the proof is incorrect, given all the assumptions of the problem, including the ones about how the agents work. I’m just not impressed with the agents, with their ability to achieve their goals. Leave A unchanged and toss in 4 reasonably bright real humans as B C D E, at least some of them will leave with more money.
...because it’s very hot in Pirate Island’s shark-infested Pirate Bay, not out of any fear or distress at an outcome that put her in a better position than she had occupied before.
Whereas A had to build an outright majority to get his plan approved, B just had to convince one other pirate. D was the natural choice, both from a strict logico-mathematical view, and because D had just watched C and E team up to throw their former captain overboard. It wasn’t that they were against betraying their superiors for a slice of the treasure, it was that the slice wasn’t big enough! D wasn’t very bright—B knew from sharing a schooner with him these last few months—but team CE had been so obliging as to slice a big branch off D’s decision tree. What was left was a stump. D could take her offer of 1 coin, or be left to the mercy of the outrageously blood-thirsty team CE.
C and E watched on, dreaming of all the wonderful things they could do with their 0 coins.
[I think the “rationality = winning” story holds here (in the case where A’s proposal passes, not in this weird counterfactual cul-de-sac) but in a more subtle way. The 98 0 1 0 1 solution basically gives a value of the ranks, i.e., how much a pirate should be willing to pay to get into that rank at the time treasure will be divided. From this perspective, being A is highly valuable, and A should have been willing to pay, say, 43 coins for his ship, 2 coins for powder, 7 coins for wages to B, C, D, E, etc., to make it into that position. C, on the other hand, might turn down a promotion to second-in-command over B, unless it’s paired with a wage hike of one 1 coin; B would be surprisingly happy to be given such a demotion, if her pay remained unchanged. All the pirates can win even in a 98 0 1 0 1 solution, if they knew such a treasure would be found and planned accordingly.]
It seems to me that the extent to which B C D E will be able to get more money is to some extent dependent on their ability to plausibly precommit to rejecting an “unfair” deal… and possibly their ability to plausibly precommit to accepting a “fair” one.
Emphasis on “plausibly” and “PIRATES.”
At minimum, if they can plausibly precommit to things, I’d expect at the very least CDE to precommit to tossing A B overboard no matter what is offered and splitting the pot three ways. There are quite possibly better commitments to make even than this.
Heh, if I can’t convince you with “any possible math and logic”, let me try with this great video (also that one) that shows the consequences of “reasoning via outrage”.
Watched the first one. It was very different from the scenario we’re discussing. No one’s life was at stake. Also the shares were unequal from the start, so there was no fair scenario being denied, to get outraged about.
I’m not in favor of “reasoning via outrage” in general. I’m simply in favor of possessing a (known) inclination to turn down overly skewed deals (like humans generally have, usefully I might add); if I have it, and your life is at stake, you’d have to be suicidal to propose a 98 0 1 0 1 if I’m one of the people whose vote you need.
What makes it different from the video example is that, in the pirate example, if I turn down the deal the proponent loses far, far more than I do. Not just 98 coins to my 1, but their life, which should be many orders of magnitude more precious. So there’s clearly room for a more fair deal. The woman in that case wasn’t like my proposed E or C, she was like a significantly stupider version of A, wanting an unfairly good deal in a situation when there was no reason for her to believe her commitment could reliably prevail over the other players’ ability to do the same.
Watched the first one. It was very different from the scenario we’re discussing. No one’s life was at stake. Also the shares were unequal from the start, so there was no fair scenario being denied, to get outraged about.
A suggestion to randomize was made and denied. They fail at thinking. Especially Jo, who kept trying to convince herself and others that she didn’t care about money. Sour grapes—really pathetic.
Whenever I come across highly counterintuitive claims along these lines, I code them up and see how they perform over many iterations.
This is a lot trickier to do in this case compared to, say, the Monty Hall problem, but if you restricted it just to cases in which Pirate A retained 98 of the coins, you could demonstrate whether the [98, 0, 1, 0, 1] distribution was stable or not.
Also, I’d suggest thinking about this in a slightly different way to the way you’re thinking about it. The only pirate in the scenario who doesn’t have to worry about dying is pirate E, who can make any demands he likes from pirate D. What distribution would he suggest?
Edit: Rereading the wording of the scenario, pirate E can’t make any demands he likes from pirate D, and pirate D himself also doesn’t need to worry about dying.
Heh, if I can’t convince you with “any possible math and logic”, let me try with this great video (also that one) that shows the consequences of “reasoning via outrage”.
That shows one aspect of the consequences of reasoning via outrage. It doesn’t indicate that the strategy itself is bad. In a similar way the consequences of randomizing and defending Podunk 1⁄11 times is that 1⁄11 times (against a good player) you will end up on youtube losing Metropolis.
“I will stand firm on A until A becomes worth less than C is now, then I will accept B or C. You get more by accepting B or C now than you get by trying to get more.”
One rational course of action is to mutually commit to a random split, and follow through. What’s the rational course of action to respond to someone who makes that threat and is believed to follow through on it? If it is known that the other two participants are rational, why isn’t making a threat of that nature rational?
With the caveat that this ‘proof’ relies on the same assumptions that ‘prove’ that the rational prisoners defect in the one shot prisoners dilemma—which they don’t unless they have insufficient (or inaccurate) information about each other. At a stretch we could force the “do not trust each other” premise to include “the pirates have terrible maps of each other” but that’s not a realistic interpretation of the sentence. Really there is the additional implicit assumption “Oh, and all these pirates are agents that implement Causal Decision Theory”.
Amazing, isn’t it? :-)
It gets even more interesting when there are more than 200 pirates (and still only 100 coins).
Actually, does the traditional result work? E would rather the result be (dead, dead, 99, 0, 1) than (98, 0, 1, 0, 1). I think it has to be (97, 0, 1, 0, 2).
[edit]It appears that E should assume that B will make a successful proposition, which will include nothing for E, and so (dead, dead, 99, 0, 1) isn’t on the table.
The actual equilibria can seem truly mind boggling at first glance. Consider this famous example:
There are 5 rational pirates, A, B, C, D and E. They find 100 gold coins. They must decide how to distribute them.
The pirates have a strict order of seniority: A is superior to B, who is superior to C, who is superior to D, who is superior to E.
The pirate world’s rules of distrubution are thus: that the most senior pirate should propose a distribution of coins. The pirates, including the proposer, then vote on whether to accept this distribution. If the proposed allocation is approved by a majority or a tie vote, it happens. If not, the proposer is thrown overboard from the pirate ship and dies, and the next most senior pirate makes a new proposal to begin the system again.
Pirates base their decisions on three factors.
1) Each pirate wants to survive.
2) Given survival, each pirate wants to maximize the number of gold coins he receives.
3) Each pirate would prefer to throw another overboard, if all other results would otherwise be equal.
The pirates do not trust each other, and will neither make nor honor any promises between pirates apart from the main proposal.
It might be expected intuitively that Pirate A will have to allocate little if any to himself for fear of being voted off so that there are fewer pirates to share between. However, this is quite far from the theoretical result.
Which is …
...
...
A: 98 coins
B: 0 coins
C: 1 coin
D: 0 coins
E: 1 coin
Proof is in the article linked. Amazing, isn’t it? :-)
It’s amazing, the results people come up with when they don’t use TDT (or some other formalism that doesn’t defect in the Prisoner’s Dilemma—though so far as I know, the concept of the Blackmail Equation is unique to TDT.)
(Because the base case of the pirate scenario is, essentially, the Ultimatum game, where the only reason the other person offers you $1 instead of $5 is that they model you as accepting a $1 offer, which is a very stupid answer to compute if it results in you getting only $1 - only someone who two-boxed on Newcomb’s Problem would contemplate such a thing.)
At some point you proposed to solve the problem of blackmail by responding to offers but not to threats. Do you have a more precise version of that proposal? What logical facts about you and your opponent indicate that the situation is an offer or a threat? I had problems trying to figure that out.
I have a possible idea for this, but I think I need help working out more the rules for the logical scenario as well. All I have are examples (and It’s not like examples of a threat are that tricky to imagine.)
Person A makes situations that involve some form of request (an offer, a a series of offers, a threat, etc.). Person B may either Accept, Decline, or Revoke Person A’s requests. Revoking a request blocks requests from occurring at all, at a cost.
Person A might say “Give me 1 dollar and I’ll give you a Frozen Pizza.” And Person B might “Accept” if Frozen Pizza grants more utility than a dollar would.
Person A might say “Give me 100 dollars and I’ll give you a Frozen Pizza.” Person B would “Decline” the offer, since Frozen Pizza probably wouldn’t be worth more than 100 dollars, but he probably wouldn’t bother to revoke it. Maybe Person A’s next situation will be more reasonable.
Or Person A might say “Give me 500 dollars or I’ll kill you.” And Person B will pick “Revoke” because he doesn’t want that situation to occur at all. The fact that there is a choice between death or minus 500 dollars is not a good situation. He might also revoke future situations from that person.
Alternate examples: If you’re trying to convince someone to go out on a date, they might say “Yes”, “No”, or “Get away from me, you creep!”
If you are trying to enter a password to a computer system, they might allow access (correct password), deny access (incorrect password), or deny access and lock access attempts for some period (multiple incorrect passwords)
Or if you’re at a receptionist desk:
A: “I plan on going to the bathroom, Can you tell me where it is?”
B: “Yes.”
A: “I plan on going to a date tonight, Would you like to go out with me to dinner?”
B: “No.”
A: “I plan on taking your money, can you give me the key to the safe this instant?”
B: “Security!”
The difference appears to be that if it is a threat (or a fraud) you not only want to decline the offer, you want to decline future offers even if they look reasonable because the evidence from the first offer was that bad. Ergo, if someone says:
A: “I plan on taking your money, can you give me the key to the safe this instant?”
B: “Security!”
A: “I plan on going to the bathroom, Can you tell me where it is?”
B: (won’t say yes at this point because of the earlier threat) “SECURITY!”
Whereas for instance, in the reception scenario the date isn’t a threat, so:
A: “I plan on going to a date tonight, Would you like to go out with me to dinner?”
B: “No.”
A: “I plan on going to the bathroom, Can you tell me where it is?”
B: “Yes.”
I feel like this expresses threats or frauds to clearly me, but I’m not sure if it would be clear to someone else. Did it help? Are there any holes I need to fix?
The doctor walks in, face ashen. “I’m sorry- it’s likely we’ll lose her or the baby. She’s unconscious now, and so the choice falls to you: should we try to save her or the child?”
The husband calmly replies, “Revoke!”
In non-story format: how do you formalize the difference between someone telling you bad news and someone causing you to be in a worse situation? How do you formalize the difference between accidental harm and intentional harm? How do you determine the value for having a particular resistance to blackmail, such that you can distinguish between blackmail you should and shouldn’t give in to?
The doctor has no obvious reason to prefer you to want to save your wife or your child. On the other hand, the mugger would very much prefer you to hand him your wallet than to accept to be killed, and so he’s deliberately making the latter possibility as unpleasant to you as possible to make you choose the former; but if you had precommitted to not choosing the former (e.g. by leaving your wallet at home) and he had known it, he wouldn’t have approached you in the first place.
IOW this is the decision tree:
where the mugger makes the first choice, you make the second choices, and the numbers in parentheses are the pay-offs for the mugger and for you respectively. If you precommit not to choose the top branch, the mugger will take the bottom branch. (How do I stop multiple spaces from being collapsed into one?
An eloquent way of pointing out what I was missing. Thank you!
I will try to think on this more. The only thing that’s occurred to me so far is that if that it seems like if you have a formalization, it may not be a good idea to announce your formalization. Someone who knows your formalization might be able to exploit it by customizing their imposed worse situation to look like simply telling you bad information, their intentional harm to look like accidental harm, or their blackmail to extort the maximum amount of money out of you, if they had an explicit set of formal rules about where those boundaries were.
And for instance, it seems like a person would prefer it someone else blackmailed that person less than they could theoretically get away with because they were being cautious, rather than having every blackmailer immediately blackmail at maximum effective blackmail. (at that point, since the threshold can change)
Again, I really do appreciate you helping me focus my thoughts on this.
If I have a choice of whether or not to perform an action A, and I believe that performing A will harm agent X and will not in and of itself benefit me, and I credibly commit to performing A unless X provides me with some additional value V, I would consider myself to be threatening X with A unless they provide V. Whether that is a threat of blackmail or some other kind of threat doesn’t seem like a terribly interesting question.
Edit: my earlier thoughts on extortion/blackmail, specifically, here.
Did you ever see Shawshank Redemption? One of the Warden’s tricks is not just to take construction projects with convict labor, but to bid on any construction project (with the ability to undercut any competitor because his labor is already paid for) unless the other contractors paid him to stay away from that job.
My thought, as hinted at by my last question, is that refusing or accepting any particular blackmail request depends on the immediate and reputational costs of refusing or accepting. A flat “we will not accept any blackmail requests” is emotionally satisfying to deliver, but can’t be the right strategy for all situations. (When the hugger mugger demands “hug me or I’ll shoot!”, well, I’ll give him a hug.) A “we will not accept any blackmail requests that cost more than X” seems like the next best step, but as pointed out here that runs the risk of people just demanding X every time. Another refinement might be to publish a “acceptance function”- you’ll accept a (sufficiently credible and damaging) blackmail request for x with probability f(x), which is a decreasing (probably sigmoidal) function.
But the reputational costs of accepting or rejecting vary heavily based on the variety of threat, what you believe about potential threateners, whose opinions you care about, and so on. Things get very complex very fast.
If I am able to outbid all competitors for any job, but cannot do all jobs, and I let it be known that I won’t bid on jobs if bribed accordingly, I would not consider myself to be threatening all the other contractors, or blackmailing them. In effect this is a form of rent-seeking.
The acceptance-function approach you describe, where the severity and credibility of the threat matter, makes sense to me.
Blackmail seems to me to be a narrow variety of rent-seeking, and reasons for categorically opposing blackmail seem like reasons for categorically opposing rent-seeking. But I might be using too broad a category for ‘rent-seeking.’
Well, I agree, but only because in general the reasons for categorically opposing something that would otherwise seem rational to cooperate with are similar. That is, the strategy of being seen to credibly commit to a policy of never rewarding X, even when rewarding X would leave me better off, is useful whenever such a strategy reduces others’ incentive to X and where I prefer that people not X at me. It works just as well where X=rent-seeking as where X=giving me presents as where X=threatening me.
Can you expand on your model if rent-seeking?
Yes but I’m not sure how valuable it is to. Basically, it boils down to ‘non-productive means of acquiring wealth,’ but it’s not clear if, say, petty theft should be included. (Generally, definitional choices like that there are made based on identity implications, rather than economic ones.) The general sentiment of things “I prefer that people not X at me” captures the essence better, perhaps.
There are benefits to insisting on a narrower definition: perhaps something like legal non-productive means of acquiring wealth, but part of the issue is that rent-seeking often operates by manipulating the definition of ‘legal.’
Here’s my version of the definition used by Schelling in The Strategy of Conflict: A threat is when I commit myself to an action, conditional on an action of yours, such that if I end up having to take that action I would have reason to regret having committed myself to it.
So if I credibly commit myself to the assertion, ‘If you don’t give me your phone, I’ll throw you off this ship,’ then that’s a threat. I’m hoping that the situation will end with you giving me your phone. If it ends with me throwing you overboard, the penalties I’ll incur will be sufficient to make me regret having made the commitment.
But when these rational pirates say, ‘If we don’t like your proposal, we’ll throw you overboard,’ then that’s not a threat; they’re just elucidating their preferences. Schelling uses ‘warning’ for this sort of statement.
So if all pirates implement TDT, what happens?
I’ll guess that in your analysis, given the base case of D and E’s game being a tie vote on a (D=100, E=0) split, results in a (C=0, D=0, E=100) split for three pirates since E can blackmail C into giving up all the coins in exchange for staying alive? D may vote arbitrarily on a (C=0, D=100, E=0) split, so C must consider E to have the deciding vote.
If so, that means four pirates would yield (B=0, C=100, D=0, E=0) or (B=0, C=0, D=100, E=0) in a tie. E expects 100 coins in the three-pirate game and so wouldn’t be a safe choice of blackmailer, but C and D expect zero coins in a three-pirate game so B could choose between them arbitrarily. B can’t give fewer than 100 coins to either C or D because they will punish that behavior with a deciding vote for death, and B knows this. It’s potentially unintuitive for C because C’s expected value in a three-pirate game is 0 but if C commits to voting against B for anything less than 100 coins, and B knows this, then B is forced to give either 0 or 100 coins to C. The remaining coins must go to D.
In the case of five pirates C and D except more than zero coins on average if A dies because B may choose arbitrarily between C or D as blackmailer. B and E expect zero coins from the four-pirate game. A must maximize the chance that two or more pirates will vote for A’s split. C and D have an expected value of 50 coins from the four-pirate game if they assume B will choose randomly, and so a (A=0, B=0, C=50, D=50, E=0) split is no better than B’s expected offer for C and D and any fewer than 50 coins for C or D will certainly make them vote against A. I think A should offer (A=0, B=n, C=0, D=0, E=100-n) where n is mutually acceptable to B and E.
Because B and E have no relative advantage in a four-pirate game (both expect zero coins) they don’t have leverage against each other in the five-pirate game. If B had a non-zero probability of being killed in a four-pirate game then A should offer E more coins than B at a ratio corresponding to that risk. As it is, I think B and E would accept a fair split of n=50, but I may be overlooking some potential for E to blackmail B.
In every case of the pirates game, the decision-maker assigns one coin to every pirate an even number of steps away from himself, and the rest of the coins to himself (with more gold than pirates, anyway; things can get weird with large numbers of pirates). See the Wikipedia article Kawoomba linked to for an explanation of why.
I see 1 rational pirate and 4 utter morons who have paid so much attention to math that they forgot to actually win. I mean, if they were “less rational”, they’d be inclined to get outraged over the unfairness, and throw A overboard, right? And A would expect it, so he’d give them a better deal. “Rational” is not “walking away with less money”.
It’s still an interesting example and thank you for posting it.
Remember, this isn’t any old pirate crew. This is a crew with a particular set of rules that gives different pirates different powers. There’s no “hang the rules!” here, and since it’s an artificial problem there’s an artificial solution.
D has the power to walk away with all of the gold, and A, B, and C dead. He has no incentive to agree to anything less, because if enough votes fail he’ll be in the best possible position. This determines E’s default result, which is what he makes decisions based off of. Building out the game tree helps here.
If B, C, D, and E throw A overboard then C, D, and E are in a position to throw B overboard and any argument B, C, D, and E can come up with for throwing A overboard is just as strong an argument as C, D, and E can come up with for throwing B overboard. In fact, with fewer pirates an even split would be even more advantageous to C, D, and E. So B will always vote for A’s split out of self-preservation. If C throws A overboard he will end up in the same situation as B was originally; needing to support the highest ranking pirate to avoid being the next one up for going overboard. Since the second in command will always vote for the first in command out of self preservation he or she will accept a split with zero coins for themselves. Therefore A only has to offer C 1 coin to get C’s vote. A, B, and C’s majority rules.
In real life I imagine the pirate who is best at up-to-5-way fights is left with 100 coins. If the other pirates were truly rational then they would never have boarded a pirate ship with a pirate who is better at an up-to-5-way fight than them.
When someone asks me how I would get out of a particularly sticky situation, I often fight the urge to glibly respond, by not getting it to said situation.
I digress, if the other pirates were truly rational then they would never let anyone know how good they were at an up-to-X-way fight.
By extension, should truly rational entities never let anyone know how rational they are?
No, they should sometimes not let people know that. Sometimes it is an advantage—either by allowing cooperation that would not otherwise have been possible or demonstrating superior power and so allow options for dominance.
I doubt it. Humans almost never just have an all in winner takes all fight. The most charismatic pirate will end up with the greatest share, while the others end up with lesser shares depending on what it takes for the leader to maintain alliances. (Estimated and observed physical prowess is one component of said charisma in the circumstance).
Well if they chose to decline A’s proposal for whatever reason, that would put C and E in a worse position than if they didn’t.
Outrage over being treated unfairly doesn’t come into one time prisoner’s dilemmata. Each pirate wants to survive with the maximum amount of gold, and C and E would get nothing if they didn’t vote for A’s proposal.
Hence, if C and E were outraged as you suggest, and voted against A’s proposal, they would walk away with even less. One gold coin buys you infinitely more blackjack and hookers than zero gold coins.
If C and E—and I’d say all 4 of them really, at least regarding a 98 0 1 0 1 solution—were inclined to be outraged as I suggest, and A knew this, they would walk away with more money. For me, that trumps any possible math and logic you could put forward.
And just in case A is stupid:
“But look, C and E, this is the optimal solution, if you don’t listen to me you’ll get less gold!”
“Nice try, smartass. Overboard you go.”
B watched on, starting to sweat...
EDIT: Ooops, I notice that I missed the fact that B doesn’t need to sweat since he just needs D. Still, my main point isn’t about B, but A.
Also I wanna make it 100% clear: I don’t claim that the proof is incorrect, given all the assumptions of the problem, including the ones about how the agents work. I’m just not impressed with the agents, with their ability to achieve their goals. Leave A unchanged and toss in 4 reasonably bright real humans as B C D E, at least some of them will leave with more money.
...because it’s very hot in Pirate Island’s shark-infested Pirate Bay, not out of any fear or distress at an outcome that put her in a better position than she had occupied before.
Whereas A had to build an outright majority to get his plan approved, B just had to convince one other pirate. D was the natural choice, both from a strict logico-mathematical view, and because D had just watched C and E team up to throw their former captain overboard. It wasn’t that they were against betraying their superiors for a slice of the treasure, it was that the slice wasn’t big enough! D wasn’t very bright—B knew from sharing a schooner with him these last few months—but team CE had been so obliging as to slice a big branch off D’s decision tree. What was left was a stump. D could take her offer of 1 coin, or be left to the mercy of the outrageously blood-thirsty team CE.
C and E watched on, dreaming of all the wonderful things they could do with their 0 coins.
[I think the “rationality = winning” story holds here (in the case where A’s proposal passes, not in this weird counterfactual cul-de-sac) but in a more subtle way. The 98 0 1 0 1 solution basically gives a value of the ranks, i.e., how much a pirate should be willing to pay to get into that rank at the time treasure will be divided. From this perspective, being A is highly valuable, and A should have been willing to pay, say, 43 coins for his ship, 2 coins for powder, 7 coins for wages to B, C, D, E, etc., to make it into that position. C, on the other hand, might turn down a promotion to second-in-command over B, unless it’s paired with a wage hike of one 1 coin; B would be surprisingly happy to be given such a demotion, if her pay remained unchanged. All the pirates can win even in a 98 0 1 0 1 solution, if they knew such a treasure would be found and planned accordingly.]
It seems to me that the extent to which B C D E will be able to get more money is to some extent dependent on their ability to plausibly precommit to rejecting an “unfair” deal… and possibly their ability to plausibly precommit to accepting a “fair” one.
Emphasis on “plausibly” and “PIRATES.”
At minimum, if they can plausibly precommit to things, I’d expect at the very least CDE to precommit to tossing A B overboard no matter what is offered and splitting the pot three ways. There are quite possibly better commitments to make even than this.
Heh, if I can’t convince you with “any possible math and logic”, let me try with this great video (also that one) that shows the consequences of “reasoning via outrage”.
Watched the first one. It was very different from the scenario we’re discussing. No one’s life was at stake. Also the shares were unequal from the start, so there was no fair scenario being denied, to get outraged about.
I’m not in favor of “reasoning via outrage” in general. I’m simply in favor of possessing a (known) inclination to turn down overly skewed deals (like humans generally have, usefully I might add); if I have it, and your life is at stake, you’d have to be suicidal to propose a 98 0 1 0 1 if I’m one of the people whose vote you need.
What makes it different from the video example is that, in the pirate example, if I turn down the deal the proponent loses far, far more than I do. Not just 98 coins to my 1, but their life, which should be many orders of magnitude more precious. So there’s clearly room for a more fair deal. The woman in that case wasn’t like my proposed E or C, she was like a significantly stupider version of A, wanting an unfairly good deal in a situation when there was no reason for her to believe her commitment could reliably prevail over the other players’ ability to do the same.
A suggestion to randomize was made and denied. They fail at thinking. Especially Jo, who kept trying to convince herself and others that she didn’t care about money. Sour grapes—really pathetic.
Whenever I come across highly counterintuitive claims along these lines, I code them up and see how they perform over many iterations.
This is a lot trickier to do in this case compared to, say, the Monty Hall problem, but if you restricted it just to cases in which Pirate A retained 98 of the coins, you could demonstrate whether the [98, 0, 1, 0, 1] distribution was stable or not.
Also, I’d suggest thinking about this in a slightly different way to the way you’re thinking about it. The only pirate in the scenario who doesn’t have to worry about dying is pirate E, who can make any demands he likes from pirate D. What distribution would he suggest?
Edit: Rereading the wording of the scenario, pirate E can’t make any demands he likes from pirate D, and pirate D himself also doesn’t need to worry about dying.
That shows one aspect of the consequences of reasoning via outrage. It doesn’t indicate that the strategy itself is bad. In a similar way the consequences of randomizing and defending Podunk 1⁄11 times is that 1⁄11 times (against a good player) you will end up on youtube losing Metropolis.
Whoever came up with that game show is a genius.
“I will stand firm on A until A becomes worth less than C is now, then I will accept B or C. You get more by accepting B or C now than you get by trying to get more.”
One rational course of action is to mutually commit to a random split, and follow through. What’s the rational course of action to respond to someone who makes that threat and is believed to follow through on it? If it is known that the other two participants are rational, why isn’t making a threat of that nature rational?
With the caveat that this ‘proof’ relies on the same assumptions that ‘prove’ that the rational prisoners defect in the one shot prisoners dilemma—which they don’t unless they have insufficient (or inaccurate) information about each other. At a stretch we could force the “do not trust each other” premise to include “the pirates have terrible maps of each other” but that’s not a realistic interpretation of the sentence. Really there is the additional implicit assumption “Oh, and all these pirates are agents that implement Causal Decision Theory”.
It gets even more interesting when there are more than 200 pirates (and still only 100 coins).
Actually, does the traditional result work? E would rather the result be (dead, dead, 99, 0, 1) than (98, 0, 1, 0, 1). I think it has to be (97, 0, 1, 0, 2).
[edit]It appears that E should assume that B will make a successful proposition, which will include nothing for E, and so (dead, dead, 99, 0, 1) isn’t on the table.