I feel like many game/decision theoretic claims are most easily grasped when looking at the iterated setup:
Example 1. When one first sees the prisoner’s dilemma, the argument that “you should defect because of whatever the other person does, you are better off by defecting” feels compelling. The counterargument goes “the other person can predict what you’ll do, and this can affect what they’ll play”.
This has some force, but I have had a hard time really feeling the leap from “you are a person who does X in the dilemma” to “the other person models you as doing X in the dilemma”. (One thing that makes this difficult that usually in PD it is not specified whether the players can communicate beforehand or what information they have of each other.) And indeed, humans models’ of other humans are limited—this is not something you should just dismiss.
However, the point “the Nash equilibrium is not necessarily what you should play” does hold, as is illustrated by the iterated Prisoner’s dilemma. It feels intuitively obvious that in a 100-round dilemma there ought to be something better than always defecting.
This is among the strongest intuitions I have for “Nash equilibria do not generally describe optimal solutions”.
Example 2. When presented with lotteries, i.e. opportunities such as “X% chance you win A dollars, (100-X)% chance of winning B dollars”, it’s not immediately obvious that one should maximize expected value (or, at least, humans generally exhibit loss aversion, bias towards certain outcomes, sensitivity to framing etc.).
This feels much clearer when given the option to choose between lotteries repeatedly. For example, if you are presented with the two buttons, one giving you a sure 100% chance of winning 1 dollar and the other one giving you a 40% chance of winning 3 dollars, and you are allowed to press the buttons a total of 100 times, it feels much clearer that you should always pick the one with the highest expected value. Indeed, as you are given more button presses, the probability of you getting (a lot) more money that way tends to 1 (by the law of large numbers).
This gives me a strong intuition that expected values are the way to go.
Example 3. I find Newcomb’s problem a bit confusing to think about (and I don’t seem to be alone in this). This is, however, more or less the same problem as prisoner’s dilemma, so I’ll be brief here.
The basic argument “the contents of the boxes have already been decided, so you should two-box” feel compelling, but then you realize that in an iterated Newcomb’s problem you will, by backward induction, always two-box.
This, in turn, sounds intuitively wrong, in which case the original argument proves too much.
One thing I like about iteration is that it makes the concept of “”it really is possible to make predictions about your actions” feel more plausible: there’s clear-cut information about what kind of plays you’ll make, namely the previous rounds. I feel like in my thoughts I sometimes feel like rejecting the premise, or thinking that “sure, if the premise holds, I should one-box, but it doesn’t really work that way in real life, this feels like one of those absurd thought experiments that don’t actually teach you anything”. Iteration solves this issue.
Another pump I like is “how many iterations do there need to be before you Cooperate/maximize-expected-value/one-box?”. There (I think) is some number of iterations for this to happen, and, given that, it feels like “1″ is often the best answer.
All that said, I don’t think iterations provide the Real Argument for/against the position presented. There’s always some wiggle room for “but what if you are not in an iterated scenario, what if this truly is a Unique Once-In-A-Lifetime Opportunity?”. I think the Real Arguments are something else—e.g. in example 2 I think coherence theorems give a stronger case (even if I still don’t feel them as strongly on an intuitive level). I don’t think I know the Real Argument for example 1⁄3.
Iteration as an intuition pump
I feel like many game/decision theoretic claims are most easily grasped when looking at the iterated setup:
Example 1. When one first sees the prisoner’s dilemma, the argument that “you should defect because of whatever the other person does, you are better off by defecting” feels compelling. The counterargument goes “the other person can predict what you’ll do, and this can affect what they’ll play”.
This has some force, but I have had a hard time really feeling the leap from “you are a person who does X in the dilemma” to “the other person models you as doing X in the dilemma”. (One thing that makes this difficult that usually in PD it is not specified whether the players can communicate beforehand or what information they have of each other.) And indeed, humans models’ of other humans are limited—this is not something you should just dismiss.
However, the point “the Nash equilibrium is not necessarily what you should play” does hold, as is illustrated by the iterated Prisoner’s dilemma. It feels intuitively obvious that in a 100-round dilemma there ought to be something better than always defecting.
This is among the strongest intuitions I have for “Nash equilibria do not generally describe optimal solutions”.
Example 2. When presented with lotteries, i.e. opportunities such as “X% chance you win A dollars, (100-X)% chance of winning B dollars”, it’s not immediately obvious that one should maximize expected value (or, at least, humans generally exhibit loss aversion, bias towards certain outcomes, sensitivity to framing etc.).
This feels much clearer when given the option to choose between lotteries repeatedly. For example, if you are presented with the two buttons, one giving you a sure 100% chance of winning 1 dollar and the other one giving you a 40% chance of winning 3 dollars, and you are allowed to press the buttons a total of 100 times, it feels much clearer that you should always pick the one with the highest expected value. Indeed, as you are given more button presses, the probability of you getting (a lot) more money that way tends to 1 (by the law of large numbers).
This gives me a strong intuition that expected values are the way to go.
Example 3. I find Newcomb’s problem a bit confusing to think about (and I don’t seem to be alone in this). This is, however, more or less the same problem as prisoner’s dilemma, so I’ll be brief here.
The basic argument “the contents of the boxes have already been decided, so you should two-box” feel compelling, but then you realize that in an iterated Newcomb’s problem you will, by backward induction, always two-box.
This, in turn, sounds intuitively wrong, in which case the original argument proves too much.
One thing I like about iteration is that it makes the concept of “”it really is possible to make predictions about your actions” feel more plausible: there’s clear-cut information about what kind of plays you’ll make, namely the previous rounds. I feel like in my thoughts I sometimes feel like rejecting the premise, or thinking that “sure, if the premise holds, I should one-box, but it doesn’t really work that way in real life, this feels like one of those absurd thought experiments that don’t actually teach you anything”. Iteration solves this issue.
Another pump I like is “how many iterations do there need to be before you Cooperate/maximize-expected-value/one-box?”. There (I think) is some number of iterations for this to happen, and, given that, it feels like “1″ is often the best answer.
All that said, I don’t think iterations provide the Real Argument for/against the position presented. There’s always some wiggle room for “but what if you are not in an iterated scenario, what if this truly is a Unique Once-In-A-Lifetime Opportunity?”. I think the Real Arguments are something else—e.g. in example 2 I think coherence theorems give a stronger case (even if I still don’t feel them as strongly on an intuitive level). I don’t think I know the Real Argument for example 1⁄3.