I had a thought related to this (apologies if this has been bought up on LW before, but I haven’t seen it exactly).
Let’s say you’re playing a game theory game, and you have access to an oracle that either answers correctly or refrains from answering.
Obviously in a lot of games it makes sense to ask what move your opponent will make, so you can then optimise your own move. But in the Prisoner’s Dilemma, the opponent’s move is useless information—whatever you learn, your optimal move is to defect.
What you want to ask is not “what move will he make ?”, but “will he make the same move as me ?”. Learning which row or column you’re in doesn’t help; learning whether you’re on-diagonal or off-diagonal does.
This is approximated by the strategy called Pavlov: “Cooperate on the first turn; Cooperate if our previous-turn moves were the same; Defect if they were different.”
I had a thought related to this (apologies if this has been bought up on LW before, but I haven’t seen it exactly).
Let’s say you’re playing a game theory game, and you have access to an oracle that either answers correctly or refrains from answering.
Obviously in a lot of games it makes sense to ask what move your opponent will make, so you can then optimise your own move. But in the Prisoner’s Dilemma, the opponent’s move is useless information—whatever you learn, your optimal move is to defect.
What you want to ask is not “what move will he make ?”, but “will he make the same move as me ?”. Learning which row or column you’re in doesn’t help; learning whether you’re on-diagonal or off-diagonal does.
This is approximated by the strategy called Pavlov: “Cooperate on the first turn; Cooperate if our previous-turn moves were the same; Defect if they were different.”
This is a really nice way of looking at it.