Two comments to this: 1) The scenario described here is a Nash equilibrium but not a subgame-perfect Nash equilibrium. (IE, there are counterfactual parts of the game tree where the players behave “irationally”.) Note that subgame-perfection is orthogonal with “reasonable policy to have”, so the argument “yeah, clearly the solution is to always require subgame-perfection” does not work. (Why is it orthogonal? First, the example from the post shows a “stupid” policy that isn’t subgame-perfect. However, there are cases where subgame-imperfection seems smart, because it ensures that those counterfactual situations don’t become reality. EG, humans are somewhat transparent to each other, so having the policy of refusing unfair splits in the Final Offer / Ultimatum game can lead to not being offered unfair splits in the first place.)
2) You could modify the scenario such that the “99 equilibrium” becomes more robust. (EG, suppose the players have a way of paying a bit to punish a specific player a lot. Then you add the norm of turning temperature to 99, the meta-norm of punishing defectors, the meta-meta-norm of punishing those who don’t punish defectors, etc. And tadaaaa, you have a pretty robust hell. This is a part of how society actually works, except usually those norms typically enforce pro-social behaviour.)
Two comments to this:
1) The scenario described here is a Nash equilibrium but not a subgame-perfect Nash equilibrium. (IE, there are counterfactual parts of the game tree where the players behave “irationally”.) Note that subgame-perfection is orthogonal with “reasonable policy to have”, so the argument “yeah, clearly the solution is to always require subgame-perfection” does not work. (Why is it orthogonal? First, the example from the post shows a “stupid” policy that isn’t subgame-perfect. However, there are cases where subgame-imperfection seems smart, because it ensures that those counterfactual situations don’t become reality. EG, humans are somewhat transparent to each other, so having the policy of refusing unfair splits in the Final Offer / Ultimatum game can lead to not being offered unfair splits in the first place.)
2) You could modify the scenario such that the “99 equilibrium” becomes more robust. (EG, suppose the players have a way of paying a bit to punish a specific player a lot. Then you add the norm of turning temperature to 99, the meta-norm of punishing defectors, the meta-meta-norm of punishing those who don’t punish defectors, etc. And tadaaaa, you have a pretty robust hell. This is a part of how society actually works, except usually those norms typically enforce pro-social behaviour.)