Regret of rationality in games isn’t a mysterious phenomenon. Let’s suppose that after the one round of PD we’re going to play I have the power to destroy a billion paperclips at the cost of one human life, and Clippy knows that. If Clippy thinks I’m a rational outcome-maximizer, then he knows that whatever threats I make I’m not going to carry out, because they won’t have any payoffs when the time comes. But if it thinks I’m prone to irrational emotional reactions, it might conclude I’ll carry out my billion-paperclip threat if it defects, and so cooperate.
Regret of rationality in games isn’t a mysterious phenomenon. Let’s suppose that after the one round of PD we’re going to play I have the power to destroy a billion paperclips at the cost of one human life, and Clippy knows that. If Clippy thinks I’m a rational outcome-maximizer, then he knows that whatever threats I make I’m not going to carry out, because they won’t have any payoffs when the time comes. But if it thinks I’m prone to irrational emotional reactions, it might conclude I’ll carry out my billion-paperclip threat if it defects, and so cooperate.