jessicata comments on Reflexive Oracles and superrationality: prisoner’s dilemma

[jessicata]

I think it would help to explain how this works for prisoner’s dilemma specifically. This is NicerBot:

def NicerBot(opponent):
    with probability epsilon:
        cooperate
    otherwise:
        cooperate with the same probability that the opponent does (by querying the oracle)

It’s easy to see that NicerBot(NicerBot) will always cooperate, while NicerBot(DefectBot) will defect with $1-ϵ$ probability. This seems like the correct analogue of FairBot in reflective oracle land.

[Vanessa Kosoy]

Note that the NicerBot is exactly what you get from the threat game formalism. Specifically, see the two equations near the end.

[Caspar Oesterheld]

My paper “Robust program equilibrium” (published in Theory and Decision) discusses essentially NicerBot (under the name ϵGroundedFairBot) and mentions Jessica’s comment in footnote 3. More generally, the paper takes strategies from iterated games and transfers them into programs for the corresponding program game. As one example, tit for tat in the iterated prisoner’s dilemma gives rise to NicerBot in the “open-source prisoner’s dilemma”.

[Nisan]

See also this comment from 2013 that has the computable version of NicerBot.

[Stuart_Armstrong]

Yep! I try and not make use of symmetry when I can avoid it, to make the result more general.

Note that NicerBot is neLP() with the allowable area being half of the output area (the half delimited by CC, DD and DC).