Thanks for taking the time to think about this! But it seems to me that Löb’s theorem should not be required to prove proposition 3 for a properly defined A. I’m guessing that you inadvertently defined A asymmetrically, as described in “AI cooperation in practice”, instead of the symmetric way described in “model of UDT with a halting oracle” where swapping C and D shouldn’t affect anything except the chicken stage.
Ah, I see. The agent described in “A model of UDT with a halting oracle” is more symmetric, because it has a choice of three actions: “Cooperate”, “Defect”, and “Break down and cry” (if we count failing to act as an action).
I think I see a way to prove the proposition without using Löb’s theorem, and I don’t even need to change the definition of A. I’ll post here whether it works out.
Thanks for taking the time to think about this! But it seems to me that Löb’s theorem should not be required to prove proposition 3 for a properly defined A. I’m guessing that you inadvertently defined A asymmetrically, as described in “AI cooperation in practice”, instead of the symmetric way described in “model of UDT with a halting oracle” where swapping C and D shouldn’t affect anything except the chicken stage.
Ah, I see. The agent described in “A model of UDT with a halting oracle” is more symmetric, because it has a choice of three actions: “Cooperate”, “Defect”, and “Break down and cry” (if we count failing to act as an action).
I think I see a way to prove the proposition without using Löb’s theorem, and I don’t even need to change the definition of A. I’ll post here whether it works out.
Thanks for the comment!
Did it work out?
I still intend to write up an answer to that question.
Great!
I finally did it, and I believe it works.