“For every decision-making process X for which there is a proof in PA that my opponent cooperates against X, if X defects against my opponent, (implement X; break) if no X defects but some exist, implement any X. If no X exists, defect.”
This is basically the idea behind the earlier Masquerade algorithm. The first tricky bit is that your opponent is under no obligation to treat you the same way that it treats X. The second tricky bit is that you can’t search over all X within any formal system, because the class of cooperators is undecidable, and the more you search over, the stronger a formal system your opponent needs to figure out what you’re actually doing.
(It’s totally worth thinking about these things, of course!)
Yeah, I realized that I was talking about behavior, but that my opponent isn’t limited to my behavior and can make decisions based on my nature. I was trying to design a program using sufficiently advanced magic that would defeat the best program I think I can create, and I succeeded. However, it isn’t competitive against other programs which operate on the same level.
It does cooperate with programs one level higher than it, but those programs defect from it.
“For every decision-making process X for which there is a proof in PA that my opponent cooperates against X, if X defects against my opponent, (implement X; break) if no X defects but some exist, implement any X. If no X exists, defect.”
Reflective consistency and self-modifying FTW.
Not sure if that’s computable in finite time.
This is basically the idea behind the earlier Masquerade algorithm. The first tricky bit is that your opponent is under no obligation to treat you the same way that it treats X. The second tricky bit is that you can’t search over all X within any formal system, because the class of cooperators is undecidable, and the more you search over, the stronger a formal system your opponent needs to figure out what you’re actually doing.
(It’s totally worth thinking about these things, of course!)
Yeah, I realized that I was talking about behavior, but that my opponent isn’t limited to my behavior and can make decisions based on my nature. I was trying to design a program using sufficiently advanced magic that would defeat the best program I think I can create, and I succeeded. However, it isn’t competitive against other programs which operate on the same level.
It does cooperate with programs one level higher than it, but those programs defect from it.