Did you mean an example more detailed than this, more formal than this, or does this answer succeed in connecting the terminology?
None of the above! In scenario 1, PREFIX isn’t part of the problem statement; it is part of the solution. The problem statement said only that programs can read each other’s source code. Unless I’m misunderstanding your initial comment in this thread, you want to invent a Scenario 3 with rules distinct from 1 or 2: programs are given each other’s source code and/or Something Else. Well, what is that Something Else? What are the mathematical rules of the game for your proposed Scenario 3? Or are you proposing some new player algorithms for Scenario 1 or Scenario 2 instead of making up a new game? If so, what are they? Please answer simply and concretely.
In these terms, I’m describing a solution to scenario 1, and its limitations, that is it can’t solve all scenarios 1 (naturally, at least because of the halting problem). The Something Else are the limitations, for example your solution to scenatio 1 requires Something Else in a form of a fixed prefix. This solution is not terribly general, as it won’t accept even equivalent programs that include an additional comment in the body of the prefix.
Analyzing a formal model of the program gives more generality, and stronger methods of analysis give more generality still. If you are the program for which you introduce these properties, you’d also want to pick a property that tells that you achieve your goals, or as high in your preference order as you can get.
Ah—you want to generalize the PREFIX-test, expand the equivalence class. Sorry for being obtuse in my replies—I was giving you the benefit of the doubt, because generality is so obviously, totally useless in Scenario 1. Each program author wanting to utilize the PREFIX method of cooperation has quite enough incentive to coordinate on a common value of PREFIX with other such authors. If you prohibit vocal coordination beforehand, they will use the Schelling focal point: the lexicographically smallest PREFIX possible in a given programming language.
You may consider the scenario 3, where the trusted algorithm checks that your PREFIX is present in your opponent, and only tells you that fact, without giving you its source code or a simulator black box. But anyway, this little example is not a Totally General Solution to All Similar Problems, and so generalization is not useless. It might in a given formal problem, that is already solved by other means, but not in other cases.
For example, recall my previous remarks about how PD turns into an Ultimatum game once you allow mixed strategies (that is, any form of randomness in the programs). In which case, pure ‘Cooperate’ becomes suboptimal.
And if you do that properly, taking into account other issues like reflective consistency, you won’t need the other part of your algorithm at all, since picking fair Pareto-optimal strategies will also allow to correctly deal with defectors.
I’m still perplexed by the fairness part (and Ultimatum game in particular) though. One solution from symmetry seems obvious (not ‘cut on equal parts’, since what ‘equal part’ means also depends on the preference orders of each agent), but I don’t know in what sense it’s universally right, and if it isn’t, what should be done.
None of the above! In scenario 1, PREFIX isn’t part of the problem statement; it is part of the solution. The problem statement said only that programs can read each other’s source code. Unless I’m misunderstanding your initial comment in this thread, you want to invent a Scenario 3 with rules distinct from 1 or 2: programs are given each other’s source code and/or Something Else. Well, what is that Something Else? What are the mathematical rules of the game for your proposed Scenario 3? Or are you proposing some new player algorithms for Scenario 1 or Scenario 2 instead of making up a new game? If so, what are they? Please answer simply and concretely.
In these terms, I’m describing a solution to scenario 1, and its limitations, that is it can’t solve all scenarios 1 (naturally, at least because of the halting problem). The Something Else are the limitations, for example your solution to scenatio 1 requires Something Else in a form of a fixed prefix. This solution is not terribly general, as it won’t accept even equivalent programs that include an additional comment in the body of the prefix.
Analyzing a formal model of the program gives more generality, and stronger methods of analysis give more generality still. If you are the program for which you introduce these properties, you’d also want to pick a property that tells that you achieve your goals, or as high in your preference order as you can get.
Ah—you want to generalize the PREFIX-test, expand the equivalence class. Sorry for being obtuse in my replies—I was giving you the benefit of the doubt, because generality is so obviously, totally useless in Scenario 1. Each program author wanting to utilize the PREFIX method of cooperation has quite enough incentive to coordinate on a common value of PREFIX with other such authors. If you prohibit vocal coordination beforehand, they will use the Schelling focal point: the lexicographically smallest PREFIX possible in a given programming language.
You may consider the scenario 3, where the trusted algorithm checks that your PREFIX is present in your opponent, and only tells you that fact, without giving you its source code or a simulator black box. But anyway, this little example is not a Totally General Solution to All Similar Problems, and so generalization is not useless. It might in a given formal problem, that is already solved by other means, but not in other cases.
For example, recall my previous remarks about how PD turns into an Ultimatum game once you allow mixed strategies (that is, any form of randomness in the programs). In which case, pure ‘Cooperate’ becomes suboptimal.
Then just change PREFIX to play the Pareto-optimal symmetric mixed strategy instead of cooperating.
And if you do that properly, taking into account other issues like reflective consistency, you won’t need the other part of your algorithm at all, since picking fair Pareto-optimal strategies will also allow to correctly deal with defectors.
I’m still perplexed by the fairness part (and Ultimatum game in particular) though. One solution from symmetry seems obvious (not ‘cut on equal parts’, since what ‘equal part’ means also depends on the preference orders of each agent), but I don’t know in what sense it’s universally right, and if it isn’t, what should be done.