The issue with you-in-all-detail vs. your-decision-algorithm is that a decision algorithm can have different levels of updatelessness, it’s unclear what the decision algorithm already knows vs. what a policy it chooses takes as input. So we pick some intermediate level that is updateless enough to allow acausal coordination among relevant entities (agents/predictors), and updateful enough to make a decision without running out of time/memory while being implemented in its instances. But that level/scope is different for different collections of entities being coordinated.
So I think a boundary shouldn’t be drawn around “a decision algorithm”, but around whatever common knowledge of each other the entities being acausally coordinated happen to have (where they don’t need to have common knowledge of everything). When packaged as a decision algorithm, the common knowledge becomes an adjudicator, which these entities can allow influence over their actions. To the extent the influence they allow an adjudicator is common knowledge among them, it also becomes knowledge of the adjudicator, available for its decision making reasoning.
Importantly for the reframing, an adjudicator is not a decision algorithm belonging to either agent individually, it’s instead a shared decision algorithm. It’s a single decision algorithm purposefully built out of the agents’ common knowledge of each other, rather than a collection of their decision algorithms that luckily happen to have common knowledge of each other. It’s much easier for there to be some common knowledge than for there to be common knowledge of individually predefined decision algorithms that each agent follows.
The issue with you-in-all-detail vs. your-decision-algorithm is that a decision algorithm can have different levels of updatelessness, it’s unclear what the decision algorithm already knows vs. what a policy it chooses takes as input. So we pick some intermediate level that is updateless enough to allow acausal coordination among relevant entities (agents/predictors), and updateful enough to make a decision without running out of time/memory while being implemented in its instances. But that level/scope is different for different collections of entities being coordinated.
So I think a boundary shouldn’t be drawn around “a decision algorithm”, but around whatever common knowledge of each other the entities being acausally coordinated happen to have (where they don’t need to have common knowledge of everything). When packaged as a decision algorithm, the common knowledge becomes an adjudicator, which these entities can allow influence over their actions. To the extent the influence they allow an adjudicator is common knowledge among them, it also becomes knowledge of the adjudicator, available for its decision making reasoning.
Importantly for the reframing, an adjudicator is not a decision algorithm belonging to either agent individually, it’s instead a shared decision algorithm. It’s a single decision algorithm purposefully built out of the agents’ common knowledge of each other, rather than a collection of their decision algorithms that luckily happen to have common knowledge of each other. It’s much easier for there to be some common knowledge than for there to be common knowledge of individually predefined decision algorithms that each agent follows.