I mostly generally agree. And when there are multiple actions a procedure needs to determine, this sort of thing is much less of an issue.
My question is about getting one bit of random information for a one-off randomized decision to split indivisible gains of trade. I assume both agents are capable of fully reading and understanding each other’s code, but they don’t know why the other agent exists and whether they could’ve been designed adversarily. My question here is, can these agents, despite that (and without relying on the other agent being created by an agent that wishes to cooperate), produce one bit of information that they know to not have been manipulated to benefit either of the agents?
The other known-code agent is only an ingredient in defining the outcome you care about (the source code of the world), but not on its own a salient thing when considering coordination. Acausal coordination acts in a broader way, potential contracts you might decide to sign (and so introduce as processes with influence over the a priori defined outcome, the result of computing the code of the world) are not restricted by what’s already explicitly in the initial code of either yourself or your opponents. Coordination here refers to how a contract acts in the same way in all places where it has influence, since it’s the same thing in all these places, and in particular the same contract can act through actions of multiple agents, coordinating them. But contracts are new programs chosen by the players as they engage in setting up coordination between each other, they are not the initial players themselves.
So the distinction between the other agent and its parent rigging the outcome shouldn’t help. Even though both of these are technically potential contracts and could be involved in coordination, both are obviously bad choices for contracts. You wouldn’t want to directly give your opponent control over your own action, hence your opponent shouldn’t be an acausal contract that coordinates your action. Similarly for your opponent’s parent, by proxy of acting through your opponent’s code, it’s not a good contract to grant influence with the outcome through your action.
I don’t know how to solve this puzzle, it seems potentially important if it can be solved. What comes to mind is picking an obvious Schelling point like the procedure of taking the hash of the agent codes concatenated in ascending order as natural numbers, using it as a contract. Parents of agents might try to manipulate behavior of such a contract (by rigging otherwise behaviorally irrelevant details of codes of the agents). Or they might manipulate the choice of a contract (by rigging agents’ intuition for which procedure would be a Schelling point). But that is symmetric and the parents have an incentive to let the agents settle on some procedure rather than to never agree, and so the outcome remains pseudorandom. Which also incentivises the parents to expend less resources on competing over it.
I mostly generally agree. And when there are multiple actions a procedure needs to determine, this sort of thing is much less of an issue.
My question is about getting one bit of random information for a one-off randomized decision to split indivisible gains of trade. I assume both agents are capable of fully reading and understanding each other’s code, but they don’t know why the other agent exists and whether they could’ve been designed adversarily. My question here is, can these agents, despite that (and without relying on the other agent being created by an agent that wishes to cooperate), produce one bit of information that they know to not have been manipulated to benefit either of the agents?
The other known-code agent is only an ingredient in defining the outcome you care about (the source code of the world), but not on its own a salient thing when considering coordination. Acausal coordination acts in a broader way, potential contracts you might decide to sign (and so introduce as processes with influence over the a priori defined outcome, the result of computing the code of the world) are not restricted by what’s already explicitly in the initial code of either yourself or your opponents. Coordination here refers to how a contract acts in the same way in all places where it has influence, since it’s the same thing in all these places, and in particular the same contract can act through actions of multiple agents, coordinating them. But contracts are new programs chosen by the players as they engage in setting up coordination between each other, they are not the initial players themselves.
So the distinction between the other agent and its parent rigging the outcome shouldn’t help. Even though both of these are technically potential contracts and could be involved in coordination, both are obviously bad choices for contracts. You wouldn’t want to directly give your opponent control over your own action, hence your opponent shouldn’t be an acausal contract that coordinates your action. Similarly for your opponent’s parent, by proxy of acting through your opponent’s code, it’s not a good contract to grant influence with the outcome through your action.
I don’t know how to solve this puzzle, it seems potentially important if it can be solved. What comes to mind is picking an obvious Schelling point like the procedure of taking the hash of the agent codes concatenated in ascending order as natural numbers, using it as a contract. Parents of agents might try to manipulate behavior of such a contract (by rigging otherwise behaviorally irrelevant details of codes of the agents). Or they might manipulate the choice of a contract (by rigging agents’ intuition for which procedure would be a Schelling point). But that is symmetric and the parents have an incentive to let the agents settle on some procedure rather than to never agree, and so the outcome remains pseudorandom. Which also incentivises the parents to expend less resources on competing over it.