A while ago I described something similar on the decision-theory-workshop mailing list, with two differences (I think they are improvements but YMMV):
1) It’s easier to use UDT than VNM, because UDT agents have no beliefs, only a utility function that mixes beliefs with values (e.g. if you think a coin is/was fair, your utility is the average of your utilities in heads-world and tails-world). When two agents merge, you just take a weighted sum of their utility functions.
2) In some games, specifying the weighted sum is not enough. For example, in the dividing the dollar game, maximizing any weighted sum will give the whole dollar to one player, except the equally weighted sum which is indifferent. To achieve e.g. an equal division of the dollar, the agents need to jointly observe a coinflip while constructing the merged agent, or more generally to agree on a probability distribution over merged agents (with the restriction that it must lie on the Pareto frontier).
A while ago I described something similar on the decision-theory-workshop mailing list, with two differences (I think they are improvements but YMMV):
1) It’s easier to use UDT than VNM, because UDT agents have no beliefs, only a utility function that mixes beliefs with values (e.g. if you think a coin is/was fair, your utility is the average of your utilities in heads-world and tails-world). When two agents merge, you just take a weighted sum of their utility functions.
2) In some games, specifying the weighted sum is not enough. For example, in the dividing the dollar game, maximizing any weighted sum will give the whole dollar to one player, except the equally weighted sum which is indifferent. To achieve e.g. an equal division of the dollar, the agents need to jointly observe a coinflip while constructing the merged agent, or more generally to agree on a probability distribution over merged agents (with the restriction that it must lie on the Pareto frontier).