I think a somewhat-more-elegant toy model might look something like the following: Alice’s object-level preferences are UA, and Beth’s are UB. Alice’s all-things-considered preferences are UA+αU′B, and Beth’s are UB+βU′A. Here, U′A & U′B represent Beth’s current beliefs about Alice’s desires and vice-versa, and the α,β parameters represent how much Alice cares about Beth’s object-level desires and vice-versa. The latter could arise from admiration of the other person, fear of pissing them off, or various other considerations discussed in the next post.
I think that the most general model would be UA+ft(U′B) and UB+gt(U′A) where At,Bt are time-dependent (or past-interaction-dependent, or status-dependent; these are all fundamentally the same thing). It’s a notable feature that this model does not assume that the functions are monotonic! I suspect that most people are willing to compromise somewhat on their preferences but become less willing to do so when the other party begins to resemble a utility monster, and notably covers the situation where the weight is negative under some but not all circumstances.
I think that the most general model would be UA+ft(U′B) and UB+gt(U′A) where At,Bt are time-dependent (or past-interaction-dependent, or status-dependent; these are all fundamentally the same thing). It’s a notable feature that this model does not assume that the functions are monotonic! I suspect that most people are willing to compromise somewhat on their preferences but become less willing to do so when the other party begins to resemble a utility monster, and notably covers the situation where the weight is negative under some but not all circumstances.