A related thing that came up in our discussion after I wrote this post is how to apply the Vow of Concord in the face of utility functions that change over time. The amendment we tentatively agreed on is: if the utility functions change, we do new KS bargaining where the disagreement point is the policy that resulted from the previous bargaining. This choice of disagreement point avoids perverse incentives to change your own utility function.
On a more pedestrian note, I was previously married for 10 years, so I’m not completely naïve in that regard.
Committing to a decision algorithm now implies that you expect to do worse in the future. Even though future you will have more information and experience. And, as you noted, potentially a different utility function.
And, as a practical matter, are you even capable of making decisions as-if you were yourself in the past?
That’s why I wrote “in the counterfactual in which the source of said doubt or dispute would be revealed to us and understood by us with all of its implications at that time as well as we understand it at the time it actually surfaced”, so we do use the new information and experience.
The reason I want to anchor it to our present selves is because at present we are fairly aligned. We have pretty good common understanding of what we want form these vows. On the other hand, the appearance of a dispute in the future might be the result of us becoming unaligned. This would create the temptation for each of us to interpret the vows in their own favor. The wedding-time anchor mitigates that somewhat, because it requires us to argue with a straight face that our wedding-time-selves would endorse the given interpretation.
A related thing that came up in our discussion after I wrote this post is how to apply the Vow of Concord in the face of utility functions that change over time.
That seems like a very important point. Also, you may end up living for more than a billion years (via future technology). The fraction of your future life in which your ~preferences/goal-system will be similar to your current ones may be extremely small.
Why impossible or undesirable?
A related thing that came up in our discussion after I wrote this post is how to apply the Vow of Concord in the face of utility functions that change over time. The amendment we tentatively agreed on is: if the utility functions change, we do new KS bargaining where the disagreement point is the policy that resulted from the previous bargaining. This choice of disagreement point avoids perverse incentives to change your own utility function.
On a more pedestrian note, I was previously married for 10 years, so I’m not completely naïve in that regard.
Committing to a decision algorithm now implies that you expect to do worse in the future. Even though future you will have more information and experience. And, as you noted, potentially a different utility function. And, as a practical matter, are you even capable of making decisions as-if you were yourself in the past?
That’s why I wrote “in the counterfactual in which the source of said doubt or dispute would be revealed to us and understood by us with all of its implications at that time as well as we understand it at the time it actually surfaced”, so we do use the new information and experience.
The reason I want to anchor it to our present selves is because at present we are fairly aligned. We have pretty good common understanding of what we want form these vows. On the other hand, the appearance of a dispute in the future might be the result of us becoming unaligned. This would create the temptation for each of us to interpret the vows in their own favor. The wedding-time anchor mitigates that somewhat, because it requires us to argue with a straight face that our wedding-time-selves would endorse the given interpretation.
That seems like a very important point. Also, you may end up living for more than a billion years (via future technology). The fraction of your future life in which your ~preferences/goal-system will be similar to your current ones may be extremely small.