A related thing that came up in our discussion after I wrote this post is how to apply the Vow of Concord in the face of utility functions that change over time.
That seems like a very important point. Also, you may end up living for more than a billion years (via future technology). The fraction of your future life in which your ~preferences/​goal-system will be similar to your current ones may be extremely small.
That seems like a very important point. Also, you may end up living for more than a billion years (via future technology). The fraction of your future life in which your ~preferences/​goal-system will be similar to your current ones may be extremely small.