Alice: I want you to change to match me: conditional on your utility function being the same as mine, my expected utility would be larger.
Bob: Yes, that follows from me being a utility maximizer
Bob: I won’t change my utility function: conditional on my utility function becoming the same as yours, my expected utility as measured by my current utility function would be lower.
Alice: Yes, that follows from you being a utility maximizer
If Bob isn’t reflectively consistent, their utility functions could currently be the same in some sense, right? (They might agree on what Bob’s utility function should be—Bob would happily press a button that makes him want to donate 30%, he just doesn’t currently want to do that and doesn’t think he has access to such a button.)
Certainly! Most likely, neither of them is reflectively consistent: “I feel like I’d find it easier to be motivated and consistent if my brain wasn’t constantly looking at you and reminding me that I totally could have a cushy life like yours if I just stopped living my values.” hints at this.
Alice: Our utility functions differ.
Bob: I also observe this.
Alice: I want you to change to match me: conditional on your utility function being the same as mine, my expected utility would be larger.
Bob: Yes, that follows from me being a utility maximizer
Bob: I won’t change my utility function: conditional on my utility function becoming the same as yours, my expected utility as measured by my current utility function would be lower.
Alice: Yes, that follows from you being a utility maximizer
If Bob isn’t reflectively consistent, their utility functions could currently be the same in some sense, right? (They might agree on what Bob’s utility function should be—Bob would happily press a button that makes him want to donate 30%, he just doesn’t currently want to do that and doesn’t think he has access to such a button.)
Certainly! Most likely, neither of them is reflectively consistent: “I feel like I’d find it easier to be motivated and consistent if my brain wasn’t constantly looking at you and reminding me that I totally could have a cushy life like yours if I just stopped living my values.” hints at this.
yes, definitely!