If I think about how good the consequences of an action are, I try to think about what I expect to happen if I take that action (ie the outcome), and I think about how likely that outcome is to have various properties that I care about, since I don’t know exactly what the outcome will be with certainty… I need to consider how good and how likely various consequences are, and take the expectation of the ‘how good’ with respect to the ‘how likely’.
I don’t understand JB yet, but when I introspected just now, my experience of decision-making doesn’t have any separation between beliefs and values, so I think I disagree with the above. I’ll try to explain why by describing my experience. (Note: Long comment below is just saying one very simple thing. Sorry for length. There’s a one-line tl;dr at the end.)
Right now I’m considering doing three different things. I can go and play a videogame that my friend suggested we play together, I can do some LW work with my colleague, or I can go play some guitar/piano. I feel like the videogame isn’t very fun right now because I think the one my friend suggested not that interesting of a shared experience. I feel like the work is fun because I’m excited about publishing the results of the work, and the work itself involves a kind of cognition I enjoy. And playing piano is fun because I’ve been skilling up a lot lately and I’m going to do accompany some of my housemates in some hamilton songs.
Now, I know some likely ways that what seems valuable to me might change. There are other videogames I’ve played lately that have been really fascinating and rewarding to play together, that involve problem solving where 2 people can be creative together. I can imagine the work turning out to not actuallybe the fun part but the boring parts. I can imagine that I’ve found no traction (skill-up) in playing piano, or that we’re going to use a recorded soundtrack rather than my playing for the songs we’re learning.
All of these to me feel like updates in my understanding of what events are reachable to me; this doesn’t feel like changing my utility evaluation of the events. The event of “play videogame while friend watches bored” could change to “play videogame while creatively problem-solving with friend”. The event of “gain skill in piano and then later perform songs well with friends” could change to “struggle to do something difficult and sound bad and that’s it”.
If I think about changing my utility function, I expect that would feel more like… well, I’m not sure. My straw version is “I creatively solve problems with my friend on a videogame, but somehow that’s objectively bad so I will not do it”. That’s where some variable in the utility function changed while all the rest of the facts about my psychology and reality stay the same. This doesn’t feel to me like my regular experience of decision-making.
But, maybe that’s not the idea. The idea is like if I had some neurological change, perhaps I become more of a sociopath and stop feeling empathy and everyone just feels like objects to me rather than alive. Then a bunch of the social experiences above would change, they’d lose any experience of things like vicarious enjoyment and pleasure of bonding with friends. Perhaps that’s what VNM is talking about in my experience.
I think that some of the standard “updates to my ethics / utility function” ideas that people discuss often don’t feel like this to me. Like, some people say that reflecting onf population ethics leads them to change their utility function and start to care about the far future. That’s not my experience – for me it’s been things like the times in HPMOR when Harry thinks about civilizations of the future, what they’ll be like/think, and how awesome they can be. It feels real to me, like a reachable state, and this is what has changed a lot of my behaviour, in contrast with changing some variable in a function of world-states that’s independent from my understanding of what events are achievable.
To be clear, sometimes I describe my experience more like the sociopath example, where my fundamental interests/values change. I say things like “I don’t enjoy videogames as much as I used to” or “These days I value honesty and reliability a lot more than politeness”, and there is a sense there where I now experience the same events very differently. “I had a positive meeting with John” might now be “I feel like he was being evasive about the topic we were discussing”. The things that are salient to me change. And I think that the language of “my values have changed” is often an effective one for communicating that – even if my experience does not match beliefs|utility, any sufficiently coherent agent can be described this way, and it is often easy to help others model me by describing my values as having changed.
But I think my internal experience is more that I made substantial updates about what events I’m moving towards, and the event “We had a pleasant interaction which will lead to use working effectively together” has changed to “We were not able to say the possibly unwelcome facts of the matter, which will lead to a world where we don’t work effectively together”. So internally it feels like an update about what events are reachable, even though someone from the outside who doesn’t understand my internal experience might more naturally say “It seems like Ben is treating the same event differently now, so I’ll model him as having changed his values”.
tl;dr: While I often talk separately about what actions I/you/we could take and how valuable those actions are are, internally when when I’m ‘evaluating’ the actions, I’m just trying to visualise what they are, and there is no second step of running my utility function on those visualisations.
As I say, I’m not sure I understand JB, so perhaps this is also inconsistent with it. I just read your comment and noticed it didn’t match my own introspective experience, so I thought I’d share my experience.
I agree that the considerations you mentioned in your example are not changes in values, and didn’t mean to imply that that sort of thing is a change in values. Instead, I just meant that such shifts in expectations are changes in probability distributions, rather than changes in events, since I think of such things in terms of how likely each of the possible outcomes are, rather than just which outcomes are possible and which are ruled out.
I don’t understand JB yet, but when I introspected just now, my experience of decision-making doesn’t have any separation between beliefs and values, so I think I disagree with the above. I’ll try to explain why by describing my experience. (Note: Long comment below is just saying one very simple thing. Sorry for length. There’s a one-line tl;dr at the end.)
Right now I’m considering doing three different things. I can go and play a videogame that my friend suggested we play together, I can do some LW work with my colleague, or I can go play some guitar/piano. I feel like the videogame isn’t very fun right now because I think the one my friend suggested not that interesting of a shared experience. I feel like the work is fun because I’m excited about publishing the results of the work, and the work itself involves a kind of cognition I enjoy. And playing piano is fun because I’ve been skilling up a lot lately and I’m going to do accompany some of my housemates in some hamilton songs.
Now, I know some likely ways that what seems valuable to me might change. There are other videogames I’ve played lately that have been really fascinating and rewarding to play together, that involve problem solving where 2 people can be creative together. I can imagine the work turning out to not actuallybe the fun part but the boring parts. I can imagine that I’ve found no traction (skill-up) in playing piano, or that we’re going to use a recorded soundtrack rather than my playing for the songs we’re learning.
All of these to me feel like updates in my understanding of what events are reachable to me; this doesn’t feel like changing my utility evaluation of the events. The event of “play videogame while friend watches bored” could change to “play videogame while creatively problem-solving with friend”. The event of “gain skill in piano and then later perform songs well with friends” could change to “struggle to do something difficult and sound bad and that’s it”.
If I think about changing my utility function, I expect that would feel more like… well, I’m not sure. My straw version is “I creatively solve problems with my friend on a videogame, but somehow that’s objectively bad so I will not do it”. That’s where some variable in the utility function changed while all the rest of the facts about my psychology and reality stay the same. This doesn’t feel to me like my regular experience of decision-making.
But, maybe that’s not the idea. The idea is like if I had some neurological change, perhaps I become more of a sociopath and stop feeling empathy and everyone just feels like objects to me rather than alive. Then a bunch of the social experiences above would change, they’d lose any experience of things like vicarious enjoyment and pleasure of bonding with friends. Perhaps that’s what VNM is talking about in my experience.
I think that some of the standard “updates to my ethics / utility function” ideas that people discuss often don’t feel like this to me. Like, some people say that reflecting onf population ethics leads them to change their utility function and start to care about the far future. That’s not my experience – for me it’s been things like the times in HPMOR when Harry thinks about civilizations of the future, what they’ll be like/think, and how awesome they can be. It feels real to me, like a reachable state, and this is what has changed a lot of my behaviour, in contrast with changing some variable in a function of world-states that’s independent from my understanding of what events are achievable.
To be clear, sometimes I describe my experience more like the sociopath example, where my fundamental interests/values change. I say things like “I don’t enjoy videogames as much as I used to” or “These days I value honesty and reliability a lot more than politeness”, and there is a sense there where I now experience the same events very differently. “I had a positive meeting with John” might now be “I feel like he was being evasive about the topic we were discussing”. The things that are salient to me change. And I think that the language of “my values have changed” is often an effective one for communicating that – even if my experience does not match beliefs|utility, any sufficiently coherent agent can be described this way, and it is often easy to help others model me by describing my values as having changed.
But I think my internal experience is more that I made substantial updates about what events I’m moving towards, and the event “We had a pleasant interaction which will lead to use working effectively together” has changed to “We were not able to say the possibly unwelcome facts of the matter, which will lead to a world where we don’t work effectively together”. So internally it feels like an update about what events are reachable, even though someone from the outside who doesn’t understand my internal experience might more naturally say “It seems like Ben is treating the same event differently now, so I’ll model him as having changed his values”.
tl;dr: While I often talk separately about what actions I/you/we could take and how valuable those actions are are, internally when when I’m ‘evaluating’ the actions, I’m just trying to visualise what they are, and there is no second step of running my utility function on those visualisations.
As I say, I’m not sure I understand JB, so perhaps this is also inconsistent with it. I just read your comment and noticed it didn’t match my own introspective experience, so I thought I’d share my experience.
I agree that the considerations you mentioned in your example are not changes in values, and didn’t mean to imply that that sort of thing is a change in values. Instead, I just meant that such shifts in expectations are changes in probability distributions, rather than changes in events, since I think of such things in terms of how likely each of the possible outcomes are, rather than just which outcomes are possible and which are ruled out.
Ah, I see, that makes sense.