It seems to me that the Jeffrey-Bolker framework is a poor match for what’s going on in peoples’ heads when they make value judgements, compared to the VNM framework. If I think about how good the consequences of an action are, I try to think about what I expect to happen if I take that action (ie the outcome), and I think about how likely that outcome is to have various properties that I care about, since I don’t know exactly what the outcome will be with certainty. This isn’t to say that I literally consider probability distributions in my mind, since I typically use qualitative descriptions of probability rather than numbers in [0,1], and when I do use numbers, they are very rough, but this does seem like a sort of fuzzy, computationally limited version of a probability distribution. Similarly, my estimations of how good various outcomes are are often qualitative, rather than numerical, and again this seems like a fuzzy, computationally limited version of utility function. In order to determine the utility of the event “I take action A”, I need to consider how good and how likely various consequences are, and take the expectation of the ‘how good’ with respect to the ‘how likely’. The Jeffrey-Bolker framework seems to be asking me to pretend none of that ever happened.
If I think about how good the consequences of an action are, I try to think about what I expect to happen if I take that action (ie the outcome), and I think about how likely that outcome is to have various properties that I care about, since I don’t know exactly what the outcome will be with certainty… I need to consider how good and how likely various consequences are, and take the expectation of the ‘how good’ with respect to the ‘how likely’.
I don’t understand JB yet, but when I introspected just now, my experience of decision-making doesn’t have any separation between beliefs and values, so I think I disagree with the above. I’ll try to explain why by describing my experience. (Note: Long comment below is just saying one very simple thing. Sorry for length. There’s a one-line tl;dr at the end.)
Right now I’m considering doing three different things. I can go and play a videogame that my friend suggested we play together, I can do some LW work with my colleague, or I can go play some guitar/piano. I feel like the videogame isn’t very fun right now because I think the one my friend suggested not that interesting of a shared experience. I feel like the work is fun because I’m excited about publishing the results of the work, and the work itself involves a kind of cognition I enjoy. And playing piano is fun because I’ve been skilling up a lot lately and I’m going to do accompany some of my housemates in some hamilton songs.
Now, I know some likely ways that what seems valuable to me might change. There are other videogames I’ve played lately that have been really fascinating and rewarding to play together, that involve problem solving where 2 people can be creative together. I can imagine the work turning out to not actuallybe the fun part but the boring parts. I can imagine that I’ve found no traction (skill-up) in playing piano, or that we’re going to use a recorded soundtrack rather than my playing for the songs we’re learning.
All of these to me feel like updates in my understanding of what events are reachable to me; this doesn’t feel like changing my utility evaluation of the events. The event of “play videogame while friend watches bored” could change to “play videogame while creatively problem-solving with friend”. The event of “gain skill in piano and then later perform songs well with friends” could change to “struggle to do something difficult and sound bad and that’s it”.
If I think about changing my utility function, I expect that would feel more like… well, I’m not sure. My straw version is “I creatively solve problems with my friend on a videogame, but somehow that’s objectively bad so I will not do it”. That’s where some variable in the utility function changed while all the rest of the facts about my psychology and reality stay the same. This doesn’t feel to me like my regular experience of decision-making.
But, maybe that’s not the idea. The idea is like if I had some neurological change, perhaps I become more of a sociopath and stop feeling empathy and everyone just feels like objects to me rather than alive. Then a bunch of the social experiences above would change, they’d lose any experience of things like vicarious enjoyment and pleasure of bonding with friends. Perhaps that’s what VNM is talking about in my experience.
I think that some of the standard “updates to my ethics / utility function” ideas that people discuss often don’t feel like this to me. Like, some people say that reflecting onf population ethics leads them to change their utility function and start to care about the far future. That’s not my experience – for me it’s been things like the times in HPMOR when Harry thinks about civilizations of the future, what they’ll be like/think, and how awesome they can be. It feels real to me, like a reachable state, and this is what has changed a lot of my behaviour, in contrast with changing some variable in a function of world-states that’s independent from my understanding of what events are achievable.
To be clear, sometimes I describe my experience more like the sociopath example, where my fundamental interests/values change. I say things like “I don’t enjoy videogames as much as I used to” or “These days I value honesty and reliability a lot more than politeness”, and there is a sense there where I now experience the same events very differently. “I had a positive meeting with John” might now be “I feel like he was being evasive about the topic we were discussing”. The things that are salient to me change. And I think that the language of “my values have changed” is often an effective one for communicating that – even if my experience does not match beliefs|utility, any sufficiently coherent agent can be described this way, and it is often easy to help others model me by describing my values as having changed.
But I think my internal experience is more that I made substantial updates about what events I’m moving towards, and the event “We had a pleasant interaction which will lead to use working effectively together” has changed to “We were not able to say the possibly unwelcome facts of the matter, which will lead to a world where we don’t work effectively together”. So internally it feels like an update about what events are reachable, even though someone from the outside who doesn’t understand my internal experience might more naturally say “It seems like Ben is treating the same event differently now, so I’ll model him as having changed his values”.
tl;dr: While I often talk separately about what actions I/you/we could take and how valuable those actions are are, internally when when I’m ‘evaluating’ the actions, I’m just trying to visualise what they are, and there is no second step of running my utility function on those visualisations.
As I say, I’m not sure I understand JB, so perhaps this is also inconsistent with it. I just read your comment and noticed it didn’t match my own introspective experience, so I thought I’d share my experience.
I agree that the considerations you mentioned in your example are not changes in values, and didn’t mean to imply that that sort of thing is a change in values. Instead, I just meant that such shifts in expectations are changes in probability distributions, rather than changes in events, since I think of such things in terms of how likely each of the possible outcomes are, rather than just which outcomes are possible and which are ruled out.
Perhaps it goes without saying, but obviously, both frameworks are flexible enough to allow for most phenomena—the question here is what is more natural in one framework or another.
My main argument is that the procrastination paradox is not natural at all in a Savage framework, as it suggests an uncomputable utility function. I think this plausibly outweighs the issue you’re pointing at.
But with respect to the issue you are pointing at:
I try to think about what I expect to happen if I take that action (ie the outcome), and I think about how likely that outcome is to have various properties that I care about,
In the Savage framework, an outcome already encodes everything you care about. So the computation which seems to be suggested by Savage is to think of these maximally-specified outcomes, assigning them probability and utility, and then combining those to get expected utility. This seems to be very demanding: it requires imagining these very detailed scenarios.
Alternately, we might say (as as Savage said) that the Savage axioms apply to “small worlds”—small scenarios which the agent abstracts from its experience, such as the decision of whether to break an egg for an omelette. These can be easily considered by the agent, if it can assign values “from outside the problem” in an appropriate way.
But then, to account for the breadth of human reasoning, it seems to me we also want an account of things like extending a small world when we find that it isn’t sufficient, and coherence between different small-world frames for related decisions.
This gives a picture very much like the Jeffrey-Bolker picture, in that we don’t really work with outcomes which completely specify everything we care about, but rather, work with a variety of simplified outcomes with coherence requirements between simpler and more complex views.
So overall I think it is better to have some picture where you can break things up in a more tractable way, rather than having full outcomes which you need to pass through to get values.
In the Jeffrey-Bolker framework, you can re-estimate the value of an event by breaking it up into pieces, estimating the value and probability of each piece, and combining them back together. This process could be iterated in a manner similar to dynamic programming in RL, to improve value estimates for actions—although one needs to settle on a story about where the information originally comes from. I currently like the logical-induction-like picture where you get information coming in “somehow” (a broad variety of feedback is possible, including abstract judgements about utility which are hard to cash out in specific cases) and you try to make everything as coherent as possible in the meanwhile.
In the Savage framework, an outcome already encodes everything you care about.
Yes, but if you don’t know which outcome is the true one, so you’re considering a probability distribution over outcomes instead of a single outcome, then it still makes sense to speak of the probability that the true outcome has some feature. This is what I meant.
So the computation which seems to be suggested by Savage is to think of these maximally-specified outcomes, assigning them probability and utility, and then combining those to get expected utility. This seems to be very demanding: it requires imagining these very detailed scenarios.
You do not need to be able to imagine every possible outcome individually in order to think of functions on or probability distributions over the set of outcomes, any more than I need to be able to imagine each individual real number in order to understand the function x↦x2 or the standard normal distribution.
It seems that you’re going by an analogy like Jeffrey-Bolker : VNM :: events : outcomes, which is partially right, but leaves out an important sense in which the correct analogy is Jeffrey-Bolker : VNM :: events : probability distributions, since although utility is defined on outcomes, the function that is actually evaluated is expected utility, which is defined on probability distributions (this being a distinction that does not exist in Jeffrey-Bolker, but does exist in my conception of real-world human decision making).
It seems to me that the Jeffrey-Bolker framework is a poor match for what’s going on in peoples’ heads when they make value judgements, compared to the VNM framework. If I think about how good the consequences of an action are, I try to think about what I expect to happen if I take that action (ie the outcome), and I think about how likely that outcome is to have various properties that I care about, since I don’t know exactly what the outcome will be with certainty. This isn’t to say that I literally consider probability distributions in my mind, since I typically use qualitative descriptions of probability rather than numbers in [0,1], and when I do use numbers, they are very rough, but this does seem like a sort of fuzzy, computationally limited version of a probability distribution. Similarly, my estimations of how good various outcomes are are often qualitative, rather than numerical, and again this seems like a fuzzy, computationally limited version of utility function. In order to determine the utility of the event “I take action A”, I need to consider how good and how likely various consequences are, and take the expectation of the ‘how good’ with respect to the ‘how likely’. The Jeffrey-Bolker framework seems to be asking me to pretend none of that ever happened.
I don’t understand JB yet, but when I introspected just now, my experience of decision-making doesn’t have any separation between beliefs and values, so I think I disagree with the above. I’ll try to explain why by describing my experience. (Note: Long comment below is just saying one very simple thing. Sorry for length. There’s a one-line tl;dr at the end.)
Right now I’m considering doing three different things. I can go and play a videogame that my friend suggested we play together, I can do some LW work with my colleague, or I can go play some guitar/piano. I feel like the videogame isn’t very fun right now because I think the one my friend suggested not that interesting of a shared experience. I feel like the work is fun because I’m excited about publishing the results of the work, and the work itself involves a kind of cognition I enjoy. And playing piano is fun because I’ve been skilling up a lot lately and I’m going to do accompany some of my housemates in some hamilton songs.
Now, I know some likely ways that what seems valuable to me might change. There are other videogames I’ve played lately that have been really fascinating and rewarding to play together, that involve problem solving where 2 people can be creative together. I can imagine the work turning out to not actuallybe the fun part but the boring parts. I can imagine that I’ve found no traction (skill-up) in playing piano, or that we’re going to use a recorded soundtrack rather than my playing for the songs we’re learning.
All of these to me feel like updates in my understanding of what events are reachable to me; this doesn’t feel like changing my utility evaluation of the events. The event of “play videogame while friend watches bored” could change to “play videogame while creatively problem-solving with friend”. The event of “gain skill in piano and then later perform songs well with friends” could change to “struggle to do something difficult and sound bad and that’s it”.
If I think about changing my utility function, I expect that would feel more like… well, I’m not sure. My straw version is “I creatively solve problems with my friend on a videogame, but somehow that’s objectively bad so I will not do it”. That’s where some variable in the utility function changed while all the rest of the facts about my psychology and reality stay the same. This doesn’t feel to me like my regular experience of decision-making.
But, maybe that’s not the idea. The idea is like if I had some neurological change, perhaps I become more of a sociopath and stop feeling empathy and everyone just feels like objects to me rather than alive. Then a bunch of the social experiences above would change, they’d lose any experience of things like vicarious enjoyment and pleasure of bonding with friends. Perhaps that’s what VNM is talking about in my experience.
I think that some of the standard “updates to my ethics / utility function” ideas that people discuss often don’t feel like this to me. Like, some people say that reflecting onf population ethics leads them to change their utility function and start to care about the far future. That’s not my experience – for me it’s been things like the times in HPMOR when Harry thinks about civilizations of the future, what they’ll be like/think, and how awesome they can be. It feels real to me, like a reachable state, and this is what has changed a lot of my behaviour, in contrast with changing some variable in a function of world-states that’s independent from my understanding of what events are achievable.
To be clear, sometimes I describe my experience more like the sociopath example, where my fundamental interests/values change. I say things like “I don’t enjoy videogames as much as I used to” or “These days I value honesty and reliability a lot more than politeness”, and there is a sense there where I now experience the same events very differently. “I had a positive meeting with John” might now be “I feel like he was being evasive about the topic we were discussing”. The things that are salient to me change. And I think that the language of “my values have changed” is often an effective one for communicating that – even if my experience does not match beliefs|utility, any sufficiently coherent agent can be described this way, and it is often easy to help others model me by describing my values as having changed.
But I think my internal experience is more that I made substantial updates about what events I’m moving towards, and the event “We had a pleasant interaction which will lead to use working effectively together” has changed to “We were not able to say the possibly unwelcome facts of the matter, which will lead to a world where we don’t work effectively together”. So internally it feels like an update about what events are reachable, even though someone from the outside who doesn’t understand my internal experience might more naturally say “It seems like Ben is treating the same event differently now, so I’ll model him as having changed his values”.
tl;dr: While I often talk separately about what actions I/you/we could take and how valuable those actions are are, internally when when I’m ‘evaluating’ the actions, I’m just trying to visualise what they are, and there is no second step of running my utility function on those visualisations.
As I say, I’m not sure I understand JB, so perhaps this is also inconsistent with it. I just read your comment and noticed it didn’t match my own introspective experience, so I thought I’d share my experience.
I agree that the considerations you mentioned in your example are not changes in values, and didn’t mean to imply that that sort of thing is a change in values. Instead, I just meant that such shifts in expectations are changes in probability distributions, rather than changes in events, since I think of such things in terms of how likely each of the possible outcomes are, rather than just which outcomes are possible and which are ruled out.
Ah, I see, that makes sense.
Perhaps it goes without saying, but obviously, both frameworks are flexible enough to allow for most phenomena—the question here is what is more natural in one framework or another.
My main argument is that the procrastination paradox is not natural at all in a Savage framework, as it suggests an uncomputable utility function. I think this plausibly outweighs the issue you’re pointing at.
But with respect to the issue you are pointing at:
In the Savage framework, an outcome already encodes everything you care about. So the computation which seems to be suggested by Savage is to think of these maximally-specified outcomes, assigning them probability and utility, and then combining those to get expected utility. This seems to be very demanding: it requires imagining these very detailed scenarios.
Alternately, we might say (as as Savage said) that the Savage axioms apply to “small worlds”—small scenarios which the agent abstracts from its experience, such as the decision of whether to break an egg for an omelette. These can be easily considered by the agent, if it can assign values “from outside the problem” in an appropriate way.
But then, to account for the breadth of human reasoning, it seems to me we also want an account of things like extending a small world when we find that it isn’t sufficient, and coherence between different small-world frames for related decisions.
This gives a picture very much like the Jeffrey-Bolker picture, in that we don’t really work with outcomes which completely specify everything we care about, but rather, work with a variety of simplified outcomes with coherence requirements between simpler and more complex views.
So overall I think it is better to have some picture where you can break things up in a more tractable way, rather than having full outcomes which you need to pass through to get values.
In the Jeffrey-Bolker framework, you can re-estimate the value of an event by breaking it up into pieces, estimating the value and probability of each piece, and combining them back together. This process could be iterated in a manner similar to dynamic programming in RL, to improve value estimates for actions—although one needs to settle on a story about where the information originally comes from. I currently like the logical-induction-like picture where you get information coming in “somehow” (a broad variety of feedback is possible, including abstract judgements about utility which are hard to cash out in specific cases) and you try to make everything as coherent as possible in the meanwhile.
Yes, but if you don’t know which outcome is the true one, so you’re considering a probability distribution over outcomes instead of a single outcome, then it still makes sense to speak of the probability that the true outcome has some feature. This is what I meant.
You do not need to be able to imagine every possible outcome individually in order to think of functions on or probability distributions over the set of outcomes, any more than I need to be able to imagine each individual real number in order to understand the function x↦x2 or the standard normal distribution.
It seems that you’re going by an analogy like Jeffrey-Bolker : VNM :: events : outcomes, which is partially right, but leaves out an important sense in which the correct analogy is Jeffrey-Bolker : VNM :: events : probability distributions, since although utility is defined on outcomes, the function that is actually evaluated is expected utility, which is defined on probability distributions (this being a distinction that does not exist in Jeffrey-Bolker, but does exist in my conception of real-world human decision making).