But suppose we abandon the beliefs/values separation: what properties do we have left? Is the concept “mind without a beliefs/values separation” Simply the concept “thing”?
An agent using UDT doesn’t necessarily have a beliefs/values separation, but still has the properties of preferences and decision making. Or at least, it only has beliefs about mathematical facts, not about empirical facts. Maybe I should have made it clear that I was mainly talking about empirical beliefs in the post.
Not quite true: state of knowledge corresponds to beliefs. It’s values that don’t update (but in expected utility maximization that’s both utility and prior). Again, it’s misleading to equate beliefs with prior and forget about the knowledge (event that conditions the current state).
Yes, I agree we can interpret UDT as having its own dichotomy between beliefs and values, but the dividing line looks very different from how humans divide between beliefs and values, which seems closer to the probability/utility divide.
UDT is invariant with respect to what universe it’s actually in. This requires it to compute over infinite universes and thus have infinite computing power. It’s not hard to see why it’s going to break down as a model of in-universe, limted beings.
An agent using UDT doesn’t necessarily have a beliefs/values separation,
I am behind on your recent work on UDT; this fact comes as a shock to me. Can you provide a link to a post of yours/provide an example here making clear that UDT doesn’t necessarily have a beliefs/values separation? Thanks.
Suppose I offer you three boxes and ask you to choose one. The first two are transparent, free, and contains an apple and an orange, respectively. The third is opaque, costs a penny, and contains either an apple or an orange, depending on a coin flip I made. Under expected utility maximization, there is no reason for you to choose the third box, regardless of your probability function and utility function. Under UDT1, you can choose the third box, by preferring to and as the outcomes of world programs P1 and P2. In that case, you can’t be said to have a belief about whether the real world is P1 or P2.
This example seems unclear. Are you seriously claiming utility maximisation can’t prefer a randomised outcome in an iterated situation? If so, you take this “independence” business much too far.
Utility maximising agents can do things like prefer a diverse diet. They simply do not have to prefer either apples or oranges—thereby winding up with vitamin and mineral deficiencies. It is trivial to create a utility function which exhibits fruit preferences which depend on what you have eaten most recently.
Randomization only maximizes diversity if you have to make decisions under amnesia or coordinate without communication or some similar perverse situation. In any normal case, you’re better off choosing a deterministic sequence that’s definitely diverse, rather than leaving it to randomness and only probably getting a diverse set of outcomes.
Sure—but that seems rather tangential to the main point here.
The options were , - or a more expensive random choice. A random diet may not be perfect—but it was probably the best one on offer in the case of this example.
If the agent already has a penny (which they must if they can afford to choose the third box), they could just flip the penny to decide which of the first two boxes to take and save themselves the money.
Unless you’re being a devil’s advocate, I don’t see any reason to justify a completely rational agent choosing the random box.
Then choice C isn’t a random mixture of choice A and choice B.
Preferring that there be randomness at a point where you otherwise wouldn’t get a decision at all, is fine. What doesn’t happen is preferring one coin-flip in place of one decision.
Not to be crass, but given the assumption that Wei_Dai is not saying something utterly asinine, does your interpretation of the hypothetical actually follow?
Hang on! My last comment was a reply to your question about when it could be rational to select the third box. I have already said that the original example was unclear. It certainly didn’t suggest an infinite sequence—and I wasn’t trying to suggest that.
The example specified that choosing the third box was the correct answer—under the author’s own proposed decision theory. Surely interpretations of what it was supposed to mean should bear that in mind.
I don’t believe we’re actually arguing about anything worth caring about. My understanding was that Wei_Dai was illustrating a problem with UDT1 - in which case a single scenario in which UDT1 gives an unambiguously wrong answer suffices. To disprove Wei_Dai’s assertion requires demonstrating that no scenario of the kind proposed makes UDT1 give the wrong answer, not showing that not every scenario of the kind proposed makes UDT1 give the wrong answer.
Are you sure you are taking the fact that he is UDT’s inventor and biggest fan into account? He certainly didn’t claim that he was illustrating a problem with UDT.
Under expected utility maximization, there is no reason for you to choose the third box, regardless of your probability function and utility function. Under UDT1, you can choose the third box, by preferring to and as the outcomes of world programs P1 and P2. In that case, you can’t be said to have a belief about whether the real world is P1 or P2.
You lost me. Is ‘apples’ supposed to be plural? Can you really not choose the third box regardless of utility function? What if you prefer things that came in opaque boxes?
The opaque box was a way of framing the problem, and not part of the problem itself, which is supposed to be about your preferences for apples and oranges. I can specify the problem in terms of three identical buttons that you can press instead.
Those buttons will probably not be absolutely identical—since they will be in different spatial positions relative to each other. So an agent operating under expected utility maximization might still prefer (say) pressing the right-most button.
Real-world utility functions can literally prefer anything you can specify.
I’m actually an example of this—where I don’t otherwise care, I will pick the third option, or the option that’s related to the number three in some way (preferably related to powers of three, but multiples of 9 are preferred over other multiples of three as well). If I didn’t care very much about apples vs. oranges, I’d be fairly likely to pay a penny for the third box/button/whatever. I also know two people who feel similarly about the number 8.
In tangentially related news, I’m sad that I’m turning 28 next month. Yes, I know I’m strange.
You’re not strange. (Sorry if that sounded offensive, I didn’t mean to!) I’m likewise sad that I just turned 27. I was always the youngest in school and university, graduating with honors at 20. Getting closer to 30 now. “Where are you now golden boy, where is your famous golden touch?” Or this: “Tired of lying in the sunshine staying home to watch the rain, you are young and life is long and there is time to kill today.”
I’m not sad that I’m closer to 30. 30′s cool, it’s a multiple of three. I’m sad that the next time my age will be a power of three won’t be ’till I’m 81.
Cute idea, but I value signaling an interest in accuracy/not coming off like a loon over being associated with the number 3. The former actually affect things in my life.
I can specify the problem in terms of three identical buttons that you can press instead.
Please do, if you think that would make the problem clearer. The piece I’m not seeing is where UDT1 lets you choose something that expected utility does not. Does expected utility usually not allow you to have states of the world in your utility function?
An agent using UDT doesn’t necessarily have a beliefs/values separation, but still has the properties of preferences and decision making. Or at least, it only has beliefs about mathematical facts, not about empirical facts. Maybe I should have made it clear that I was mainly talking about empirical beliefs in the post.
How, then, would you describe its representation of empirical information—if not as “beliefs”?
Not quite true: state of knowledge corresponds to beliefs. It’s values that don’t update (but in expected utility maximization that’s both utility and prior). Again, it’s misleading to equate beliefs with prior and forget about the knowledge (event that conditions the current state).
Yes, I agree we can interpret UDT as having its own dichotomy between beliefs and values, but the dividing line looks very different from how humans divide between beliefs and values, which seems closer to the probability/utility divide.
UDT is invariant with respect to what universe it’s actually in. This requires it to compute over infinite universes and thus have infinite computing power. It’s not hard to see why it’s going to break down as a model of in-universe, limted beings.
What do you mean? It has a utility function just like most other decision theories do. The preferences are represented by the utility function.
I am behind on your recent work on UDT; this fact comes as a shock to me. Can you provide a link to a post of yours/provide an example here making clear that UDT doesn’t necessarily have a beliefs/values separation? Thanks.
Suppose I offer you three boxes and ask you to choose one. The first two are transparent, free, and contains an apple and an orange, respectively. The third is opaque, costs a penny, and contains either an apple or an orange, depending on a coin flip I made. Under expected utility maximization, there is no reason for you to choose the third box, regardless of your probability function and utility function. Under UDT1, you can choose the third box, by preferring to and as the outcomes of world programs P1 and P2. In that case, you can’t be said to have a belief about whether the real world is P1 or P2.
This example seems unclear. Are you seriously claiming utility maximisation can’t prefer a randomised outcome in an iterated situation? If so, you take this “independence” business much too far.
Utility maximising agents can do things like prefer a diverse diet. They simply do not have to prefer either apples or oranges—thereby winding up with vitamin and mineral deficiencies. It is trivial to create a utility function which exhibits fruit preferences which depend on what you have eaten most recently.
Randomization only maximizes diversity if you have to make decisions under amnesia or coordinate without communication or some similar perverse situation. In any normal case, you’re better off choosing a deterministic sequence that’s definitely diverse, rather than leaving it to randomness and only probably getting a diverse set of outcomes.
Sure—but that seems rather tangential to the main point here.
The options were , - or a more expensive random choice. A random diet may not be perfect—but it was probably the best one on offer in the case of this example.
If the agent already has a penny (which they must if they can afford to choose the third box), they could just flip the penny to decide which of the first two boxes to take and save themselves the money.
Unless you’re being a devil’s advocate, I don’t see any reason to justify a completely rational agent choosing the random box.
What—never? Say they can only make the choice once—and their answer determines which box they will get on all future occasions.
Then choice C isn’t a random mixture of choice A and choice B.
Preferring that there be randomness at a point where you otherwise wouldn’t get a decision at all, is fine. What doesn’t happen is preferring one coin-flip in place of one decision.
Not to be crass, but given the assumption that Wei_Dai is not saying something utterly asinine, does your interpretation of the hypothetical actually follow?
Hang on! My last comment was a reply to your question about when it could be rational to select the third box. I have already said that the original example was unclear. It certainly didn’t suggest an infinite sequence—and I wasn’t trying to suggest that.
The example specified that choosing the third box was the correct answer—under the author’s own proposed decision theory. Surely interpretations of what it was supposed to mean should bear that in mind.
I don’t believe we’re actually arguing about anything worth caring about. My understanding was that Wei_Dai was illustrating a problem with UDT1 - in which case a single scenario in which UDT1 gives an unambiguously wrong answer suffices. To disprove Wei_Dai’s assertion requires demonstrating that no scenario of the kind proposed makes UDT1 give the wrong answer, not showing that not every scenario of the kind proposed makes UDT1 give the wrong answer.
Are you sure you are taking the fact that he is UDT’s inventor and biggest fan into account? He certainly didn’t claim that he was illustrating a problem with UDT.
...you’re right, I’m misreading. I’ll shut up now.
Okay, let’s see if I have this straight—you’re assuming:
the axiom of independence is necessary for expected utility theory
losing a penny represents some negative amount of utility
one’s utility function can’t include terms for “the outcomes of world programs” under expected utility theory
You lost me. Is ‘apples’ supposed to be plural? Can you really not choose the third box regardless of utility function? What if you prefer things that came in opaque boxes?
It’s not supposed to be plural. Fixed.
The opaque box was a way of framing the problem, and not part of the problem itself, which is supposed to be about your preferences for apples and oranges. I can specify the problem in terms of three identical buttons that you can press instead.
Those buttons will probably not be absolutely identical—since they will be in different spatial positions relative to each other. So an agent operating under expected utility maximization might still prefer (say) pressing the right-most button.
Real-world utility functions can literally prefer anything you can specify.
I’m actually an example of this—where I don’t otherwise care, I will pick the third option, or the option that’s related to the number three in some way (preferably related to powers of three, but multiples of 9 are preferred over other multiples of three as well). If I didn’t care very much about apples vs. oranges, I’d be fairly likely to pay a penny for the third box/button/whatever. I also know two people who feel similarly about the number 8.
In tangentially related news, I’m sad that I’m turning 28 next month. Yes, I know I’m strange.
You’re not strange. (Sorry if that sounded offensive, I didn’t mean to!) I’m likewise sad that I just turned 27. I was always the youngest in school and university, graduating with honors at 20. Getting closer to 30 now. “Where are you now golden boy, where is your famous golden touch?” Or this: “Tired of lying in the sunshine staying home to watch the rain, you are young and life is long and there is time to kill today.”
I’m not sad that I’m closer to 30. 30′s cool, it’s a multiple of three. I’m sad that the next time my age will be a power of three won’t be ’till I’m 81.
I obviously can’t read. Let that comment stand as a monument to my stupidity =)
You mean your second annual 27th birthday?
Cute idea, but I value signaling an interest in accuracy/not coming off like a loon over being associated with the number 3. The former actually affect things in my life.
Why is that strange?
Please do, if you think that would make the problem clearer. The piece I’m not seeing is where UDT1 lets you choose something that expected utility does not. Does expected utility usually not allow you to have states of the world in your utility function?