I vaguely agree that preference cycles can be a good way to represent some human behavior. A somewhat grim example I like is: serial monogamists subjectively feel like each step of the process is moving toward something good (I imagine), but at no point would they endorse the whole cycle (I imagine).
I think of it this way: evolution isn’t “good enough” to bake in a utility function which exactly pursues evolutionary fitness. So why should we expect it to put in a coherent function at all? Sometimes, an incoherent preference system will be the best available way to nudge organisms into fitness-inducing behavior.
However, your binge-eating example has a different character, in that the person may endorse the whole cycle explicitly while engaging in it. This character is essential to your story: you claim that incoherent preferences might be fine in a normative sense.
Here’s my intuitive problem with your argument.
I think of the preference relation, >, in VNM, Savage, Jeffrey-Bolker, and other preference representation theorems (all of which justify some form of utility theory) as indicating behavior during deliberation.
We’re in the process of deciding what to do. Currently, we’d default to A. We notice the option B. We check whether B > A. If yes, we switch to B. Now we notice C, and C > B, so we switch it to our default and keep looking. BUT WAIT: we immediately notice that A > C. So we switch back to A. We then do all these steps over and over again.
If there’s a cycle, we get stuck in indecision. This is bad because it wastes cognition. It’s strictly better to set A=B=C, so that the computation can move on and look for a D which is better than all of them. So, in my view, the true “money” in the money pump is thinking time.
Eliezer’s example with pizza fits with this, more or less: we see someone trying to decide between options, with some switching cost.
Your example does not fit: the person would want to actually eat the food before switching to being hungry again, not just think about it.
Why do I choose to operationalize A>B via deliberation, and think it a mistake to operationalize A>B as a fact about an agent already in state B, as you do? Well, part of the big problem with money-pump arguments is that the way they’re usually framed, they seem to require a magical genie who can swap any reality A for a different one B (for a fee, of course). This is dumb. In the absence of such a genie, are incoherent preferences OK?
So it makes a lot more sense to interpret A>B as a fact about cognition. Being “offered” C just means you’ve thought to examine the possibility C. No magical genie required to define preferences.
Another justification is that, importantly, A, B and C can be lotteries, rather than fully-fleshed-out-worlds. Suppose C is just “the Riemann hypothesis is true”. We have a lot of uncertainty about what else is going on in that world. So what does it mean to ask whether an agent “prefers” C or not-C, from the behavioral perspective? We have to come up with a concrete instantiation, like giving them a button which makes the conjecture true or false. But obviously this could skew the results (EG, the agent likes pushing red buttons and hates pushing blue ones).
On the other hand, if we define > cognitively, it becomes just a matter of whether the agent prefers one or the other hypothetically—IE we only have to suppose that the agent can compare the desirability of abstract statements. This is still a bullet to bite (it constrains what cognition can look like, in a potentially unnecessary way), but it is cleaner.
Bottom line: you’re operationalizing A>B as information about what an agent would be willing to do if it was already in situation A, and was offered to switch to situation B. I think this is a misconception propagated by the way philosophers talk about money-pump arguments. I prefer to operationalize A>B as information about deliberation behavior, which I think fits better with most uses of >. Money-pumps are then seen as infinite loops in cognition.
Money-pumps are then seen as infinite loops in cognition.
And setting A=B=C is deciding not to allocate the time to figure out their values (hard to decide → similar). Usually, such a thing indicates there are multiple things you want (and as bad as ‘buy 3 pizzas, one of each might sound’ it seems like it resolves this issue).
If someone feels like formalizing a mixed pizza theorem, or just testing this one experimentally, let me know.
This doesn’t seem like a problem that shows up a lot, outside of ‘this game has strategies that cyclically beat each other/what option should you play in Rock Paper Scissors?’
And setting A=B=C is deciding not to allocate the time to figure out their values (hard to decide → similar).
This sentence seems to pre-suppose that they have “values”, which is in fact what’s at issue (since numerical values ensure transitivity). So I would not want to put it that way. Rather, cutting the loop saves time without apparently losing anything (although to an agent stuck in a loop, it might not seem that way).
Usually, such a thing indicates there are multiple things you want
I think this is actually not usually an intransitive loop, but rather, high standards for an answer (you want to satisfy ALL the desiderata). When making decisions, people learn an “acceptable decision quality” based on what is usually achievable. That becomes a threshold for satisficing. This is usually good for efficiency; once you achieve the threshold, you know returns for thinking about this decision are rapidly diminishing, so you can probably move on.
However, in the rare situations where the threshold is simply not achievable, this causes you to waste a lot of time searching (because your termination condition is not yet met!).
I vaguely agree that preference cycles can be a good way to represent some human behavior. A somewhat grim example I like is: serial monogamists subjectively feel like each step of the process is moving toward something good (I imagine), but at no point would they endorse the whole cycle (I imagine).
I think of it this way: evolution isn’t “good enough” to bake in a utility function which exactly pursues evolutionary fitness. So why should we expect it to put in a coherent function at all? Sometimes, an incoherent preference system will be the best available way to nudge organisms into fitness-inducing behavior.
However, your binge-eating example has a different character, in that the person may endorse the whole cycle explicitly while engaging in it. This character is essential to your story: you claim that incoherent preferences might be fine in a normative sense.
Here’s my intuitive problem with your argument.
I think of the preference relation, >, in VNM, Savage, Jeffrey-Bolker, and other preference representation theorems (all of which justify some form of utility theory) as indicating behavior during deliberation.
We’re in the process of deciding what to do. Currently, we’d default to A. We notice the option B. We check whether B > A. If yes, we switch to B. Now we notice C, and C > B, so we switch it to our default and keep looking. BUT WAIT: we immediately notice that A > C. So we switch back to A. We then do all these steps over and over again.
If there’s a cycle, we get stuck in indecision. This is bad because it wastes cognition. It’s strictly better to set A=B=C, so that the computation can move on and look for a D which is better than all of them. So, in my view, the true “money” in the money pump is thinking time.
Eliezer’s example with pizza fits with this, more or less: we see someone trying to decide between options, with some switching cost.
Your example does not fit: the person would want to actually eat the food before switching to being hungry again, not just think about it.
Why do I choose to operationalize A>B via deliberation, and think it a mistake to operationalize A>B as a fact about an agent already in state B, as you do? Well, part of the big problem with money-pump arguments is that the way they’re usually framed, they seem to require a magical genie who can swap any reality A for a different one B (for a fee, of course). This is dumb. In the absence of such a genie, are incoherent preferences OK?
So it makes a lot more sense to interpret A>B as a fact about cognition. Being “offered” C just means you’ve thought to examine the possibility C. No magical genie required to define preferences.
Another justification is that, importantly, A, B and C can be lotteries, rather than fully-fleshed-out-worlds. Suppose C is just “the Riemann hypothesis is true”. We have a lot of uncertainty about what else is going on in that world. So what does it mean to ask whether an agent “prefers” C or not-C, from the behavioral perspective? We have to come up with a concrete instantiation, like giving them a button which makes the conjecture true or false. But obviously this could skew the results (EG, the agent likes pushing red buttons and hates pushing blue ones).
On the other hand, if we define > cognitively, it becomes just a matter of whether the agent prefers one or the other hypothetically—IE we only have to suppose that the agent can compare the desirability of abstract statements. This is still a bullet to bite (it constrains what cognition can look like, in a potentially unnecessary way), but it is cleaner.
Bottom line: you’re operationalizing A>B as information about what an agent would be willing to do if it was already in situation A, and was offered to switch to situation B. I think this is a misconception propagated by the way philosophers talk about money-pump arguments. I prefer to operationalize A>B as information about deliberation behavior, which I think fits better with most uses of >. Money-pumps are then seen as infinite loops in cognition.
And setting A=B=C is deciding not to allocate the time to figure out their values (hard to decide → similar). Usually, such a thing indicates there are multiple things you want (and as bad as ‘buy 3 pizzas, one of each might sound’ it seems like it resolves this issue).
If someone feels like formalizing a mixed pizza theorem, or just testing this one experimentally, let me know.
This doesn’t seem like a problem that shows up a lot, outside of ‘this game has strategies that cyclically beat each other/what option should you play in Rock Paper Scissors?’
This sentence seems to pre-suppose that they have “values”, which is in fact what’s at issue (since numerical values ensure transitivity). So I would not want to put it that way. Rather, cutting the loop saves time without apparently losing anything (although to an agent stuck in a loop, it might not seem that way).
I think this is actually not usually an intransitive loop, but rather, high standards for an answer (you want to satisfy ALL the desiderata). When making decisions, people learn an “acceptable decision quality” based on what is usually achievable. That becomes a threshold for satisficing. This is usually good for efficiency; once you achieve the threshold, you know returns for thinking about this decision are rapidly diminishing, so you can probably move on.
However, in the rare situations where the threshold is simply not achievable, this causes you to waste a lot of time searching (because your termination condition is not yet met!).