There is a distinction people often fail to make, which is commonly seen in analyses of fictional characters’ actions but also those of real people. It is the distinction between behaving irrationally and having extreme preferences.
If we look at actions and preferences the way decision theorists do, it is clear that preferences cannot be irrational. Indeed, rationality is defined as tendency toward preference-satisfaction. To say preferences are irrational is to say that someone’s tastes can be objectively wrong.
Example: Voldemort is very stubborn in JKR’s Harry Potter. He could have easily arranged for a minion to kill Harry, but he didn’t, and this is decried as irrational. Or even more to the point, he could have been immortal if only he hid in a cave somewhere and didn’t bother anyone.
But that is ignoring Voldemort’s revealed preference relation and just treating survival as his chief end. What is the point of selling your soul to become the most powerful lich of all time so you can live as a hermit? That would be irrational, as it would neglect Voldemort’s preferences.
It’s possible to behave in a way you don’t endorse on reflection. Or even merely wouldn’t endorse, especially using modes of reflection you lack skill/knowledge for. Calling this condition “revealed preferences” is erring in the other direction from what you point out, an overly behaviorist view that ignores the less observable process of coming up with preferences. There is also something to be said about lacking preferences in some ways where it would be healthier to have them, even absent their spontaneous emergence.
I’ve read a good chunk of Eliezer’s paper on TDT, and it’s in that context that I am interpreting reflection. Forgive me if I misunderstand some of it; it’s new to me.
TDT is motivated by requiring a decision rule that is consistent under reflection. It doesn’t seem to pass judgment on preferences themselves, only on how actions ought to be chosen given preferences. Am I mistaken here?
Perhaps I should have been clearer with Voldemort’s “revealed” preferences. JKR writes him as a fairly simple character and I did take for granted that what we saw was what we got. I agree that in general actions aren’t indicative of beliefs.
EDIT: Ah, there is an exception. Eliezer is quite critical in the paper of preferring a decision rule for its own sake.
Preferences CAN be inconsistent (over time, or across different modes of preference-identification), which is definitely irrational. And sufficiently extreme preferences, even if consistent and consistently pursued, can be judged evil, insane, or harmful to many others.
Revealed preference is a very useful obfuscation over the fact that humans aren’t actually rational in the VNM sense. Not a one of us has a consistent ordering over potential world-states, and we’re full of contradictions between how we think about our wants (which varies in different modes and frames of asking the question) and what we actually do. It’s quite a bit stronger, predictively, especially averaged over a population, than polling or discussion. But it’s not the only valid conception of preferences, and not what most people mean when talking about preferences.
Voldemort can more easily be understood (and predicted) if you model his preference as immortality being instrumental to his quest for being powerful and feared (respected, in his worldview). But remember two things:
it’s fiction—there’s no requirement that it correlate with any possible real person’s behavior or belief patterns.
Even real people are mostly fiction. All such analysis is about models. All models are wrong, some models are useful. For real people, there are tens of billions of nodes in a fairly complicated network changing weights all the time based on a crazy amount of inputs. The underlying reality isn’t a complete and consistent preference ordering—it only considers a small subset of choices at a time. And it’s beliefs of what it’s doing are even less accurate. Other brains’ modeling of what this one is thinking still further from reality.
I would argue that inconsistency of preferences isn’t necessarily a sign of irrationality. Come to think of it, it may hinge greatly on how you frame the preference.
Consider changing tastes. As a child, I preferred some sweets to savory items, and those preferences reversed as I aged. Is that irrational? No and, indeed, you needn’t even view it as a preference reversal. The preference “I prefer to eat what tastes good to me” has remained unchanged, after all. Is my sense of taste itself a preference? It seems like this would devolve into semantics quickly.
My reluctance to characterize preferences as rational or irrational is that I see these as prescriptive terms. But you can’t prescribe preferences. You either have them or you don’t. Only decision rules are chosen.
There is a distinction people often fail to make, which is commonly seen in analyses of fictional characters’ actions but also those of real people. It is the distinction between behaving irrationally and having extreme preferences.
If we look at actions and preferences the way decision theorists do, it is clear that preferences cannot be irrational. Indeed, rationality is defined as tendency toward preference-satisfaction. To say preferences are irrational is to say that someone’s tastes can be objectively wrong.
Example: Voldemort is very stubborn in JKR’s Harry Potter. He could have easily arranged for a minion to kill Harry, but he didn’t, and this is decried as irrational. Or even more to the point, he could have been immortal if only he hid in a cave somewhere and didn’t bother anyone.
But that is ignoring Voldemort’s revealed preference relation and just treating survival as his chief end. What is the point of selling your soul to become the most powerful lich of all time so you can live as a hermit? That would be irrational, as it would neglect Voldemort’s preferences.
It’s possible to behave in a way you don’t endorse on reflection. Or even merely wouldn’t endorse, especially using modes of reflection you lack skill/knowledge for. Calling this condition “revealed preferences” is erring in the other direction from what you point out, an overly behaviorist view that ignores the less observable process of coming up with preferences. There is also something to be said about lacking preferences in some ways where it would be healthier to have them, even absent their spontaneous emergence.
I’ve read a good chunk of Eliezer’s paper on TDT, and it’s in that context that I am interpreting reflection. Forgive me if I misunderstand some of it; it’s new to me.
TDT is motivated by requiring a decision rule that is consistent under reflection. It doesn’t seem to pass judgment on preferences themselves, only on how actions ought to be chosen given preferences. Am I mistaken here?
Perhaps I should have been clearer with Voldemort’s “revealed” preferences. JKR writes him as a fairly simple character and I did take for granted that what we saw was what we got. I agree that in general actions aren’t indicative of beliefs.
EDIT: Ah, there is an exception. Eliezer is quite critical in the paper of preferring a decision rule for its own sake.
Preferences CAN be inconsistent (over time, or across different modes of preference-identification), which is definitely irrational. And sufficiently extreme preferences, even if consistent and consistently pursued, can be judged evil, insane, or harmful to many others.
Revealed preference is a very useful obfuscation over the fact that humans aren’t actually rational in the VNM sense. Not a one of us has a consistent ordering over potential world-states, and we’re full of contradictions between how we think about our wants (which varies in different modes and frames of asking the question) and what we actually do. It’s quite a bit stronger, predictively, especially averaged over a population, than polling or discussion. But it’s not the only valid conception of preferences, and not what most people mean when talking about preferences.
Voldemort can more easily be understood (and predicted) if you model his preference as immortality being instrumental to his quest for being powerful and feared (respected, in his worldview). But remember two things:
it’s fiction—there’s no requirement that it correlate with any possible real person’s behavior or belief patterns.
Even real people are mostly fiction. All such analysis is about models. All models are wrong, some models are useful. For real people, there are tens of billions of nodes in a fairly complicated network changing weights all the time based on a crazy amount of inputs. The underlying reality isn’t a complete and consistent preference ordering—it only considers a small subset of choices at a time. And it’s beliefs of what it’s doing are even less accurate. Other brains’ modeling of what this one is thinking still further from reality.
I would argue that inconsistency of preferences isn’t necessarily a sign of irrationality. Come to think of it, it may hinge greatly on how you frame the preference.
Consider changing tastes. As a child, I preferred some sweets to savory items, and those preferences reversed as I aged. Is that irrational? No and, indeed, you needn’t even view it as a preference reversal. The preference “I prefer to eat what tastes good to me” has remained unchanged, after all. Is my sense of taste itself a preference? It seems like this would devolve into semantics quickly.
My reluctance to characterize preferences as rational or irrational is that I see these as prescriptive terms. But you can’t prescribe preferences. You either have them or you don’t. Only decision rules are chosen.