We’ve all met people who are acting as if “Acquire Money” is a terminal goal, never noticing that money is almost entirely instrumental in nature. When you ask them “but what would you do if money was no issue and you had a lot of time”, all you get is a blank stare.
Even the LessWrong Wiki entry on terminal values describes a college student for which university is instrumental, and getting a job is terminal. This seems like a clear-cut case of a Lost Purpose: a job seems clearly instrumental. And yet, we’ve all met people who act as if “Have a Job” is a terminal value, and who then seem aimless and undirected after finding employment …
You can argue that Acquire Money and Have a Job aren’t “really” terminal goals, to which I counter that many people don’t know their ass from their elbow when it comes to their own goals.
Why does politics strike rationalists as so strangely shaped? Why does rationalism come across as aggressively apolitical to smart non-rationalists?
Part of the answer: Politics is absolutely rife with people mixing their ends with their means and vice versa. It’s pants-on-head confused, from a rationalist perspective, to be ultimately loyal to a particular set of economic or political policies. There’s something profoundly perverse, something suggesting deep confusion, about holding political identities centered around policies rather than goals. Instead, you ought to be loyal to your motivation for backing those policies, and see those policies as disposable means to achieve your motivation. Your motives want you to be able to say (or scream) “oops” and effortlessly, completely drop previously endorsed policies once you learn there’s a better path to your motives. It shouldn’t be a big psychological ordeal to dramatically upset your political worldview; this too is just a special case of updating your conditional probabilities (of outcomes given policies). Once you internalize this view of things, politicized debates should start to really rub you the wrong way.
I often wonder if this framing (with which I mostly agree) is an example of typical mind fallacy. The assumption that many humans are capable of distinguishing terminal from instrumental goals, or in having terminal goals more abstract than “comfort and procreation”, is not all that supported by evidence.
In other words, politicized debates DO rub you the wrong way, but on two dimensions—first, that you’re losing, because you’re approaching them from a different motive than your opponents. And second that it reveals not just a misalignment with fellow humans in terminal goals, but an alien-ness in the type of terminal goals you find reasonable.
Why does politics strike rationalists as so strangely shaped? Why does rationalism come across as aggressively apolitical to smart non-rationalists?
Part of the answer: Politics is absolutely rife with people mixing their ends with their means and vice versa. It’s pants-on-head confused, from a rationalist perspective, to be ultimately loyal to a particular set of economic or political policies. There’s something profoundly perverse, something suggesting deep confusion, about holding political identities centered around policies rather than goals. Instead, you ought to be loyal to your motivation for backing those policies, and see those policies as disposable means to achieve your motivation. Your motives want you to be able to say (or scream) “oops” and effortlessly, completely drop previously endorsed policies once you learn there’s a better path to your motives. It shouldn’t be a big psychological ordeal to dramatically upset your political worldview; this too is just a special case of updating your conditional probabilities (of outcomes given policies). Once you internalize this view of things, politicized debates should start to really rub you the wrong way.
I often wonder if this framing (with which I mostly agree) is an example of typical mind fallacy. The assumption that many humans are capable of distinguishing terminal from instrumental goals, or in having terminal goals more abstract than “comfort and procreation”, is not all that supported by evidence.
In other words, politicized debates DO rub you the wrong way, but on two dimensions—first, that you’re losing, because you’re approaching them from a different motive than your opponents. And second that it reveals not just a misalignment with fellow humans in terminal goals, but an alien-ness in the type of terminal goals you find reasonable.