Lies, Damn Lies, and Fabricated Options

Duncan Sabien (Deactivated)17 Oct 2021 2:47 UTC
298 points
132 comments14 min readLW link2 reviews

This is an essay about one of those “once you see it, you will see it everywhere” phenomena. It is a psychological and interpersonal dynamic roughly as common, and almost as destructive, as motte-and-bailey, and at least in my own personal experience it’s been quite valuable to have it reified, so that I can quickly recognize the commonality between what I had previously thought of as completely unrelated situations.

The original quote referenced in the title is “There are three kinds of lies: lies, damned lies, and statistics.”

Background 1: Gyroscopes

Gyroscopes are weird.

Except they’re not. They’re quite normal and mundane and straightforward. The weirdness of gyroscopes is a map-territory confusion—gyroscopes seem weird because my map is poorly made, and predicts that they will do something other than their normal, mundane, straightforward thing.

In large part, this is because I don’t have the consequences of physical law engraved deeply enough into my soul that they make intuitive sense.

I can imagine a world that looks exactly like the world around me, in every way, except that in this imagined world, gyroscopes don’t have any of their strange black-magic properties. It feels coherent to me. It feels like a world that could possibly exist.

“Everything’s the same, except gyroscopes do nothing special.” Sure, why not.

But in fact, this world is deeply, deeply incoherent. It is Not Possible with capital letters. And a physicist with sufficiently sharp intuitions would know this—would be able to see the implications of a world where gyroscopes “don’t do anything weird,” and tell me all of the ways in which reality falls apart.

The seeming coherence of the imaginary world where gyroscopes don’t balance and don’t precess and don’t resist certain kinds of motion is a product of my own ignorance, and of the looseness with which I am tracking how different facts fit together, and what the consequences of those facts are. It’s like a toddler thinking that they can eat their slice of cake, and still have that very same slice of cake available to eat again the next morning.

Background 2: H2O and XYZ

In the book Labyrinths of Reason, author William Poundstone delves into various thought experiments (like Searle’s Chinese Room) to see whether they’re actually coherent or not.

In one such exploration, he discusses the idea of a Twin Earth, on the opposite side of the sun, exactly like Earth in every way except that it doesn’t have water. Instead, it has a chemical, labeled XYZ, which behaves like water and occupies water’s place in biology and chemistry, but is unambiguously distinct.

Once again, this is the sort of thing humans are capable of imagining. I can nod along and say “sure, a liquid that behaves just like water, but isn’t.”

But a chemist, intimately familiar with the structure and behavior of molecules and with the properties of the elements and their isotopes, would be throwing up red flags.

“Just like water,” they might say, and I would nod.

“Liquid, and transparent, with a density of 997 kilograms per meter cubed.”

“Sure,” I would reply.

“Which freezes and melts at exactly 0º Celsius, and which boils and condenses at exactly 100º Celsius.”

“Yyyyeahhhh,” I would say, uneasiness settling in.

“Which makes up roughly 70% of the mass of the bodies of the humans of Twin Earth, and which is a solvent for hydrophilic substances, but not hydrophobic ones, and which can hold ions and polar substances in solution.”

“Um.”

The more we drill down into what we mean by behaves exactly like water, the more it starts to become clear that there just isn’t a possible substance which behaves exactly like water, but isn’t. There are only so many configurations of electrons and protons and neutrons (especially while remaining small enough to mimic water’s molarity, and to play water’s role in various chemical interactions).

Once again, our ability to imagine “a substance that behaves exactly like water, but isn’t” is a product of our own confusion. Of the fuzziness of our concepts, the fast-and-loose-ness of our reasoning, our willingness to overlook a host of details which are actually crucially relevant to the question at hand.

(Tickling at the back of my mind is the axiom “your strength as a rationalist is your ability to be more confused by fiction than by reality.” The thing I’m gesturing toward seems to be a corollary of sorts.)

Of key importance:

Until we actually zero in on the incoherence, the imagined thing feels coherent. It seems every bit as potentially-real as actually-potentially-real options.

We have no internal feeling that warns us that it’s a fabrication masquerading as a possibility. Our brains do not tell us when they’re playing fast and loose.

Fabricated Options

Claim: When people disagree with one another, or are struggling with difficult decisions, they frequently include, among their perceived options, at least one option which is fake-in-the-way-that-XYZ-is-fake. An option that isn’t actually an option at all, but which is a product of incoherent thinking.

This is what this essay seeks to point out, and to give you taste and triggers for. I would like to establish fabricated options as a category in your mind, so that you are more likely to notice them, and less likely to be taken in by them.

Example 1: Price gouging

This example is one that many of my readers will already be familiar with; it’s the kind of topic that gets covered in Econ 101. I’m not trying to teach it to you from scratch so much as get you to see it as an instance of the class of fabricated options, so that you can port your intuitions about price gouging over to other situations.

In short: during natural disasters or other market disruptions, it often becomes difficult to deliver things like food, water, clothing, toilet paper, medical supplies, gasoline, transportation, etc., to the people who need them.

Sometimes there simply isn’t enough supply, and sometimes there’s plenty of supply but the logistics become complicated (because, for instance, the act of physically delivering things becomes significantly more dangerous).

In those situations, the price of the needed items often goes through the roof. Toilet paper selling for $100 a roll, Ubers costing $500 for a ten-mile drive, things like that.

People watching from the outside see this, and feel horror and sympathy and dismay, and often propose (and sometimes successfully enact) legal barriers to price gouging. They make it illegal to raise the price on goods and services, or put a ceiling on how much it can be raised.

Most such interventions do not produce the desired effect.

The desired effect is that people will just continue to deliver and sell items for a reasonable price, as if nothing has happened.

But that option was never really on the table. In the middle of a wildfire, or a massive flood, or raging citywide riots, or global supply chain disruption, it simply isn’t possible. The actual price of the goods and services, in the sense of “what does it take to provide them?” has gone up, and the market price will necessarily follow.

If you successfully prevent people from selling toilet paper at $100 a roll (rather than simply driving the transactions underground into a black market), the actual effect is usually that there’s no one selling toilet paper at all.

The critical insight for this essay is that the thinking of the lawmakers is confused. It is insufficiently detailed; insufficiently in touch with the reality of the situation.

The lawmakers seem to think that the options are:

  • [Do nothing], and bad people will continue doing a bad thing, and ludicrously jacking up the price on critically necessary items.

  • [Pass laws forbidding/​punishing sharp price increases in times of trouble], and the bad people will just not do the bad thing, and the critically necessary items will be available for reasonable prices.

… and in that world, given that menu of options, of course we should choose the second one!

But in reality, that is not the menu. The second option is fabricated. The story in which [passing that law] results in goods being available at normalish prices is an incoherent fairy tale. It falls apart as soon as you start digging into the details, and realize that there are forces at work which cannot be dispersed by the stroke of a lawmaker’s pen, just as there are physical laws which prevent non-weird gyroscopes and non-water XYZ.

(No matter how easy it is to imagine these things, when we gloss over the relevant details.)

In fact, the true options in most such situations are:

  • [Do nothing], and people will be able to get access to the critically necessary items, but it will be much harder and more expensive because there is low supply and high logistical difficulty.

  • [Pass laws forbidding/​punishing sharp price increases in times of trouble], and people won’t be able to get anything at all, because someone erected an artificial barrier to trade.

And given that menu of options, the first is obviously (usually) better.

Caveat 1: this could be misinterpreted (both in the specific case of price gouging and in the more general case of fabricated options) as encouraging a sort of throw-up-your-hands, if-we-can’t-solve-everything-we-shouldn’t-bother-to-try-anything helplessness.

That’s not the point. There are often ways to break the tradeoff dynamics at play, in any given situation. There are often third paths, and ways to cheat, and ways to optimize within the broken system to minimize negative effects and maximize positive outcomes.

There are, in other words, some versions of anti-price-gouging laws that do marginal good and avoid the outright stupid failure modes.

But in order to have those intelligent effects, you first have to see and account for the relevant constraints and tradeoffs, and what I am attempting to point at with the above example is the common human tendency to not do so. To simply live in the fantasy world of what we could “just” accomplish, if people would “just” do [simple-sounding but not-actually-possible thing].

Most anti-price-gouging proposals are naive in exactly the way described above; this is not meant to imply that non-naive proposals don’t exist. They do. I’m just focusing on the central tendency and ignoring the unusually competent minority.

Caveat 2: in this example and many others, the fabricated option is less a made-up action and more a made-up story about the consequences of that action. In both versions of the above dilemma, the listed actions were the same. The difference was the valence assigned to the “pass laws” option, and the story emerging from it.

This is not always the case. Sometimes people think the options are A or B, and they are in fact B or C, and sometimes people think the options are A or B and they are, but their imagination distorts the impact of option A into something utterly unrealistic.

For the sake of thinking about the category “fabricated options,” this distinction is not especially relevant, and will mostly be ignored in the rest of the essay. The important thing to note is that in either case, the fabricated option has inflated relative appeal.

Either it’s a genuinely available action A wrapped up in an incoherent and unrealistic story that makes it sound better than the unappealing B, or it’s an entirely made-up option A which makes the actual best option B look bad in comparison (causing us to fail to shoot for B over an even worse default C).

In both cases, the result in practice is that option B, which is usually sort of dour and uninspiring and contains unpleasant costs or tradeoffs, gets something like disproportionately downvoted. Downvoted relative to an impossible standard—treated as worse than it ought to be treated, given constraints.

It’s a common assumption among both rationalists and the population at large that people tend to flinch away from things which are unpleasant to think about. However, people rarely take the time to spell out just what “flinching” means, in practice, or just what triggers it.

The fabrication of options is, I claim, one example of flinching. It’s one of the things we do, as humans, when we feel ourselves about to be forced into choosing an uncomfortable path. There’s a sense of “surely not” that sends our minds in any other available direction, and if we’re not careful—if we do not actively hold ourselves to a certain kind of stodgy actuarial insistence-on-clarity-and-coherence—we’ll more than likely latch onto a nearby pleasant fiction without ever noticing that it doesn’t stand up to scrutiny.

“If only they would just [calm down/​listen/​take a deep breath/​forgive me/​let it go/​have a little perspective/​not be so jealous/​not be so irrational/​think things through more carefully/​realize how much I love them/​hang on just a little bit longer], everything would be fine.”

Pleasant fictions always outnumber pleasant truths, after all.

Example 2: An orphan, or an abortion?

This is the question posed by John Irving’s excellent novel The Cider-House Rules. The point of the question, within the novel, is to break the false dichotomy wherein the choices are framed as “a living baby or a dead/​murdered one?”

A living baby: 🙂

A dead one: 🙁, or perhaps 😡

But “living baby” in the sense often pushed for by pro-life advocates is something of a motte-and-bailey. It’s a naive, fabricated option. It hand-waves away all of the inconvenient and uncomfortable detail, in exactly the same fashion as “gyroscopes, but not weird.”

John Irving’s novel doesn’t take a stand on which is better—rather, it tries to force the reader to consider the decision at all, instead of getting confused by alluring falsehoods. The footing of the two sides, in the novel, is less uneven-by-design, which seems to me like a step in the right direction.

Example 3: Drowning

I have a longtime friend who I’ll refer to here as Taylor, who’s got a longtime romantic partner who I’ll refer to here as Kelly.

Kelly struggles with various mental health issues. They genuinely do their best, but as is so often the case, their best is not really “enough.” They spend the better part of each year depressed and mildly delusional, with frequent dangerous swerves into suicidality.

As a side effect of these issues, Kelly—who is at their core an excellent partner for Taylor—also puts Taylor through the wringer. Kelly has destroyed multiple of Taylor’s possessions, multiple times. Kelly has screamed and yelled at Taylor, multiple times. Over and over, Taylor has asked Kelly what would help, what they can do, how they could change their own behavior to be a better partner for Kelly—and over and over, granting Kelly’s explicit requests has resulted in Taylor being yelled at, punished, told to go away.

This has been rough.

Taylor is already the sort of person who doesn’t give up on people—the sort of person who would willingly sacrifice themselves for a friend or a family member, the sort of person who will go to genuinely extreme lengths to save a fellow human in trouble.

And on top of that, Taylor genuinely loves Kelly, and has plenty of evidence that—when things are okay—Kelly genuinely loves Taylor.

But for years now, the situation has been spiraling, and Taylor has been getting more and more exhausted and demoralized, and it has become increasingly clear that neither Taylor’s direct efforts, nor any of the other resources they’ve funneled Kelly’s way (therapists, medication, financial stability, freedom of movement), are going to be sufficient. It no longer seems reasonable to expect things to get better.

Taylor and I have talked about the situation a lot, and one of the metaphors that has come up more and more often is that of a drowning person out in rough waters.

From Taylor’s point of view, saving Kelly is worth it. Saving Kelly is worth it even if it means Taylor goes under. From Taylor’s point of view, the options have always been “help save Kelly, or watch Kelly drown.”

But this frame is broken. At this point, it’s clear that “help save Kelly” is not a real option. It’s a fabrication, conjured up because it is deeply uncomfortable to face the real choice, which is “let Kelly drown, or drown with them.”

(Alternatively, and a little less harshly: “let Kelly figure out how to swim on their own, or keep trying to help them and drown, yourself, without actually having helped them float.”)

Example 4: Block lists

I’ve previously had disagreements with a few people in various bubbles over block lists, and coordination, and what the defaults should be, and where various obligations lie.

In my (probably straw) characterization of the other side, they’re fabricating options. They hold a position that (probably deserves steelmanning, but given my current state of understanding) looks like:

  • Option A, everyone keeps the lines of communication open, and people don’t block each other except under extraordinary circumstances (which will tend to be legible and obvious and which basically everyone will agree upon), and that way everyone can see all of the important discussion and there aren’t confusing non-overlapping bubbles of fragmented common knowledge.

  • Option B, some people defect on the project of maintaining a clear and open commons, and block people, and make everything worse for everybody.

Option A is 🙂

Option B is clearly 🙁

In my trying-to-look-at-the-actual-tradeoffs perspective, though—

(Which is not meant to imply that the other people aren’t also trying, it just seems to me like if they are trying, they’re not quite managing to do so.)

—it seems to me that the actual options are:

  • Option B, which is very much just as 🙁 as they think it is, in which the world is imperfect and communication and coordination are tricky and costly and often go sideways, and some people need to block other people for all sorts of valid and self-protective reasons, and yep, this makes it harder to coordinate and establish common knowledge but it’s the actual best we can do—

  • or Option A, which is 😱, in which the self-protective blocking option is outlawed or disincentivized-on-the-margin, and people are either punished when they do it anyway (analogous to people being fined for selling toilet paper at inflated prices) or somehow compelled not to, in which they are either constantly exposed to triggers and to attacks from their enemies and abusers and all sorts of other things that are horrible for their mental health, or they just go dark and disappear from the conversation altogether.

The version of option A where [everyone just manages to be in the same room all the time and it’s just never disastrously problematic] is obviously better than either of the two options described above.

But it’s a substance identical to water that isn’t water. It’s not actually on the table.

Example 5: Parental disapproval

Your kid wants to hang out with another kid who you’re pretty sure is a bad influence.

Your kid wants to quit their piano lessons, sinking their previous three years of effort.

Your kid seems like they’re about to start having sex, or using drugs, or playing Magic: the Gathering.

Your kid doesn’t want to go to the family reunion.

Your kid doesn’t want to eat that.

I see parents’ hopes and expectations come up against the reality of their kids’ preferences all the time, and I always have this sucking-in-a-breath, edge-of-my-seat anticipation, because it so often seems to me like parents fabricate options rather than dealing with the tradeoffs with eyes open.

If I just tell them they can’t hang out with that kid anymore, the problem will be solved.

If I just make them keep playing piano, they’ll thank me later.

I can just tell them no.

I can just tell them they have to.

I can ground them until they comply.

As with the example of price gouging, it’s not that there aren’t good ways to intervene on the above situations. The claim is not “the options, as they are at this exact moment, are the only options that will ever be on the table.”

Rather, it’s “there are a certain limited number of options on the table at this exact moment. If none of them are satisfactory, someone will have to actively create or uncover new ones. They can’t be willed into being by sheer stubborn fiat.”

Option A, in each of the above scenarios, comes with massive costs, usually taken out of the value of the parent-child relationship.

Sure, you can ban your child from a given friendship, but what’s going to actually happen is that your child will stop viewing you as their ally and start treating you as a prison warden or appointed overseer—as obstacle to be dealt with. They’ll either succeed at getting around your edict, and you’ll have sacrificed a significant part of your mutual trust for nothing, or they’ll fail, and resent you for it.

Some parents would argue that this is fine, it’s worth it, better the kid be mad at me than suffer [bad outcome].

And in some cases that’s genuinely true.

But most of the time, the thing the parent implicitly imagines—that they can get [good outcome] and it won’t cost anything in terms of relationship capital—it’s not really on the table.

It’s not “I’ll make them play piano and everything will be fine” versus “they’ll lose their piano-playing potential.”

It’s “I’ll make them play piano by using our mutual affection as kindling” or “I’ll let them do what they want and preserve our relationship.”

Neither option is great, viewed through that lens. It’s an orphan on the one hand and an abortion on the other.

But that’s the thing. Most of the time, neither option is great. In difficult situations, it’s wise to be at least a little suspicious of straightforward, easy Options A that are just so clearly better than those uncomfortably costly tradeoff-y Options B.

Example 6: 2020

(This section left as an exercise for the reader.)

Conclusion

A likely thought on the minds of some readers is that this isn’t exactly new ground, and we already have all of the pieces necessary to individually identify each instance of fabricated options based on their inherent falsehood, and therefore don’t actually need the new category.

I disagree; I find that fine distinctions are generally useful and have personally benefitted from being able to port strategies between widely-spaced instances of option fabrication, and from being able to train my option-fabrication-recognizer on a broad data set.

That being said: beware the failure mode of new jargon, which is thinking that you now recognize [the thing], rather than that you are now equipped to hypothesize [maybe the thing?]. The world would be a better place if people’s response to the reification of concepts like “sealioning” or “DARVO” or “attention-deficit disorder” were to ask whether that’s what’s happening here, and how we would know as opposed to immediately weaponizing them.

(Alas, that’s a fabricated option, and the real choice is between “invent good terms but see them misused a bunch” and “refuse to invent good terms.” But maybe LessWrong can do better than genpop.)

As for what to do about fabricated options (both those your own brain generates and those generated by others), the general recommendation is pretty much “use your rationality” and there isn’t room in this one essay to operationalize that. My apologies.

If you’re looking for e.g. specific named CFAR techniques that might come in handy here, I’d point you toward TAPs (especially TAPs for noticing fabricated options as they come up, or booting up your alert awareness in situations where they’re likely to) and Murphyjitsu (which is likely to improve people’s baseline ability to both recognize glossed-over fairy tales and patch the holes therein). You might also work on building your general noticing skill, perhaps starting with any number of writings by Logan Strohl, and on double crux and similar tools, which will make it easier to make disagreements over the menu-of-options productive rather than not.

In the meantime, I would deeply appreciate it if any comments sharing examples of the class contained the string #EXAMPLE, and if any comments containing concrete recommendations or stories about how-you-responded contained the string #TOOLS. This will make it easier for the comment section to stand as an enduring and useful appendix to this introduction.

Good luck.

Followup from Logan Strohl: Investigating Fabrication
Related content from Ray Arnold: Things I’ve Grieved

Duncan Sabien (Deactivated)17 Oct 2021 2:47 UTC
298 points
132 comments14 min readLW link2 reviews