There is a classic distinction between between epistemic and instrumental rationality, and I like to think of epistemic rationality as a special case of instrumental rationality where map-territory correspondence is the terminal goal rather than an instrumental goal.
From LessWrong’s Wiki:
Epistemic rationality [...] can be seen as a form of instrumental rationality in which knowledge and truth are goals in themselves, whereas in other forms of instrumental rationality, knowledge and truth are only potential aids to achieving goals. Someone practicing instrumental rationality might even find falsehood useful.
What a pure and beautiful creature the epistemic rationalist is. Imagine how useful it would be if everyone thought I was one.
Indeed, it is instrumentally useful for instrumental rationalists to portray themselves as epistemic rationalists. And so this is a common pattern in human politics—“[insert political coalition] care only about themselves, while [insert political coalition] are merely trying to spread truth” is one of the great political cliches for a reason. And because believing one’s own lies can be instrumentally useful, falsely believing oneself to have a holy devotion to the truth is a not-uncommon delusion.
I try to dissuade myself of this delusion.
To the extent I am rational at all, I am instrumentally rational. I do not want to hold beliefs that will cause me harm, even if they are true. Having said that, to the extent I can modify myself to safely hold these true beliefs, I would like to do so.
What does it look like to fail at this?
The Emperor’s New Clothes is a nice parable, though the ending needs to be modified; the naively epistemically rational little boy beaten to death, or perhaps institutionalized, the Emperor’s legendary invisible fabric becoming ritually significant for generations.
We can look at this boy as a hero who bravely attempted to instigate preference falsification; we can look at him as a fool who naively believed everyone’s lie about terminally caring about the truth, or just a tragic figure too young to understand the full weight of his actions.
For the purposes of this post, I want you to see him as a rationalist who has discovered a taboo truth, and feels an unfortunate urge to shout it from the rooftops. This is almost always a pathological urge, a symptom of being in the Valley of Bad Rationality.
To the extent one desires to change social consensus on taboo topics, this should be done strategically and very, very carefully, by those with the social and political clout needed to get away with it.
If you want to be the hero who instigates preference falsification on your taboo topic of choice, your first goal should be acquiring social and material capital. But note, it is easy, too, to delude yourself into thinking you have more clout than you have, and it is often wiser to support the efforts of someone more capable or more specialized than yourself.
And indeed, it often wiser still to bow out entirely.
It is worth asking yourself why you are focused on this particular taboo truth. Why the Emperor’s clothes in particular? Why not the Pope’s supposed relationship with god? Why not the Cobbler’s guild’s propaganda on the safety hazards of unauthorized shoe repair? Is it possible you are just latching on to the first “Big Lie” you have encountered since you read a bunch about the virtue of truth?
And should you think your taboo truth is of great importance, ask yourself again if action on your part is necessary? What sort of action?
Noble lies are usually cheap signals for the individual, even if deleterious for the society as whole. Is technology changing material conditions in such a way to make this signal no longer cheap to the individual? Would you be advocating, at great personal cost, for the inevitable? If it is not, could it? Is there a less socially-legible path to the same end than public advocacy?
If you realize public advocacy of your taboo truth is not worth the trouble, pretend to believe in the noble lie if convenient. And if you can pretend without allowing yourself to believe your own pretense (which few can) even better.
When falsehoods are requisite, hypocrisy is often a virtue.
Taboo Truth
There is a classic distinction between between epistemic and instrumental rationality, and I like to think of epistemic rationality as a special case of instrumental rationality where map-territory correspondence is the terminal goal rather than an instrumental goal.
From LessWrong’s Wiki:
What a pure and beautiful creature the epistemic rationalist is. Imagine how useful it would be if everyone thought I was one.
Indeed, it is instrumentally useful for instrumental rationalists to portray themselves as epistemic rationalists. And so this is a common pattern in human politics—“[insert political coalition] care only about themselves, while [insert political coalition] are merely trying to spread truth” is one of the great political cliches for a reason. And because believing one’s own lies can be instrumentally useful, falsely believing oneself to have a holy devotion to the truth is a not-uncommon delusion.
I try to dissuade myself of this delusion.
To the extent I am rational at all, I am instrumentally rational. I do not want to hold beliefs that will cause me harm, even if they are true. Having said that, to the extent I can modify myself to safely hold these true beliefs, I would like to do so.
What does it look like to fail at this?
The Emperor’s New Clothes is a nice parable, though the ending needs to be modified; the naively epistemically rational little boy beaten to death, or perhaps institutionalized, the Emperor’s legendary invisible fabric becoming ritually significant for generations.
We can look at this boy as a hero who bravely attempted to instigate preference falsification; we can look at him as a fool who naively believed everyone’s lie about terminally caring about the truth, or just a tragic figure too young to understand the full weight of his actions.
For the purposes of this post, I want you to see him as a rationalist who has discovered a taboo truth, and feels an unfortunate urge to shout it from the rooftops. This is almost always a pathological urge, a symptom of being in the Valley of Bad Rationality.
To the extent one desires to change social consensus on taboo topics, this should be done strategically and very, very carefully, by those with the social and political clout needed to get away with it.
If you want to be the hero who instigates preference falsification on your taboo topic of choice, your first goal should be acquiring social and material capital. But note, it is easy, too, to delude yourself into thinking you have more clout than you have, and it is often wiser to support the efforts of someone more capable or more specialized than yourself.
And indeed, it often wiser still to bow out entirely.
It is worth asking yourself why you are focused on this particular taboo truth. Why the Emperor’s clothes in particular? Why not the Pope’s supposed relationship with god? Why not the Cobbler’s guild’s propaganda on the safety hazards of unauthorized shoe repair? Is it possible you are just latching on to the first “Big Lie” you have encountered since you read a bunch about the virtue of truth?
And should you think your taboo truth is of great importance, ask yourself again if action on your part is necessary? What sort of action?
Noble lies are usually cheap signals for the individual, even if deleterious for the society as whole. Is technology changing material conditions in such a way to make this signal no longer cheap to the individual? Would you be advocating, at great personal cost, for the inevitable? If it is not, could it? Is there a less socially-legible path to the same end than public advocacy?
If you realize public advocacy of your taboo truth is not worth the trouble, pretend to believe in the noble lie if convenient. And if you can pretend without allowing yourself to believe your own pretense (which few can) even better.
When falsehoods are requisite, hypocrisy is often a virtue.