I was recently thinking about a failure mode that classical rationality often recognizes and even reasonably competently challenges, yet nearly all the heuristics it uses to detect it, seem remarkably easy to use to misuse. Not only that they seem easily hackable to win a debate. How much has the topic been discussed on LW? Wondering about this I sketched out my thoughts in the following paragraphs.
On conspiracy theories
What does the phrase even mean? They are generally used to explain events or trends as the results of plots orchestrated by covert groups. Sometimes people use the term to talk about theories that important events are the products of secret plots that are largely unknown to the general public. Conspiracy in a somewhat more legal sense is also used to describe agreement between persons to deceive, mislead, or defraud others of their legal rights, or to gain an unfair advantage in some endeavour. And finally it is a convenient tool to clearly and in vivid colours paint something as low status, it is a boo light applied to any explanation that has people acting in anything that can be described as self-interest and is a few inferential jumps away. One could argue this is the primary meaning of calling an argument a conspiracy theory in on-line debates.
But putting aside the misuse of the label and associated cached thoughts, people do engage in constructing conspiracy theories when they just aren’t needed. Note that we do have plenty of historical examples of real conspiracies with pretty high stakes, so we do know they can be the right answer. Sometimes entire on-line communities fixate on them or just don’t call such bad thinking out. Why does this happen? Groups are complicated since we are social monkeys. This is something I don’t feel like going into right now, since plenty of fancy phrases like tribal attire or bandwagon effect would abound not to mention the obligatory Hansonian status based explanations, packed in a even bigger wall of text. Let us then first take a look at why individuals may be biased towards such explanations.
First off we have a hard time understanding that coordination is hard. If we see a large pay off available and thinking it easily in reach if “we could just get along” seems like a classical failing. Our pro-social sentiments lead us to downplay such barriers in our future plans. Motivated cognition on behalf of assessing the threat potential of perceived enemies or strangers likely shares this problem. Even if we avoid this, we may still be lost since the second big relevant thing is our tendency for anthropomorphizing things that better not be. A paranoid brain seeing agency in every shadow or strange sound, seems like something evolution would favour over one that fails to see it every now and then. In other words the cost of false positives was reasonably low. Also our brains are just plain lazy, the general population is pretty good at modelling other human minds and considering just how hard the task is, we do a pretty remarkable job of it. So when you want rain you do a rain dance to appease the sky spirits since the weather is pretty capricious and angry sky spirits is a model that makes as much sense as any other (when you are stuck in relative ignorance) and is cheap to run on your brain. The modern world is remarkably complex. Our Dunbarian minds probably just plain can’t get how a society can be that complex and unpredictable without it being “planned” by a cabal of Satan or Heterosexual White Males or the Illuminati (but I repeat myself twice) scheming to make weird things happen in the small stone age tribe. Learning about and gaining confidence in some models helps people escape anthropomorphizing human society (this might sound strange but here on LW we are wary of doing this to people, ha beat that!) or the economy or government. The latter is particularly salient since the idea that say something like the United States government can be successfully modelled as a single agent to explain most of its actions is something I dare say most people slip up on occasionally. And lastly… naughty secret conspiracy and malignant agency just plain make a good story.
An argument against a conspiracy theory is probabilistic, because we don’t deny that conspiracies exist, only that in this specific case, a non-conspiracy explanation is more probable than a conspiracy explanation, therefore focusing on the conspiracy explanation is privileging a hypothesis.
People are not very good at probabilistic reasoning. So some of them prefer an interesting story. And others try to reverse stupidity by making fully general counterarguments against conspiracies.
The situation is further complicated by not having a precise definition of what conspiracy is. Does it require mutual verbal agreement, or does a silent cooperation on Prisonner’ Dilemma also count as a conspiracy? (Two duopolistic producers decide to avoid lowering their product prices, without ever speaking with each other.) Somewhere between this is a cooperation organized by people who avoid to speak about the topic directly. (Each of the duopolistic producers publishes a press article “we try to provide the best quality, because making cheap junk would be bad for our customers”.) Actually, the players can even deceive themselves that they are really following a different goal, and the resulting cooperation is just a side effect.
Related to: List of public drafts on LessWrong
Article based on this draft: Conspiracy Theories as Agency Fictions
I was recently thinking about a failure mode that classical rationality often recognizes and even reasonably competently challenges, yet nearly all the heuristics it uses to detect it, seem remarkably easy to use to misuse. Not only that they seem easily hackable to win a debate. How much has the topic been discussed on LW? Wondering about this I sketched out my thoughts in the following paragraphs.
On conspiracy theories
What does the phrase even mean? They are generally used to explain events or trends as the results of plots orchestrated by covert groups. Sometimes people use the term to talk about theories that important events are the products of secret plots that are largely unknown to the general public. Conspiracy in a somewhat more legal sense is also used to describe agreement between persons to deceive, mislead, or defraud others of their legal rights, or to gain an unfair advantage in some endeavour. And finally it is a convenient tool to clearly and in vivid colours paint something as low status, it is a boo light applied to any explanation that has people acting in anything that can be described as self-interest and is a few inferential jumps away. One could argue this is the primary meaning of calling an argument a conspiracy theory in on-line debates.
But putting aside the misuse of the label and associated cached thoughts, people do engage in constructing conspiracy theories when they just aren’t needed. Note that we do have plenty of historical examples of real conspiracies with pretty high stakes, so we do know they can be the right answer. Sometimes entire on-line communities fixate on them or just don’t call such bad thinking out. Why does this happen? Groups are complicated since we are social monkeys. This is something I don’t feel like going into right now, since plenty of fancy phrases like tribal attire or bandwagon effect would abound not to mention the obligatory Hansonian status based explanations, packed in a even bigger wall of text. Let us then first take a look at why individuals may be biased towards such explanations.
First off we have a hard time understanding that coordination is hard. If we see a large pay off available and thinking it easily in reach if “we could just get along” seems like a classical failing. Our pro-social sentiments lead us to downplay such barriers in our future plans. Motivated cognition on behalf of assessing the threat potential of perceived enemies or strangers likely shares this problem. Even if we avoid this, we may still be lost since the second big relevant thing is our tendency for anthropomorphizing things that better not be. A paranoid brain seeing agency in every shadow or strange sound, seems like something evolution would favour over one that fails to see it every now and then. In other words the cost of false positives was reasonably low. Also our brains are just plain lazy, the general population is pretty good at modelling other human minds and considering just how hard the task is, we do a pretty remarkable job of it. So when you want rain you do a rain dance to appease the sky spirits since the weather is pretty capricious and angry sky spirits is a model that makes as much sense as any other (when you are stuck in relative ignorance) and is cheap to run on your brain. The modern world is remarkably complex. Our Dunbarian minds probably just plain can’t get how a society can be that complex and unpredictable without it being “planned” by a cabal of Satan or Heterosexual White Males or the Illuminati (but I repeat myself twice) scheming to make weird things happen in the small stone age tribe. Learning about and gaining confidence in some models helps people escape anthropomorphizing human society (this might sound strange but here on LW we are wary of doing this to people, ha beat that!) or the economy or government. The latter is particularly salient since the idea that say something like the United States government can be successfully modelled as a single agent to explain most of its actions is something I dare say most people slip up on occasionally. And lastly… naughty secret conspiracy and malignant agency just plain make a good story.
Humans loooove stories.
Polish this and it will make a decent discussion post.
I have since expanded and polished it into an article. I hope it isn’t unworthy!
Related: Reversed stupidity is not intelligence; Knowing About Biases Can Hurt People.
An argument against a conspiracy theory is probabilistic, because we don’t deny that conspiracies exist, only that in this specific case, a non-conspiracy explanation is more probable than a conspiracy explanation, therefore focusing on the conspiracy explanation is privileging a hypothesis.
People are not very good at probabilistic reasoning. So some of them prefer an interesting story. And others try to reverse stupidity by making fully general counterarguments against conspiracies.
The situation is further complicated by not having a precise definition of what conspiracy is. Does it require mutual verbal agreement, or does a silent cooperation on Prisonner’ Dilemma also count as a conspiracy? (Two duopolistic producers decide to avoid lowering their product prices, without ever speaking with each other.) Somewhere between this is a cooperation organized by people who avoid to speak about the topic directly. (Each of the duopolistic producers publishes a press article “we try to provide the best quality, because making cheap junk would be bad for our customers”.) Actually, the players can even deceive themselves that they are really following a different goal, and the resulting cooperation is just a side effect.