Human Evil and Muddled Thinking
George Orwell saw the descent of the civilized world into totalitarianism, the conversion or corruption of one country after another; the boot stamping on a human face, forever, and remember that it is forever. You were born too late to remember a time when the rise of totalitarianism seemed unstoppable, when one country after another fell to secret police and the thunderous knock at midnight, while the professors of free universities hailed the Soviet Union’s purges as progress. It feels as alien to you as fiction; it is hard for you to take seriously. Because, in your branch of time, the Berlin Wall fell. And if Orwell’s name is not carved into one of those stones, it should be.
Orwell saw the destiny of the human species, and he put forth a convulsive effort to wrench it off its path. Orwell’s weapon was clear writing. Orwell knew that muddled language is muddled thinking; he knew that human evil and muddled thinking intertwine like conjugate strands of DNA:1
In our time, political speech and writing are largely the defence of the indefensible. Things like the continuance of British rule in India, the Russian purges and deportations, the dropping of the atom bombs on Japan, can indeed be defended, but only by arguments which are too brutal for most people to face, and which do not square with the professed aims of the political parties. Thus political language has to consist largely of euphemism, question-begging and sheer cloudy vagueness. Defenceless villages are bombarded from the air, the inhabitants driven out into the countryside, the cattle machine-gunned, the huts set on fire with incendiary bullets: this is called PACIFICATION . . .
Orwell was clear on the goal of his clarity:
If you simplify your English, you are freed from the worst follies of orthodoxy. You cannot speak any of the necessary dialects, and when you make a stupid remark its stupidity will be obvious, even to yourself.
To make our stupidity obvious, even to ourselves—this is the heart of Overcoming Bias.
Evil sneaks, hidden, through the unlit shadows of the mind. We look back with the clarity of history, and weep to remember the planned famines of Stalin and Mao, which killed tens of millions. We call this evil, because it was done by deliberate human intent to inflict pain and death upon innocent human beings. We call this evil, because of the revulsion that we feel against it, looking back with the clarity of history. For perpetrators of evil to avoid its natural opposition, the revulsion must remain latent. Clarity must be avoided at any cost. Even as humans of clear sight tend to oppose the evil that they see; so too does human evil, wherever it exists, set out to muddle thinking.
1984 sets this forth starkly: Orwell’s ultimate villains are cutters and airbrushers of photographs (based on historical cutting and airbrushing in the Soviet Union). At the peak of all darkness in the Ministry of Love, O’Brien tortures Winston to admit that two plus two equals five:2
“Do you remember,” he went on, “writing in your diary, ‘Freedom is the freedom to say that two plus two make four’?”
“Yes,” said Winston.
O’Brien held up his left hand, its back towards Winston, with the thumb hidden and the four fingers extended.
“How many fingers am I holding up, Winston?”
“Four.”
“And if the party says that it is not four but five—then how many?”
“Four.”
The word ended in a gasp of pain. The needle of the dial had shot up to fifty-five. The sweat had sprung out all over Winston’s body. The air tore into his lungs and issued again in deep groans which even by clenching his teeth he could not stop. O’Brien watched him, the four fingers still extended. He drew back the lever. This time the pain was only slightly eased.
I am continually aghast at apparently intelligent folks—such as Robin Hanson’s colleague Tyler Cowen—who don’t think that overcoming bias is important.3 This is your mind we’re talking about. Your human intelligence. It separates you from an orangutan. It built this world. You don’t think how the mind works is important? You don’t think the mind’s systematic malfunctions are important? Do you think the Inquisition would have tortured witches, if all were ideal Bayesians?
Tyler Cowen apparently feels that overcoming bias is just as biased as bias: “I view Robin’s blog as exemplifying bias, and indeed showing that bias can be very useful.” I hope this is only the result of thinking too abstractly while trying to sound clever. Does Tyler seriously think that scope insensitivity to the value of human life is on the same level with trying to create plans that will really save as many lives as possible?
Orwell was forced to fight a similar attitude—that to admit to any distinction is youthful naiveté:
Stuart Chase and others have come near to claiming that all abstract words are meaningless, and have used this as a pretext for advocating a kind of political quietism. Since you don’t know what Fascism is, how can you struggle against Fascism?
Maybe overcoming bias doesn’t look quite exciting enough, if it’s framed as a struggle against mere accidental mistakes. Maybe it’s harder to get excited if there isn’t some clear evil to oppose. So let us be absolutely clear that where there is human evil in the world, where there is cruelty and torture and deliberate murder, there are biases enshrouding it. Where people of clear sight oppose these biases, the concealed evil fights back. The truth does have enemies. If Overcoming Bias were a newsletter in the old Soviet Union, every poster and commenter of Overcoming Bias would have been shipped off to labor camps.
In all human history, every great leap forward has been driven by a new clarity of thought. Except for a few natural catastrophes, every great woe has been driven by a stupidity. Our last enemy is ourselves; and this is a war, and we are soldiers.
1George Orwell, “Politics and the English Language,” Horizon, 1946.
2George Orwell, 1984 (Signet Classic, 1950).
3See Tyler Cowen, “How Important is Overcoming Bias?,” Marginal Revolution (blog), 2007, http://marginalrevolution.com/marginalrevolution/2007/08/how-important-i.html.
- Doublethink (Choosing to be Biased) by (14 Sep 2007 20:05 UTC; 98 points)
- Stop Voting For Nincompoops by (2 Jan 2008 18:00 UTC; 76 points)
- Protected From Myself by (19 Oct 2008 0:09 UTC; 47 points)
- False Laughter by (22 Dec 2007 6:03 UTC; 42 points)
- Time-Binding by (14 Aug 2015 17:38 UTC; 27 points)
- 's comment on Base your self-esteem on your rationality by (22 Jul 2015 11:22 UTC; 17 points)
- Rationality Reading Group: Part F: Politics and Rationality by (29 Jul 2015 22:22 UTC; 14 points)
- [SEQ RERUN] Human Evil and Muddled Thinking by (27 Aug 2011 3:43 UTC; 11 points)
- Summarizing the Sequences Proposal by (4 Aug 2011 21:15 UTC; 9 points)
- 's comment on Large Language Models will be Great for Censorship by (23 Aug 2023 23:57 UTC; 1 point)
- Where “the Sequences” Are Wrong by (7 May 2023 20:21 UTC; -15 points)
In my opinion, this is Establishment writing (Mao and Stalin as the “bad guys”) only problematic to me because the strength against challenges of your presented ideas nontransparently rest on power alignment, rather than just their accuracy of modeling apparent reality and devising solutions to challenges we face.
This is the great case against hypocrisy, that hypocrisy allows us to act contrary to our ideals, and at times our ideals could have prevented holocausts. I suppose on the other side of the ledger must go the various “social graces” where hypocrisy supposedly smooths social interactions and lets us save face. Is there a way to weigh these two sides against each other, or is there a way to distinguish them, so we could have the good hypocrisy without excessive risk of the bad slipping in too?
Hypocrisy is a protection against bad ideals as well as an impediment to achieving good ideals.
If religious people we not hypocrites, we would all be burned at the stake.
This depends how the counterfactual is constructed—ie. when they stopped being hypocrites and whether the non-hypocrisy is prevented from causing the no-longer-hypocritical people to lose their religion. I mean—we might win and kill all the religious people!
The assumption is that people start doing things that match with their stated beliefs- so, for instance, people who claim to oppose genocide would actually oppose genocide in all cases, which is the whole point of thinking hypocrisy is bad. Causing people to no longer be hypocrites by making them instead give up their stated beliefs would just make for a world which was more honest but otherwise not dramatically improved.
Incidentally, on the joking side: If atheists did win the religious war, they could then use this statement in a completely serious and logical context: https://www.youtube.com/watch?v=FmmQxXPOMMY
Granted. We might hope that they would stop being hypocrites not by abandoning their behavior-motivating beliefs (e.g. “it is wrong to murder someone for being an atheist”) but instead by abandoning their professed “beliefs” (e.g. “nonbelievers will suffer in Hell for eternity”); but it’s hard for me to say which would actually happen. We do know that under certain circumstances, human beings definitely will burn other humans to death in the name of religious belief.
Well, professed beliefs and behavior-motivating beliefs are often integral. However, usually conversion rather than immolation is the goal of a believer who believes that atheists will burn in hell for eternity. Generally, the goal is to get everyone to heaven, and most people realize that by burning an atheist, they are not being rational about their faith, and secondly not making it look good. In fact, rarely in any religious text (perhaps excluding the Koran—correct me if I’m wrong) does one find an instruction to kill atheists.
That rather depends on what tenets of their religion they choose to be hypocritical about.
That depends on the religion. Not all religions have a strong concern with heresy or other sorts of non-belief, and not all religions permit violence.
Most religions do not dictate that heretics be burned at the stake. And if all religious people were non-hypocritical to the basic tenets of a religion (see Ten Commandments, Five Pillars of Islam, etcetera) rather than to specific instructions that are open to interpretation, the world would probably be a much better place.
Ten Commandments are just a part of a larger whole. Why would non-hypocritical people accept this part, and reject the parts about “if you see someone doing this or that, kill them”? Both parts come from the same book.
I assume that what you are referring to are some of the laws encountered in the Old Testament, which were part of a legal structure designed to apply to the Israelite nation (and no one else, point of interest). From a Judaism perspective, the law is supposed to apply only to Jews—those who are part of the religion and the race. Only Jews are affected, unless people attack the Jews and the Jews fight back (see most of the Israeli conflicts). From a Christian perspective, there is an explanation about the relevance of the Law to Christianity, explaining that as Christianity replaced Judaism, for obvious reasons sacrifices did not need to be carried out anymore. For similar reasons, the whole of the Mosaic Law did not need to be adhered to; the important things according to Jesus were to ‘love God, and love your neighbour as yourself’. So, those are the operatives on which Christians are supposed to base their behavior. I think you would agree that Buddhism, Sikhism, Hinduism, Taoism, and Confucianism, along with most of the minor religions and cults do not pose much of a threat to humanity in terms of the instructions contained within them. Islam, granted, is a whole other story. There are a number of contradictions in the Koran for which complicated rules have been devised to find out the ‘correct’ interpretation, based on which part was written first and whether the Hadith applies, etcetera. There is particular conflict over whether to act on Surat At-Tabwah 9:5 (‘kill the Polytheists wherever you find them… and sit in wait for them at every place of ambush’) or Surat Al-Baqarah 2:256 (‘Let there be no compulsion in religion’). The Hadith has even more to say on the issue. Most moderate Muslims place the latter Surat over the former, many Shiites however interpret it a different way.
In any case, the number of non-hypocritical people who would be doing massive amounts of good would outweigh the number of non-hypocrites doing evil—although the definitions of ‘Good’ and ‘Evil’ have to be looked at in the context of—wait for it! religion. However if, as I assume (please correct me if I’m wrong) you are basically a Utilitarian, it can be deduced that the net amount of human suffering would decrease if people were non-hypocritical about their religion. Of course, interpretation will always play a part. I hope that answers your comment/question.
Yes, because murder and genocide make perfect sense as long as you restrict them to a particular place and time! And there are such things as “races” and it makes sense for them to be units of moral analysis. And obviously “she must marry her rapist” (Deuteronomy 22:28-29) is a totally sensible rule for an ancient culture, and neither the Greeks nor the Chinese had figured out anything even remotely better by that time period. Yes, obviously, it was totally fair for Moses to be talking about slaughtering the Amalekites (and their children, and their cattle; Deuteronomy 20:16-17) at the same time in history when Demosthenes and Epicurus were debating about the proper form of democratic government. And no one today takes those ideas seriously, and certainly there aren’t millions of Americans who use passages from Leviticus (18:22 and 20:13) to argue against gay marriage.
And of course Jesus came to change the rules; that’s why he put it so plainly in Matthew 5:17-19:
\end{sarcasm}
Witness what religion does to a human mind; it makes an otherwise intelligent and reasonable person defend the obviously indefensible, because they cannot bear to accept the obvious fact that what they were told isn’t true. Suddenly genocide becomes “a different time” and rape becomes “their culture”, because the thought that so many people’s precious beliefs are false is simply too much to bear. Contradictions in holy books are somehow seen as a good thing, because they let you take whatever meaning you want and declare the result infallible (when it’s obvious from basic logic that contradictions in beliefs are always bad).
Of course no religion is harmless. Delusions are never a good thing. Some religions are more harmful than others, I’ll grant you that; but if you want to know why Islam is particularly bad, it’s because it actually follows the book. Jews and Christians have largely given up on the crazy evil books, and so they can behave (mostly) like reasonable human beings. Muslims haven’t, and that’s why they do things like hang gay people and keep women covered head to toe. Confucians are an interesting case, in that their books contain falsehoods, but are not genocidally insane, so that counts for something. Jain are also crazy, but crazy in a way that makes them relatively harmless—like the Amish. So if I could make every Muslim in the world suddenly turn Jain, I would; but I’d rather turn them atheist. What’s more, I find it’s easier to make people atheist, because the rational part of their brains already wants to.
As for what evil means, no, it has nothing to do with religion (other than the obvious fact that religion makes assertions about it, just as religion makes unfounded assertions about literally everything). Evil is found in human suffering, particularly when it could be easily prevented. It is found in death and destruction, especially when we are in a position to avoid them. Am I a utilitarian? Yes, I suppose I am—if you are not, you must be saying that your decisions can’t be made to fit a Von Neumann-Morgenstern decision utility… and isn’t that a lot like saying your decisions are irrational? If you meant to say that human beings rarely engage in intentional evil (accidental and negligent evil is far more common), that’s actually a very good point; but then, this is just one more problem with religion, because religion often asserts that our enemies are servants of demons whose only goals are pure evil.
The net amount of human suffering would be decreased if people abandoned religion altogether. If they continued to believe in religion and stopped being hypocrites, no… I stand by my previous claim. They would burn people like me and most of the rest of Less Wrong at the stake. The war between Catholics and Protestants in Ireland would flare up once again, and really if theology is as important as people say, even Baptists and Methodists should be torturing each other over doctrinal differences. It is only a lack of religious fervor that defends civilization as we know it; and if given the choice between fanaticism and hypocrisy I wholemindedly express my preference for hypocrisy.
I suppose actually reading the book is out of the question. :) The Bible is not a book of rules and ideals to follow, it’s a history book. What Moses or anyone did isn’t right just because it’s in the Bible. Plenty of people take these ideas seriously. But what they take seriously (most of them) are the important things, like how to treat other people, not what to do if your second cousin kills your brother’s cow or something like that. Contrairy to popular belief, there is scientific, historical, and philisophical evidence to support the Bible. The problem is, a lot Christians are really deluded and unclear as to what they are, and which ones are actually important. Even when I’m in church listening to a sermon, I see things that I don’t like, things that make assumptions and are confusing and not welcoming if you don’t already believe it. And yes, that sucks, and yes, that’s a problem, but that does not mean the Bible is untrue, and it does not mean that religious people are unintelligent and unreasonable. They don’t think that genocide and rape are good things, either. Again, the Bible is a history book. Learn from its example and don’t make the same mistakes. And I havn’t investigated it myself, but many reasonable, intelligent people have read the Bible and tried to find contradictions. A lot of Atheists-turned-Christians did so because of the phrase “intellectual honesty”. They investigated the Bible for themselves and found that they couldn’t honestly tell themselves that it wasn’t true. It’s true that terrible things happen because of the Christian religion, but just because something is done in the name of a religion doesn’t mean that that religion supports it. Enemies of Christianity are not evil. They’re people too, and they’re good people. Christians try to evangilize people because they believe it’s for their own good. Burning you at the stake would probably kill you, and that defeats the purpose. If a Christian ever beats you up for dissing his faith, tell him he’s a hypocryte, becuase he is. The Catholic-Protestant thing is not supposed to be important. According to the Bible, the Catholic-Protestant argument thing is wrong. Rationality is all about TRUTH. If people lied and killed in the name of science, that wouldn’t make science evil and it wouldn’t make science wrong. I’m only in junior high, and I didn’t understand all of your comment, but the things I did understand I tried to . . . I don’t know, correct. And by the way, atheism is a religion too.
Worth elaborating: If all religious people were non-hypocritical and do exactly what the religion they claim to follow commands, there would probably be an enormous initial drop in violence, followed by any religions that follow commandments like “thou shalt not kill” without exception being wiped out, with religions advocating holy war and the persecution of heretics getting the eventual upper hand (although imperfectly adapted religions might potentially be able to hold off the better-adapted ones through strength of numbers- for instance, if a large area was controlled by a religion with the burning of heretics and defensive, cooperative religious wars, they could hold off smaller nations with religions advocating offensive wars).
One good thing about hypocrisy is that it makes a massive buffer against certain types of virulent memes. On the other hand, a world where everyone took a burn-the-heretics interpretation of Christianity or Islam 100% seriously would certainly have some advantages over ours, and especially over our middle ages- things like no un-sanctioned killing, most notably, no wars against others of the same religion, etc. Probably lots of things that would be decent ideas if you could get everyone to follow them, at the cost of an occasional burnt heretic (and possibly constant holy wars, until one religion gains the upper hand and overwhelms the others).
Sounds like the history of Europe and the Islamic world. Except that no-one ever did get the upper hand, neither for Christianity vs. Islam, nor the splits within those faiths.
Anyone want to go back to the time of the Crusades?
If the only thing in favour of an idea is how wonderful the world would be if everyone followed it, it’s a bad idea.
Almost entirely agreed. The one class of exceptions are cases where a single standard avoids some severe problem with a mix. “Elbonia will switch from driving on the left to driving on the right. The change will be made gradually.”
In a bit more general case, you would like to standardise things with a huge network effect. Like TCP/IP, for example.
Is it correct to say that he who is not coherent is hypocritical? I’m used to think of hypocrisy as someone who does not apply to himself the criteria he wants to apply to others. I can think of some reasons why it is not plausible that there can be found people completely coherent:
people are not aware of all their ideas at the same time;
people change their minds;
people can hold inconsistent ideas, at least when they are not aware of it;
Another thought: human minds are the environment where memes develop, but one should notice that memes are also the environment in which humans act. That means that even firmly believing something to be wrong someone can still decide to do it, and vice-versa.
I observe that the cases where hypocrisy is beneficial are usually cases where a negative action is recommended and cases where hypocrisy has negative value are usually cases where a positive action is recommended.
I wonder if hypocrisy is simply a patch on reasoning to include risk aversion—or even inaction!
If we are in a situation which necessitates hypocrisy with regard to our current ideals in order to maintain ‘social graces’, we have to ask ourselves whether the integrity of our ideals is more important than preserving said social graces. Hypocrisy is more often a way for us to evade the more onerous parts of our ideals than it is a way to preserve ‘social graces’; in these cases we have no excuse for our hypocrisy, and must see it as negative. If ‘preservation of social graces’ is the purpose of the said hypocrisy, then ‘preservation of social graces’ has become an ideal for us, and we must decide whether our former ideological system will throw out this new ideal, or whether we pin our life on our social interactions. If we include the concept of ‘ideals’, we must see new ideals as ideals and measure them against each other. Of course, this can be a circular process and often relies on a gut feeling, but if something is an ‘ideal’, we cannot allow hypocrisy, because if we think that the hypocrisy in a situation is a good thing, our ideals have changed without us knowing it and we should revise, and make a conscious decision regarding this.
tl;dr: Hypocrisy is compartmentalization of ideals for preserving ‘social graces’.
In this way it is similar to democracy, which was (in the U.S. among other places) originally intended to, and certainly does now prevent the government from accomplishing much of anything, on the logic that one black-swan period of tyranny is worse than considerable amounts of efficient authoritarian development. Consider the idea of the “separation of powers”: it cannot be called anything other than a handicap on all activities and conscious long-term sabotage (except that I am told that it affects little in practice, for good or ill).
I don’t really endorse that line of thinking—anyways, the vast majority of tyranny operates in a rather “democratic” grass-roots way and starts in the family (see The White Ribbon by Michael Haneke, an excellent film) - but it certainly has many proponents.
When did Tyler say that overcoming bias is not important? Are you talking about this post here: http://www.marginalrevolution.com/marginalrevolution/2007/08/how-important-i.html
He says it’s not the most important thing in the world, not that it’s not important. So, is your reading of Cowen biased? Or did I miss something?
J, that’s Tyler quoting a summary of Hanson, not Tyler himself.
Robin, I think it’d be pretty silly to trade off social graces against the Inquisition, especially when some creative thinking could devise alternative social graces.
Seriously, we’re trading off social graces against what?
In all human history, every great leap forward has been driven by a new clarity of thought. Except for a few natural catastrophes, every great woe has been driven by a stupidity. Our last enemy is ourselves; and this is a war, and we are soldiers.
Amazing quote.
Summary: if they can make you believe absurdities, they can make you commit atrocities.
Amen.
“In my opinion, this is Establishment writing (Mao and Stalin as the “bad guys”) only problematic to me because the strength against challenges of your presented ideas nontransparently rest on power alignment, rather than just their accuracy of modeling apparent reality and devising solutions to challenges we face.”
Don’t you love post modernists? their rhetoric is frankly baffling to common sense (what they would term a mental power-structure—I’m not sure even they understand what it is their writing half the time)
Its obviously been long since humanity had to strive for objectivity and earnest disinterest because human bias was creating so much suffering in the world.
There isn’t much known about Hopefully Anonymous, but I’m fairly certain he does not consider himself a post-modernist.
To be fair, imprisonment-as-punishment does indeed lead towards advocating torture, if you’re avoiding hypocrisy and keeping an open mind. I hadn’t noticed this before, because it’s too abhorrent to have occurred to me as a strategy.
However, I’d wager that imprisonment-as-punishment is the flaw in the argument here. Imprisonment-to-protect-others and imprisonment-as-rehabilitation are far more effective, in my opinion (I haven’t actually looked into this much; it just sounds sensible to me), and I think a lot of people primarily advocate imprisonment-as-punishment because they have a desire for revenge.
The artist is not the Art. Many an artist will fail, but the Art does not.
mtraven—To be against bias is not the same as to be without bias, so even if we were to grant that James is “comfortably against bias and for torture without losing a moment’s sleep”, that would not constitute a counterexample to Eliezer’s claim that “where there is...torture...there are biases enshrouding it.”
“This is the great case against hypocrisy, that hypocrisy allows us to act contrary to our ideals, and at times our ideals could have prevented holocausts. I suppose on the other side of the ledger must go the various “social graces” where hypocrisy supposedly smooths social interactions and lets us save face. Is there a way to weigh these two sides against each other, or is there a way to distinguish them, so we could have the good hypocrisy without excessive risk of the bad slipping in too?”
I look at this a bit differently. Let’s say there are four of us (Robin, Eliezer, TGGP, and me, HA) that want to persist as subjective conscious entities. What actions that the 4 of us take will maximize our persistence odds? I think every ideal should be subordinated to that. It’s on this basis that I would criticize (or promote) hypocricies against ideals, and the occurance (or non-occurance) of holocausts.
“persist as subjective conscious entities. What actions that the 4 of us take will maximize our persistence odds? I think every ideal should be subordinated to that.”
I’m not entirely sure I know what that means, but it SOUNDS like “each of us wants to be different—it’s more important than anything else that we be different from each other.”
If my reading is correct, I fail to see why I would want to consider that my overriding priority. Particularly in placing over, say, the truth.
And Eliezer, it just keeps getting better and better. I had to stop reading Marginal Revolution precisely because, brilliant as Tyler is, he really does try to be too clever for his own good. It becomes frustrating to watch.
Disregard—retracted
Flynn, response on my blog within the next ten minutes (“being different” isn’t what I mean).
http://www.hopeanon.typepad.com
Flynn, HA is a near-total egoist who wants to live forever.
Why would a near-total egoist hope to be anonymous? :-)
HA, replied on your blog. Sorry about the confusion.
I feel you exaggerate the case here Eliezer. Overcoming bias will not solve all the problems in the world. There’s even a chance it could make them worst. Let’s look at hypocrisy, for instance:
This is the great case against hypocrisy, that hypocrisy allows us to act contrary to our ideals, and at times our ideals could have prevented holocausts. I suppose on the other side of the ledger must go the various “social graces” where hypocrisy supposedly smooths social interactions and lets us save face. Is there a way to weigh these two sides against each other, or is there a way to distinguish them, so we could have the good hypocrisy without excessive risk of the bad slipping in too?
There’s a huge positive chunk of hypocrisy that we’re missing there—hypocrisy allows us to have ideals higher than we can (and do) attain in our actions. It can have a tremendously aspirational effect. The phrase “all men are created equal” was written by rich, white slaveowners. Eliezer feels that if hypocrisy had been banned, they would have written the same phrase, and set all their slaves free. I fear that if hypocrisy had been banned, they would have kept their slaves and instead written “all rich, white men are created equal”. And future progress would have been ruled out.
To summarise: hypocrisy is the distance between ideals and actions. Erasing that distance does not tell us whether our actions will rise or if our ideals will fall—especially over several generations. Without hypocrisy, people may just become more tolerant of those “arguments which are too brutal for most people to face”.
Related:
“Sometimes a hypocrite is nothing more than a man in the process of changing.” ― Brandon Sanderson, Oathbringer (By Dalinar Kholin)
Flynn,
Egoist!=egotist.
There’s a huge positive chunk of hypocrisy that we’re missing there—hypocrisy allows us to have ideals higher than we can (and do) attain in our actions. It can have a tremendously aspirational effect. The phrase “all men are created equal” was written by rich, white slaveowners. Eliezer feels that if hypocrisy had been banned, they would have written the same phrase, and set all their slaves free. I fear that if hypocrisy had been banned, they would have kept their slaves and instead written “all rich, white men are created equal”. And future progress would have been ruled out.
Ah, now there’s a powerful argument.
But at the same time, if hypocrisy had not decreased, we would still have rich white slaveowners.
It should be assumed by default that when I talk about the benefits of overcoming bias, I am talking about the sort of human being who comes into existence when they set out to overcome their own biases by acts of mental will and training. Not, necessarily, the sort of entity that you get if you do neurosurgery on a human; nor the sort of entity that would have evolved if deception and self-deception had not been part of the ancestral environment.
The sort of human being who makes a continual effort to overcome hypocrisy, and who manages to do so, will probably set the slaves free.
(Claiming that “ideal Bayesians” would not have sponsored an Inquisition obscures this point, since it launches too great a counterfactual; this was probably a mistake of writing, if not of fact.)
I regret that I have to disagree with the post, even though I am a great fan of Orwell.
Stalin and Hitler did not suffer from lack of clarity. They knew exactly what they were doing, knew why they were doing it, and were glad of the outcome. More logic and better writing would simply have helped them be even more effectively evil. Teaching clear thinking is important; but it will not stop evil people from having evil intentions or acting evil. Evil emerges from the heart and soul, not the head. Intellectuals who supported, and support, Lenin, Stalin, Hitler, Mao, Castro, Ho Chi Minh, Osama, Saddam, and so forth, knew what they doing. They got a vicarious thrill from the results, even if they did not get their hands dirty. Yes, they may have used wretched writing to hide the consequences from others, but they knew what they wanted. I might add that Orwell’s hands were not clean; he fought with the Communists in Spain, he advocated hard-line total socialism in England of a type that would make George Mason economists gag.
Maybe you suffer from “intellectualist bias.” Academics commonly do. That is a bias, that might go all the way back to Socrates, that the world only needs education to be good. A few courses in logic, rhetoric, and good writing, and everthing will be okay. So, sorry, but intellectualist bias may be the hardest to overcome.
So, in a sense, I am with Tyler on this one. It is good to overcome confirmation bias, or attribution bias, and so forth, but they are not at the top of my list.
Helping people to open their eyes and see human suffering, raising children to be compassionate, will do far more to get rid of the Hitlers and Castros than logic and writing classes. Maybe that is just my bias.
Yes, hypocrisy is not the problem with them.
No, I don’t think that they did or do! Orwell was writing to intellectuals who were in denial about what Stalin was doing and why. Here is where hypocrisy causes problems.
One might say that, in a general sense, Stalin and Hitler were in possession of ‘cold logic’, much like ‘cold empathy’. One can know both how to, and how not to, steal the cookie, and the effects that will have, and the moral consequences of that, but in the end, if one’s intent is evil, then one’s actions will be evil, especially if one knows all the consequences of their actions. Logic is blind; in the end, it is subservient to the will of the wielder, and merely amplifies the actions of the individual, whether good or evil. In the same way, hypocrisy is only a ‘good’ or ‘evil’ thing when it contradicts ‘evil’ or ‘good’ ideals. One can think one’s way into and out of any situation one wishes; who here has read Donne’s poetry?
I disagree. They managed to convince themselves that the people they were killing weren’t really people.
Helping people open their eyes means making them stop lying to themselves. People lying to themselves is one of the largest causes of bias. That is a very good example of why overcoming bias is important.
How do you know that to be true? Especially as I’m not sure which German word you are referring to when you speak of ‘people’ if you are referring to any at all.
You have an attractor for “rube” and “blegg”. If something is “really a blegg”, that means that, once you know everything about it, you’d sort it as a blegg. You might currently sort it as “unknown”, but since you would sort it as a blegg, it’s really a blegg.
You also have an attractor for “person”. You feel empathy for people. You care if they die. If you know everything about a human, they are sorted into “person”. It’s not really rational. They obviously have a name, and every name sorts them into “person”, but somehow they only get sorted into there if you know what it is. Nonetheless, since everyone would get sorted into “person” if you knew enough about them, they’re all people.
If Hitler personally knew the people he was killing, he wouldn’t be okay with killing them.
??? Knowing people doesn’t mean you like them.
I think that’s wrong for Hitler. It’s my impression that Hitler was willing to kill anyone he considered a traitor whether or not he knew the person personally. He didn’t killed as much people he knew personally as Stalin but I think he was capable of that feat.
I generally don’t like “taboo this word” but you could make a good case for tabooing “people” here.
If by “people”, DanielC meant “entities which have rights and whose rights deserve to be respected”, then of course Hitler thought he wasn’t killing people, but that is just vacuously true.
That assumes that Hitler believed in the principle of respecting rights in the first place. I don’t think that’s true.
It’s imaginable that Hitler might have discovered that Jews are people after all, if he had been just slightly more rational and spotted a flaw in his racist ideology. It’s also imaginable that Hitler might have been tricked into believing that his racial ideas were wrong, if he had been just slightly less rational and unable to spot the fallacy in the ideas of someone who objected to his racist policies.
It’s important that we recognize both of these as realistic possibilities.
If someone is insufficiently rational to spot the problems in an argument against genocide, they’ll also be insufficiently rational to spot the problems in an argument in favor of genocide.
How does that follow? Certainly, “if someone is insufficiently rational to spot the problems with an argument for ~X, they are insufficiently rational to spot the problems with an argument for X” is not true in the general case.
It’s possible that being more intelligent will make you go from a true position to a false position, but it’s not something that will happen consistently. If you want someone to be more likely to believe a true thing, it’s better to make them smarter rather than stupider.
I agree with this, but this is a more nuanced position than what Yudkowsky’s above words express.
Hitler had a plausible argument that doing dreadful things was urgent, right, and necessary. Stalin had a plausible argument that he was not doing evil things. The overwhelming majority of American intellectuals before 1956 believed that Stalin was saintly, superhuman, and distinctly godlike, and if they doubted, were careful not to express such doubts, therefore it is probable that Stalin plausibly believed himself at least somewhat saintly.
Pol Pot clearly believed that he was a saint, and everyone who had personal contact with him, as a child or as an adult either believed in his saintliness, or believed that he suffered from delusions of saintliness.
If it is so obvious that Stalin was consciously and intentionally evil, why is it that no respectable person in the US could express this view before 1946, and no respectable properly academic public intellectual could express this view before 1956?
I’m unconcerned with theorizing on the earnestness of Hitler and Stalin, because I don’t think it matters. The power they had was lent to them by millions who were conceptually confused. Let me take Marx, who I’ve read more of. His materialist conception of history is riddled with the worst kind of idealistic piffle—the kind that mistakes itself for reality.
On the more pragmatic, prescriptive side, how many believers in Marxism had a clear idea what the Dictatorship of the Proletariat would in fact be? I doubt many. There was just a fuzzy collective “We” that would be in charge, which left the simple minded to project their own values onto “We”, and since their values were consistent with their values, the Dictatorship would be a wonderful thing.
People committed to clarity of thought would have wanted to know how that Dictatorship of the Proletariat was going to work in reality, as opposed to a political slogan. “Let’s empower some subset of primates to control the rest without limit.” Yeah, that’ll work out well.
Having read dozens autobiographies, memoirs, and other primary sources (including translations, I don’t read German, Russian, etc.), as well as many more secondary sources about the Second World War (and the Spanish Civil War, and the Winter War, etc.), I can only wonder as to what you might mean by “clarity”, “logic”, and the other terms you use and/or how you came to your conclusions, and what sources were the input you interpreted.
This is misleading. Wikipedia.
Good intentions do not stop people from acting evilly.
Non sequitur.
I think Lessdazed meant that even if rationality doesn’t stop people with bad intentions from doing bad things, it can stop people with good intentions from doing bad things. And there are probably more bad things done by irrational, well-intentioned people than by evil people.
In order to accomplish gigantic evil deeds, it is necessary for people to work together in teams. Consciously evil people are not team players.
Thus gigantic evil deeds are invariably accomplished by teams of people full of clever rationalizations for evil constructed by clever intellectuals.
They don’t need to rationalize when they just don’t have sam0345′s concept of ‘evil’ anywhere in their brain in the first place. It get the impression, for example, that Genghis Khan just did what he wanted to do—no excuses, no need to rationalize. Hearing this ‘evil’ concept of Sam from the future wouldn’t have just been something he rejected it would be completely dumbfounding.
Seems like he would have a label like “evil” for stabbing an ally in the back or the like. It just mightn’t apply to outgroups whatsoever.
Yes, there’s clearly something dubious about assuming that not only Genghis Khan, but his entire army, consisted of weird mutants who somehow lack moral intuitions. Much more likely is that they had normal human moral intuitions, but failed to apply them generally to all people, rather than (say) people in their own cultural group.
Stalin actually was a psychopath (probably diagnosable, as he fits all the standard criteria: flat affect, deceives people easily and without remorse, indifferent to suffering, superficially charming). Genghis Khan may have been (we know far less about him). But the average Soviet soldier? The average Mongol warrior? Clearly not—there are simply too many of them for that to be plausible.
Can you explain?
The article says things such as:
B.H. replies:
B.H. seems to think that intellectuals of the past century suffered from a lack of compassion and desire to do good, and were good at thinking clearly. I think that they were compassionate and had good intentions, yet had muddled thinking. It is this contrast that I was trying to bring out, and this is not anon sequitur.
You had a valid point to make (or at least an interesting point that would appeal to the lesswrong philosophy). It did not apply to the quote you set it up as a refutation of. Refuting a somewhat different position that what you are quoting is the foundation of debating but I consider it bad form. Particularly because it works so well against human minds.
If you just made your point without the vaguely relevant quote then I would not have commented.
For my part I don’t particularly agree with either of you. I wouldn’t focus on ‘teaching compassion’ or ‘teaching clear thinking’. I would focus on setting up institutions and power structures in which corruption and things-I-call-evil just aren’t the most efficient way to gain power.
I think you read into what I said some things that weren’t there.
I agree with the value of that approach to group and societal problems, but the smaller the scale, the less relevant that approach is and the more relevant overcoming bias is, so which I think better to focus on for a situation depends on specifics. B.H. was discounting clear thought in favor of good intentions, I addressed that, without intending to malign auxiliary approaches. I do not believe that many people are the villains of their personal narrative, and so think that “teaching compassion” is not too important, and that teaching clear thinking is. Teaching clear thinking isn’t always the right approach.
It is indeed misleading to describe Orwell’s Catalonian comrades-in-arms as capital-C “Communists,” since this would imply that they were controlled from Moscow, which they weren’t. (They were a mix of local independent communists and anarchists.) However, in Homage to Catalonia, there are several passages where Orwell presents clear evidence of their terror, murder, vandalism, and forcible suppression of all opposition, which he however excuses and rationalizes away, never toning down his utterly idealistic appraisal of them. His general comments about the war are also clearly remote from reality and biased in the pro-communist (small-c) direction, and on occasions he obviously relays the communist propaganda as a complete dupe. On the whole, as a propagandist for his favored side, he commits pretty much all the sins for which he would later bitterly excoriate the orthodox Stalinists, if perhaps in a less blatant manner.
So on the whole, I wouldn’t say that his hands are that clean. He certainly didn’t deserve the place in intellectual history he was eventually awarded, in the sense of being remembered as the unwavering fighter for truth, clear thinking, intellectual honesty, and opposition to political lies and gangsterism. Certainly some of his contemporaries were far more deserving of such description, and yet hardly anyone remembers them today.
I would say that he’s remembered as the writer of two of the most influential books opposing tyranny, rather than as an unwavering fighter against truth, clear thinking, intellectual honesty, and opposition to political lies and gangsterism.
Homage to Catalonia came out in 1938. 1984 came out in 1949. Is it possible that his experiences (perhaps including realizing what he’d been excusing) had something to do with 1984?
Well, the article that started this discussion describes him in these terms. It is true that most people who have heard of him know him only as the writer of these two books. But among people who know more about him, as far as I’ve seen, he typically has this illustrious image.
As far as I can tell, he never ceased admiring the “revolutionary” regime that ruled over Catalonia before the Communists took over. However, even regardless of that particular issue, his views have definitely never been particularly free of ideological bias—and here I’m not comparing him with some unreachable ideal, but with other people who lived and wrote at the same time. Yes, he was certainly much better than the typical Stalinist intellectual of the time, but that’s an awfully low bar to clear, and some other people managed to clear much higher ones.
Orwell was still something of a totalitarian to the day he died, but that is what made him great. He understood totalitarianism from the inside, and so condemned it in an accurate and insightful way that no outsider could condemn it.
In “Nineteen eighty four” Orwell whitewashes Trotsky’s disagreement with Stalin as merely a matter of technical details, represented by the windmill, but in actual fact, it was over terror, torture, and mass murder. Trotsky complained that Stalin was not tough enough on the peasants, and objected to torture and murder being slowed down and obstructed by bureaucracy and red tape. But we do not look to the book for an accurate history of Russia. We read the book to understand how totalitarians think.
No one who had not been well and truly totalitarian could have written such a book.
Fair enough. I didn’t check back to the article, and only went with my impression of his reputation—the latter is a mistake I should watch out for, since I seem to be less inclined to think of famous people as comprehensively wonderful than most.
The Republicans in general and anarchists in particular should not be conflated with the communists; communists gradually and somewhat steadily took over the leftist side from being a tiny minority at the outset of the war to being in control of a lost cause.
Orwell’s unit was almost all anarchists. The communists were just one group against the fascists, his propaganda is pro-Republican generally and pro-anarchist in particular, so pro-communist is not the best description.
Fighting among anarchist allies of communists and doing as the anarchists do, until the communists turn on them and kill them, does not make him associated with communism in a very important way and especially not with Communism.
Orwell’s “anarchists” set up a totalitarian terror state in Catalonia within hours of seizing power. See The Anarcho-Statists of Spain and What really happened in Catalonia
The analysis at the first link is pretty decent factually, and not a flat caricature either. I don’t completely agree, but the objective picture feels correct. Indeed, when reading Homage to Catalonia, it felt obvious to me that Orwell was mostly charmed by the contrast between his comrades in arms’ heartfelt quasi-religious attitude and the emotionally stunted life of Western middle class. He was conscious that they were in essense a barbarian tribe crossed with a Puritan sect—seeing all out-groups as not quite human unless proven otherwise—but chose not to apply all the boring ethical standards to them. Even later in his life he showed a certain insensitivity to slaughter of “innocents”, coldly pointing out that there shouldn’t be an ethical difference between soldiers & civillians. Indeed, he was a little bit of a fascist, although closer to Nazism than to Stalinism in his darker moments.
However, calling the Spanish Anarchist rule “totalitarian” is pointless abuse of the term, of which I prefer Arendt’s strict and horrifying definition. (See her work Origins of Totalitarianism.) It was, in essense, the rebirth of some scavenger values, painted red mostly for political utility and planting a few Marxist ideas into rich soil. And they certainly were the heroes of their own stories—it’s moral myopia and not everyday heartlessness that appears to be their cardinal sin.
I’m not especially a fan of anarcho-syndicalism, as you can see. Even in theory it can threaten to throw out civilization’s baby along with the bathwater. And even disregarding the out-group interactions (which are psychologically imperilled whenever in-group consciousness strengthens), it depends too much on morale, high spirits and good leadership.
Which is also my answer to Eugine_Nier’s criticism of Alinsky’s work—his approach to everything was heavily Syndicalist (not Socialist), he was proud and stiff-necked and it could’ve rubbed off on the black communities he sought to unite; without guidance, their new-found voice and political power might’ve served to plaster over long-standing internal problems and reduce the relative attractiveness of self-improvement in blacks’ eyes (the material incentive for “breaking out of the hood” shrank as life got better, but it’s a tall order to cultivate ideological and cultural incentives during a short window of a community’s eagerness to change).
Yet I feel certain that doing nothing for those benighted, long-suffering people was morally unacceptable. And I haven’t heard any better counter-factual proposals from anywhere right of center—it’s just “Segregation was not so bad, leftists are whining over good old ways, equality of outcome is horrible anyway” from what I’ve seen of their criticisms.
(I’m not going to read the second link, as I’ve had enough of Comrade Sam for the next few centuries.)
Edit: oh, the author is an Anarchist himself, and looks fairly broad-minded too. I was afraid he’s got an orthodox libertarian bottom line, given who linked to him.
I include the anarchists (CNT) and the Catalonian independent Marxists (POUM) among the “small-c communists.” We can quibble about this designation, but I think it’s fair, especially since I have emphasized that they were not Moscow-controlled. I’m also sure that members of POUM would not have had any problem with this label, being self-proclaimed orthodox Leninists.
Also, Orwell served in POUM’s militia, not with the anarchists.
In any case, however you choose to call them, it is indisputable that the parties for which Orwell fought were guilty of political terror and murder, that they were violently intolerant of any opposition, and that Orwell clearly excused, rationalized, and even praised these acts and attitudes, which he witnessed first-hand. Sure, they eventually ended up as loser underdogs who got crushed by even bigger and meaner political gangsters, but this is no valid reason to excuse and romanticize them the way Orwell did.
The unit’s members, not its flag, hence “almost all”, which would make no sense describing the unit’s affiliation.
...
...
...
Those are from Homage to Catalonia.
The minority U.G.T. Leninists wouldn’t, but the Catalan draftees who were members of anarchist unions (which were strongest in Catalonia) would.
If they were so violent, they wouldn’t have let the Communist minority grow in power until they killed them. They were really violently intolerant of some opposition, which is not the same quality of thing, for many are violently intolerant of some opposition, the extreme stances being violent intolerance to no opposition or all opposition.
This isn’t really relevant for the main point of the discussion, but the official ideological self-designation of the CNT was “libertarian communism” (comunismo libertario). See for example this declaration from the 1936 CNT congress:
http://www2.uah.es/jmc/comunismolibertario.pdf
I do remember the stories of Mao’s failed agricultural policies, and that he was generally either deceived or not made aware of this fact. Are you saying that he actively caused famines and poverty to test his control over China?
According to Wikipedia, he was aware of the problem.
Eliezer, you seem to say that trying to reduce hypocrisy is good for people who are trying to overcome their biases. That seems a tautology, and doesn’t seem that related to the issue you seemed to raise in the post, which is whether hypocrisy in society tends to promote holocausts.
“Helping people to open their eyes and see human suffering, raising children to be compassionate, will do far more to get rid of the Hitlers and Castros than logic and writing classes.”
’Cause no one ever thought that Che Guivera or Lenin were acting out of a compassionate desire to end suffering. Nobody EVER claimed that.
Confucius on the Rectification of Names written some 2500 years ago.
“Tzu-lu asked ’If the Lord of Wei entrusts the government to you, what will you do first?”
“Correct names, surely!” the Master said.
“How can you stray so far from the point! What would that correct?”
“Tzu-lu, you are a boor. On matters of which he is ignorant a gentleman expresses no opinion. If names are incorrect, it is impossible to speak. When it is impossible to speak, work is not done. When work is not done, society breaks down and punishment is misapplied. When punishment is misapplied, the people do not know how to act. Therefore what the gentleman names is sure to be accurate, and what he says is sure to be actionable. It is simply that the gentleman is never careless in what he says.”
Analects 13⁄3. Adaptation of A.C. Graham’s translation in Disputers of the Tao.
Eliezer, you seem to say that trying to reduce hypocrisy is good for people who are trying to overcome their biases. That seems a tautology...
“The sort of human being who makes a continual effort to overcome hypocrisy, and who manages to do so, will probably set the slaves free.” I don’t quite see how this is tautological...
Stalin and Hitler did not suffer from lack of clarity.
Hitler certainly did. Stalin… maybe not, I don’t know his case in as much detail. But it is their followers, and the rest of the world, who they managed to confuse. This is what Orwell opposed. He was not speaking to Stalin, but to the people who tried to excuse Stalin.
Helping people to open their eyes and see human suffering, raising children to be compassionate, will do far more to get rid of the Hitlers and Castros than logic and writing classes.
I’m supposed to applaud now, right?
Opening eyes is what I do. It’s a lot more complicated than telling your kids, “Suffering is bad, m’kay?”
There’s a long, long distance between being told by your parents not to murder, and learning how to actually see a “murder” taking place rather than an “alternative justice process”. Morality without logic will be flushed down a toilet by self-deception.
Or an ordinary justice process, for that matter.
David Brin has a nice analysis in his book The Transparent Society of what makes open societies work so well (no doubt distilled from others). Essentially it is the freedom to criticize and hold accountable that keeps powerful institutions honest and effective. While most people do not care or dare enough there are enough “antibodies” in a healthy open society to maintain it, even when the “antibodies” themselves may not always be entirely sane (there is a kind of social “peer review” going on here among the criticisms).
Muddled thinking affects this process in several ways. It weakens the ability to perform and react to criticism, and may contribute to reducing the signal-to-noise ratio among whistleblowers by reducing the social “peer review”. This is how muddled thinking can promote the loss of openness, democracy and accountability, in the long run leading to non-accountable leaders that have little valid feedback or can just ignore it.
But are biases the main source of muddled thinking? I think muddle is the sum of many different factors: biases, lack of knowledge, communications problems etc. In any situation one or a few factors are the most serious causes of muddle, but they may differ between issues—the biases we have discussed relating to new technology are different from the biases in conspiracy theory or everyday political behavior. To reduce muddle in a situation we ought to reduce the main muddling component(s), but that may be very different in different situations. Sometimes biases are the main problem, sometimes it might just be lack of communication ability. It might be more cost-effective giving people in a developing country camera cellphones than teaching them about availability biases—while in another country the reverse may be true. But clearly overcoming biases is a relevant component in attacking many forms of societally dangerous muddle.
Eliezer, do you see the distance between how you are discussing this topic and how Anders is? Invoking “Good and Evil”, with Stalin and Mao (losers of conflicts with the anglosphere?) being “evil” seems to me to me to be an appropriation of the overcomingbias space more so than a good faith effort to intelligently expand that space. Anders approach in general and in this post in particular serves as a good contrast, in my opinion.
I’m also bothered that all of the villains cited are enemies of the Anglosphere (although these two actually won). B.H. gives a long list which is quite biased in this way. Even Orwell seems to reference Churchill and Truman, and any list ending with Osama and Saddam practically begs for a reference to Bush.
Nevertheless, it’s clear to me that all of the people on B.H.’s list (as well as the others that I mentioned) committed great evils that have been (and still are) defended with hypocrisy and other forms of muddled thinking. And I agree with Orwell’s and Eliezer’s indictment of this.
In my post here you can find links to defenses of Stalin and Mao. They do not deny that both killed a huge amount of people, or that a great many of those people were completely innocent and it was tragic that they were killed. The author instead states that Stalin and Mao gave enough benefits to people, who had previously been faced with even worse rule, that those unfortunate deaths should be viewed as cons outweighed by the pros. I can’t do an effective job of presenting his case (for one thing I completely disagree with it and think the two were among the worst people in history) but I would like if he could explain more here, including why our view of them is so off.
While Eliezer and I may be approaching the topic differently, I think we have very much the same aim. My approach will however never produce anything worthy to go into anybody’s quote file.
Anders & TGGP, Response on my blog within 10 minutes (so as not to flood overcomingbias with posts).
How come this blog won’t remember my info? I keep clicking that damn checkbox to no effect.
I read Orwell’s essay last night. Quite impressive, but I didn’t immediately understand some of his criticisms. I did later on, and would like to share.
One thing that I had a revelation about is this pair of equivalent passages:
“I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all.”
“Objective considerations of contemporary phenomena compel the conclusion that success or failure in competitive activities exhibits no tendency to be commensurate with innate capacity, but that a considerable element of the unpredictable must invariably be taken into account.”
At first, I didn’t understand why Orwell thought that the second passage conveys only part of the meaning of the first. But later, I imagined each of them read by someone. The Bible quote was read with feeling by a beautiful woman. The modern pretentious BS was read by Microsoft Dave. The first evoked images of an army defeated because of bad luck and a wiseman who died poor because his ideas fell on deaf ears. The second didn’t evoke any images or emotions.
And I realized that such abstract writing is dangerous because of this quality: it transmits all the information, and may make perfect sense, but creates no emotions. You can describe cities being burned to the ground like this, and people will react to this description as “somebody else’s problem” or “necessary sacrifice”.
But concrete imagery is not a cure-all:
“All the “best people” from the gentlemen’s clubs, and all the frantic fascist captains, united in common hatred of Socialism and bestial horror at the rising tide of the mass revolutionary movement, have turned to acts of provocation, to foul incendiarism, to medieval legends of poisoned wells, to legalize their own destruction of proletarian organizations, and rouse the agitated petty-bourgeoise to chauvinistic fervor on behalf of the fight against the revolutionary way out of the crisis.”
Lots of imagery here, and it does provoke emotions. But the problem is that 90% of it is trite slogans. They already have a predetermined value: revolution good, bourgeoise ungood. If you avoid triteness and invent your own metaphors, people will actually have to decide whether your metaphor resembles what you’re describing.
Also, I think that the word “clarity”, if it was meant to convey what I wrote above, ironically doesn’t do so very clearly. It suggests that you’re fine as long as your words are easy to parse.
I think the recent 1984 quotes are close to encouraging generalization from fictional evidence. Bias and totalitarianism are not more connected because Orwell wrote a work of fiction connecting them; you may understand this on a gut level, but many readers probably don’t. If there’s a duty to make history available, isn’t there a converse duty to make fiction unavailable?
Anyway, here are my thoughts on whether it’s a good idea to choose bias.
The rules of the game are much different when you have bad faith, as in totalitarian societies, as opposed to comparing Democrats vs Republicans. Or are they?
steven, your criticism is valid only if Eliezer used the passage from 1984 in some way that properly requires that it actually have happened. It is not at all clear that he did so. For example, if he used it as a way to illustrate a point, clarifying his meaning, then it was not improperly used. If he used it as evidence of Orwell’s own well-thought-out views on the subject, then again, it was not improperly used. There are many ways to use fiction that are not improper.
“For example, if he used it as a way to illustrate a point, clarifying his meaning, then it was not improperly used. If he used it as evidence of Orwell’s own well-thought-out views on the subject, then again, it was not improperly used. There are many ways to use fiction that are not improper.”
What his intentions were doesn’t matter; what matters is the expected reaction of the audience, which in this case is going to see a vivid example supporting Eliezer’s claim that human evil sets out to muddle thinking. This happens to be true, but it doesn’t have anything whatsoever to do with Winston Smith or O’Brien, both of whom were made up by Orwell with (I suspect) one of Orwell’s intentions being exactly that their fake historical authority would be used to support Orwell’s (true) opinions. I wish we could all just agree on a norm banning references to fiction from any serious discussion about the real world. People seem completely incapable of handling the stuff.
Copy of my post defending Stalin and Mao from Hopeanon.
Hello, my defense of Stalin and Mao are simple and the links can be found here on the Entitled To an Opinion blog. My argument is simple. Stalin saved far more lives than he took. In fact, Czarism was three times more deadly on a per capita basis than the average for the Stalin years. Plus, Stalin set a world record for the fastest doubling of life expectancy in any land. This amazing feat was only broken by Mao in 1976. Therefore, based on those records, I hold that Stalin and Mao were two of the greatest humanitarians that the world has ever known.
There are plenty of ways to kill a man. You can do it with a bullet, work him to death, or you can kill him with hunger and disease. Dead is dead, one way or another. I liken Stalin and Mao to vaccines that, say, kill 50 people every year, but without the vaccine, 1000 would die. Or the death rate of abortions versus childbirth.
Further, the notion of the worst killers of all time is simply insane. Over his time, Stalin killed maybe 4 million. Yet he saved so many lives that at the end, he had saved a net 35 million lives, NET. In contrast, the transition to capitalism with its collapsing life expectancy may have killed up 15 million, NET. I suggest we look at net losses and gains of life when making lists of these “killers”. Increase in life expectancy is not a given. Near the end of Stalin’s rule, life expectancy was still about 35 in both Albania and China. There are places now that have life expectancy that low.
Let us look at China and India. They had similar figures in 1949. Since India took the capitalist road over the socialist one, we can compare nations by life expectancy and death rate. India is killing 4 million Indians per year, compared to China. That is the penalty for India not taking the Chinese path to development. Further, 14 million starve to death every year in the world, mostly in South Asia—India, Bangladesh, Pakistan, Nepal, Afghanistan. I do not understand why killing 14 million a year is not chalked up to the worst killers of all, but S and M are? Figuring over time, we can see that compared to China, Indian capitalism has killed about 170 million Indians. And that comparison INCLUDES all of those killed by Mao (maybe 20 million).
I will also argue that there were really no deliberate famines in Ukraine or China. China’s famine in 1958 was caused by gross stupidity. The one in the Ukraine had many causes, but deliberate starvation was not really among them.
Stalin’s victim toll has been exaggerated. It is really 4 million instead of 20 or 40 or 60 or 120 or however many million.
Mao’s looks the same—maybe 20 million as opposed to 40, 60, 80 or however many million.
Indian capitalism kills 4 million a year, minimum. I would argue that replacing it with a killing system that killed a lot less people would be a net good.
I really do not support most of the killing that Stalin and Mao did, but many of those killed by Stalin had taken up arms against the state, or were trying to overthrow the government, so they were not totally innocent. The initial killing of 3 million landlords, not by Mao, but by the outraged peasants themselves, was not a killing of innocents either. Almost of those landlords were serious criminals with a ton of blood on their hands.
This is a pretty interesting discussion. While the overall topic of this blog is the worthwhileness of overcoming bias, I think how that relates to “evil” is a pretty important facet of what we should talk about. Some of the comments on this post reminded me of a passage from a novel (a fantasy novel if you must know) on the nature of evil. I found it to be very profound and I think everyone here might find it sort of interesting. You’ll have to excuse the use of some of the plot specific names, the beginning of each chapter of the book opens with a few paragraphs from the point of view of a historian who lives several thousand years after the events in the story take place.
“What is the nature of evil? Evil is mistakenly portrayed as a unified force. Tresserhorn, Stromgald, the Adnates of Soldevi, Varchild, all of these, we are told, were cut from the same cloth and share that same corrupted fabric. Yet these forces have battled each other as often as they have fought the forces of “good” and so can hardly be considered members of the same side. Some researches state that all who bear the stamp of evil are imperfect or faulty beings from the onset. The more generous scholar might classify them as being merely weak, corrupted by potent forces that poisoned them. Yet in the end, the historians label them all as “evil,” as if that state were a disease that could be contracted through mere proximity. So goes traditional thought. In reality, the various “forces of evil” all chose their paths for reasons that sounded good, even beneficial, at the time. They might have chosen their paths for personal advancement, for a particular belief or religion, for the desire of power, or even for patriotism or devotion. What unified them all, from Lim-Dûl to the Adnates, was usually the belief that they had “the answer” to whatever problem beset them. This answer often had the advantage of requiring minimal sacrifice on the part of the solver. Further, when the solution did require some degree of sacrifice, that sacrifice would be on the part of others who were unaware of a need for sacrifice in the first place. These other people, the ones making the sacrifices, often took umbrage when they learned of their situation. As a result, such unpleasant information was kept from them until the last moment. In this fashion, the first steps of deception, regardless of the reason, was evil born.”
I tend to agree with what the author of that passage said about evil. When you look at people like Hitler, Mao, and Stalin it is clear they believed they had “the answer” or “the solution” to the problems they faced. But they lacked the ability to objectively see that their plans called for suffering on levels few people alive today can fathom (I know I certainly can’t). They let their biases close their eyes to the deaths they caused because they simply had to be right in their own mind. Teaching everyone to be compassionate and to empathize with others will certainly help people become less cruel, but overcoming our biases is no less important a step.
Clear thinking is a necessary but insufficient condition for avoiding evil. Eichmann is a paradigmatic case of local rationality in the pursuit of evil ends. And right here on this blog, we see proudly rational thinkers advocating what most normal people would think of as evil, namely the employment of torture as a judicial punishment. I’ve argued against them, but perhaps my arguments aren’t any good. Maybe it is more rational to apply shocks to the genitals or waterboarding than to lock someone in a cell. Maybe we don’t have anything better than instinctive revulsion to keep us from evil. In which case, we should not be overcoming our biases, but listening closely to them.
Ah, it’s good to see that even in OB’s early days there were people who saw the ugliness of naive utilitarianism and “rationality”-worship, who plainly spoke out against this blight.
Just look at how in-your-face this crap used to be. It’s subtler now, of course… I’d say that the community has progressed in ethics and spirit, yet this proportion of people that I fear has not been wasting time either, probably.
mtraven, You’re ignoring concepts like inquiring empirically whether situationally employed torture can reduce net torture in the world. I brought it up in the torture thread and I think every commenter ignored the concept. Also, I’m unconvinced that the desire to label indivuduals as “good” and “evil” comes from a good faith attempt to accurately model reality, or even optimally solve existential challenges we face. It seems to come more from a desire to use morality to create status heirarchies, although in some cases it could also create representational heirarchies privileging both the people framed as “good” and as “evil”, to the detriment of those that embody neither archetypes.
I’m surprised no one else in engaging me on these ideas of moral and representational heirarchies as ends in and of themselves.
HA, that’s exactly the sort of argument I’m talking about. It is too easy to convince oneself by some bit of reasoning that doing an evil act is OK—maybe it reduces some other evil, maybe it gets rid of the Jews who you have convinced yourself are a source of evil. Maybe you can convince yourself that implementing some torture will reduce the total amount of suffering in the world. I would be extremely dubious. Reasoning from first principles about practical affairs is extremely unreliable, and has to be augmented with heuristics, intuition, and gut feelings. You have to be extremely suspicious when supposedly rational arguments lead you to morally revolting conclusions. (BTW, Ursula Leguin wrote a short story on these themes which is worth reading).
Now, “trusting your gut” is something of a metaheuristic, and not a very reliable one. For example, one of my least favorite philosphers, Leon Kass, uses similar arguments to argue against homosexuality, assissted reproduction, and eating ice-cream in public. His gut is apparently very sensitive to things like that; mine isn’t. How do we reconcile our positions?
The answer is, we don’t. Such disagreements can’t be resolved by pure reason and thus enter the realm of politics. I know politics isn’t popular around here, but that’s how things work. Pro-torture and anti-torture factions will have to fight it out politically, just as the pro- and anti-homosexuality factions do. Which side are you on?
Tiiba, I’ve always considered this bible quote
“I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all.”
to be a paradigm of bad writing. The nasty trick it exemplifies is using a paradox to get one up on the reader without committing to a specific meaning.
If the race is not to the swift, who does win? The lucky? Contrast two aphorisms “the race is to the swift” and “the race is to the lucky”. The second of these is useless. You cannot assess some-one’s luck ahead of time, you have to discover who is lucky by waiting to see who wins and then say smugly “see I told you the lucky one would win.”
This passage irritates me because the writer forces the reader to create the meaning of the passage from nothing and then the writer appropriates the readers efforts. There is something important to be said on this theme:
The race is to the swift and the battle to the strong, but the odds are never quite so short as the book maker offers and the greatest number are drowned when the sea takes the unshinkable ship with no lifeboats.
I’ve no doubt that the author of Ecclesiastes would say “that’s it, you have got my meaning exactly” but that would be a lie; it is my meaning, I said it and he didn’t.
“Pro-torture and anti-torture factions will have to fight it out politically, just as the pro- and anti-homosexuality factions do. Which side are you on?”
haha, that reads as deliberate bait to me. Presenting 2 options as if they’re the only options seems to be preying on a dualistic/binary bias common in most people, and that I think stems from our primate roots. That sentence reads like it could be lifted from a George W. Bush speech, that type that’s widely mocked for performing a lack of nuance.
When we think in ways that hide the consequences of our actions, there’s no way we can make good choices based on consequences.
But when that isn’t blurred we still might disagree about the choices. So if you don’t think DDT affects songbirds you’re likely to think DDT is entirely a good thing. But even when you know, you might decide that perfect fruits are more important than birds.
Moral choices aren’t inevitable when you understand the facts, but without the facts it’s even harder.
There’s often a conflict between immediate things and distant implication. You can see the immediate good effects of DDT, it took years to work out the over-all bad consequences. HA makes the opposite claim about Stalin, the immediate deaths are obviously bad, but it takes effort to see the long-term good that came from them. So, who’s right?
Scientific method is good where it works. Peer review is good for science when the science works. When you can’t do controlled experiments it gets harder to tell what’s going on.
Still, sometimes the language is central. When we say the US military is in iraq “protecting iraqis”, it might work better to reduce the level of abstraction and say we’re there “killing people and blowing things up”. Or maybe we’re “killing people that we think are about to kill the ones we don’t want killed”. When we say “train the iraqi army” we might say “train selected iraqis to kill people and blow things up under our direction”. The clearer we say what we’re doing, the easier it might get to create a better strategy.
J Thomas, I really like your last paragraph “Still, sometimes the language is central. When we say the US military is in iraq “protecting iraqis”, it might work better to reduce the level of abstraction and say we’re there “killing people and blowing things up”. Or maybe we’re “killing people that we think are about to kill the ones we don’t want killed”. When we say “train the iraqi army” we might say “train selected iraqis to kill people and blow things up under our direction”. The clearer we say what we’re doing, the easier it might get to create a better strategy.”
I’m sure TGGP will agree with my request to you: Start a blog!
that reads as deliberate bait to me. Presenting 2 options as if they’re the only options seems to be preying on a dualistic/binary bias common in most people, and that I think stems from our primate roots.
Everything we do stems from our primate roots. I don’t know you well enough to deliberately bait you, but using that expression (which comes from ) is meant to signify that at some point, when dealing with politics, you have to stop spouting hot air, choose which side of an issue you are on, and fight for it. That attitude seems somewhat antithetical to the prevaiing ethos of this blog, but I’m a contrarian.
If you like, you can see it as a dominance battle between two large coalitions of primates, with Amnessty International, other anti-torture groups, the authors of the 8th amendment, and those who have some measure of respect for basic human rights on one side, with the other side consisting of various shadowy govenment organziations (parts of the CIA and military), fans of 24, Alan Dershowitz, and the hopelessly authoritarian. Which side are you on? Which band of primates do you think deserves to win this battle, which side do you want to associate yourself with?
I think what I am most opposed to is the epistemologically naive idea that you can talk about matters like this in an objective, apolitical way. So—I have a bias against torture, I have no desire to overcome it. Orwell is not around to ask, but I think he’d feel the same way.
Whoops, messed up a link there, was supposed to read: comes from an old union song. The link is to my own blog where I am basically taking the opposing view, that forced-side-choosing is bad, in many circumstances. But I did say I was contrarian...
mtraven, my position is pretty transparent. I want to persist as a subjective conscious entity, forever, and everything else is subsidiary to that. To the degree torture (of myself or others) maximizes my persistence odds, then I perceive it to be in my interest to welcome it. To the degree torture (of myself or others) reduces my persistence odds, I oppose it.
To the degree you mtraven would place opposing torture over your own persistence as a subjective conscious entity is perhaps a triumph of meme over a subjective conscious individual, in my opinion.
Although if you’re non-anonymous, then I understand how it can be in your interest to perform opposition to torture, regardless of how it affects your persistence odds.
mtraven, I’m not sure what it means to be “objective, apolitical” but I certainly hope that when issues arise here people here are willing to set aside what they think they know enough to imagine being uncertain and then trying to evaluate which way the evidence and analysis leans.
I’m puzzled, as usual, but perhaps more so: this post has helped clarify the lack of clarity in my understanding of “bias” as the word is used here. You see, I don’t in general see an a priori distinction between “bias” and other kinds of heuristics; we are talking about computational shortcuts, ways to save on reasoning, and all of them go wrong sometimes. I’m glad to have scope insensitivity pointed out to me, and having seen the discussion on this blog may even keep me from some error at some time, but my reasoning will always be incomplete, my models will always fall short of being “isomorphic with reality.” (A modeler’s joke, along the lines of using a territory as its own map.) I am tempted to think of “bias” as a label for the systematic errors introduced by any given heuristic, such as valuation by prototype; is this fair?
If so, “overcoming bias” is one of those journey-rather-than-destination unattainables that at any given moment faces me with an economic sort of choice: I can spend more resources trying to overcome the bias introduced by each heuristic I use, i.e. in meta-inference, or I can spend more resources actually carrying out inferences by those heuristics—expanding and pruning nodes in my actual current search-graph, so to speak, and coming to conclusions even though I know that some of them will turn out to have been wrong.
Is this an obviously bad way to think? Probably so—because it leans me in Tyler’s direction. Well, I dunno. This comment is an extremely imperfect representation of what I want to say...but I post it, on the grounds that I have to go listen to Dougie MacLean, a Scottish singer and songwriter, and achieving a more perfect (but never actually perfect) representation of what I want to say has a finite value, which at the margin somewhere loses out to getting on with the next thing. I thought that was true for everybody.
HA, you may have your personal single life goal worked out but I’m not sure why the rest of us should be interested. My goal is emphatically not persisting my subjective reality as far forward in time as possible, let alone yours. I have other people I care about, I probably care more about the quality of my life than quantity, I do have memes that I place a great deal of value on. Most other normal people feel the same way, I suspect.,
Robin, what evidence has been brought forth to support the proposition that torture may be a good idea? I have cited books that study the history and deployment of torture, and groups that work with actual torture victims and would be happy to supply you with data. There’s some evidence for you. It may not be objective, but objectivity about such issues is unobtainable even in theory—you can’t even define torture without taking a stance about what constitutes torture as opposed to “harsh treatment”.
You say I certainly hope that when issues arise here people here are willing to set aside what they think they know enough to imagine being uncertain and then trying to evaluate which way the evidence and analysis leans.. Carl Sagan (I think) said we should be open-minded, but not so open that our brains fall out. It’s even more important when discussing issues as morally fraught as torture, that we don’t open our minds so far that our souls fall out.
Carl Sagan (I think) said we should be open-minded, but not so open that our brains fall out. It’s even more important when discussing issues as morally fraught as torture, that we don’t open our minds so far that our souls fall out.
Well said.
To me, that sounds a lot like saying “We should be rational, but we should stop before it feels too unintuitive or against social norms”.
“Carl Sagan (I think) said we should be open-minded, but not so open that our brains fall out. It’s even more important when discussing issues as morally fraught as torture, that we don’t open our minds so far that our souls fall out.
Well said.
Posted by: Eliezer Yudkowsky | September 16, 2007 at 06:07 PM”
What does that even mean? The quote and concept seem to distract from empirical inquiry and rational problem solving, rather than add to them. I’m not sure I see it as different in kind from “If it doesn’t fit, then you must acquit.”
HA, this is not the forum for it but perhaps somewhere else you might explain where you stand.
The way US law used to work, you or anybody else got to torture whenever you thought it was a good idea, and then you could expect to stand trial. If the police thought you did the right thing they might choose not to investigate, if the DA thought you did the right thing he might not prosecute, if the grand jury thought you did the right thing they might not continue, if the jury thought you did the right thing they might not convict, if the judge thought you did the right thing he might give a suspended sentence, and then if the governor or president thought you did the right thing he might pardon you.
The new way, the Bush administration gets to torture you if they sort of imagine you might fit some sort of criteria, with no oversight by anybody but them, and then if it turns out they were wrong and you don’t fit their criteria they suffer no consequences, they don’t owe you any money or apology or even an oops, in many cases they don’t even have to let you go.
Somewhere along the continuum between those two is the approach that maximises your survival. I wonder where along that line it is. But as I understand it this blog is devoted to studying bias and not to a straightforward study of what’s good for you. Maybe somewhere else....
Tom Myers,
I think the convention on this blog, among the small set of people who have such a precise definition, is that not every heuristic is a bias, that only heuristics whose errors are worth overcoming should be called biases. I don’t like this usage. For one thing, it’s really hard to know the cost of overcoming particular biases. It’s easy to look at an isolated experiment and say: here’s a procedure that would have been better, but that doesn’t cover cost of actually changing your behavior to look out for this kind of error.
Also, there are other people on this blog that use bias to refer to systematic, but unexplained errors, where it’s hard to call them heuristics. Without a mechanism, it’s probably hard to overcome these, although not always impossible.
mtraven, do you really believe in the existence of the soul, or are you just using it because it is in common usage? At my blog I was thinking of writing a post whose title began “Thank god”, then remembered I had already declared I was an agnotheist, and then considered “Thank goodness”, but remembered I didn’t believe in objective good either.
Do not take my rhetorical flourishes overly literally. We may be meat machines without an inhabiting ghost, but that doesn’t mean that “soul” doesn’t have meaningful connotations. In this context, it (roughly) means “our deep-seated representations of ourselves, and others, and morality, and our ability to empathize with others.” It means that a rationality that only considers your own personal ends (as HA seems to advocate) is deficient. It means that people who blithely talk about torture as an option probably haven’t spent too much effort imagining themselves, or a loved one, as the torturer or torture victim in the scenarios they are planning.
“Losing your soul” means getting out of touch with your basic humanity. Rationality is all well and good, but rationality alone can’t suffice to make moral judgments—as the case of Eichmann illustrates. Consider “soul” as shorthand for a whole bunch of basic, irrational, but vital aspects of thought.
Tom and Douglas, I’d take “bias” to mean something like “common feature of human thinking that gives us predictably wrong answers”. Some biases might (for any given person) not be worth correcting, if correcting them requires sustained effort and similar effort could be more effective if deployed to other ends. I don’t see why whether something’s a “bias” should depend on whether it’s the result of a heuristic operating out of the regime it was designed for, or on whether we can correct it.
I have the feeling that the term “bias” gets overused here (possible explanations if I’m right: mission creep plus rationalization, and the satisfaction of labelling others as “biased”), but I don’t actually have concrete examples in mind. Perhaps I’m suffering from overidentification of overidentification of bias bias bias.
(Remark: this construction generalizes to produce noun phrases ending with an arbitrary number of instances of “bias”. Similar but nicer construction, not due to me: “oysters oysters oysters split split split” is an English sentence, at least notionally, for all values of 3.)
Mtravern,
In my opinion you’re preying on bias to achieve status advantage for yourself/posting name. But I don’t think it will be as effective a social strategy in this overcomingbias medium as it would be in the general population.
Hopefully Anon, I can’t imagine what great status or survival advantages you think I’m going to get by posting here. If I was interested in advancing my status I’d pick better activities than flaming pseudonymously on blogs, you may be sure.
But in a way you are right, but trivially. Presumably everybody who posts here or otherwise engages in human communication has, at some level, an underlying motive of increasing their status. Everybody hopes to make themselves look good. I don’t see how my postings differ from anybody else’s in this regard.
mtravern,
I’ll have a response up to your post on my blog within 10 minutes.
http://www.hopeanon.typepad.com
getting out of touch with your basic humanity
I am a homo sapien, therefore my characteristics are human. Perhaps I should wonder why you have an inhuman bias against torture, but of course that is human as well.
Homo sum: humani nil a me alienum puto
Douglas, with regard to systematic but unexplained errors, would you agree that we can (usually) describe these as due to unidentified heuristics? I’m feeling very unsure about that, but I would like to have some fairly concrete way of thinking about this blog’s subject matter, and at least this way it’s something I’ve taught. :-) I’m not about to insist that all thought can be modeled with symbol-processing. It may even be that the most fundamental errors are those that arise without any symbol-processing—I’ve just been reading a dog-trainer’s book which emphasizes the errors we make in dealing with our dogs, simply because we automatically do what other primates do even when we’ve verbally worked it out. Still, by the time you get to correcting a bias found in less-obviously-symbolic process, say a neural net or genetic learning-algorithm, I think you’re describing that bias and its correction via rules on patterns of symbols—and that’s the way I’ve been seeing this blog. (Is everybody asleep yet? Well, I suspect this thread was drying up anyway—as you can see, I have a bias in favor of mixed metaphors.)
Douglas and g both respond in terms of cost, apparently agreeing that the value of bias-correction (in any given context) will be limited, and it may not be the best use of whatever resources you’ve got...this again seems Cowenesque to me, and I’m reasonably happy with it; we have few absolutes showing except for (a) HA’s survival (which I tentatively rate as a rhetorical position, unlikely to show itself in HA’s real-world behavior), and (b) anti-torture.
I was going to leave the torture theme strictly alone because I’ve never seen it come to any good end, but maybe it’s worth saying: when you say you’re “against torture”, you haven’t actually told me much at all because too many people mean too many things by “torture”. In a blog post apparently dedicated to clarity, I think that’s extraordinarily important.
Let me clarify: consider a long interrogation under bright lights when you want to sleep; is that torture? Some will very sincerely say yes, some will very sincerely (I believe) say no, and some will draw the line at some specific number of hours of sleep-deprivation. If someone opposes more than 10 hours as “torture”, and you oppose more than 5 hours as “torture”, you can class them as “pro-torture” but it strikes me as an anti-clarity sort of rhetorical move. Is (fake? real?) menstrual blood-smearing torture? Again, apparently sincere disagreement exists. Is (some specific kind of) fraternity hazing torture? Is boot camp torture? Do I have an opinion on any of those? Not really: my primary torture stance is not anti-torture or pro-torture, it’s pro-clarity.
Specifically, I would focus on interrogation—trying to get leave out O’Brien and Winston, just as I would leave out Saddam’s people-shredders, on the grounds that the purposes involved are not obviously those of “enhanced interrogation”—O’Brien isn’t at that point looking for information, he’s destroying Winston prior to execution for reasons I’ve never understood, but which I suspect the pyramid-of-prisoners people understood perfectly well; it has something to do with dominance. (Yes?) And I would not start out with rules that say “10 hours yes, 11 hours no” or “uncomfortable chair yes, stress position no”. I would not even start out with a rule against waterboarding. I would start out with a rule that says that (1) interrogation should always be on video, that (2) any sequence of interrogation techniques (insults? lights? Madonna videos? waterboarding?) should be precisely described, that (3) any interrogator should have previously undergone any technique he or she applies on a publicly-available “licensing video”, that (4) the interrogation video should immediately begin a chain of viewings and authorizations which ends with actual publication no more than N years (10?) later. I could go on about what I mean by “precisely described” and how the authorizers would be authorized and all that, but I suspect that (a) most people who describe themselves as “anti-torture” are, by now, angry, even though (b) any such pro-clarity rule would end up ruling out whatever it is that ordinary people think of as torture, while allowing the anti-torture people a maximal opportunity to explain specifically what they meant and why all of these (or all but X, Y, and Z) should be outlawed. So...gee. Am I biased in favor of clarity? Yes, but only to a limited extent. And I’m glad to learn from people who are biased against bias more than I am, especially if I can figure out what they mean.
Tom Myers,
Systematic but unexplained: sure, most errors are probably due to heuristics, but I’m not sure that’s a useful statement. A number of posts here have been so specific, they don’t seem like useful starting points for searching for heuristics.
Cost:
Most people don’t seem to have sufficiently clear goals to make sense of whether something benefits or costs them, let alone balancing the two.
People live normal lives by not thinking too much about things, so it shouldn’t be so surprising that they don’t think in psych experiments in which it is often clear that analysis will help. But if one is interested in producing answers to questions that don’t come up in normal life (eg, how much is medicine worth), avoiding everyday heuristics is probably worth the cost. Heuristics may well be worth overcoming in everyday life, as well, but I don’t think any experiments I’ve heard about shed any light on this.
Torture:
Your proposals are too detailed. I don’t imagine an opportunity to experiment enough to figure out how to structure torture, and if I do get an opportunity to experiment on government structure, torture is not going to be high on my list of variables. A government is an extremely expensive experimental apparatus. At least I can imagine how to experiment with corporal punishment, but I don’t really have much of an idea of how one could go about comparing the efficacy of different interrogation methods or the general investigative qualities of, say, American and Japanese police.
I’m not inclined to find out what you mean by Saddam’s people-shredders, but I imagine that one effect was a deterrent to crime, especially crime that would get Saddam’s attention. Torture, especially creative torture with vivid imagery, may well exploit salience(?) biases to be a more effective deterrent (aside from the rationally greater desire to avoid torture+death than to avoid death). The role of whim on one’s fate may also have (irrationally) increased the deterrent effect. The vague beliefs people hold about prison rape may play a very similar role in the US system. We do have arbitrary torture in our criminal justice system already.
Winston survives.
Douglas Knight: for me, thinking of “bias” (as used on this blog) as a result of heuristic processing is moderately useful ’cos (a) mainly, it just gives a general framework, a set of very concrete metaphors and therefore heuristics (and therefore biases) that I’ve worked with over the years; (b) it suggests that the problem of bias can be ameliorated but not solved, because you’ll never get perfect heuristics and you’ll never be able to do all the computing that’s required to do without heuristics; (c) ah, well, I forget what (c) was gonna be. But it’s useful for me, and I wanted to know if anybody here would point to a reason why I shouldn’t use it. Nobody did, yet.
When you speak of the cost/value of “overcoming” heuristics, that’s interesting...it jars slightly, which is good. I’m used to ideas of balancing out heuristics, of using meta-heuristics (i.e., explicit knowledge of the bias introduced by a particular heuristic) and such for overcoming the bias of a heuristic, but overcoming heuristics...strange. I’m not sure why that jars, but I thank you.
My mention of people-shredders was merely to distinguish that kind of torture, punitive/deterrent torture, from interrogative torture. The distinction matters because when I see people (say, Max Boot) defending practices which others class as torture, they aren’t defending punitive/deterrent torture (or confession-inducing torture) at all; they’re defending “interrogation techniques”. Posts such as this one, I believe, lose some of their impact because they don’t go as far as they could in achieving clarity; the people who might be criticized will, if exposed to this post, think of it as a straw-man argument: “That’s not me at all”. BTW, the people-shredders specifically may never have existed; if I’d remembered that as I typed, I’d have used Saddam’s deterrent amputations instead:
In any case, I don’t think it’s worth attacking this kind of torture except in a context where someone’s defending it; if Joe Schmoe defends an interrogator who tries to track down a house where IEDs are being made, Joe is not going to recognize himself in this criticism (nor should he.) It might be relevant in a discussion of the US prison system and its acceptance of prison rape, but even then, I don’t think that a defender of that system (I’m not one) would recognize his own motives in O’Brien’s. Having made that distinction, I would then drop the term “torture”, as I said, because it does not help clarity; “pro-torture” and “anti-torture” have non-empty intersections. And my proposal may well be too detailed...but our current attempts to regulate interrogation are even more detailed, and I think they’re detailed the wrong way. Whatever. The principle, I thought, was pretty simple: everything done by X should be documented, X must have voluntarily had it done to him first, and the documents should after some time-lapse be public so that X and his commanders are accountable. And I thought it an interesting kind of principle, in that it avoids the current kind of detail. Okay, probably not.Yes, of course Winston “survives”, in a sense. He’s not executed. I was remembering, quite probably misremembering (can’t find my copy, I think my eldest son took it years ago) a passage in which execution is represented as too easy: first he has to Love Big Brother, and then after that it doesn’t matter if he’s executed or not. Something about dominance, as I tried to say. (But is the Winston at the end...hmm. Is that survival? Maybe HA would consider it so; personally I felt that Winston the person had been destroyed. The politics of personal destruction, as it were. Unless I’m misremembering quite drastically, which is always possible.)
You probably won’t go far wrong if you assume I agree with you on the points I don’t respond to. I probably shouldn’t have talked about them in the first place.
overcoming heuristics:
If we know a bias is caused by a heuristic, then we should use that heuristic less. But articulating a meta-heuristic about when to use it is very different implementing that meta-heuristic. Human minds aren’t eurisko that can dial up the strength on heuristics. Even if we implement a heuristic, as in Kahneman’s planning anecdote, and get more accurate information, we may simply ignore it.
The basic problem is system 1 vs 2, when to analyze a problem and when to trust unconscious systems. Perhaps we have control over the conscious analysis (but still, it has unconscious inputs). But even starting the process, turning on the conscious process, must be done from an unconscious start.
I guess we’re not disagreeing about much, at this point, though I think that you’re basically more optimistic than I, and this might cause us to form different conceptions of the “overcoming bias” enterprise. I agree that we’re not Eurisko (and suddenly I’m remembering Lenat’s talk at IJCAI-77, explaining AM’s fixed-heuristics problem that then led him to Eurisko...I was a graduate student) but my feeling is that we don’t in general even have the choice of using a given heuristic less: we don’t in general have the choice of becoming a less initially biased person. Sometimes we do, and it’s worth a try, I’ll admit that. In general, however, I don’t think much of my own rationality in speech or action or even writing: it’s mainly in proofreading, especially shared proofreading, that we have the chance to overcome our biases. For this purpose, it’s perfectly possible to say “this is a valuation by prototype” or whatever, and then think a meta-thought about errors found in association with that heuristic. (Nor do I really believe that we commonly have heuristics that aren’t associated with bias—systematic error—it’s just a question of identification and of doing the best we can. Not error-free, but error-correction.
Of course in order to do that, you need to be conscious of your heuristics, which isn’t always possible either, but when you try to explain your opinions to somebody else, sometimes you notice the rule of inference you’re applying, and then take a step backwards. And another. And another, until the metaphor falls off the cliff. :-) But until transhumanism actually works, or until Lenat successfully mixes Eurisko and Cyc (and, as he said in 1977: “It’s our last problem. They’ll handle the next one”), I think it’s the best we can do, and I get the feeling you think we can do better. But I have no confidence in such feelings.
:The German text of the taped police examination, each page corrected and approved by EIchmann, constitutes a veritable gold mine for a psychologist—provided he is wise enough to understand that the horrible can be not only ludicrous but outright funny. Some of the comedy cannot be conveyed in English, because it lies in Eichmann’s heroic fight with the Germna language, which invariably defeats him. It is funny when he speaks, passim, of “winged words” (geflugelte Worte, a Gemran colloquialism of famous quotes from the classics) when we means stock phrases, Redensarten, or slogans, Schlagworte....
Dimly aware of a defect that much have plagued him even in school, he apologized, saying “Officialese [Amtssprache] is my own language. But the point here is that officialese became his language because he was genuinely incapable of uttering a single sentence that was not a cliche...
Eichmann’s mind was filled to the brim with such statements.....his memory proved to be quite unreliable about what had actually happen; the [reason], of course, was that Eichmann remembered the turning points in his own career rather well, but they did not necessarily correspond to the turning points in the Jewish extermination or, as a matter of fact, with a lot of the turning points in the history....
But the point of the matter is that he had not forgotten a single one of his sentences that at one time or another had served to give him ‘elation’. Hence, whenever, during the cross-examination, the judged tried to appeal to his conscience, they were met with ‘elation,’ and they were outrage and disconcerted when they learned that at his disposal he had a different elating cliche for each period of his life and each of his activities...” (Eichmann in Jerusalem, Hannah Arendt, Chapter III)
Clearly, Winston was just an actor, and O’Brien was being tested by a researcher at Miniluv, whose name was Stanley Milgram.
I find it eerie how intellectually close O’Brien’s mindset is to that of the pure Post-Modernist of today (for the purpose of comparison, we will disregard the fact that the word ‘pure’ in relation to post-modernism is an oxymoron). If a meteor smashed through a roof, someone then cleaned up the mess and patched a hole in the roof, then erased the memories of those in proximity at the time, then the meteor never fell. This is exactly how O’Brien deals with Winston’s refusal to depart from the bare physicality of arithmetic on the fingers; the key similarity is that both the State in 1984 and modern philosophers of today deny the existence of an objective truth—even a physical one; this is why not many scientists are prepared to become post-modernists, because the degree of doublethink required is too great. Hawking, one of the greatest contributors to physics in the 20th and 21st century, searches for a ‘Theory of Everything’; to have a theory at all, one must believe in some sort of exterior truth, wherever it is from.
New Speak always chilled me. To limit the abilities of population by limiting their vocabulary seems like a sneaky and underhanded thing to do that should never be attempted by any government. Needless to say the idea of 1984 being real freaks me out.
I really dislike this post. It is essentially propaganda. It claims (without providing any kind of evidence based assessment!) that all the good things in human history are the result of rational thinking, while all the bad things in human history are the result of stupidity. I think that’s quite clearly false.
First, most deaths in human history could not have been avoided, only delayed at best. No one person could have created an industrial society on their own, no matter how clever they might have been. If Da Vinci couldn’t save the world and stop death, then no one else should be perceived as having failed. They did what they could, mostly, even if their lives were not lived perfectly.
Second, stupidity has caused some good things. Many humans are prone to naivety. They trust strangers to handle their wealth, or decide to cooperate in one shot instances of the prisoner’s dilemma even when there are large rewards that might await them. The effect of such stupidity is on the whole good for our society. Rational, highly intelligent beings cannot take the same kinds of shortcuts that more limited beings can. Thus, they can sometimes face heightened transaction costs.
Third, intelligence has caused some bad things. The United States would never have atomically bombed Japan if no scientist ever invented nuclear weapons, for example. Every major evildoer in history was only able to do evil due to certain ideas which entered their minds. Ultimately, knowledge is simply power, and power is neither totally safe nor totally dangerous. We are not soldiers in a war against evil irrationality; we are human beings simply trying to live and evil is hard to identify or fight without looking at specifics and context.
Of course, there is an imaginable utopia in which every single human being is perfectly rational and everyone cooperates and there is perfect happiness. But envisioning utopias is a bad way to decide which causes to support, as utopias of all types are very easy to imagine. A conservative might claim that if everyone perfectly adhered to conservative principles the world would be better off, and a progressive might claim that if everyone perfectly adhered to progressive principles the world would be better off, but such assumptions of perfection are so unrealistic that they don’t help us to decide which principles we should actually believe in or support.
Vague aesthetic insight I’ve just had upon rereading this comment of mine, which I don’t want to forget:
“Trust, then verify” works much better for building good social systems than “verify, then trust”.
Overcoming bias in itself will be useless if there would be people with the power to decide what is bias and what is not.
Just as the fight against extremism (or racism, intolerance, etc.) can be skewed if there are people with enough political power to decide the exact meanings of these terms.
begging the question
From epistemic rationality how can one infer a particular route for instrumental rationality? I can’t think of any way since there are unaccounted for variables independent of rationality skills, like the complexity of one’s values (there’s a sequence post on this, too!)
Blue or green?. It’s this kind of post, I hypothesise, which scared Mark away.
Presumptuous. And, compulsory history lessons indoctrinated this fear into me. Moreover, politics is broadly along liberal-conservatives (manned by anti-totalitarians) vs progressive-socialists in my country, so totalitarianism is a salient concept in many political debates. I identify as a liberal (in the classical liberal sense), but don’t precommit to any abstract notions, including those of fellow liberals, one might assume follows from there, because only I can stop my mind from being killed. It’s only significance to me is concrete – it’s permission to participate in the social activities of the one of 2 major political parties in my country.
How are rationalist writers any different? Consider Tyler’s argument again:
Rationalist taboo?
Strawmanning your competition, steelmanning your own side – unfair audience manipulation here
I suppose you think that’s justified because it advances the rationalist course, or the MIRI course, but ends don’t justify means amongst humans.
Then again, maybe I’m expecting too much from debate, reasoning isn’t about logic, it’s about arguing, even if that’s about improving our logical ability to get better at arguing. It’s not going to work unless we’re at the same interal inferential step.
Scope sensitivity, as I wrote on that sequence article too, eternalism isn’t the rational position. I, for one, discount the value of anything in the future because of my capacity to die, or to change my mind to a position unaccountable with present information.
Woes and leaps forward can be intertwined. Peter the Great deliberately sent many workers (not to mention soldiers) to certain deaths to build St. Petersburg and fleet to win over Charles XII of Sweden. Nor was he ever hypocritical about it—he was strict enough to witness executions and not care. But in the end Peter’s actions do seem like a leap forward. No hypocrisy, no stupidity, but huge woes.
Orwell is frequently cited by both the left and right as the most clear-headed social critic of the millennium. But he is not a corrective to muddle-headedness. He was, indeed pretty muddle-headed himself. His early works, Down and Out, and The Road to Wigan Pier, were, he later admitted, largely fictionalized, lavishly embellished accounts of his adventures among the exploited and desperately poor. The conditions he described were not untrue, but they were dramatized. He entered the Spanish Civil War on the highly romanticized left and quickly became disillusioned and revealed the Soviet Union’s betrayal of the Left, the side he had joined. Prior to that. shortly after graduating from the foremost establishment preparatory school, he joined the British Colonial police in Burma. He became disillusioned with colonialism. What did he expect? A policeman no less. Yes,1984 is a prophetic masterpiece, but he failed to recognize that the totalitarian state had already arrived in the USSR. His “future” predictions were already the present. He died, as left literary and political commentators are at pains the underline, as a man of the left, and a socialist. The USSR died when Gorbachev tried the impossible, to invent socialism, or communism, with a “human face,” That is, free of intolerable coercion. Not even the fantasizers, from Bacon on, ever imagined that a utopian “paradise on earth” could be fashioned without totalitarianism. He had great insight into the mechanisms of totalitarianism, but his life was rife with contradictions.