Rationality Quotes January—March 2017

[Sabiola]

Another quarter, another rationality quotes thread. The rules are:

• Provide sufficient information (URL, title, date, page number, etc.) to enable a reader to find the place where you read the quote, or its original source if available. Do not quote with only a name.

• Post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)

• Do not quote yourself.

• Do not quote from Less Wrong itself, HPMoR, Eliezer Yudkowsky, or Robin Hanson. If you’d like to revive an old quote from one of those sources, please do so here.

[PhilGoetz]

One lesson I’ve learned in life is that when someone thinks that something you’re doing is crazy, having a logical, multi-point defense of said action will not make you look less crazy to them.

• Scott Meyer, Basic Instructions, “How to be Eccentric”

[Gram_Stone]

This is rather as if you imagine a puddle waking up one morning and thinking, ‘This is an interesting world I find myself in, an interesting hole I find myself in, fits me rather neatly, doesn’t it? In fact, it fits me staggeringly well, must have been made to have me in it!’ This is such a powerful idea that as the sun rises in the sky and the air heats up and as, gradually, the puddle gets smaller and smaller, it’s still frantically hanging on to the notion that everything’s going to be all right, because this World was meant to have him in it, was built to have him in it; so the moment he disappears catches him rather by surprise. I think this may be something we need to be on the watch out for.

Douglas Adams, The Salmon of Doubt

[Viliam]

If you are playing a game according to certain rules and set the playing-machine to play for victory, you will get victory if you get anything at all, and the machine will not pay the slightest attention to any consideration except victory according to the rules. If you are playing a war game with a certain conventional interpretation of victory, victory will be the goal at any cost, even that of extermination of your own side, unless this condition of survival is explicitly contained in the definition of victory according to which you program the machine.

While it is always possible to ask for something other than what we really want, this possibility is most serious when the process by which we are to obtain our wish is indirect, and the degree to which we have obtained our wish is not clear until the very end. Usually we realize our wishes, insofar as we do actually realize them, by a feedback process, in which we compare the degree of attainment of intermediate goals with our anticipation of them. In this process, the feedback goes through us, and we can turn back before it is too late. If the feedback is built into a machine that cannot be inspected until the final goal is attained, the possibilities for catastrophe are greatly increased.

A goal-seeking mechanism will not necessarily seek our goals unless we design it for that purpose, and in that designing we must foresee all steps of the process for which it is designed, instead of exercising a tentative foresight which goes up to a certain point, and can be continued from that point on as new difficulties arise. The penalties for errors of foresight, great as they are now, will be enormously increased as automation comes into its full use.

-- Norbert Wiener, God and Golem, 1964

[g_pepper]

Nice, succinct statement of the Unfriendly AGI argument, and written 53 years ago!

[Stabilizer]

In the United States, constructivist views of knowledge are closely linked to such progressive movements as post-colonialism and multiculturalism because they supply the philosophical resources with which to protect oppressed cultures from the charge of holding false or unjustified views.

Even on purely political grounds, however, it is difficult to understand how this could have come to seem a good application of constructivist thought: for if the powerful can’t criticize the oppressed, because the central epistemological categories are inexorably tied to particular perspectives, it also follows that the oppressed can’t criticize the powerful.

• Paul Boghossian, Fear of Knowledge: Against Relativism and Constructivism, pg. 130

[WalterL]

“Playing to win involves viewing a loss as an opportunity to learn and improve.”

David Sirlin, Playing To Win

https://​​sirlin.squarespace.com/​​ptw-book/​​sportsmanship

[Viliam]

One thing at any rate is clear. The physical identity of an individual does not consist in the matter of which it is made. Modern methods for tagging the elements participating in metabolism have shown a much higher turnover than was long thought possible, not only of the body as a whole, but of each and every component part of it. The biological individuality of an organism seems to lie in a certain continuity of process, and in the memory by the organism of the effects of its past development. This appears to hold also of its mental development. In terms of the computing machine, the individuality of a mind lies in the retention of its earlier tapings and memories, and in its continued development along the lines already laid out.

Under these conditions, [...] there is no inconsistency in a living individual forking or divaricating into two individuals sharing the same past, but growing more and more different. This is what happens with identical twins; [...]

To recapitulate: the individuality of the body is that of a flame rather than that of a stone, of a form rather than of a bit of substance. This form can be transmitted or modified and duplicated, [...] there is no absolute distinction between the types of transmission which we can use for sending a telegram from country to country and the types of transmission which at least are theoretically possible for transmitting a living organism such as a human being.

-- Norber Wiener: The Human Use of Human Beings

[cousin_it]

I try to let my projects live and die by their own momentum with a few caveats.

Edward Kmett

[Stabilizer]

True listening requires giving up the prerogative of your own mental model. You have to allow them to set the rules of engagement, no matter how bizarre, so that they let their guard down and realize you are not a threat, because you have no intention of blaming them for anything. The way they set these rules will reveal their assumptions and constraints, which thoughts and actions are open to them. If you can tell an authentic story that speaks to these assumptions, you can break through, because stories speak to emotions expressed in the body, which fortunately refuses to go along with even our most well-reasoned rationalizations.

The price of such effective action is we have to be willing to give up the petty payoffs we cherish in our arguments with each other: not only the blaming but the cynicism, the martyrdom, the self-righteous indignation, the outrage, the winning, the making others lose, the being right, the making others wrong.

And after all that, you may still fail to convince them to your point of view. Are you willing to not win in order to keep the conversation going?

• Tiago Forte, The Throughput of Learning

[PhilGoetz]

Three inventions which may perhaps be long delayed, but which possibly are near at hand, will give to this overcrowded island the prosperous condition of the United States. The first is the discovery of a motive force which will take the place of steam, with its cumbrous fuel of oil and coal; the second, the invention of aerial locomotion which will transport labour at a trifling cost of money and of time to any part of the planet, and which by annihilating distance will speedily extinguish national distinctions; the third, the manufacture of flesh and flour from the elements by a chemical process in the laboratory, similar to that which is now performed within the bodies of animals and plants. Food will then be manufactured in unlimited quantities at a trifling expense, and our enlightened prosperity will look back upon us who eat oxen and sheep just as we look back upon cannibals. Hunger and starvation will then be unknown, and the best part of human life will no longer be wasted in a tedious process of cultivating the fields. … [claims that everyone will embrace Victorian morality omitted] … These bodies which now we wear belong to the lower animals; our minds have already outgrown them; already we look upon them with contempt. A time will come when science will transform them by means which we cannot conjecture, and which, even if explained to us, we could not now understand, much as the savage cannot understand electricity, magnetism, or steam. Disease will be extirpated; the causes of decay will be removed; immortality will be invented. And then, the earth being small, mankind will migrate into space, and will cross the airless Saharas which separate planet from planet and sun from sun. The earth will become a Holy Land which will be visited by pilgrims from all the quarters of the universe. Finally, men will master the forces of Nature; they will become themselves architects of system, manufacturers of worlds.

-- Winwood Reade, “The Martyrdom of Man”, 1872

quoted (and ridiculed) by Patrick Allitt in 2002

[Stabilizer]

The intuitive standard for rational decision-making is carefully considering all available options and taking the best one. At first glance, computers look like the paragons of this approach, grinding their way through complex computations for as long as it takes to get perfect answers. But as we’ve seen, that is an outdated picture of what computers do: it’s a luxury afforded by an easy problem. In the hard cases, the best algorithms are all about doing what makes the most sense in the least amount of time, which by no means involves giving careful consideration to every factor and pursuing every computation to the end. Life is just too complicated for that.

In almost every domain we’ve considered, we have seen how the more real-world factors we include—whether it’s having incomplete information when interviewing job applicants, dealing with a changing world when trying to resolve the explore/​exploit dilemma, or having certain tasks depend on others when we’re trying to get things done—the more likely we are to end up in a situation where finding the perfect solution takes unreasonably long. And indeed, people are almost always confronting what computer science regards as the hard cases. Up against such hard cases, effective algorithms make assumptions, show a bias toward simpler solutions, trade off the costs of error against the costs of delay, and take chances.

These aren’t the concessions we make when we can’t be rational. They’re what being rational means.

• Brian Christian and Tom Griffiths, Algorithms to Live By

[Sarunas]

From doing this internet propaganda in the early years of the internet, I learned how to do propaganda. You don’t appeal to emotion, or to reason, or anything. You just SHOUT. And REPEAT, and explain the position, and let the reader defend it for himself.

In the end, most readers agree with you (if you are right), but they will come up to you, much as you did, and say “While you are right, I see that, you are doing yourself a disservice by being so emotional—you aren’t persuasive....”

But I persuaded this reader! The fact is, I am persuasive, and maximally so. When there is a hostile political environment, if a paper is called “bullshit” or “pseudoscience”, you need to first MOCK the idiots calling it that, so as to establish a level playing field. That means calling them “douchebag”, “fuckwit”, “turd-brain”, etc, so that both you and the other person sound like children fighting in the playground, no authority.

Then you need to state the objective case (after the name-calling and cussing, or simultaneously), and then wait. If you are objectively right, people will sort it out on their own time, you don’t have to do anything. The people who didn’t sort it out will say “oh my, there’s a controversy” and will keep an open mind.

It’s classic propaganda techniques, and it can be used for good as easily as it can be used for evil. Of course, when calling people idiots for not agreeing with material that is called crackpot, you had better be careful, because if you are not right about the material, if it is crackpot, you are gone for good. The main difficulty is evaluating the work well, understanding it fully, and making sure that it is not crackpot, before posting the first cussword.

Ron Maimon

I have found it interesting and thought provoking how this quote basically inverts the principle of charity. Sometimes, for various reasons, one idea is considered much more respectable than the other. Since such unequal playing field of ideas may make it harder for the correct idea to prevail, it might be desirable to establish a level playing field. In situations when there are two people who believe different things and there is no cooperation between them, the person who holds the more respectable opinion can unilaterally apply the principle of charity and thus help to establish it.

However, the person who holds the less respectable opinion cannot unilaterally level a playing field by applying the principle of charity, therefore they resort to shouting (as the quote describes) or, in other contexts, satire, although just like shouting it is often used for other, sometimes less noble purposes.

To what extent do you think these two things are symmetrical?

[Tyrrell_McAllister]

Of course, when calling people idiots for not agreeing with material that is called crackpot, you had better be careful, because if you are not right about the material, if it is crackpot, you are gone for good.

But you aren’t “gone for good”. You will have your own tribe of believers who will still support you. Before they had been called “fuckwits” they might have deserted you when the evidence didn’t go your way. But they’re not going to desert you now, not when doing so would be tantamount to admitting that they were fuckwits all along.

[cousin_it]

Yeah, if you want to be heard, it certainly helps to be a loudmouth. You can do it in different ways though. Eliezer’s trick of being infuriatingly certain works as well or better than Ron’s rudeness.

On the other hand, I feel that being a loudmouth can sometimes hurt your ability to do interesting intellectual work. For that purpose Wei’s communication style is the best I know, and I’ve adopted it wholeheartedly.

[Good_Burning_Plastic]

We need data, but we also must prioritize understanding the data we have over collecting ever more data.

-- Bruce Schneier

[27chaos]

Disasters and miracles follow similar rules. Charles Babbage, in his Ninth Bridgewater Treatise of 1837, considered the nature of miracles (which, as a computer scientist, he viewed as pre-determined but rarely-called subroutines) and urged us “to look upon miracles not as deviations from the laws assigned by the Almighty for the government of matter and of mind; but as the exact fulfilment of much more extensive laws than those we suppose to exist.” It’s that question of characteristic scale.

George Dyson, comment on Taleb’s “The Fourth Quadrant”.

[Tyrrell_McAllister]

If someone who wanted to learn to dance were to say: For centuries, one generation after the other has learned the positions, and it is high time that I take advantage of this and promptly begin with the quadrille—people would presumably laugh a little at him, but in the world of spirit [i.e., development of one’s soul] this is [thought to be] very plausible. What, then, is education? I believed it is the course the individual goes through in order to catch up with himself, and the person who will not go through this course is not much helped by being born in the most enlightened age.

Søren Kierkegaard, Fear and Trembling, III 96 (trans. H. V. Hong and E. H. Hong). Annotations are mine.