Rationality Quotes: July 2010
This is our monthly thread for collecting these little gems and pearls of wisdom, rationality-related quotes you’ve seen recently, or had stored in your quotesfile for ages, and which might be handy to link to in one of our discussions.
Please post all quotes separately, so that they can be voted up/down separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
Do not quote yourself.
Do not quote comments/posts on LW/OB.
No more than 5 quotes per person per monthly thread, please.
-- Steve Gilham
The Salvation War Web Original trilogy is based on this premise. And boy does it makes good use of it.
I’ve just been reading it.
Oh dear… this series looks to be fun and really really bad.
“Good Omens” (Pratchett and Gaiman) is an excellent book along such apocalyptic lines.
I’ve been finding it to be rather terrible, in terms of plausibility. I won’t rip into it at length, since the tvtropes link seems to do a decent job.
Relevant xkcd: http://www.xkcd.com/808/
Now that you mention it, this is kind of the premise of the Star Ocean series. It has “symbology” (as it’s called in the third installment, Till the End of Time), which is basically the ability to manipulate nature to do “magical”-seeming things by formation of particular symbols. The people in these worlds harness this capability for standard engineering purposes: they build air-conditioning units that draw their coldness from application of specific symbols.
The plot of End of Time revolves around a professor combining symbological powers with those of genetics.
TTEoT is a superb game, and that’s not the only LW-relevant theme it contains.
Spoilers:
Gur cynlre punenpgref ner NVf jub cebprrq, va gur pybfvat npgf bs gur tnzr, gb rfpncr gurve obk. Gur raqvat vf n Crezhgngvba Pvgl-yvxr fpranevb jurer gur “birefrref” fuhg qbja gur jbeyq-fvzhyngvba—ohg vg pbagvahrf naljnl qhr gb vagreany frys-pbafvfgrapl.
So it’s the same as the plot of Harry Potter and the Methods of Rationality? >.>
Oh god, I hope not. You can see in my above comment what that would actually entail.
As it was presented in the game, I personally found that to be among the worst plot twists I had ever experienced.
Fnlvat gung gur znva punenpgref ner nyy NVf fbhaqf n ybg pbbyre guna gur zber cerpvfr eriryngvba, gung gurl’er nyy ACPf sebz na ZZBECT cynlrq ol orvatf sebz “4 qvzrafvbany fcnpr” juvpu gheaf bhg gb or culfvpnyyl nyzbfg rknpgyl yvxr gurve bja havirefr.
Oh, hush you. With my LessWrongian superpowers, I know what the game was meant to be about. It’s not our fault that the scriptwriters just had to be rephrased with the “sbhegu qvzrafvba” herpaderp so that lesser mortals could follow it. :)
(I’m now imagining a MethodsOfRationality!Fayt who had been expecting the plot twist based on ubj fhfcvpvbhfyl uhzna nyy bs gur “nyvra” enprf va gur tnynkl jrer...)
:D I was rather disappointed in Eliezer for not giving Star Ocean: Till the End of Time a mention in his ultimate crossover fanfic. My cat is named Fayt. (The other one’s named Lyra.)
It does, and it is.
Are you referring to psychology?
I posted without having thought of any examples, still confident that the statement is true.
The nearest example I can think of is the alchemical concept of transmuting one metal into another. This process is of course central to nuclear reactions, but pre-20th Century was considered physically impossible.
Lightning was the weapon of Zeus. Now it can be controlled by electrical engineers.
The Aztecs thought the sun was a god. Now plasma physicists can produce light via similar means.
Those are good examples.
Magnetism?
People have cited some very modern inventions, but ever since the first invention, people have been inventing things that were previously relegated to magic. I mean, what else is technology?
Some technology is for replacing things ascribed to magic. But most technology is fulfilling needs that a wizard doesn’t know he has, or is there to solve a need or problem that wouldn’t except for some other bit of technology.
Look at a few thousand (out of the millions of existing) patents. How many of them have a direct analogue in magic such that their utility really is something ‘previously relegated to magic’? - not vaguely falling in a general class of functionality which you once saw a spell kinda-sorta like.
For example. Recent news about an environmentally friendly rocket fuel made out of just nitrogen and oxygen. This is a very modern invention. It’s very useful. What magic does it replace?
‘Oh, rocket fuel replaces flying on a broomstick!’ Ah, but I wasn’t talking about rocket fuel in general and certainly not rockets+rocket-fuel, because this invention was not of ‘rocket fuel in general’. This invention was of a particular kind of rocket fuel. What magic does this particular kind of rocket fuel replace? The answer is none. Flying magic is conceived of as intrinsically environmentally friendly. Harry Potter never asks himself how he will pay for the bioremediation of the hydrochloric acid left in the wake of his broomstick.
Take a look at Kevin Kelly’s discussion of sparkcatchers. What magic is this technology replacing? This isn’t even particularly esoteric stuff like I could start pulling out from biology or chemistry. (What wizard—or author—ever had the imagination to think of bacteria glowing green like fireflies?)
This kind of statement reminds me of an upvoted quote here somewhere which went ‘since when has the majority ever been right about anything?’ This irritates me, because the majority is right. Of the infinite class of propositions the majority holds, most are right, and the ones the majority has been wrong about, like geocentrism, have approximately measure 0. It’s right about everything else. The majority in the room I am in think I have 2 hands; think I have 2 feet; think I have 2 lungs; think I am there; think.… etc.
For a group that’s supposed to be so awfully wrong, the majority seems to get things awfully right.
Very much agreed. By the way, this one one of the points W. Brian Arthur makes in The Nature of Technology which I recommend as a well written exposition of quite a number of the dynamics in technology development.
One very broad range of technology which has no analog to magic is high precision instrumentation. There are very few situations in daily life when I need to know a measurement to within 0.1%, and they are many technological and scientific situations where that knowledge is necessary.
Actually, Arthur is already on my reading list. ;) I think I must have seen him recommended before by Kevin Kelly or perhaps Don Ihde.
I was going to point out that another good example is all the sensory modalities that science knows of, like electromagnetism in all its forms and frequencies, or gravity itself, but I thought that they were too arguably close to magical ‘second sight’ or chi skills like sensing someone nearby.
You have to be careful that you don’t go from “the majority is right in most cases” to “the majority is right in this case” without good reason. Prior to any information about this case, sure, it’s a good guide. But if you have reason to believe that this case isn’t a trivial inference from overwhelming direct observational evidence, then the majority is much less of a guide. When it comes to a socially-accepted conclusion, a severe dearth of evidence, and motivations to hold a certain point of view, 70-90% - the majority—get it wrong.
On the phrase “since when has the majority ever been right about anything?”: on most points of difference, the majority has been wrong first and right only later. It is easy to look at only the points of difference: when it comes to points of similarity, you are part of the majority, and so you’re not really interested in knocking them down. Perhaps a better formulation would be “since when has the majority ever been right about anything that isn’t simultaneously important and obvious ?”
The problem with saying
is that most are boring. On the class of interesting propositions, the majority does not have a good track record.
Voted down because of epistemic strutting. Not citing examples in your first comment is one thing, but you could at least have done so in your follow-up.
-L. A. Rollins, Lucifer’s Lexicon
Wow. I’m going to have to get myself a copy of that book!
-- K. Eric Drexler, Engines of Creation
-- Steve Moore, former FBI agent
From the Wikipedia article about perverse incentives:
and
[context added]
-- Soul Music, by Terry Pratchett
Needs more context.
--Friedrich Nietzsche
“I am free, no matter what rules surround me. If I find them tolerable, I tolerate them; if I find them too obnoxious, I break them. I am free because I know that I alone am morally responsible for everything I do.”
Robert A. Heinlein
-- Isaac Asimov
Repeat?
I only judge something as a repeat if it’s already appeared in a quotes thread. Sometimes it’s worth pulling out something quoted in the middle of a discussion and presenting it here.
Apparently, yeah. Not in a quotes thread, though.
--Mikhail Bakunin, God and the State
Reminded me of some posts here by Academician.
I wish I could upvote this, maybe 10 or 20 times. It’s essentially a concise statement of my religion and I’m putting it on Facebook today.
-- Plutarch
A recent one from Linux Weekly News that gives insight into rationality:
Side note: when a respected information source covers something where you have on-the-ground experience, the result is often to make you wonder how much fecal matter you’ve swallowed in areas outside your own expertise. -- Rusty Russell
Similar to what professor Karel Culik said about Time magazine. He thought Time was really good, until he read something about computer science, and he started to wonder about the rest of it.
This was the original sense of TVTropes/DanBrowned, although the scope seems to have broadened a bit.
-- Mark Twain, “Old Times on the Mississippi”
What frightens us most in a madman is his sane conversation.
--Anatole France
I can’t find any source for this, so it may be apocryphal.
-- George Iles
“Anyone who believes that the theory of evolution implies moral darwinism, and who also believes in the theory of gravity, has a moral duty to go jump off a cliff.”—Ari Rahikkala
Is there anyone who actually believes in moral darwinism under that self-description, or is it just a straw man position that people like to claim as the underpinning of the ideology of people they don’t like? When I tried to google the term I found the book Moral Darwinism: How We Became Hedonists which is described thusly in a five star review:
I think that the quote is best read as a critique of any appeal to nature. Maybe you don’t see such brazen appeals to nature in careful arguments, but they seem common enough in informal arguments.
Many people really do seem to think that you’ve made a substantive point about what we ought to do when you point out that something happens “in nature”. For example, I knew someone once who was uncomfortable with tolerating homosexuality, but who felt obliged to seriously reconsider his position when he learned that animals sometimes exhibit homosexual behavior. I was in the strange position of trying to explain why he ought not to be persuaded by that argument, even though it led him to what I thought was the right conclusion.
Did you then tell him about chimpanzee war, murder, and infanticide?
The appeal to nature is very common, and I’d rather get rid of that than change someone’s mind about same-sex behavior, which seems to be a terminal value difference anyway.
So, Wiker thinks that the eventually-dominant (?) Epicurean movement staged a deliberate 2-millennium (?) campaign to get a 60s-style social revolution (?) going, and thereby steal credit (?) for scientific advancement from the Church, which was doing so well at it (?) for the first 1500 years.
Before reading it: what are the odds Wiker has marshaled enough evidence to even get that hypothesis on the radar?
Oh, and check this out from the description:
Yeah, we sure don’t have any evidence of a Greek culture practicing infanticide, do we?
Edit: Also, the review you refer to is by a “Phillip Johnson”, who looks to be this Phillip Johnson, founder of the Intelligent Design movement.
I think that you’re agreeing with me? My point was that Wike clearly has an axe to grind and was attributing belief in “moral darwinism” to people who don’t even know what it means. You’re logically taking apart the theory I was quoting… thus we’re in agreement… or not? Or something.
The thing I was puzzled by was the way the original quote had a surface layer of congruence (darwinism, suicide for stupid people, etc) but was difficult for me to coherently parse when I tried to work out the implications and examine the logic that inspired it.
I came up wanting when I did that, and that lead me to question whether the ideas themselves were well grounded. If not, where did the ideas even come from?
Yes, we’re in agreement. I was just shocked by how bizarre the claim is, given how many improbable pieces he’s stitched together.
I think Johnson—or Wiker, if that’s what the book really argues—came to that belief by fitting his own view sympathetically into a larger narrative, and found it convenient to stretch it as far back in time as possible.
In fairness, when reading about Epicurus, a lot of his ideas do match modern post-60s beliefs, but there isn’t a common demographic that endorses the whole package.
When I was 14, my father was stationed in Japan. I went rock climbing with this kid from school. He fell and got injured, and I had to bring him to the hospital. We came in through the wrong entrance, and passed this guy in the hall. He was a janitor. My friend came down with an infection, and the doctors didn’t know what to do. So they brought in the janitor. He was a doctor. And a Buraku—one of Japan’s untouchables. His ancestors had been slaughterers, gravediggers. And this guy knew that he wasn’t accepted by the staff, didn’t even try. He didn’t dress well. He didn’t pretend to be one of them. People around that place didn’t think he had anything they wanted, except when they needed him—because he was right, which meant that nothing else mattered. And they had to listen to him.
-- Dr. Greg House
-- Gregory House
Once we have achieved unerring correctness in all domains obscure or not, all we have to do is contrive a situation in which our correctness is useful and in which others will require our knowledge. Then we will be respected.
Yeah, I really thought that was true at some point… but as it turns out, being good at fixing broken Windows computers hasn’t made me respected, just commodified. :-)
That’s probably because the people whose computers you fix do not understand how difficult what you do is—or if they do, they also understand that another way to fix it is to reinstall the operating system.
-- H.G. Wells, A Modern Utopia
Silas will like this one:
-- David Gemmell “Legend”
I was actually starting another article that presents a solution (well, a research program) for qualia. [1] The idea is this:
The concept of qualia becomes mysterious when we have a situation in which sensory data (edit: actually, cognition of the sensory data) is incommensurable (not comparable) between beings. So the key question is, when would this situation arise?
If you have two identical robots with idential protocols, you have no qualia problem. They can directly exchange their experiences and leave no question about whether “my red” is “your red”.
But here’s the kicker: imagine if the robots don’t use identical protocols. Imagine that they instead simply use themselves to collect and retain as much information about their experiences as physically possible. They optimize “amount I remember”.
In that case, they will use every possible trick to make efficient use of what they have, no longer limited by the protocols. So they will eventually use “encoding schemes” for which there is no external rulebook; the encoding is implicitly “decompressed” by their overall functionality. They have not left a “paper trail” that someone else can use and make sense of (without significant reverse engineering effort).
In that case, you can no longer directly port one’s experience over into the other’s. To each other, the encoding looks like meaningless garbage. But if they’re still alive, they can still achieve some level of commensurability. They can look at the same uniform surface and ask each other, “how does your photo-modality respond to this thingamajig?” [2] They can then synchronize internal experiences across each other and have a common conception of “red”, even as it still may differ from what exactly the other robot is doing internally upon receiving red-data.
(And they can further constrain the environment to make sure they are talking about the same thing if e.g. one robot has tightly-coupled sensory cognition in which sensation of color varies with acoustics of the environment.)
This, I claim, is the status of humans with respect to each other: We have very similar general “body plans” but also use a no-holds-barred, standards-free method for creating (encoding) memories that puts up a severe—but partially circumventable—barrier to comparing internal experiences.
(Oh, and since you guys are probably still wondering: even I wouldn’t fault you for failing to explain color a blind man. The best I would expect is that you can say, “Alright, you know how smelling is different from hearing? Well, seeing is as different from both of those as they are from each other.”)
[1] Yes, I start a lot of articles but don’t finish them … have about three times as many in progress as I have posted.
[2] Remember: even though they have different internal experiences, they can still tell that a particular observation depends on a particular sensor by turning it on and off, and thus meaningfully talking about how their cognition relates to a particular sensor.
Before finishing (or perhaps as a sequel?) you should make sure to watch Cristof’s Koch’s “neural correlates of consciousness” talk. He’s been giving variations on this talk for something like 10 years that I know of and its pretty polished. Its gotten better over the years and the speaker is the source of my current working definition of consciousness (quoted below). Which is not about language I/O and compression but about internal experiences themselves and what systems implement them.
The core insight is that you can show someone a visual trick (like the faces or goblet image) and you can go back and forth “seeing different interpretations”. When you’re in one or the other state “internal state” this is you having different kinds of “qualia”, and presumably these distinct perceptual states have “biological correlates”.
Manipulation of these internal mental states and study of the associated physical systems become the “object of study” in order to crack the mind-body problem. Once you’ve got the neural level you can ask about high level issues like algorithms or ask about deeper mechanisms like neurotransmitters and genes and so on. Full understanding would imply that we could create mutant “zombie mice” and that they would have no qualia (of certain sorts) and be incapable of whatever behavior was “computed” in a way that involved (that sort of) qualia.
Ideally we would have a theory to predict and and explain such phenomenon and perhaps we’d be able to invent things like a pill that lets you “become a p-zombie” for an hour (though I suspect part of that would involve shutting down enough memory formation processes that you would not be able to remember the experience except via something external like videotape).
The Q&A has much more sophisticated objections/questions than you usually get on the subject of minds. The final question ends with Koch’s working theory which sounds about right to me. Quoting Koch when asked why he thinks bees are probably conscious and why he became vegetarian:
Thanks for the pointer! This will help me to connect my speculations to the existing literature.
Any text version/transcript of this lecture, or paper that explains the points in the talk?
My summary above cuts to what I think is the core insight about focusing clearly and experimentally on exactly the elements of interest: consciousness and neurons. The talk itself is a summary of the main points of many papers with different points and some demonstrations of the experimental manipulations.
The talk itself (rather than the intros) starts about 4 minutes in. I recommend just watching it. Koch is a pretty good speaker and this is sort of his “dog and pony show” where he summarizes an entire research program in a way that’s been iteratively optimized for years. Your 60 minutes will not be wasted :-)
Up voted, looking forward to you posting some of these in-progress articles. Should:
Be vice-versa?
Thanks! And yes it should, I’ll correct that.
This seems to match up with an argument I made against writing oneself into the future, though I think your formulation is more general. I’m certainly quite interested in hearing more on the subject; I think you’re headed in a good direction.
Thank you much, and I definitely see the similarity with what you posted.
I may indeed be running into the problem of letting “good enough” become the enemy of “at all”. I’ll try to get these articles up in some presentable form soon.
The drafts you’ve been posting lately are interesting to me, and I’d like to see them fleshed out into top-level posts. I would also suggest adding external material and references to give more context to your thought experiments.
Thanks. But that’s one of my difficulties. I read a lot of stuff and so these ideas just “come together”. I don’t even know if there is a source that agrees with this idea. As it stands now, the only sources I believe I’d be able to cite are some of Gary Drescher’s discussion of qualia, and the information-theoretic basics of how compression works, and what makes it more or less effective.
Any suggestions (specific suggestions) for how to find the external references that would be relevant to this topic or the other one’s I’ve posted recently?
By the way: I started keeping a list of planned top-level articles on my Wiki page.
I ‘get’ the other two, but the red-silk analogy eludes me.
I have heard of a blind man comparing red (as he had heard of it) to the sound of a trumpet.
Hmm… can a blind man possess synæsthesia? It seems possible if the source of the blindness is in the eye rather than the brain.
More likely color-words as concepts exist in the brains of blind people because there are interestingly different distributions of other directly meaningful concepts associated in proximity with others’ usage of the color-words, and with actual objects in the world (presuming that their color is sometimes described to you).
I think Richard put this very well when he said “as he had heard of it”.
Probably. A color-blind person can.
Me too. Silk is more like beige or cream color. How can you describe red without talking about fire?
Or this infographic from Information is beautiful.
The texture of silk is smooth and easily slips from the grasp. It is exciting and sensuous as a fabric. Anger, courage, love, heat, and passion, according to the chart.
Perhaps it might be easier to read them the referenced chart’s correspondences.
That infographic would have been much better as a regular table, instead of this circular thing. It seems to me as if it is not intended to be actually used, only to look nice.
It’s a dreadful graphic. No information leaps out at the viewer, you have to hunt through two tables for the meanings of the letters and numbers. It takes an effort to find the letter for any given block, or to find the block for any given letter, in radii far from where the letters appear. It’s difficult to tell apart yellow and gold, or grey and silver: the key only serves to highlight how indistinguishable the colours are.
And since this graphic does not work, I cannot see it as beautiful. It is an ugly sacrifice of function to superficial prettiness.
Agreed. I wish they’d stick to calling hard-to-read graphics like this ‘visualizations’ - the word ‘infographics’ implies a graphic designed to efficiently display information.
The worst part is it wouldn’t be hard to improve the graphic. They could drop the annoying 84-item list and just directly write the emotions in the 84 slots around the circle instead of using numbers. Enlarge the circle and blow up the font size a bit—then they can put the A to J list of cultures into the empty middle of the circle so you don’t have to keep looking off the side to cross-reference it. That’d help, even if it wouldn’t fix it.
Edit—I see that when they used that infographic as their book’s cover, they gave up on the idea of making it a real infographic and just made it into a pretty flower!
So looking at numbers 31, 2, and 46, a friendly AI should be orange and blue...
I don’t associate red with smoothness, or silk with any of those emotional qualities.
Are you sure about that? Your emotions don’t ask you for permission before they make associations.
I can’t tell if this is wordplay based on the ambiguity in the quoted sentence (in which case, I like the joke :) ) or if you’re serious. If you’re serious, then yes, I’m sure: while emotions may not ask me for permission, I’m aware of what they’re associated with.
Strange. I have associations for green (foliage, leafs), yellow (sun), blue(water). Red requires conscious effort to select domain for association to pop up.
This is me saying that yes, you do have these associations with red, they are hardwired into your brain.
To be clear, we’re talking about associating red with smoothness? Why do you think this is hardwired into my brain?
And silk? You think silk is hardwired into people’s brains? What about humans who lived before the cultivation of silkworms was discovered?
My response referred to the emotional qualities, not to smoothness. The list I read in your comment was:
This is approximately in line with what studies on human responses depending on colour stimulus have found.
End of my contribution to this conversation.
Sounds like it could just as well be cultural. “Red”, in particular, could easily get associated with “heat” in various cultures because of fire, but that doesn’t mean it’s hard-wired to the brain.
You don’t need to respond, but you seem to have misread the original comment. The David Gemmell quote compared red to silk. I said I didn’t associate that list of emotional qualities with silk. I also said that I did not associate red with smoothness. Those were the associations being discussed.
Or blood?
-- Philip Gourevitch
-- Thucydides, Greek Historian, ca. 5th century BCE (Book IV, 108)
I like Thucydides for the way he tries to explain history in terms of real-politik, people, their drives and especially without including the gods in an explanation, somewhat similar to Hippocrates.
Interestingly, a modern version of this appeared in Neal Stephenson’s Anathem:
where it’s called Diax’s Rake.
Anathem is a great book, I’d like to add, and quite well aligned with many of the LW themes.
No love for Wizard’s First Rule?
I have to disagree on two counts. First, Diax’s Rake is explicitly a reference to Thucydides, (or, more spoilerifically, Thucydides’ “referenced” Diax for some value of reference), so it’s not really interesting that it appeared in Anathem.
Secondly, Anathem isn’t actually aligned with LW themes at all. It might appear that way at the beginning, but Stephenson undoes all of it with the spoiler twist at the end.
Hmm.. i can’t remember the specific reference to Thucydides from the book, and I don’t have it handy right now… did the book mention him? I just found the parallel quite interesting.
Regarding the other point, I meant ‘aligned with LW themes’, that is discusses many of the same things that are discussed here—and in many cases seems to agree. Not always—but apart from the parallel universe mixups which are a bit… suspect, I got the idea that NS has been lookin at LW (well, OB) and similar sources.
Jonathan Swift (also attributed to Pope)
Abraham Lincoln
=> A man who is not ashamed of being wrong will be thought of much by Abraham Lincoln?
As an aside, one should be ashamed of being wrong when there was sufficient time and information to be right, and still admit being wrong. It should lead to better care taken next time.
You should resolve to take better care next time, and evaluate where your mistake was, but I don’t see how shame helps anything.
Getting singed helps avoid hot objects. Shame helps avoid stupidity.
Shame leads to a variant on guessing the teacher’s password—an effort to not piss people off, without asking them what might be problematic. After all, you’re supposed to know better than to make that mistake.
“The shameful does not learn”—Talmud
I agree with your point, but I do not think the emotion should be dispensed with altogether, if that’s really possible. I think you can separate social embarrassment, which in this context is counterproductive, from deep very personal private embarrassment of having been stupid.
For an even somewhat rational person, pain is far stronger than necessary as a warning sign. As someone generally concerned with my own body’s welfare, the mental equivalent of popping up a politely worded dialog box would be sufficient. I find that shame is likewise overkill for solving this problem.
Personally, I don’t mind pain being a strong enough warning that it’s hard to ignore. I can see the need for that.
I think the problem with pain is that it’s like those stupid car alarms. You know the ones that should be programmed to turn off after five minutes because any good they might do will have been done by then if it’s to be done at all, but they actually keep going all night?
That’s what pain should have: a way of saying, after some appropriate enforced time delay, fine, I’ve got the message, I’m doing everything I can about the problem, you can stop now.
I can’t. I’ve had problems with pains that are demonstrably unrelated to any threat to bodily integrity and for which there is no known technique that removes it. If pain were limited to real threats, I’d agree, but it’s not.
So it’s not even an issue of “yeah, I get the message, you can stop reminding me”; often times, there is no message to be given, just suffering.
Have you never been tempted to push ahead doing something you want, and ignored a minor pain? We’d just ignore pop-up boxes if we were in the middle of something we considered important.
Absolutely. But this is not a way in which pain is superior to a pop-up box. If the pop-up box that replaced intense pain had alarms and flashing lights attached, and the one for more minor pains did not, I would pay attention to the alarms and flashing lights.
Speaking in terms of real pop-up boxes, you might be surprised at how easy it is for people to ignore the content of even the most blaring, attention-grabbing error messages.
A typical computer user’s reaction to a pop-up box is to immediately click whatever they think will make it go away, because a pop-up box is not a message to be understood but a distraction from what they’re actually trying to accomplish. A more obnoxious pop-up box just increases the user’s agitation to get rid of it.
As rationalists, we try hard to avoid falling into traps like these (I’m not sure if there’s a name for the fallacy of ignoring information because it’s annoying, but it’s not exactly a high-utility strategy), but part of the way we should do that is to design systems that encourage good habits automatically.
I like Firefox’s approach; when it wants you to choose between Yes or No on an important question (“Really install this unsigned plugin?”), it actually disables the buttons on the pop-up for the first 3 seconds. You see the pop-up box, your well-honed killer instinct kicks in and attempts to destroy it by mindlessly clicking on Yes so you can get back to work already… but that doesn’t work, you’re surprised, and that jolt out of complacency inspires you to actually read the message.
I suspect a “Hey, have you noticed that something has penetrated the skin of your left foot?” warning might benefit from having the same mechanism.
I appear to be a mutant: I always read pop-up boxes.
By all means, please adapt my analogy to something that you would actually pay attention to.
That doesn’t seem a lot better—I know what it wants me to do with an unsigned plugin prompt—review the site, verify the download hash, review the plugin source code, look for reviews of the plugin, author, site...
What I actually do is wait 5 seconds, then click “yes do it” as soon as possible. So the utility is … well intentioned, but still ineffective.
It’s true that it’s less than perfectly effective, but it serves some purpose: I almost always install plugins from the Mozilla plugin site, where a rating is immediately available, and where a virused plugin would probably get removed very quickly. Under those conditions, I know that I’m fairly safe just installing it anyways.
However, a malicious site could attempt to infect my browser by installing a plugin, which is where the timer comes in handy. It could even attempt to hide the plugin dialog with lots of other useless dialogs (“Really submit this comment?” Yes. “Really really submit this comment?” Yes. “Really really REALLY submit this comment?” Yes. “Install this plugin?” Yes. Oh, hold on, wait! Crap.)
More generally, timed dialogs are helpful because they increase the chance that you notice what it is you’re confirming. If you know you’re doing something risky and want to do it anyways, so be it… but at least you know what it is you’re accepting, and are given a greater opportunity to back out if you are surprised by the level of risk.
The about:config option “security.dialog_enable_delay” allows one to reduce the delay to 0.
It seems pretty clear to me that this was not true in our ancestral environment. It may be the case in our present artificially benign environment however.
That is precisely what I mean; but also note that there are circumstances in the ancestral environment in which pain is entirely useless, such as when one has been mortally wounded. So even in the worst case we can do better than pain, and in the current case I suspect we can do much, much better.
Evolution has the problem of path-dependence, though. Once the “don’t do that” / “pay attention to that” mechanism builds up slowly over many generations, it cannot refactor in such a way that it surgically cuts out the internal feeling of pain in precisely those circumstances where, “hey, might as well give up”.
It’s hard to see what reproductive benefit there would be to reduced suffering when dying either so there is unlikely to be any evolutionary pressure in that direction.
Thanks for the explanation. I like the argument, but still willing to play the devil’s advocate: popup box is nice when you’re paying attention, but it does not produce learning. Imagine working on a plant where a wrong move can cause a serious injury. Popup boxes will not produce the muscle memory needed to navigate.
I would argue this strongly about pain, but I am not sure how well the analogy transfers to mental errors.
Well, it would have to be a popup box that interrupts whatever you’re currently thinking about. It grabs focus, to continue the analogy.
Does pain produce muscle memory? I haven’t heard that before.
Maybe a popup would be better, but until we can hack our brains the question remains whether you’re better off with shame as a learning mechanism.
Does pain produce learning? Probably yes.
http://www.nytimes.com/1991/08/27/news/pain-and-learning-may-be-close-cousins-in-chain-of-evolution.html?pagewanted=all
To continue the computer analogy, it would have to be a popup box that steals focus, so that you can’t do anything else until you acknowledge it’s there.
Does pain aid in creating muscle memory? I hadn’t heard that before.
Should it pop up again if pain increases from mild to strong? Should it pop up periodically when pain is extreme?
I’d rather prefer to consciously disable drive to remove pain source, but stay informed of kind and intensity of pain.
Edit: AFAIK insects use that kind of pain processing, they react on pain, but they don’t get overwhelmed by it.
Sure. The analogy to computers is not a perfect one, because brains don’t function like modern computer operating systems. My objection to pain is not that it is uninformative, it’s that it’s overwhelmingly unpleasant even when we do not wish it to be.
Yes, it can inflict more harm by forcing you into suboptimal decisions. Shame can be alike too. So, I vote for insect-like shame and pain processing, I’ve mentioned in grandparent.
There are cases of people with no sense of pain. Here’s an article about it:
http://findarticles.com/p/articles/mi_m1200/is_25_170/ai_n26713199/
...and a link to the primary it references, if you have a subscription to Nature:
http://www.nature.com/nature/journal/v444/n7121/full/nature05413.html
A relevant quote from the review:
“The team’s first research subject, a 10-year-old boy, was well known in his community for street performances in which he placed knives through his arms and walked on hot coals. Despite tissue damage, he apparently felt no discomfort.”
Other mutants for the same gene were similar—they generally enjoyed showing off their lack of pain by deliberately injuring themselves. One killed himself by jumping off a roof, I believe just after the above article was published (sorry, I’m not really sure where I read that). Living without pain is generally an unhealthy idea. Without visceral negative feedback, humans don’t place value on bodily integrity and tend to self-terminate, which is probably why those mutants are incredibly rare.
Perhaps once aversion reactions have been formed, knocking out pain receptors would be a good thing. I’d certainly be curious to see if people still avoid injuring themselves if they lose the ability to feel pain AFTER they’ve experienced it all their lives.
They don’t. People who develop pain asymbolia from various insults to the brain are not strongly motivated to avoid pain. (And these are people who have functioning nociception, unlike the family described in the article you link.)
Yeah, knocking out pain receptors in children seems like a very bad idea. Pain is unnecessarily harsh if you already have a strong interest in preserving your body; then all you really need is to know that you’re doing something bad to yourself, and that’s more than enough reason to stop. But for those who are not yet sufficiently rational, like (most) children, it’s probably not something to be messed with without serious consideration.
It sounds like that kid has some of the worst parents in human history. How do you let your kid get a reputation for self-mutilation and not, y’know, stop him?
Stop your child from getting status for himself and your family? Inconceivable!
Knocking out nociceptors? It’s not what I thought about. Knocking them out reduce available infomation to make decisions. I meant deliberate suppression of goal-shifting and brain resource allocation effects of nociceptors activity (in a sense proposed by Marvin Minsky). It is hard to evaluate effects of this change on child’s pain aversion behavior, but I can hypothesize that this kind of pain control could be ineffective until “central executive” is sufficiently developed.
Edit: In the case of mild pain this way of dealing with it can be exercised on our current brainware.
I disagree—perverse incentives. If I was ashamed every time I was wrong, I might be more careful. Or, I might stop admitting I was wrong. I make an effort to congratulate myself for admitting I’m wrong, for this reason.
Now that I think about it, it would be even more helpful to find out which of these I would do.
For most people, it mostly makes them avoid admitting their stupidity.
You might find Peter McWilliams’s Do It: Let’s Get Off Our Buts interesting and useful, at least in spots. Several section headings are:
Actually, “No valid inference.”
A man who has admitted in the wrong may or may not be thought much of by Abraham Lincoln is the only thing we can infer, and it’s too indefinite to count as a useful deductive truth.
-L. A. Rollins, Lucifer’s Lexicon
He who knows not and knows not that he knows not is a fool—shun him
He who knows not, and knows that he knows not is a child—teach him
He who knows, and knows not that he knows is asleep—wake him
He who knows, and knows that he knows is wise—follow him
Persian proverb
I’ve always enjoyed three-fourths of this quote, but the first line still bothers me. If one knows not that they know not, should we not guide them so they at least know that they know not? Then we can teach them, possibly awake them, and finally have more wise people for others to follow.
Good point. I guess the extra effort and subtlety required to guide the obliviously ignorant makes it tempting to just walk away.
Also, the last line presupposes that the knowingly knowledgeable will be wise, which may not be the case if wisdom is taken to have a moral dimension. They could be rational but evil. (Gah—analysing proverbs …)
--Analects 7.8
If you were in China and were confronted with the top 1 billion, would it be worthwhile to try to teach the ignorant who are ignorant even of being ignorant?
George Box
Does this count as a repeat?
I am shamed by my failure. I will master the Search, so that the Search can not master me.
-- The Dead Zone, 1983
-- Mass Effect 2
I am stunned by the relatively high mod-points of this exchange.
I agree that the quotes are moderately funny. (Albeit the M.S. quote was much more funny in the specific context within the game, but even there it was his white-wash response to an action that earned Shepard renegade points.)
Still, I can’t see, how all this is related to the “art of human rationality”...
“Killing is wrong, no matter what,” is a very powerful and standard meme for heroes.
It is counter intuitive for someone who “loves people” to kill someone. It requires a less-biased assessment of expected utility than is typically performed. That’s why I enjoyed the original quote; in the context of the movie, it made sense in the way of typical human failings for him to say no, and his body language and tone highly suggested he would do so right until the end.
It’s also convenient for writers. Imagine what would happen to the Batman comic book series if someone finally got around to putting a bullet through The Joker’s brain. (In a Discworld story, it’s suggested that “heroes” and Dark Lords have a bit of an understanding: Dark Lords keep on making all of the mistakes on the Evil Overlord list, and heroes keep on letting Dark Lords escape after the day has been saved.)
And nauseating. Don’t forget nauseating.
In my reading, the assessment was funny exactly because it was emotional and therefore biased. That’s what use of “son of a bitch” suggested as well.
Emotion drives value and purpose; logic is compatible with emotion; Spock is a bad example for rationalists.
It’s related because it portrays someone disregarding the omission/commission distinction.
Among consequentialists (who seem to be quite common on LW), how something happens is not directly relevant to its moral value. Untutored intuition, in contrast, seems to say that killing is worse than letting die.
Therefore, if consequentialism is right about this, then many humans’ moral intuitions are wrong in a predictable way. Thus they are biased. Thus they are irrational. Thus this is related to the art of human rationality.
The situations in which a habitual killer-of-humans will cause death come up more often than the situations in which a habitual letter-of-humans-die will cause death. If you’re a consequentialist who negatively values death, it seems to follow that a habit of killing humans is worse than a habit of letting them die.
Thank you for pointing out this argument.
I’m not sure I agree that “let-die” situations arise less often than “kill” situations. It seems that every moment you have disposable income (i.e. more than you and yours need to survive) involves a choice between saving someone’s life and not saving anyone’s life.
(nods) That’s fair.
And yes, if it makes sense to classify what I’m doing right now as choosing not to avert avoidable deaths, it follows that my lifestyle is morally worse (from a consequentialist perspective) than that of a poor murderer.
An interesting related fact: the British considered assassinating Hitler in Operation Foxley in ’44. It was kaiboshed, mostly because he was seen as a really terrible strategist.
-- H. G. Wells
--Samuel Johnson, Rambler #175, November 19, 1751
-- Robert C. W. Ettinger, The Prospect of Immortality
On the other hand...
-- Friedrich Schiller
Seems to me that Schiller’s been better vindicated so far.
Both of them are inaccurate and useless.
I can not speak for usefulness but “Human stupidity is formidable but not invincible” doesn’t seem too off the mark. I suppose it depends on which territory we are considering the stupidity to be formidable in defense of.
From Fine Structure, by Sam Hughes.
I think that while this quotation is true if we take “SCIENCE!” to mean intelligent optimization pressure, it is far more likely to create affective death spirals around anything that calls itself science than get people to try to fix problems.
Fine Structure is a great piece of fiction, I started following it around the middle and stayed till the end… but I feel it could’ve been so much more. Sam started out with some awesome premises, but gradually wrote himself into a corner as the powers in play kept escalating. All the while, his writing skill was noticeably growing stronger and more confident, which is why the series stays readable up to the finale. IMO, the high point is the short story Failure Mode from the middle of the series. It reads like a description of what would happen to Eliezer’s Harry Potter if one of his experiments went awry.
-- TV Tropes, A Nuclear Error
One day four boys approached Hodja and gave him a bag of walnuts.
”Hodja, we can’t divide these walnuts among us evenly. So would you help us, please?”
Hodja asked, “Do you want God’s way of distribution or mortal’s way?”
″God’s way,” the children answered.
Hodja opened the bag and gave two handfuls of walnuts to one child, one handful to the other, only two walnuts to the third child and none to the fourth.
”What kind of distribution is this?” the children asked, baffled.
”Well, this is God’s way,” he answered. “He gives some people a lot, some people a little and nothing to others. If you had asked for mortal’s way I would have given the same amount to everybody.”
-- Marine General James Mattis, to Iraqi leaders in every area his men served in, after sending his tanks and artillery home following the invasion of Iraq
Why is this a rationality quote?
My guess is that it’s intended as an example of the Prisoner’s Dilemma. Then again, nearly every morality-related situation is...
Tit-for-tat in the iterated Prisoner’s Dilemma, specifically. It also poses the question: how good a strategy is it to combine humor with mortal threats?
It looked to me like a perfect, textbook example of a Schelling-type Threat; but maybe that’s just because I’m reading The Strategy of Conflict.
Well spotted; I missed that.
-- “Olyander”
Charlie Munger
-- George Orwell
I think it would be more clear if it included the previous sentence:
Or perhaps just substituting “[Scientific education]” for “It”.
-- Often attributed to Pablo Picasso, but I can’t find a reliable source.
I’m quite curious to hear what LW thinks of this one.
I don’t think it would be controversial to say that “useless” is way too strong a term for describing shortcomings of computers.
I can’t find the quote’s context either, but consider this—why would someone ask Picasso about computers?
If the quote is correct, I wouldn’t be surprised if it was in response to something like “Do you believe that computers can [be made to] create art [on their own]?”. In which case the quote becomes much less categoric.
Well, in that case, it still sounds to me like View 2: “Only humans will be able discriminate against art upon learning a computer / monkey / child / prankster made it, pendejo!”
But that discussion was about science. Nonhuman science is the same thing as human science, so discriminating is irrational. Nonhuman art is (will) not be (necessarily) the same as human art, and it is quite possible that it will not be at all enjoyable by humans.
But it will (likely) be the case that people’s opinions about particular artwork will dive sharply downward upon learning it was mostly the work of a computer, even as the pre-revelation opinion is higher than average.
And the experimental evidence for this is what?
More substantially—it is perfectly possible to have a great deal of difference in the emphasis placed on various subfields in the sciences. If we’d gone directly from vacuum tubes to Drexler/Merkle nanotechnology, do you think semiconductor device physics would have been studied as deeply as it has been?
He was no computer scientist, but he presumably knew a lot about specific non-computer things. The more important those things are, the less important computers are likely to be (relative to computers, that is). So I don’t think Picasso had to have known much about computers to denigrate them rationally.
But I am assuming that Picasso knew some pretty important non-computer things. And it does help to remember when he stopped being able to learn more about computers.
Particularly if you consider the term literally rather than as a way to say “not particularly important”.
What question was the Avatar movie an answer to?
How about the last flash game you played?
This conversation?
It may be worth noting, if the quotation’s attribution is accurate, that Picasso died in 1973.
Okay—assuming that the quote’s claim is accurate for its time period, that still leaves the fact that streamlining the process of getting accurate answers leaves more time for figuring out good questions or doing other valuable things.
I interpreted the quote as being more of a point about answers than about computers. But YMMV.
I saw it as saying “garbage in, garbage out.”
Peoples’ values vary. People who don’t value answers wouldn’t be very popular if they turned up here, but they do exist, and I don’t see much point in passing judgment on them.
“It’s not that I judge them, I just, just...”
“Don’t see any reason for them to exist?”
“Exactly.”
HP:MoR.
To be fair, I expect that most of them don’t see much reason for people like us to exist, either.
A practical arrangement all round.
Hey, but I like you! ;-)
Seriously, in my line of work, answers only take you a step or two forward. The lasting value is in the questions, which can be reused over and over again to produce change as a side effect of the answering, while the actual answers can be consciously discarded once the process of answering is complete.
--‘Mentat Zensufi admonition’, Chapterhouse Dune; Frank Herbert
I think, the quote is useless and in rooted in a fundamental misunderstanding of IT.
E.g. experimental mathematics would not exist without computers. Computer simulation is fantastic way to empirically produce and check hypotheses.
“The conformist benefits from the pooled knowledge of it’s companions.”
-E.O. Wilson in Sociobiology the new synthesis
“The only thing worse than generalizing from one example is generalizing from no examples at all.”
— Bob Scheifler and Jim Gettys, 1984
C.S. Lewis
-- Billy Joel, “Keeping the Faith”
Robert Ettinger, The Prospect of Immortality
—H.P. Lovecraft, “The Silver Key”
The following dialogue is excerpted from Chapter 16 of “Becoming a Technical Leader”, by Jerry Weinberg (warmly recommended). It has been edited out of the post Kaj Sotala and I are working on, but I think it’s worth having it somewhere around.
Some background: Jerry is interviewing Edrie, a (presumably somewhat fictionalized) senior woman engineer at a client of his. Edrie has just butted horns with a male peer over his conception of what makes a good technical leader: “we have nothing against competent unmarried women, like you, but...”. But upon finding herself alone with Jerry, Edrie acknowledges that her colleague does have a point, taking Jerry somewhat by surprise.
J: “Why do you think fathers are such good leadership material?” [...]
E: “That’s easy. It’s a matter of power conversion”.
J: “Power conversion?”
E:”Yes, you know. The ability to convert one form of power into another that you value more. Like converting water power from a stream into electric light for your house.”
J: “What does that have to do with married men being better leaders?”
“Well, in this country, married men have an advantage over single men. They have power over a woman, which they convert into services that support them in their work. Single men have much more work to do just taking care of themselves, so they are at a disadvantage.” [...]
J: “A lot of men would see it differently. They would say that the married woman is converting her sexual power over the man into money power—a guaranteed lifetime of support.” [...]
E:”There’s no contradiction in both people using power at the same time and converting it into something they both want more.”
Is that high endorsement or medium endorsement?
High—pretty much everything Jerry Weinberg has ever published is golden. It comes in roughly two main categories—things you ought to read if you want to pursue a worthwhile career in software, and things you ought to read if you want to be a manager or leader. There is some overlap, but this book falls squarely in the second.
-- Peter Marshall
(Quoted in Robert Ettinger’s “The Prospect of Immortality.” Ettinger says that Gene Lund credited it to Peter Marshall. I haven’t been able to find a more direct source that confirms this.)
Edit: How do you force a line return so that a poem will read correctly? Thanks, fixed now.
End the line with two (or more) spaces.
Thanks very much.
Excellent! My kids get this version:
Twinkle Twinkle little star,
We all know just what you are,
You’re a sun that’s far away,
Far too faint to see by day
Twinkle Twinkle little star,
We all know just what you are
Put a double space at the end of the line.
“They never do tests. Not many real deeds either. Oh, conversation with your grandmother’s shade in a darkened room, the odd love potion or two, but comes a doubter, why, then it’s the wrong day, the planets are not in line, the entrails are not favorable, we don’t do tests!” -Tyrian, Dragonslayer
--Théophile Gautier, Preface (1834) to Gautier, Mademoiselle de Maupin (1835), http://berlin.wolf.ox.ac.uk/lists/quotations/quotations_by_ib.html
(Assassin’s Creed II, Ezio Auditore’s speech)
Okay, I’ll choose my own way.
What else does Auditore say I should do?
This problem of “I want to teach you how to do something by yourself—to start with, do what I say because I say so” seems to apply to teaching children / non-rational people to be more rational, teaching unhappy / non-biologically-broken depressed people (i.e. broken world view, not causatively abnormal brain chemistry) to be happier, as well as dependent people to be more independent.
Taking better control of yourself, starting by giving more control to someone else to get you past the initial obstable that you can’t get past and onto the proverbial ladder that you can then continue cimbing yourself.
It looks like a more legitimate problem than your dismissal gives it credit for.
Why do you care? You should not follow it anyways. ;)
-L. A. Rollins, Lucifer’s Lexicon
From Give Well:
I don’t think the problem is limited to the world of giving.
Suzzette Haden Elgin
-- Ambrose Bierce, The Devil’s Dictionary
I have a few questions.
1) What’s “Bayescraft”? I don’t recall seeing this word elsewhere. I haven’t seen a definition on LW wiki either.
2) Why do some people capitalize some words here? Like “Traditional Rationality” and whatnot.
-- Hunter S. Thompson
-- Al Ghazzali, “The Alchemy of Happiness”
If he were speaking to us through the chronophone, we might hear him continue:
How wise it was of Al-Ghazzali to recognize, a thousand years ago, that our bodily desires exist only to induce us, in a rather stupid way, to satisfy our needs.
-- Dmitry Galkovsky (in my crude translation)
“I too, breaking out of old ways, had discovered solitude and melancholy which is at the basis of religion. Religion turns the melancholy into uplifting fear and hope. But I had rejected the ways and comforts of religion. I couldn’t turn to them again, just like that. That melancholy about the world remained something I had to put up with on my own. At some times it was sharp; at some times it wasn’t there.”
Salim, the narrator of V.S. Naipauls A Bend in the River
An irrational animal whose irrationality is best demonstrated by his irrational believed.