Does anyone have a reputable source for Feynman’s 137? google makes it look very concentrated in this group, probably the result of a single confabulation.
Sykes and Gleick’s biographies both give 12x. Sykes quotes Feynman’s sister remembering sneaking into the records as a child. This seems important to me: Feynman didn’t just fabricate the 12x.
Eliezer’s password is [correct answer deleted, congratulations Douglas—EY].
the dominant consensus in modern decision theory is that one should two-box...there’s a common attitude that “Verbal arguments for one-boxing are easy to come by, what’s hard is developing a good decision theory that one-boxes”
Those are contrary positions, right?
Robin Hason:
Punishment is ordinary, but Newcomb’s problem is simple! You can’t have both.
The advantage of an ordinary situation like punishment is that game theorists can’t deny the fact on the ground that governments exist, but they can claim it’s because we’re all irrational, which doesn’t leave many directions to go in.
Nick Tarleton,
Yes, it is probably correct that one should devote substantial resources to low probability events, but what are the odds that the universe is not only a simulation, but that the containing world is much bigger; and, if so, does the universe just not count, because it’s so small? The bounded utility function probably reaches the opposite conclusion that only this universe counts, and maybe we should keep our ambitions limited, out of fear of attracting attention.
Luis Enrique, See above about “We Change Our Minds Less Often Than We Think”; my interpretation is that the people are trying to believe that they haven’t made up their minds, but they are wrong. That is, they seem to be implementing the (first) advice you mention. Maybe one can come up with more practical advice, but these are very difficult problems to fix, even if you understand the errors. On the other hand, the main part of the post is about a successful intervention.
Science was weaker in these days
Could you elaborate on this? What do you mean by Science? (reasoning? knowledge?)
The thing whose weakness seems relevant to me is a cultural tradition of doubting religion. Also, prerequisites which I have trouble articulating because they are so deeply buried: perhaps a changing notion of benevolence.
You probably won’t go far wrong if you assume I agree with you on the points I don’t respond to. I probably shouldn’t have talked about them in the first place.
overcoming heuristics:
If we know a bias is caused by a heuristic, then we should use that heuristic less. But articulating a meta-heuristic about when to use it is very different implementing that meta-heuristic. Human minds aren’t eurisko that can dial up the strength on heuristics. Even if we implement a heuristic, as in Kahneman’s planning anecdote, and get more accurate information, we may simply ignore it.
The basic problem is system 1 vs 2, when to analyze a problem and when to trust unconscious systems. Perhaps we have control over the conscious analysis (but still, it has unconscious inputs). But even starting the process, turning on the conscious process, must be done from an unconscious start.
Tom Myers,
Systematic but unexplained: sure, most errors are probably due to heuristics, but I’m not sure that’s a useful statement. A number of posts here have been so specific, they don’t seem like useful starting points for searching for heuristics.
Cost:
Most people don’t seem to have sufficiently clear goals to make sense of whether something benefits or costs them, let alone balancing the two.
People live normal lives by not thinking too much about things, so it shouldn’t be so surprising that they don’t think in psych experiments in which it is often clear that analysis will help. But if one is interested in producing answers to questions that don’t come up in normal life (eg, how much is medicine worth), avoiding everyday heuristics is probably worth the cost. Heuristics may well be worth overcoming in everyday life, as well, but I don’t think any experiments I’ve heard about shed any light on this.
Torture:
Your proposals are too detailed. I don’t imagine an opportunity to experiment enough to figure out how to structure torture, and if I do get an opportunity to experiment on government structure, torture is not going to be high on my list of variables. A government is an extremely expensive experimental apparatus. At least I can imagine how to experiment with corporal punishment, but I don’t really have much of an idea of how one could go about comparing the efficacy of different interrogation methods or the general investigative qualities of, say, American and Japanese police.
I’m not inclined to find out what you mean by Saddam’s people-shredders, but I imagine that one effect was a deterrent to crime, especially crime that would get Saddam’s attention. Torture, especially creative torture with vivid imagery, may well exploit salience(?) biases to be a more effective deterrent (aside from the rationally greater desire to avoid torture+death than to avoid death). The role of whim on one’s fate may also have (irrationally) increased the deterrent effect. The vague beliefs people hold about prison rape may play a very similar role in the US system. We do have arbitrary torture in our criminal justice system already.
Winston survives.
Tom Myers,
I think the convention on this blog, among the small set of people who have such a precise definition, is that not every heuristic is a bias, that only heuristics whose errors are worth overcoming should be called biases. I don’t like this usage. For one thing, it’s really hard to know the cost of overcoming particular biases. It’s easy to look at an isolated experiment and say: here’s a procedure that would have been better, but that doesn’t cover cost of actually changing your behavior to look out for this kind of error.
Also, there are other people on this blog that use bias to refer to systematic, but unexplained errors, where it’s hard to call them heuristics. Without a mechanism, it’s probably hard to overcome these, although not always impossible.
While many people have mentioned similar disappointments, no one has echoed “I’ll get that theorem eventually...even though my first try failed!” That was what seemed like a really bad sign when I read the essay before the comments. But I think we’re really bad at communicating feelings, so I don’t know how the feelings relate, how strong they were, and especially, how the commenters see the parallels with their reactions.
The ancient Greeks themselves played around with the rules. Archimedes used a “marked straightedge” to trisect an angle.
The first hit on google for trisect an angle is about ways to do it, not discussions of impossibility.
I think people should be more careful about the word “science.” Here are some meanings I see attached to it:
knowledge of nature
naturalism
“the scientific method”
institutions practicing the scientific method
rules of a particular institution
the output of institutions
I feel compelled to add that what I mean by “the scientific method” is that observation should drive belief and that we can put effort into obtaining useful observations (experiments, stamp collecting). Also, it may be useful to distinguish between institutional rules intended to protect the institution from cheaters and rules intended to protect people from their own biases.
Eliezer Yudkowsky, The word “normative” has stood in the way of my understanding what you mean, at least the first few times I saw you use it, before I pegged you as getting it from the heuristics and biases people. It greatly confused me many times when I first encountered them. It’s jargon, so it shouldn’t be surprising that different fields use it to mean rather different things.
The heuristics and biases people use it to mean “correct,” because social scientists aren’t allowed to use that word. I think there’s a valuable lesson about academics, institutions, or taboos in there, but I’m not sure what it is. As far as I can tell, they are the only people that use it this way.
My dictionary defines normative as “of, relating to, or prescribing a norm or standard.” It’s confusing enough that it carries those two or three meanings, but to make it mean “correct” as well is asking for trouble or in-groups.
(parody, I think) story
The story was for real. The site, I dunno, but it does accept money through paypal.
Your conclusion matches your data, but the data is suspiciously focused on charity. Is scope neglect easier to elicit in such contexts? Other explanations include it being hard to make large numbers relevant, and lack of imagination by researchers.
interpretations which postulate an infinitely sliced spatial manifold which is fundamentally real
Strictly speaking, I suppose that is part of the interpretation, but it’s a pretty mild part of the interpretation of QM, or at least QFT. Many people expect that this to stop being true in a unification with GR, but that’s about physical law, not interpretation.
randomly subject to anthropic constraints, for instance
That might lead us to simulations, quite close to the operating system example.
TGGP, You used an example of moral progress produced by a philosopher: the word consequentialist.
TGGP, What do you mean that you are a consequentialist, if you are so sure ethics is meaningless?
Eliezer tries to derive his morality from stated human values.
In theory, Eliezer’s morality (at least CEV) is insensitive to errors along these lines, but when Eliezer claims “it all adds up to normality,” he’s making a claim that is sensitive to such an error.