Heuristic is not a bad word
An insect tries to escape through the windowpane, tries the same again and again, and does not try the next window which is open and through which it came into the room. A man is able, or at least should be able, to act more intelligently. —George Polya, How To Solve It
Intelligence makes humans capable of many impressive feats. Unlike flies and birds, we don’t bang up against windows multiple times trying to get out of our houses. We can travel to the moon. We have taken over the planet. Why? Because intelligence enables us to solve problems.
All problems start the same way. They start unsolved. Each fact humans have figured out was initially unfigured out by us. Then we did something, which converted the unknown fact into a known fact, changed the state of a problem from unsolved to solved.
I emphasize the unknown starting state of problems to make a point: problem solving, the basis of human achievement, depends on a process of discovery, discovery of new facts, new possibilities, new methods, and new ways of thought.
Heuristic—the art and science of discovery—has been integral for human progress. The word “heuristic” is related to “Eureka!”
Heuristics and biases
Unfortunately, heuristic is a bad word. At least, that’s the impression you might get, seeing it hand-in-hand with “bias” in the psychological literature. In Judgment under Uncertainty: Heuristics and Biases, Tversky and Kahneman acknowledge that “in general, these heuristics are quite useful, but sometimes they lead to severe and systematic errors.” On Overcoming Bias, heuristics seem primarily discussed as resulting in biases.
Bias-reduction is a form of skepticism that is a critical part of rationality. Due to the uncertain nature of the territory of reality, many notions of the territory are wrong. Rational skepticism helps us identify false assumptions, areas where our map will depart from the territory.
While bias-reduction is necessary in the search for rationality, it is not sufficient. It’s a mistake in cartography to have areas of your map that are filled in wrong, but it’s also a mistake to have areas on your map blank that you could have filled in, at least with something approximate. A map with wrong patches will not take you to your destination, but neither will a map with blank patches. Believing things that are false is one error which will prevent us from finding the truth or winning in our endeavors, yet another error in rationality is failing to recognize or believe things that are true, probable, or useful.
How can we draw our maps more accurately in the first place, so that they need less corrections? This is a job for heuristic.
Heuristic as rational creativity
Rationality depends on both bias-reduction and heuristic. Heuristic is the creative faculty, while overcoming bias and other skeptical techniques are the critical faculty. As Ben Kovitz proposes on the Heuristic Wiki, “Heuristic is about how to steer your attention so that you find things that meet the criteria of logic.” From the start, heuristic depends on avoiding bias, or else it will be based on false assumptions and spiral off in the wrong direction. The results of even well-calibrated heuristics require critical scrutiny. Yet no matter how good your ability to critique ideas may be—to separate the wheat from the chaff—you will never learn anything if your attention is wasted on ideas that are overwhelmingly chaff; heuristic is about growing better wheat in the first place, making your winnowing efforts more productive.
While granting and emphasizing the fallibility of heuristic and its danger of taking us away from truth, and that most applications of heuristic will be crap, I also want to explore the potential of heuristic to take us towards truth. I want to understand how heuristic works in practice, not just acknowledge the benefits of heuristic in principle. Heuristic enabled Tversky and Kahneman to make new discoveries about bias; it enabled Einstein to formulate General Relativity and arrogantly state his confidence in it regardless of future experiments.
While many of the heuristics currently scrutinized for bias seem like quaint quirks of human psychology to which we condescendingly admit usefulness in some situations, we must recognize that all of human knowledge came from heuristic and started off as a guess. To the extent that we think that humans have solved any problems—albeit approximately or provisionally—we should value heuristic. Perhaps the best heuristics are so far unarticulated or undiscovered.
The varied results of heuristic lead me not to pessimism about heuristic, but rather to optimism about how we might identify the strong heuristics currently in use and develop even stronger ones. In future posts, I intend to delve deeper into what heuristic is, why we need it, and how to practice specific heuristics. I don’t yet know a great deal about heuristic on a conscious level, but I want to figure it out.
- How Much Thought by (12 Apr 2009 4:56 UTC; 49 points)
- 's comment on [SEQ RERUN] Where Recursive Justification Hits Bottom by (27 Jun 2012 5:09 UTC; 7 points)
- 's comment on The Sin of Underconfidence by (22 Apr 2009 23:02 UTC; 3 points)
- 's comment on Rationality Toughness Tests by (6 Apr 2009 19:50 UTC; 1 point)
Shall I take the lack of comments to mean that:
a) Everyone agrees with my points and thinks that they are obvious or trivial
b) Nobody is interested in this topic
c) TL:DR
d) This post is very general and people are waiting for me to get more specific
e) Nobody knows what heuristic is, so they don’t read the post
f) ???
Mostly D. A little A, but mostly the first, less negative half of A.
I originally skimmed the first two paragraphs, failed to see stuff I expected to learn from, and moved on. Largely because the content looked fairly general and familiar. Don’t take offense, but it looked like a bland opinion article for a position I already agreed with. I just scanned back through more carefully since I saw your comment asking for responses, and it still looks like that to me.
Try adding personal anecdotes of occasions that point to principles that surprised you and so might surprise us, links to the research literature, or specific practical suggestions for forming better beliefs. Or, if you think posts are ignoring this point, quote from specific posts or comments that you think get this wrong; debates add interest.
Another thing you might try is asking for comments that help with some specific question that’s bothering you. My post on how not to get caught by consistency pressures got no comments except “nice”, basically, until I added a comment that specifically asked for the kinds of comments I was looking for. And then it got fifty-some comments; telling people what I needed help with may have given people an entry point for useful conversation.
The article looks as if you are arguing, but at the same time the arguments seem so obvious that you can only argue against an invisible strawman.
I’m curious about why you think I’m setting up a straw man. The heuristics and biases literature in psychology seems to focus on the costs of heuristic, but not on its potential. I illustrated this point with the quote from Tversky and Kahneman: “in general, these heuristics are quite useful, but sometimes they lead to severe and systematic errors.” When I searched Overcoming Bias about heuristic, which I linked to in my post, it is mainly discussed the same way it is discussed in the psychological literature. Perhaps computer science, AI research, and mathematics take a more positive view of heuristic than psychology does.
I didn’t have any “aha” moments reading this—I felt like I already appreciated that heuristics are indispensible. I wouldn’t use terms as strong as “obvious or trivial”, but “known to me” would cover it.
Mostly A, and a little of D. The term heuristic has positive connotations to me, since it brings to mind Polya and related ideas more than it does cognitive bias. I know that many cognitive biases are a result of heuristics being applied inappropriately, but I think of that more in terms of our abusing heuristics (which are still great) than there being some inherent problem or negative aspect of the heuristics themselves.
Very general. It’s a hard problem to make much headway on such a broad topic—starting with heuristics for a narrow range of problem solving might give good direction for how to think about the wider problem.
Got that one from proving things in combinatorics. :D
Incidentally, we will see what comes of Tricki, the mathematical problem-solving technique wiki—it goes fully live in a week or so.
I read it. It’s a good post; I just didn’t have anything specific to say in reaction.
Similarly inductive bias is a necessary thing for a machine learning algorithm.
Yes, this would be an excellent example of heuristic.
It seems to me that every heuristic presupposes some form of inductive bias in the form of simplifying assumptions that are arrived at via induction on past experience. A good heuristic makes assumptions that simplify (or optimize) the task it is designed for, with at most minor inaccuracies for the vast majority of common instances of the task (until a black swan arrives, at least). Determining which assumptions to use, in turn, involves relying on some more general form(s) of inductive bias, like Occam’s Razor or Minimum Description Length.
Thanks for the feedback, everyone. Since the psychological literature on heuristic focuses on its costs but not its potential (a focus seemingly shared by Overcoming Bias), I thought that I would set the stage by explaining why we should care about heuristic as anything more than something that leads us astray.
It seems that I’m preaching to the choir about the benefits of heuristic, which I am glad about. Since we seem to be on the same page, I’ll be ready to get more specific in future posts.