1) You are assuming that your mind did or did not do certain things in those moments when it was quietly answering the question. In particular, you make assumptions like ”....your answer probably didn’t take any of the above complications into account. It’s as if your brain, while generating an answer, never even considered them …..”. Why are you so sure that it probably didn’t take any of the above considerations into account?
To illustrate how wrong this might be, consider that when a cognitive psychologist gives someone a visual priming task, with (for example) masked cues, the subject reports that she did not take ANY account of the masked cues. And yet, she shows clear evidence that she did very much take the cue into account! The proof is right there in the reaction times (which depend on what was in the cue).
So if someone can be that wrong in their self-report assessment of what factors they are taking into account in a situation as siimple as masked priming, what is the chance that a person in one of the scenarios you describe above is also doing all kinds of assessments that actually happen below the reporting threshold? At the very least this seems likely. But even if you don’t accept that it is likely, you still have to give reasons why we should believe that it is not happening.
So, when Kahneman cites substitutions, his evidence clearly distinguishes substitutions from complex assessments that may be interpreted as substitutions, or which are correlated with substitutions? I don’t buy that.
My second objection has to do with the oversimplification of the analysis:
2) You seem to be framing a lot of scenarios as if they were all instances of the same type of problem. As if the same mechanism was operating in most or all of these circumstances. Your mechanism involves a “target question”, a “substituted question” (assumed to be of dubious validity) and a resulting answer that is assumed to be of sub-optimal quality. While there may be some situations where this frame neatly applies to a situation, I do not believe that it applies to all, and nor do I believe that it helps to try to oversimplify all instances of “bias” so they can be squeezed into this narrow frame.
At the very least, there appear to be situations that do not fit the pattern. Chess skill, for example. The question asked by the chess player is “How do I take the opponent’s King?”. But rather than address this question directly (as I did, in my very first chess game, when I imagined a sequence of moves that culminated in me taking that King, then started executing my planned sequence of moves …..), the expert chess player knows that a different set of questions have to be asked: to wit, “How do I make pleasing, coherent patterns of support and strength on the board?” and “Do I recognize anything about the current pattern as similar or identical to a situation I have seen in the past?”
That particular “substitution” happens to be extremely optimal. It also happens to be not how chess computers work (by and large: let’s not get sidetracked by the finer points of chess programming … the fact is that machines rely on depth to a very large extent, whereas humans rely on pattern). So it makes no sense to talk about substitution as a “problem” in this case. Far from it, substitution seems to be why a tiny little lookahead device (human mind) can give the massive supercomputer a run for its money.
3) Finally, this analysis overall has the feel (like almost all “human bias is a problem” arguments) of fitting the data to the theory. So many people (here on LW, and in the biasses community generally) want to see “Biasses” as a big deal, that they see them everywhere. All the evidence that you see supports the idea that the concept of “biasses and heuristics” is a meaningful one, and that there are many instances of that concept, and that thos instances have certain ramifications.
But, like people who see images of Jesus in jars of Marmite, or evidence of divine intervention whenever someone recovers from an illness, I think that you see evidence of bias (and, in this case, substitution) primarily because you have trained yourself to theorize about the world that way. Not because it is all real.
“It also happens to be not how chess computers work ”
Not to sidetrack this with some unimportant fine point but that is by large a significantly invalid assertion and in so much as this assertion is relevant it needs to be corrected.
It actually happens to be, to significant extent, how good chess programs work. Also, for the best, most effective chess programs (e.g. crafty), you need to download gigabyte sized datasets before they’ll play their best. Even the badly playing naive programs try to maximize the piece dis-balance rather than consider just the taking of the king. It is literally impossible (on today’s hardware) to play chess by considering just taking the king—whenever you are human or supercomputer. Maybe an extremely powerful supercomputer could play chess by considering just the king—but that is an extreme case of playing chess perfectly.
There is no practical solution that does not involve substitution, not even for a well defined, compact problem like chess. Computers, even now, literally cannot win versus humans without relying on the human mind to perform good substitutions. Furthermore, if you run competitions between chess programs, on same hardware, the one with best substitutions and heuristics will win.
I think that actually strenghtens your point. Nothing can play chess yet without heuristics. Something that can play chess without heuristics would also have power to play chess perfectly (it would have to see the demise of the king or the tie right from the first move, to make any first move—that’s the depth of, for some of the tree, 100+ moves) and give the answer to long standing unsolved problem—which side wins with the perfect play? Or is it a tie?
It actually happens to be, to significant extent, how good chess programs work… Nothing can play chess yet without heuristics.
Citation needed. The best chess-playing computer AFAIK is still Deep Blue. Deep Blue evaluated 200 million positions per second. That means it could look about 7 ply ahead per move, exhaustively, with no heuristics. But I agree that this does not weaken Richard’s point.
Even if you can look 7 ply ahead, how do you evaluate which of the possible resulting positions is better with you without heuristics?
Precisely.
The king is still alive 7 moves ahead (usually).
At very least you use the heuristics like e.g. summing the piece values, and penalties for the e.g. two pawns of yours on same column (and bonuses for the pawn that threatens to become a queen, etc).
Also, God only knows how many millions positions per second Garry Kasparov evaluates. Possibly quite many as they can be evaluated in parallel.
I would think that Kasparov’s evaluation, in a sense, substitutes less than Deep Blue’s—Kasparov can meta-strategise to king’s death or to tie from any position—when you are playing for tie you try to make computer exchange pieces, when you are playing for win you try to put computer into situation where you will be able to plan more moves ahead than the computer (computer tends to act as if it desperately hopes that some bad-looking move wins the game).
(To end a block quote, leave a blank line (i.e. two consecutive line breaks) after it.)
Precisely. The king is still alive 7 moves ahead.
In the endgame this is useful, but in the opening/early mid-game, not so much. The king is still alive 7 moves ahead in the overwhelming majority of possible legal sequences of moves.
At very least you use the heuristics like e.g. summing the piece values, and penalties for the e.g. two pawns of yours on same column (and bonuses for the pawn that threatens to become a queen, etc).
I’d bet Deep Blue does that, too.
EDIT: The “usually” you added makes it clear that I had completely missed your point. I’m retracting this.
I’ve participated some in computer contests, not chess related, where you can’t solve the problem exactly. It’s generally the case that you need clever “substitutions” to get anywhere at all.
Most problems that aren’t outright trivial are too hard to solve from first principles.
Definitely so for the real world behaviours. A supposedly ‘rational’ calculation of best move, from first principles (death of king), but without substitutions and hunch based heuristics, done by human, over course of centuries, will not be even remotely powerful enough to match the immediate move that Kasparov would somehow make playing 1-minute blitz (and if you actually played chess real fast, it does become apparent that you just can’t match this play without massively parallel, rather deep evaluation).
I think all aspiring rationalists should learn boardgames like Chess, Go, and perhaps some RTS (like starcraft, though my favourite is springrts. In the starcraft the AI has too much advantage due to human bottlenecking on the output) to appreciate the difficulties. Ideally, try writing an AI that fights against other AIs (in a timed programming contest) to appreciate the issues with advance metastrategies and heuristics.
I perform best on that sort of stuff if I am not tired and I act on feelings. If I am tired or bored and I act on feelings that leads to a quick loss which is also rational because—what the hell am I doing playing game when tired?
edit: where the hell did my wikipedia link disappear? ahh nevermind.
I have a friend who is much better at starcraft than I am; he says that he’s largely better because he’s worked out a lot of things like exactly the most efficient time to start harvesting gas and the resource collection per minute harvesters under optimal conditions, and he uses that information when he plays. It works better than playing based on feelings (by which I mean that he beats me).
If you don’t have way too much time on your hands, though, it’s about as much fun to not bother with all of that.
Also, I notice you cited a Wikipedia page. Naughty, naughty, naughty.
Well yea the start timings are important… they correspond to ‘book’ knowledge of chess openings. I did those a fair lot when i were playing chess more seriously (which was long long ago when i was 10-12). You have to do those before you start actually playing, and in RTS those openings are not so well developed as to read ’em off an old book like you do in chess.
Chess computer btw also uses openings, i think none of the 16 possible first moves leads to loss of king in 10ply (i’d wager a bet that none of possible white’s first moves leads to inevitable loss or victory at all, and certainly not in less than 50ply), and the computer does e2-e4 (or other good first move) purely by book. It doesn’t figure out that e2-e4 is better move than say a2-a3 from the rules alone. There’s a LOT of human thought about chess that chess AI relies on to beat humans.
I used to play balanced annihilation with springrts. I didn’t play it a whole lot but I did program some lua scripts for it and contributed to lobby development. Not playing it much any more but it is very interesting to make scripts for.
1) You are assuming that your mind did or did not do certain things in those moments when it was quietly answering the question.
This is a fair criticism, and you’re right—I can’t say with definite certainty that these things were actually never considered at all. Still, if those things were considered, they don’t seem to be reflected in the final output. If instead of sayng “the thing is, it probably didn’t”, I said “the thing is, it probably didn’t—or if it did, it’s difficult to notice from the provided answer”, would you consider that acceptable?
2) You seem to be framing a lot of scenarios as if they were all instances of the same type of problem...
I think you might be somewhat misinterpreting me here. I didn’t say that substitution is necessarily a problem—I specifically said it probably works pretty well most of the time. Heck, I imagine that if I was building an AI, I would explicitly program something like a substitution heuristic into it, to be used most of the time—because difficult problems are genuinely difficult, both in the sense of being computationally expensive and requiring information that isn’t usually at hand. A system that always tried to compute the exact answer for everything would never get anything done. Much better to usually employ some sort of quick heuristic that tended to at least point in the right direction, and then only spend more effort on the problem if it seemed to be important.
For that matter, you could say that a large part of science consists of a kind of substitution. Does System 1 actually ignore all those complicated considerations when considering its answer? Well, we can’t really answer that directly… but we can substitute the question with “does it seem to be ignoring them in certain kinds of experimental setups”, and then reflect upon what the answers to that question seem to tell us. This is part of the reason why I felt confident in saying that the brain probably never did take all the considerations into account—because simplifying problems to easier ones is such an essential part of actually ever getting anything done that it would seem odd if the brain didn’t do that.
So I agree that there are many cases (including your chess example) where substitution isn’t actually a problem, but rather the optimal course of action. And I agree that there are also many cases of bias where the substitution frame isn’t the best one.
The reason why I nevertheless brought it up was that, if I were building my hypothetical AI, there’s still one thing that I’d do differently than how the human brain seems to do it. A lot of the time, humans seem to be completely unaware of the fact that they are actually making a substitution, and treat the answer they get as the actual answer to the question they were asking. Like I mentioned in my other comment, I think the substitution principle is useful because it gives a useful rule of thumb that we can use to notice when we might be mistaken, and might need to think about the matter a bit more before assigning our intuitive result complete confidence.
You are making many unanalyzed assumptions here.
1) You are assuming that your mind did or did not do certain things in those moments when it was quietly answering the question. In particular, you make assumptions like ”....your answer probably didn’t take any of the above complications into account. It’s as if your brain, while generating an answer, never even considered them …..”. Why are you so sure that it probably didn’t take any of the above considerations into account?
To illustrate how wrong this might be, consider that when a cognitive psychologist gives someone a visual priming task, with (for example) masked cues, the subject reports that she did not take ANY account of the masked cues. And yet, she shows clear evidence that she did very much take the cue into account! The proof is right there in the reaction times (which depend on what was in the cue).
So if someone can be that wrong in their self-report assessment of what factors they are taking into account in a situation as siimple as masked priming, what is the chance that a person in one of the scenarios you describe above is also doing all kinds of assessments that actually happen below the reporting threshold? At the very least this seems likely. But even if you don’t accept that it is likely, you still have to give reasons why we should believe that it is not happening.
So, when Kahneman cites substitutions, his evidence clearly distinguishes substitutions from complex assessments that may be interpreted as substitutions, or which are correlated with substitutions? I don’t buy that.
My second objection has to do with the oversimplification of the analysis:
2) You seem to be framing a lot of scenarios as if they were all instances of the same type of problem. As if the same mechanism was operating in most or all of these circumstances. Your mechanism involves a “target question”, a “substituted question” (assumed to be of dubious validity) and a resulting answer that is assumed to be of sub-optimal quality. While there may be some situations where this frame neatly applies to a situation, I do not believe that it applies to all, and nor do I believe that it helps to try to oversimplify all instances of “bias” so they can be squeezed into this narrow frame.
At the very least, there appear to be situations that do not fit the pattern. Chess skill, for example. The question asked by the chess player is “How do I take the opponent’s King?”. But rather than address this question directly (as I did, in my very first chess game, when I imagined a sequence of moves that culminated in me taking that King, then started executing my planned sequence of moves …..), the expert chess player knows that a different set of questions have to be asked: to wit, “How do I make pleasing, coherent patterns of support and strength on the board?” and “Do I recognize anything about the current pattern as similar or identical to a situation I have seen in the past?”
That particular “substitution” happens to be extremely optimal. It also happens to be not how chess computers work (by and large: let’s not get sidetracked by the finer points of chess programming … the fact is that machines rely on depth to a very large extent, whereas humans rely on pattern). So it makes no sense to talk about substitution as a “problem” in this case. Far from it, substitution seems to be why a tiny little lookahead device (human mind) can give the massive supercomputer a run for its money.
3) Finally, this analysis overall has the feel (like almost all “human bias is a problem” arguments) of fitting the data to the theory. So many people (here on LW, and in the biasses community generally) want to see “Biasses” as a big deal, that they see them everywhere. All the evidence that you see supports the idea that the concept of “biasses and heuristics” is a meaningful one, and that there are many instances of that concept, and that thos instances have certain ramifications.
But, like people who see images of Jesus in jars of Marmite, or evidence of divine intervention whenever someone recovers from an illness, I think that you see evidence of bias (and, in this case, substitution) primarily because you have trained yourself to theorize about the world that way. Not because it is all real.
“It also happens to be not how chess computers work ”
Not to sidetrack this with some unimportant fine point but that is by large a significantly invalid assertion and in so much as this assertion is relevant it needs to be corrected.
It actually happens to be, to significant extent, how good chess programs work. Also, for the best, most effective chess programs (e.g. crafty), you need to download gigabyte sized datasets before they’ll play their best. Even the badly playing naive programs try to maximize the piece dis-balance rather than consider just the taking of the king. It is literally impossible (on today’s hardware) to play chess by considering just taking the king—whenever you are human or supercomputer. Maybe an extremely powerful supercomputer could play chess by considering just the king—but that is an extreme case of playing chess perfectly.
There is no practical solution that does not involve substitution, not even for a well defined, compact problem like chess. Computers, even now, literally cannot win versus humans without relying on the human mind to perform good substitutions. Furthermore, if you run competitions between chess programs, on same hardware, the one with best substitutions and heuristics will win.
I think that actually strenghtens your point. Nothing can play chess yet without heuristics. Something that can play chess without heuristics would also have power to play chess perfectly (it would have to see the demise of the king or the tie right from the first move, to make any first move—that’s the depth of, for some of the tree, 100+ moves) and give the answer to long standing unsolved problem—which side wins with the perfect play? Or is it a tie?
Citation needed. The best chess-playing computer AFAIK is still Deep Blue. Deep Blue evaluated 200 million positions per second. That means it could look about 7 ply ahead per move, exhaustively, with no heuristics. But I agree that this does not weaken Richard’s point.
Even if you can look 7 ply ahead, how do you evaluate which of the possible resulting positions is better with you without heuristics?
Precisely. The king is still alive 7 moves ahead (usually).
At very least you use the heuristics like e.g. summing the piece values, and penalties for the e.g. two pawns of yours on same column (and bonuses for the pawn that threatens to become a queen, etc).
http://en.wikipedia.org/wiki/Computer_chess#Leaf_evaluation
Also, God only knows how many millions positions per second Garry Kasparov evaluates. Possibly quite many as they can be evaluated in parallel.
I would think that Kasparov’s evaluation, in a sense, substitutes less than Deep Blue’s—Kasparov can meta-strategise to king’s death or to tie from any position—when you are playing for tie you try to make computer exchange pieces, when you are playing for win you try to put computer into situation where you will be able to plan more moves ahead than the computer (computer tends to act as if it desperately hopes that some bad-looking move wins the game).
(To end a block quote, leave a blank line (i.e. two consecutive line breaks) after it.)
In the endgame this is useful, but in the opening/early mid-game, not so much. The king is still alive 7 moves ahead in the overwhelming majority of possible legal sequences of moves.
I’d bet Deep Blue does that, too.
EDIT: The “usually” you added makes it clear that I had completely missed your point. I’m retracting this.
ahh btw to satisfy the citation request:
Computer chess on wikipedia
I’ve participated some in computer contests, not chess related, where you can’t solve the problem exactly. It’s generally the case that you need clever “substitutions” to get anywhere at all. Most problems that aren’t outright trivial are too hard to solve from first principles.
Definitely so for the real world behaviours. A supposedly ‘rational’ calculation of best move, from first principles (death of king), but without substitutions and hunch based heuristics, done by human, over course of centuries, will not be even remotely powerful enough to match the immediate move that Kasparov would somehow make playing 1-minute blitz (and if you actually played chess real fast, it does become apparent that you just can’t match this play without massively parallel, rather deep evaluation). I think all aspiring rationalists should learn boardgames like Chess, Go, and perhaps some RTS (like starcraft, though my favourite is springrts. In the starcraft the AI has too much advantage due to human bottlenecking on the output) to appreciate the difficulties. Ideally, try writing an AI that fights against other AIs (in a timed programming contest) to appreciate the issues with advance metastrategies and heuristics.
I perform best on that sort of stuff if I am not tired and I act on feelings. If I am tired or bored and I act on feelings that leads to a quick loss which is also rational because—what the hell am I doing playing game when tired?
edit: where the hell did my wikipedia link disappear? ahh nevermind.
I have a friend who is much better at starcraft than I am; he says that he’s largely better because he’s worked out a lot of things like exactly the most efficient time to start harvesting gas and the resource collection per minute harvesters under optimal conditions, and he uses that information when he plays. It works better than playing based on feelings (by which I mean that he beats me).
If you don’t have way too much time on your hands, though, it’s about as much fun to not bother with all of that.
Also, I notice you cited a Wikipedia page. Naughty, naughty, naughty.
Well yea the start timings are important… they correspond to ‘book’ knowledge of chess openings. I did those a fair lot when i were playing chess more seriously (which was long long ago when i was 10-12). You have to do those before you start actually playing, and in RTS those openings are not so well developed as to read ’em off an old book like you do in chess.
Chess computer btw also uses openings, i think none of the 16 possible first moves leads to loss of king in 10ply (i’d wager a bet that none of possible white’s first moves leads to inevitable loss or victory at all, and certainly not in less than 50ply), and the computer does e2-e4 (or other good first move) purely by book. It doesn’t figure out that e2-e4 is better move than say a2-a3 from the rules alone. There’s a LOT of human thought about chess that chess AI relies on to beat humans.
Oh, hey, wow. I am a huge fan of Total Annihilation; this is really exciting! Which game do you recommend using with springrts?
I used to play balanced annihilation with springrts. I didn’t play it a whole lot but I did program some lua scripts for it and contributed to lobby development. Not playing it much any more but it is very interesting to make scripts for.
This is a fair criticism, and you’re right—I can’t say with definite certainty that these things were actually never considered at all. Still, if those things were considered, they don’t seem to be reflected in the final output. If instead of sayng “the thing is, it probably didn’t”, I said “the thing is, it probably didn’t—or if it did, it’s difficult to notice from the provided answer”, would you consider that acceptable?
I think you might be somewhat misinterpreting me here. I didn’t say that substitution is necessarily a problem—I specifically said it probably works pretty well most of the time. Heck, I imagine that if I was building an AI, I would explicitly program something like a substitution heuristic into it, to be used most of the time—because difficult problems are genuinely difficult, both in the sense of being computationally expensive and requiring information that isn’t usually at hand. A system that always tried to compute the exact answer for everything would never get anything done. Much better to usually employ some sort of quick heuristic that tended to at least point in the right direction, and then only spend more effort on the problem if it seemed to be important.
For that matter, you could say that a large part of science consists of a kind of substitution. Does System 1 actually ignore all those complicated considerations when considering its answer? Well, we can’t really answer that directly… but we can substitute the question with “does it seem to be ignoring them in certain kinds of experimental setups”, and then reflect upon what the answers to that question seem to tell us. This is part of the reason why I felt confident in saying that the brain probably never did take all the considerations into account—because simplifying problems to easier ones is such an essential part of actually ever getting anything done that it would seem odd if the brain didn’t do that.
So I agree that there are many cases (including your chess example) where substitution isn’t actually a problem, but rather the optimal course of action. And I agree that there are also many cases of bias where the substitution frame isn’t the best one.
The reason why I nevertheless brought it up was that, if I were building my hypothetical AI, there’s still one thing that I’d do differently than how the human brain seems to do it. A lot of the time, humans seem to be completely unaware of the fact that they are actually making a substitution, and treat the answer they get as the actual answer to the question they were asking. Like I mentioned in my other comment, I think the substitution principle is useful because it gives a useful rule of thumb that we can use to notice when we might be mistaken, and might need to think about the matter a bit more before assigning our intuitive result complete confidence.