I haven’t thought about the bat and ball question specifically very much since writing this post, but I did get a lot of interesting comments and suggestions that have sort of been rolling around my head in background mode ever since. Here’s a few I wanted to highlight:
Is the bat and ball question really different to the others? First off, it was interesting to see how much agreement there was with my intuition that the bat and ball question was interestingly different to the other two questions in the CRT. Reading through the comments I count four other people who explicitly agree with this (1, 2, 3, 4) and three who either explicitly disagree or point out that they find the widget problem hardest (5, 6, 7). I’d be intrigued to know if other people also disagree that the bat and ball feels different to them.
Concrete vs abstract quantities. Out of the people who agreed with that the bat and ball is different, this comment from @awbery does a particularly good job of giving a potential explanation for why:
The problem is a ‘two things’ problem. The first sentence presents two things, a bat and a ball. The language correctly reflects there are two things we should consider. The first sentence is ‘this plus that equals $1.10’. It correctly sounds like a + b; two things. The first sentence presents the state of affairs, not the problem itself. The second sentence presents the problem. The language of the second sentence reinforces the two things idea because there’s still the bat and the ball and they’re compared against each other: ‘there’s this one and it’s more than that one’. The trickiness is that it is a two things problem, but the two things we need to consider are not the most object level single units, but the bat, and the bat-plus-ball. Our brains are pulled toward the object level division of things by the language and the visual nature of the problem. We have to think really hard to understand that the abstract construct of the problem is the same shape as the state of affairs – there are two things to consider in relation to each other – but while the bat and the ball are still involved, they’re reconfigured by a non-intuitive/non-object-like division.
There’s no object level mirror trick in the other two problems, they’re straight forward maths mapping an object level visual representation. The widget problem presents a process which doesn’t change how the machines and widgets relate to each other in its solution. Our brains don’t have to mash up the pond and the lilies to separate the visual presentation to an abstract level. We can see that the pond is the same pond, half covered with lilies then fully covered with lilies at the next step. We don’t suddenly have some new abstract unreal configuration of lilies and pond to contend with.
I think this is why Kyzentun and Ander’s methods help get at the bat and ball problem intuitively – because they bypass the conflict between object level and abstract and translate it into the formal algebra realm. The problem as presented is non-intuitive because the objects visualization it suggests doesn’t reflect the shape of the formal solution.
So I think this is a particular type of problem, one in which visual shape and language of the presentation collude to obfuscate the visualization of the solution at an abstract/formal level. It’s a different type of problem to the other two in this sense, because the objects they present can be used as given in the solution.
Closeness to correct answer. Another interesting possibility is in TheManxLoiner’s comment—that the bat and ball problem is difficult because the incorrect answer is ‘close to the real one’, whereas for the other two problems the incorrect answer is ‘wildly off’. I’ve written a comment in response but I need to think about this more.
Ethnomethodology. David Chapman pointed out that these introspective accounts of what people are thinking when they solve maths problems are very unreliable, and that I’d probably be better concentrating strictly on what people do, as in ethnomethodology:
Yes, the fundamental principle of ethnomethodological methodology is “look at what people say and do, and don’t ever speculate about what’s happening in their head, because we can’t know.” At first that seems like a straitjacket, and highly unintuitive; but it forces you to really look, and then you see what is going on.
This sounds promising. I’m only just getting round to reading some ethnomethodology, and I haven’t got my bearings yet.
Cognitive decoupling. There’s a link with cognitive decoupling (in Stanovich’s original sense) that could be worth exploring further. Success in the bat and ball problem seems to involve decoupling from the noisy wrong answer. David Chapman recommended Formal Languages in Logic by Dutilh Novaes for more background on this. So far I’ve read maybe a third of it. I’ve also written a bit more about cognitive decoupling and the history of the term here.
Next steps. I’m not sure where I’m going to take this next. Probably nowhere much for a while, as I have other priorities. But some options are:
Anders came up with a load of similar problems in the comments. These are designed to be cognitively unpleasant in the same way as the bat and ball, so I keep putting them off. I should actually go through them!
I’m going to continue reading Dutilh Novaes and some ethnomethodology.
Connect more specifically to Stanovich’s idea of cognitive decoupling.
Testing theories? Further out, it could be interesting to actually test some theories by trying alternative, disguised versions of the question, on Mechanical Turk or something. Right now I’ve barely considered this, because I haven’t thought through what I’d want carefully enough yet, but it might be interesting to test variations in:
how concrete the things the quantities refer to are (e.g. really concrete like ‘the price of the bat’, or more abstract like ‘the difference between the price of the bat and ball’. Some of Anders’ variant questions might fit the bill
how close in magnitude the intuitive-but-wrong answer is, as in TheManxLoiner’s comment
I’m very ignorant about experiment design, so to do this I’d to get help from someone more knowledgeable. And psych research sounds like a gigantic minefield even if you are knowledgeable, so I’d probably end up wasting my time. But probably I’d learn something from going through the process, and it’s something that could maybe happen in the future.
I haven’t thought about the bat and ball question specifically very much since writing this post, but I did get a lot of interesting comments and suggestions that have sort of been rolling around my head in background mode ever since. Here’s a few I wanted to highlight:
Is the bat and ball question really different to the others? First off, it was interesting to see how much agreement there was with my intuition that the bat and ball question was interestingly different to the other two questions in the CRT. Reading through the comments I count four other people who explicitly agree with this (1, 2, 3, 4) and three who either explicitly disagree or point out that they find the widget problem hardest (5, 6, 7). I’d be intrigued to know if other people also disagree that the bat and ball feels different to them.
Concrete vs abstract quantities. Out of the people who agreed with that the bat and ball is different, this comment from @awbery does a particularly good job of giving a potential explanation for why:
Closeness to correct answer. Another interesting possibility is in TheManxLoiner’s comment—that the bat and ball problem is difficult because the incorrect answer is ‘close to the real one’, whereas for the other two problems the incorrect answer is ‘wildly off’. I’ve written a comment in response but I need to think about this more.
Ethnomethodology. David Chapman pointed out that these introspective accounts of what people are thinking when they solve maths problems are very unreliable, and that I’d probably be better concentrating strictly on what people do, as in ethnomethodology:
This sounds promising. I’m only just getting round to reading some ethnomethodology, and I haven’t got my bearings yet.
Cognitive decoupling. There’s a link with cognitive decoupling (in Stanovich’s original sense) that could be worth exploring further. Success in the bat and ball problem seems to involve decoupling from the noisy wrong answer. David Chapman recommended Formal Languages in Logic by Dutilh Novaes for more background on this. So far I’ve read maybe a third of it. I’ve also written a bit more about cognitive decoupling and the history of the term here.
Next steps. I’m not sure where I’m going to take this next. Probably nowhere much for a while, as I have other priorities. But some options are:
Anders came up with a load of similar problems in the comments. These are designed to be cognitively unpleasant in the same way as the bat and ball, so I keep putting them off. I should actually go through them!
I’m going to continue reading Dutilh Novaes and some ethnomethodology.
Connect more specifically to Stanovich’s idea of cognitive decoupling.
Testing theories? Further out, it could be interesting to actually test some theories by trying alternative, disguised versions of the question, on Mechanical Turk or something. Right now I’ve barely considered this, because I haven’t thought through what I’d want carefully enough yet, but it might be interesting to test variations in:
how concrete the things the quantities refer to are (e.g. really concrete like ‘the price of the bat’, or more abstract like ‘the difference between the price of the bat and ball’. Some of Anders’ variant questions might fit the bill
how close in magnitude the intuitive-but-wrong answer is, as in TheManxLoiner’s comment
I’m very ignorant about experiment design, so to do this I’d to get help from someone more knowledgeable. And psych research sounds like a gigantic minefield even if you are knowledgeable, so I’d probably end up wasting my time. But probably I’d learn something from going through the process, and it’s something that could maybe happen in the future.