Does this phenomenon happen at all, or is it a product of selective memory? It seems to me that incidents where a solution occurs to us while we are thinking of something else will be more memorable (and make better stories) than incidents where we find a solution while explicitly working on it.
How would we distinguish a world in which stepping away from a problem makes a solution more probable to occur from a world in which stepping away from a problem makes a solution more awesome-feeling and therefore more likely to be remembered and repeated as a story?
You make a good point. Thank you for bringing it up!
The phenomenon definitely happens. There’s no question that insights pop into mind out of context for at least some rather large chunk of the population. Ask just about any math professor or graduate student: I’m willing to give 85% odds that they’ll indicate that key insights to problems they had been working on have occurred to them more than once during times when they weren’t thinking about the problems in question.
Subjectively and in memory, I think there is a similarity between what happens when an insight pops into mind unexpectedly & out of context as compared to when it pops into mind in the middle of working on a problem. (“Oh! Right! I can just think of these geometric structures as elements of a basis for a vector field!”) This is different in character from working out different permutations of the problem and coming across one that happens to work. It’s possible that the method of derivation is basically the same but one is done automatically in the background—but I really don’t know! I wouldn’t be surprised one way or the other.
However, many, many mathematicians have stumbled on roughly the same strategy for dealing with tough problems: walk away from it and work on something else. There are lots and lots of surveys of mathematicians’ reports of their working behavior, most of which jive with what I recall seeing while in graduate school. For instance, it’s pretty typical for working mathematicians to have several largely unrelated problems they’re tackling in parallel, and many of them say that this is specifically so that they can cycle through the problems and come back to them with fresh eyes, possibly with insight appearing in the pattern Hadamard describes.
This strikes me as Bayesian evidence for us being in the universe where stepping away increases the probability of solution over one where the narrative just happens to be interesting and compelling. Otherwise wouldn’t mathematicians who just stick with one problem until it’s done have better productivity overall?
Also, I want to be clear that I’m not claiming that all insight happens this way. I’m claiming that some insight does, and that it seems to play a pretty key role in a lot of human functioning and problem-solving. I don’t need this method in order to solve a quadratic equation by hand, for instance, but I almost certainly would if I had to figure out (rather than look up) how to solve an arbitrary cubic equation.
The phenomenon definitely happens. There’s no question that insights pop into mind out of context for at least some rather large chunk of the population. Ask just about any math professor or graduate student: I’m willing to give 85% odds that they’ll indicate that key insights to problems they had been working on have occurred to them more than once during times when they weren’t thinking about the problems in question.
The question was whether the insight is more likely to pop into mind when stepping away from the problem.
I seem to recall an experiment in which subjects told to think about a hard problem for two hours had a significantly lower success rate than subjects told to think about it for half an hour, go play sudoku for an hour, then think about it for another half an hour. But grep and google are coming up empty for me at the moment, so take this with a grain of salt.
Do you think that the point about mathematicians coming consistently to this strategy does not constitute evidence? It certainly isn’t overwhelming evidence, but it seems suspicious that virtually all mathematicians who talk about mathematical process talk about the importance of walking away from problems. I’m personally not aware of a single mathematician who thinks that such a practice is unnecessary.
(My dissertation was on mathematicians’ methods for navigating struggle in their research, and as part of that I did a fair amount of looking both at mathematicians’ accounts and at summaries of such accounts. The closest thing to denying this phenomenon I’ve encountered is the strong insistance of a very tiny minority of mathematicians that “intuition” has nothing to do with mathematics—but those same people still reported needing multiple problems to work on in parallel so that they could turn their attention away from a given problem they were stuck on.)
Mathematicians’ claims may too be explained by selective memory effects mentioned by fubarobfusco in the first comment in this thread. The question is how to discriminate between the case when the mathematicians’ testimonies are reflecting an existing phenomenon and the case when they result from a bias. Even if the insights were less likely to materialise after stepping away, there would be plenty of cases of this happening, so the fact that virtually every mathematician can remember few of them wouldn’t be surprising.
Even if the insights were less likely to materialise after stepping away, there would be plenty of cases of this happening, so the fact that virtually every mathematician can remember few of them wouldn’t be surprising.
Point taken. I guess the likelihood ratio for this strategy being actively helpful is closer to 1 than I had previously thought.
However, it’s not just a few incidences. It’s remarkably frequent. And it’s also still valuable to note that problem-solving can occur in the background without the need for conscious attention. Even if the background process turns out not to be as efficient as conscious reflection, freeing up attention while still working on the problem looks like an obvious win to me.
Does this phenomenon happen at all, or is it a product of selective memory? It seems to me that incidents where a solution occurs to us while we are thinking of something else will be more memorable (and make better stories) than incidents where we find a solution while explicitly working on it.
How would we distinguish a world in which stepping away from a problem makes a solution more probable to occur from a world in which stepping away from a problem makes a solution more awesome-feeling and therefore more likely to be remembered and repeated as a story?
You make a good point. Thank you for bringing it up!
The phenomenon definitely happens. There’s no question that insights pop into mind out of context for at least some rather large chunk of the population. Ask just about any math professor or graduate student: I’m willing to give 85% odds that they’ll indicate that key insights to problems they had been working on have occurred to them more than once during times when they weren’t thinking about the problems in question.
Subjectively and in memory, I think there is a similarity between what happens when an insight pops into mind unexpectedly & out of context as compared to when it pops into mind in the middle of working on a problem. (“Oh! Right! I can just think of these geometric structures as elements of a basis for a vector field!”) This is different in character from working out different permutations of the problem and coming across one that happens to work. It’s possible that the method of derivation is basically the same but one is done automatically in the background—but I really don’t know! I wouldn’t be surprised one way or the other.
However, many, many mathematicians have stumbled on roughly the same strategy for dealing with tough problems: walk away from it and work on something else. There are lots and lots of surveys of mathematicians’ reports of their working behavior, most of which jive with what I recall seeing while in graduate school. For instance, it’s pretty typical for working mathematicians to have several largely unrelated problems they’re tackling in parallel, and many of them say that this is specifically so that they can cycle through the problems and come back to them with fresh eyes, possibly with insight appearing in the pattern Hadamard describes.
This strikes me as Bayesian evidence for us being in the universe where stepping away increases the probability of solution over one where the narrative just happens to be interesting and compelling. Otherwise wouldn’t mathematicians who just stick with one problem until it’s done have better productivity overall?
Also, I want to be clear that I’m not claiming that all insight happens this way. I’m claiming that some insight does, and that it seems to play a pretty key role in a lot of human functioning and problem-solving. I don’t need this method in order to solve a quadratic equation by hand, for instance, but I almost certainly would if I had to figure out (rather than look up) how to solve an arbitrary cubic equation.
The question was whether the insight is more likely to pop into mind when stepping away from the problem.
I seem to recall an experiment in which subjects told to think about a hard problem for two hours had a significantly lower success rate than subjects told to think about it for half an hour, go play sudoku for an hour, then think about it for another half an hour. But grep and google are coming up empty for me at the moment, so take this with a grain of salt.
Do you think that the point about mathematicians coming consistently to this strategy does not constitute evidence? It certainly isn’t overwhelming evidence, but it seems suspicious that virtually all mathematicians who talk about mathematical process talk about the importance of walking away from problems. I’m personally not aware of a single mathematician who thinks that such a practice is unnecessary.
(My dissertation was on mathematicians’ methods for navigating struggle in their research, and as part of that I did a fair amount of looking both at mathematicians’ accounts and at summaries of such accounts. The closest thing to denying this phenomenon I’ve encountered is the strong insistance of a very tiny minority of mathematicians that “intuition” has nothing to do with mathematics—but those same people still reported needing multiple problems to work on in parallel so that they could turn their attention away from a given problem they were stuck on.)
Mathematicians’ claims may too be explained by selective memory effects mentioned by fubarobfusco in the first comment in this thread. The question is how to discriminate between the case when the mathematicians’ testimonies are reflecting an existing phenomenon and the case when they result from a bias. Even if the insights were less likely to materialise after stepping away, there would be plenty of cases of this happening, so the fact that virtually every mathematician can remember few of them wouldn’t be surprising.
Point taken. I guess the likelihood ratio for this strategy being actively helpful is closer to 1 than I had previously thought.
However, it’s not just a few incidences. It’s remarkably frequent. And it’s also still valuable to note that problem-solving can occur in the background without the need for conscious attention. Even if the background process turns out not to be as efficient as conscious reflection, freeing up attention while still working on the problem looks like an obvious win to me.