Asking other people who have solved a similar problem to evaluate your answer is very powerful and simple strategy to follow.
Also, most evidence I have seen is that you can only learn how to do a small number of things well. So if you are solving something outside of your area of expertise (which probably includes most problems you’ll encounter during your life) then there is probably somebody out there who can give a much better answer than you (although the cost to find such a person may be too great).
Edit: The software does not seem to allow for links that have parentheses, so you would need to copy the whole link—including the ”.pdf” at the end—in order to actually pull up the document.
Edit Jan 18: Hex-escaped the parentheses so it should work better.
Your Gian-Carlo Rota link talks about tricks used in proofs, which is different from sub-fields of mathematics. It’s certainly true that modern mathematicians and scientists are hyperspecialized, but that’s not the same thing.
I agree that my mathematics example is insufficient to prove the general claim: “One will master only a small number of skills”. I suppose a proper argument would require an in-depth study of people who solve hard problems.
I think the essential point of my claim is that there is high variance with respect to the subset of the population that can solve a given difficult problem. This seems to be true in most of the sciences and engineering to the best of my knowledge (though I know mathematics best). The theory I believe that explains why this variation occurs is that the subset of people which can solve a given problem use unconscious heuristics borne out of the hard work they put into previous problems over many years.
Admittedly, the problems I am thinking about are kind of like NP problems: it seems difficult to find a solution, but once a solution is found we can know it when we see it. There tends to be a large number of such problems that can be solved by only a small number of people. And the group of people that can solve them varies a lot from problem to problem.
There are also many hard problems for which it is hard to say what a good solution is (e.g. it seems difficult to evaluate different economic policies), or the “goodness” of a solution varies a lot with different value systems (e.g. abortion policy). It does seem that in these instances politicians claim they can give good answers to all the problems as do management consulting companies. Public intellectuals and pundits also seem to think they can give good answers to lots of questions as well. I suppose that if they are right then my claim is wrong. I argue that such individuals and organizations claim to be able to solve many problems but since its hard to verify the quality of the solutions we should take the claim with a grain of salt. We know that individuals who can solve lots of problems would have a lot of status so there is a clear incentive to claim to be able to solve problems that one cannot actually solve if verifying the solution is sufficiently costly.
I also think there is a good reason to think that even for those problems whose solutions are difficult to evaluate we should expect only a small number of people to actually give a good solution. The reason relates to a point made by Robin Hanson (and myself in another comment) which is that in solving a problem you should try to solve many at once. A good solution to a problem should give insight to many problems. Conversely, to understand and recognize a good solution to a given hard problem one should understand what it says about many other problems. The space of problems is too vast for any human being to know but a small portion, so I expect that people who are able to solve a given problem should only be those aware of many related problems and that most people will not be aware of the related problems. Given that in our civilization different people are exposed to different problems (no matter in which field they are employed) we should expect high variance of who can solve which hard problems.
Asking other people who have solved a similar problem to evaluate your answer is very powerful and simple strategy to follow.
Also, most evidence I have seen is that you can only learn how to do a small number of things well. So if you are solving something outside of your area of expertise (which probably includes most problems you’ll encounter during your life) then there is probably somebody out there who can give a much better answer than you (although the cost to find such a person may be too great).
Post Note: The fact that you can only learn a few things really well seems to be true with mathematics: as in here. More generally, mastering a topic seems to take ten years or so [PDF] (see Edit below).
Edit: The software does not seem to allow for links that have parentheses, so you would need to copy the whole link—including the ”.pdf” at the end—in order to actually pull up the document.
Edit Jan 18: Hex-escaped the parentheses so it should work better.
Your Gian-Carlo Rota link talks about tricks used in proofs, which is different from sub-fields of mathematics. It’s certainly true that modern mathematicians and scientists are hyperspecialized, but that’s not the same thing.
There is a trick for your parenthesized link: hex-escape the parentheses. Here’s the link itself: http://graphics8.nytimes.com/images/blogs/freakonomics/pdf/DeliberatePractice%28PsychologicalReview%29.pdf Here are cute fluffy kittens.
Thanks for the link assistance.
I agree that my mathematics example is insufficient to prove the general claim: “One will master only a small number of skills”. I suppose a proper argument would require an in-depth study of people who solve hard problems.
I think the essential point of my claim is that there is high variance with respect to the subset of the population that can solve a given difficult problem. This seems to be true in most of the sciences and engineering to the best of my knowledge (though I know mathematics best). The theory I believe that explains why this variation occurs is that the subset of people which can solve a given problem use unconscious heuristics borne out of the hard work they put into previous problems over many years.
Admittedly, the problems I am thinking about are kind of like NP problems: it seems difficult to find a solution, but once a solution is found we can know it when we see it. There tends to be a large number of such problems that can be solved by only a small number of people. And the group of people that can solve them varies a lot from problem to problem.
There are also many hard problems for which it is hard to say what a good solution is (e.g. it seems difficult to evaluate different economic policies), or the “goodness” of a solution varies a lot with different value systems (e.g. abortion policy). It does seem that in these instances politicians claim they can give good answers to all the problems as do management consulting companies. Public intellectuals and pundits also seem to think they can give good answers to lots of questions as well. I suppose that if they are right then my claim is wrong. I argue that such individuals and organizations claim to be able to solve many problems but since its hard to verify the quality of the solutions we should take the claim with a grain of salt. We know that individuals who can solve lots of problems would have a lot of status so there is a clear incentive to claim to be able to solve problems that one cannot actually solve if verifying the solution is sufficiently costly.
I also think there is a good reason to think that even for those problems whose solutions are difficult to evaluate we should expect only a small number of people to actually give a good solution. The reason relates to a point made by Robin Hanson (and myself in another comment) which is that in solving a problem you should try to solve many at once. A good solution to a problem should give insight to many problems. Conversely, to understand and recognize a good solution to a given hard problem one should understand what it says about many other problems. The space of problems is too vast for any human being to know but a small portion, so I expect that people who are able to solve a given problem should only be those aware of many related problems and that most people will not be aware of the related problems. Given that in our civilization different people are exposed to different problems (no matter in which field they are employed) we should expect high variance of who can solve which hard problems.
Thanks