I do agree that there are fields where the overall standards of the academic mainstream are not that high, but I’m not sure about the heuristics—I tend to use a different set.
One confusing factor is that in almost any field, the academic level of an arbitrary academic paper is not that high—average academic papers are published by average scientists, and are generally averagely brilliant—in other words, not that good. The preferred route is typically to prove something that’s actually already well known, but there are also plenty of flawed papers. There are also plenty of papers that are perhaps interesting if you’re interested in some particularly small niche of some particularly minor topic, but are of no relevance to the average reader. None of this says anything much about the quality of the mainstream orthodoxy, which can be very much higher than the quality of the average paper.
My main principle is that human beings are just not that intelligent. They are intelligent enough to follow a logical argument that is set into a system where there are tightly defined rules from which one can reason. They are NOT intelligent enough to reason sensibly AT ALL in regions where such rules are not defined. Well, perhaps a logical step or two is plausible, but anything beyond that becomes very dubious indeed—it is like trying to build a tower on a foundation of jello.
Reasoning based on vague definitions is a red flag—it encourages people to come up with any answer they want, and believe they’ve logically arrived at it. Reasoning based on a complicated set of not particularly related facts is a red flag, as nobody is intelligent enough to do it correctly.
Someone once said that all science is either physics or stamp collecting. It’s close—you have to have some organising principles of decent mathematical quality to do reasoning with any certainty. Without that, stamp collecting is the limit of the possible.
Equally, maths is not a panacea. It’s quite possible, in an academic paper, to spend a great deal of time developing a mathematical argument based on assumptions that aren’t really connected to the question you’re trying to answer—the maths is probably correct, but the vague and fuzzy bit where the maths is trying to connect to the problem is where it all goes wrong. To take the example everyone knows, financial models that assume average house prices can’t go down as well as up may have perfectly correct mathematics, but will not predict well what will happen to those investments when house prices do go down.
In summary, those fields with widely accepted logical systems are probably doing something right. Those fields where there are multiple logical systems that are competing are probably also doing something right—the worst they can do is to reason correctly about the wrong thing. Fields where there is an incumbent system which is vague are bad, as are those fields where freeform reason is the order of the day.
Someone once said that all science is either physics or stamp collecting. It’s close—you have to have some organising principles of decent mathematical quality to do reasoning with any certainty. Without that, stamp collecting is the limit of the possible.
I disagree with this. In many areas there are methodologies that don’t approach a mathematical level of formalization, and nevertheless yield rock-solid insight. One case in point is the example of historical linguistics I cited. These people have managed to reach non-obvious conclusions as reliable as anything else in science using a methodology that boils down to assembling a large web of heterogeneous common-sense evidence carefully and according to established systematic guidelines. Their results are a marvelous example of what some people call “traditional rationality” here.
In a way making a forum post is an example of the very kind of thing that I’m criticising—it’s a piece of freeform expression, and it’s a medium in which mistakes creep in easily.
I think you’re right to disagree with my statement there. The key thing isn’t the presence of mathematics—it’s the existence of some kind of set rational process—the “established systematic guidelines” that you mentioned.
It has a germ of truth, but I think it’s deeply misleading. In particular, it needs some kind of nod to the importance of relevance to everyday life. E.g., it would be more serious to claim “all science is either physics, or the systematizing side of some useful discipline like engineering, or stamp collecting.” Pure stamp collecting endeavors have nothing to stop them from veering into the behavior stereotypically associated with modern art or the Sokal hoax. Fields like paleobotany or astronomy (or, indeed, physics itself in near-unobservable limits) can become arbitrarily pure stamp collecting when the in-group controls funding. More applied fields like genetics or immunology or synthetic chemistry or geology are messy and disordered compared to pure physics, and do resemble stamp collecting in that messiness. But true stamp collecting is not merely messy, but also arbitrarily driven by fashion. To the extent that a significant amount of the interest (and money) associated with an academic field flows from applications like agriculture and medicine and resource extraction, it tends not to dive so deeply into true free-floating arbitrariness of pure stamp collecting.
I’m not as hard on stamp-collecting as you are. Admittedly, you need some sort of theory for why the information you’re collecting is of interest, but if the information isn’t widely and and carefully collected, the theoreticians don’t have anything to work with.
I wasn’t trying to be hard on that kind of collecting, though I was making a distinction. To me, choosing stamps (as opposed to, e.g., butterflies or historical artifacts) as a type specimen suggests that the collecting is largely driven by fashion or sentiment or some other inner or social motive, not because the objects are of interest for piecing together a vast disorderly puzzle found in the outer physical world. Inner and social motives are fine with me, though my motivation in such things tends to things other than collecting. (E.g., music and Go and Chess.)
E.g., it would be more serious to claim “all science is either physics, or the systematizing side of some useful discipline like engineering, or stamp collecting.”
As far as I can see, sitting in the mechanical engineering department of a state university, engineering research is a combination of physics and stamp collecting.
I do agree that there are fields where the overall standards of the academic mainstream are not that high, but I’m not sure about the heuristics—I tend to use a different set.
One confusing factor is that in almost any field, the academic level of an arbitrary academic paper is not that high—average academic papers are published by average scientists, and are generally averagely brilliant—in other words, not that good. The preferred route is typically to prove something that’s actually already well known, but there are also plenty of flawed papers. There are also plenty of papers that are perhaps interesting if you’re interested in some particularly small niche of some particularly minor topic, but are of no relevance to the average reader. None of this says anything much about the quality of the mainstream orthodoxy, which can be very much higher than the quality of the average paper.
My main principle is that human beings are just not that intelligent. They are intelligent enough to follow a logical argument that is set into a system where there are tightly defined rules from which one can reason. They are NOT intelligent enough to reason sensibly AT ALL in regions where such rules are not defined. Well, perhaps a logical step or two is plausible, but anything beyond that becomes very dubious indeed—it is like trying to build a tower on a foundation of jello.
Reasoning based on vague definitions is a red flag—it encourages people to come up with any answer they want, and believe they’ve logically arrived at it. Reasoning based on a complicated set of not particularly related facts is a red flag, as nobody is intelligent enough to do it correctly.
Someone once said that all science is either physics or stamp collecting. It’s close—you have to have some organising principles of decent mathematical quality to do reasoning with any certainty. Without that, stamp collecting is the limit of the possible.
Equally, maths is not a panacea. It’s quite possible, in an academic paper, to spend a great deal of time developing a mathematical argument based on assumptions that aren’t really connected to the question you’re trying to answer—the maths is probably correct, but the vague and fuzzy bit where the maths is trying to connect to the problem is where it all goes wrong. To take the example everyone knows, financial models that assume average house prices can’t go down as well as up may have perfectly correct mathematics, but will not predict well what will happen to those investments when house prices do go down.
In summary, those fields with widely accepted logical systems are probably doing something right. Those fields where there are multiple logical systems that are competing are probably also doing something right—the worst they can do is to reason correctly about the wrong thing. Fields where there is an incumbent system which is vague are bad, as are those fields where freeform reason is the order of the day.
DuncanS:
I disagree with this. In many areas there are methodologies that don’t approach a mathematical level of formalization, and nevertheless yield rock-solid insight. One case in point is the example of historical linguistics I cited. These people have managed to reach non-obvious conclusions as reliable as anything else in science using a methodology that boils down to assembling a large web of heterogeneous common-sense evidence carefully and according to established systematic guidelines. Their results are a marvelous example of what some people call “traditional rationality” here.
In a way making a forum post is an example of the very kind of thing that I’m criticising—it’s a piece of freeform expression, and it’s a medium in which mistakes creep in easily.
I think you’re right to disagree with my statement there. The key thing isn’t the presence of mathematics—it’s the existence of some kind of set rational process—the “established systematic guidelines” that you mentioned.
I thought it was Heinlein, but it’s actually Ernest Rutherford.
It has a germ of truth, but I think it’s deeply misleading. In particular, it needs some kind of nod to the importance of relevance to everyday life. E.g., it would be more serious to claim “all science is either physics, or the systematizing side of some useful discipline like engineering, or stamp collecting.” Pure stamp collecting endeavors have nothing to stop them from veering into the behavior stereotypically associated with modern art or the Sokal hoax. Fields like paleobotany or astronomy (or, indeed, physics itself in near-unobservable limits) can become arbitrarily pure stamp collecting when the in-group controls funding. More applied fields like genetics or immunology or synthetic chemistry or geology are messy and disordered compared to pure physics, and do resemble stamp collecting in that messiness. But true stamp collecting is not merely messy, but also arbitrarily driven by fashion. To the extent that a significant amount of the interest (and money) associated with an academic field flows from applications like agriculture and medicine and resource extraction, it tends not to dive so deeply into true free-floating arbitrariness of pure stamp collecting.
I’m not as hard on stamp-collecting as you are. Admittedly, you need some sort of theory for why the information you’re collecting is of interest, but if the information isn’t widely and and carefully collected, the theoreticians don’t have anything to work with.
I wasn’t trying to be hard on that kind of collecting, though I was making a distinction. To me, choosing stamps (as opposed to, e.g., butterflies or historical artifacts) as a type specimen suggests that the collecting is largely driven by fashion or sentiment or some other inner or social motive, not because the objects are of interest for piecing together a vast disorderly puzzle found in the outer physical world. Inner and social motives are fine with me, though my motivation in such things tends to things other than collecting. (E.g., music and Go and Chess.)
As far as I can see, sitting in the mechanical engineering department of a state university, engineering research is a combination of physics and stamp collecting.