I’m not going to get into a status competition with you over who is in a position to determine what.
OK, I will phrase it in different terms that make it explicit that I am making several claims here (one about what Bayesianism can determine, and one about what science can determine). It’s much like I said above:
It’s adequately suited for the accumulation of not-false beliefs, but it both could be better instrumentally designed for humans and is not the bedrock of thinking by which anything works. The thing that is essential to the method you described, “Scientists...have an informal sense of what P(A) is likely to be and are more inclined to question a conclusion if it is unlikely than if it is likely”. What abstraction describes the scientist’s thought process, the engine within the scientific method? I suggest it is Bayesian reasoning but even if it is not, one thing it cannot be is more of the Scientific method, as that would lead to recursion. If it is not Bayesian reasoning, then there are some things I am wrong about, and Bayesianism is a failed complete explanation, and the Scientific method is half of a quite adequate method—but they are still different from each other.
Some people claim Bayesian reasoning models intelligent agents’ learning about their environments, and agents’ deviations from it is failure to learn optimally. This model encompasses choosing when to use something like the scientific method and deciding when it is optimal to label beliefs not as “X% likely to be true, 1-X% likely to be untrue,” but rather “Good enough to rely on by virtue of being satisfactorily likely to be true,” and “Not good enough to rely on by virtue of being satisfactorily likely to be true”. If Bayesianism is wrong, and it may be, it’s wrong.
The scientific method is a somewhat diverse set of particular labeling systems declaring ideas “Good enough to rely on by virtue of being satisfactorily likely to be true,” and “Not good enough to rely on by virtue of being satisfactorily likely to be true.” Not only is the scientific method incomplete by virtue of using a black-box reasoning method inside of it, it doesn’t even claim to be able to adjudicate between circumstances in which it is to be used and in which it is not to be used. It is necessarily incomplete. Scientists’ reliance on intuition to decide when to use it and when not to may well be better than using Bayesian reasoning, particularly if Bayesianism is false, I grant that. But the scientific method doesn’t, correct me if I am wrong, purport to be able to formally decide whether or not a person should subject his or her religious beliefs to it.
The most obvious interpretation of your statement that science is “an imperfect human construct designed to accommodate the more biased of scientists” and that “it’s a cost and a deviation from ideal thinking to minimize the influence of scientists who receive no training in debiasing” is that you think your LW expertise means that you wouldn’t need those safeguards.
I disagree but here is a good example of where Bayesians can apply heuristics that aren’t first-order applications of Bayes rule. The failure mode of the heuristic is also easier to see than where science is accused of being too strict (though that’s really only a part of the total claim, the other parts are that science isn’t strict enough, that it isn’t near Pareto optimal according to its own tradeoffs in which it sacrifices truth, and that it is unfortunately taken as magical by its practitioners).
In those circumstances in which the Bayesian objection to science is that it is too strict, science can reply by ignoring that money is the unit of caring and declare its ideological purity and willingness to always sacrifice resources for greater certainty (such as when the sacrifice is withholding FDA approval of a drug already approved in Europe), “Either way you’re spending resources, but spending resources in the cause of epistemological purity is okay with me. Spending resources on junk because you are not practising the correct purification rituals is not.”
Here, however, the heuristic is “reading charitably”, in which the dangers of excess are really, really obvious. Nonetheless, even if I am wrong about what the best interpretation is, the extra-Bayesian ritual of reading (more) charitably would have had you thinking it more likely than you did that I had meant something more reasonable (and even more so, responding as if I did). It is logically possible that you were reading charitably ideally and my wording was simply terrible. This is a good example of how one can use heuristics other than Bayes’ rule once one discovers one is a human and therefore subject to bias. One can weigh the costs and benefits of it just like each feature of scientific testing.
For “an imperfect human construct designed to accommodate the more biased of scientists”, it would hardly do to assume scientists are all equally biased, and likewise for assuming the construct is optimal no matter the extent of bias in scientists. So the present situation could be improved upon by matching the social restrictions to the bias of scientists and also decreasing that bias. If mostly science isn’t strict enough, then perhaps it should be stricter in general (in many ways it should be) but the last thing to expect is that it is perfectly calibrated. It’s “imperfect”, I wouldn’t describe a rain dance as an “imperfect” method to get rain, it would be an “entirely useless” method. Science is “imperfect”, and it does very well to the extent thinking is warped to accommodate the more biased of scientists, and so something slightly different would be more optimal for the less biased ones.
″...it’s a cost and a deviation from ideal thinking to minimize the influence of scientists who receive no training in debiasing,” and less cost would be called for if they received such training, but not zero. Also, it is important to know that costs are incurred, lest evangelical pastors everywhere be correct when they declare science a “faith”. Science is roughly designed to prevent false things from being called “true” at the expense of true things not being called “true”. This currently occurs to different degrees in different sciences, and it should, and some of those areas should be stricter, and some should be less strict, and in all cases people shouldn’t be misled about belief such that they think there is a qualitative difference between a rigorously established base rate and one not so established, or science and predicting one’s child’s sickness when it vomits a certain color in the middle of the night.
My point is that the evidence on the table to support PUA theories is vulnerable to all the same problems as the evidence supporting claimed psychic powers
It’s not too similar since psychic powers have been found in controlled scientific studies, and they are (less than infinitely, but nearly) certainly not real. PUA theories were formed from people’s observations, then people developed ideas they thought based on the theories, then tested what they thought were the ideas, tested them insufficiently rigorously. Each such idea is barely more likely than the base rate for being correct due to all the failure nodes, but each is more likely, the way barely enriched uranium’s particles are more likely to be U-235 than natural uranium’s are. This is in line with “However I also suspect that PUAs have no idea which bits of PUA are the efficacious bits and which are superstition, and that they could achieve the modest gains possible much faster if they knew which was which”.
When it comes to action, as in psychological experiments in which one is given a single amount of money for correctly guessing the color of something between red and blue, and one determines 60% of the things are red, one should always guess red, one should act upon ideas most likely true if one must act, all else equal.
Any chance of turning this (and some of your other comments) into a top-level post? (perhaps something like, “When You Can (And Can’t) Do Better Than Science”?)
I think the first section should ignore the philosophy of science and cover the science of science, the sociology of it, and concede the sharpshooter’s fallacy, assuming that whatever science does it is trying to do. The task of improving upon the method is then not too normative, since one can simply achieve the same results with fewer resources/better results with the same resources. Also, that way science can’t blame perceived deficiencies on the methods of philosophy, as it could were one to evaluate science according to philosophy’s methods and standards. This section would be the biggest added piece of value that isn’t tying together things already on this site.
A section should look for edges with only one labeled node in the scientific methods where science requires input from a mystery method, such as how scientists generate hypotheses or how scientific revolutions occur. These show the incompleteness of the scientific method as a means to acquire knowledge, even if it is perfect at what it does. Formalization and improvement of the mystery methods would contribute to the scientific method, even if nothing formal within the model changes.
A section should discuss how science isn’t a single method (according to just about everybody), but instead a family of similar methods varying especially among fields. This weakens any claim idealizing science in general, as at most one could claim that a particular field’s method is ideal for human thought and discovery. Assuming each (or most) fields’ methods are ideal (this is the least convenient possible world for the critic of the scientific method as practiced), the costs and benefits of using that method rather than a related scientific method can be speculated upon. I expect to find, as policy debates should not be one sided, that were a field to use other fields’ methods it would have advantages and disadvantages; the simple case is choice of stricter p-value modulating wrong things believed at the expense of true things not believed.
Sections should discuss abuses of statistics, one covering violations of the law (failing to actually test P(B|~A) and instead testing P((B + (some random stuff) - (some other random stuff)|~A) and another covering systemic failures such as publication bias and failure to publish replications. This would be a good place to introduce intra-scientific debates about such things to show both that science isn’t a monolithic outlook that can be supported and how one side in the civil war is aligned with Bayesian critiques. To the extent science is not settled on what the sociology of science is, that is a mark of weakness—it may be perfectly calibrated, but it isn’t too discriminatory here.
A concession I imagine pro-science people might make is to concede the weakness of soft science, such as sociology. Nonetheless, sociology’s scientific method is deeply related to hard sciences’, and its shortcomings somewhat implicate them. What’s more, if sociology is so weak, one wonders whence the pro-science person gets their strong pro-science view. One possibility is that they get it purely from philosophy of science, (a school of which) they wholly endorse, but if that is the case they don’t have an objection in kind to those who also predict science as is works decently but have severe criticisms of it and ideas on how to improve upon it, i.e. Bayesians.
I think it’s fair to contrast the scientific view of science with a philosophical view of Bayesianism to see if they are of the same scope. If science has no position on whether or not science is an approximation of Bayesian reasoning, and Bayesianism does, that is at least one question addressed by the one and not the other. It would be easy to invent a method that’s not useful for finding truth that has a broader scope than science, e.g. answering “yes” to every yes or no question unless it would contradict a previous response. This alone would show they are not synonymous.
A problem with the title “When You Can (And Can’t) Do Better Than Science” is that it is binary, but I really want three things explicitly expressed: 1) When you can do better than science by being stricter than science, 2) when you can do better than science by being more lenient than science, 3) when you can’t do better than science. The equivocation and slipperiness surrounding what it is reasonable to do is a significant part of the last category, e.g. one doesn’t drive where the Tappan Zee Bridge should have been built. The other part is near-perfect ways science operates now according to a reasonable use of “can’t”; I wouldn’t expect science to be absolutely and exactly perfect anywhere, any more than I can be absolutely sure with a probability of 1 that the Flying Spaghetti Monster doesn’t exist.
Second order Bayesianism deserves mention as the thing being advocated. A “good Bayesian” may use heuristics to counteract bias other than just Bayes’ rule, such as the principle of charity, or pretending things are magic to counteract the effort heuristic, or reciting a large number of variably sized numbers to counteract the anchoring effect, etc.
Is there a better analogy than the driving to the airport one for why Bayes’ Rule being part of the scientific toolbox doesn’t show the scientific toolbox isn’t a rough approximation of how to apply Bayes’ Rule? The other one I thought of is light’s exhibiting quantum behavior directly, it being a subset of all that is physical, but all that is physical actually embodying quantum behavior.
A significant confusion is discussing beliefs as if they weren’t probabilistic and actions in some domains as if they ought not be influenced by anything not in a category of true belief “scientifically established”. Bayesianism explains why this is a useful approximation of how one should actually act and thereby permits one to deviate from it without having to claim something like “science doesn’t work”.
OK, I will phrase it in different terms that make it explicit that I am making several claims here (one about what Bayesianism can determine, and one about what science can determine). It’s much like I said above:
Some people claim Bayesian reasoning models intelligent agents’ learning about their environments, and agents’ deviations from it is failure to learn optimally. This model encompasses choosing when to use something like the scientific method and deciding when it is optimal to label beliefs not as “X% likely to be true, 1-X% likely to be untrue,” but rather “Good enough to rely on by virtue of being satisfactorily likely to be true,” and “Not good enough to rely on by virtue of being satisfactorily likely to be true”. If Bayesianism is wrong, and it may be, it’s wrong.
The scientific method is a somewhat diverse set of particular labeling systems declaring ideas “Good enough to rely on by virtue of being satisfactorily likely to be true,” and “Not good enough to rely on by virtue of being satisfactorily likely to be true.” Not only is the scientific method incomplete by virtue of using a black-box reasoning method inside of it, it doesn’t even claim to be able to adjudicate between circumstances in which it is to be used and in which it is not to be used. It is necessarily incomplete. Scientists’ reliance on intuition to decide when to use it and when not to may well be better than using Bayesian reasoning, particularly if Bayesianism is false, I grant that. But the scientific method doesn’t, correct me if I am wrong, purport to be able to formally decide whether or not a person should subject his or her religious beliefs to it.
I disagree but here is a good example of where Bayesians can apply heuristics that aren’t first-order applications of Bayes rule. The failure mode of the heuristic is also easier to see than where science is accused of being too strict (though that’s really only a part of the total claim, the other parts are that science isn’t strict enough, that it isn’t near Pareto optimal according to its own tradeoffs in which it sacrifices truth, and that it is unfortunately taken as magical by its practitioners).
In those circumstances in which the Bayesian objection to science is that it is too strict, science can reply by ignoring that money is the unit of caring and declare its ideological purity and willingness to always sacrifice resources for greater certainty (such as when the sacrifice is withholding FDA approval of a drug already approved in Europe), “Either way you’re spending resources, but spending resources in the cause of epistemological purity is okay with me. Spending resources on junk because you are not practising the correct purification rituals is not.”
Here, however, the heuristic is “reading charitably”, in which the dangers of excess are really, really obvious. Nonetheless, even if I am wrong about what the best interpretation is, the extra-Bayesian ritual of reading (more) charitably would have had you thinking it more likely than you did that I had meant something more reasonable (and even more so, responding as if I did). It is logically possible that you were reading charitably ideally and my wording was simply terrible. This is a good example of how one can use heuristics other than Bayes’ rule once one discovers one is a human and therefore subject to bias. One can weigh the costs and benefits of it just like each feature of scientific testing.
For “an imperfect human construct designed to accommodate the more biased of scientists”, it would hardly do to assume scientists are all equally biased, and likewise for assuming the construct is optimal no matter the extent of bias in scientists. So the present situation could be improved upon by matching the social restrictions to the bias of scientists and also decreasing that bias. If mostly science isn’t strict enough, then perhaps it should be stricter in general (in many ways it should be) but the last thing to expect is that it is perfectly calibrated. It’s “imperfect”, I wouldn’t describe a rain dance as an “imperfect” method to get rain, it would be an “entirely useless” method. Science is “imperfect”, and it does very well to the extent thinking is warped to accommodate the more biased of scientists, and so something slightly different would be more optimal for the less biased ones.
″...it’s a cost and a deviation from ideal thinking to minimize the influence of scientists who receive no training in debiasing,” and less cost would be called for if they received such training, but not zero. Also, it is important to know that costs are incurred, lest evangelical pastors everywhere be correct when they declare science a “faith”. Science is roughly designed to prevent false things from being called “true” at the expense of true things not being called “true”. This currently occurs to different degrees in different sciences, and it should, and some of those areas should be stricter, and some should be less strict, and in all cases people shouldn’t be misled about belief such that they think there is a qualitative difference between a rigorously established base rate and one not so established, or science and predicting one’s child’s sickness when it vomits a certain color in the middle of the night.
It’s not too similar since psychic powers have been found in controlled scientific studies, and they are (less than infinitely, but nearly) certainly not real. PUA theories were formed from people’s observations, then people developed ideas they thought based on the theories, then tested what they thought were the ideas, tested them insufficiently rigorously. Each such idea is barely more likely than the base rate for being correct due to all the failure nodes, but each is more likely, the way barely enriched uranium’s particles are more likely to be U-235 than natural uranium’s are. This is in line with “However I also suspect that PUAs have no idea which bits of PUA are the efficacious bits and which are superstition, and that they could achieve the modest gains possible much faster if they knew which was which”.
When it comes to action, as in psychological experiments in which one is given a single amount of money for correctly guessing the color of something between red and blue, and one determines 60% of the things are red, one should always guess red, one should act upon ideas most likely true if one must act, all else equal.
Any chance of turning this (and some of your other comments) into a top-level post? (perhaps something like, “When You Can (And Can’t) Do Better Than Science”?)
Yes.
I think the first section should ignore the philosophy of science and cover the science of science, the sociology of it, and concede the sharpshooter’s fallacy, assuming that whatever science does it is trying to do. The task of improving upon the method is then not too normative, since one can simply achieve the same results with fewer resources/better results with the same resources. Also, that way science can’t blame perceived deficiencies on the methods of philosophy, as it could were one to evaluate science according to philosophy’s methods and standards. This section would be the biggest added piece of value that isn’t tying together things already on this site.
A section should look for edges with only one labeled node in the scientific methods where science requires input from a mystery method, such as how scientists generate hypotheses or how scientific revolutions occur. These show the incompleteness of the scientific method as a means to acquire knowledge, even if it is perfect at what it does. Formalization and improvement of the mystery methods would contribute to the scientific method, even if nothing formal within the model changes.
A section should discuss how science isn’t a single method (according to just about everybody), but instead a family of similar methods varying especially among fields. This weakens any claim idealizing science in general, as at most one could claim that a particular field’s method is ideal for human thought and discovery. Assuming each (or most) fields’ methods are ideal (this is the least convenient possible world for the critic of the scientific method as practiced), the costs and benefits of using that method rather than a related scientific method can be speculated upon. I expect to find, as policy debates should not be one sided, that were a field to use other fields’ methods it would have advantages and disadvantages; the simple case is choice of stricter p-value modulating wrong things believed at the expense of true things not believed.
Sections should discuss abuses of statistics, one covering violations of the law (failing to actually test P(B|~A) and instead testing P((B + (some random stuff) - (some other random stuff)|~A) and another covering systemic failures such as publication bias and failure to publish replications. This would be a good place to introduce intra-scientific debates about such things to show both that science isn’t a monolithic outlook that can be supported and how one side in the civil war is aligned with Bayesian critiques. To the extent science is not settled on what the sociology of science is, that is a mark of weakness—it may be perfectly calibrated, but it isn’t too discriminatory here.
A concession I imagine pro-science people might make is to concede the weakness of soft science, such as sociology. Nonetheless, sociology’s scientific method is deeply related to hard sciences’, and its shortcomings somewhat implicate them. What’s more, if sociology is so weak, one wonders whence the pro-science person gets their strong pro-science view. One possibility is that they get it purely from philosophy of science, (a school of which) they wholly endorse, but if that is the case they don’t have an objection in kind to those who also predict science as is works decently but have severe criticisms of it and ideas on how to improve upon it, i.e. Bayesians.
I think it’s fair to contrast the scientific view of science with a philosophical view of Bayesianism to see if they are of the same scope. If science has no position on whether or not science is an approximation of Bayesian reasoning, and Bayesianism does, that is at least one question addressed by the one and not the other. It would be easy to invent a method that’s not useful for finding truth that has a broader scope than science, e.g. answering “yes” to every yes or no question unless it would contradict a previous response. This alone would show they are not synonymous.
A problem with the title “When You Can (And Can’t) Do Better Than Science” is that it is binary, but I really want three things explicitly expressed: 1) When you can do better than science by being stricter than science, 2) when you can do better than science by being more lenient than science, 3) when you can’t do better than science. The equivocation and slipperiness surrounding what it is reasonable to do is a significant part of the last category, e.g. one doesn’t drive where the Tappan Zee Bridge should have been built. The other part is near-perfect ways science operates now according to a reasonable use of “can’t”; I wouldn’t expect science to be absolutely and exactly perfect anywhere, any more than I can be absolutely sure with a probability of 1 that the Flying Spaghetti Monster doesn’t exist.
Second order Bayesianism deserves mention as the thing being advocated. A “good Bayesian” may use heuristics to counteract bias other than just Bayes’ rule, such as the principle of charity, or pretending things are magic to counteract the effort heuristic, or reciting a large number of variably sized numbers to counteract the anchoring effect, etc.
Is there a better analogy than the driving to the airport one for why Bayes’ Rule being part of the scientific toolbox doesn’t show the scientific toolbox isn’t a rough approximation of how to apply Bayes’ Rule? The other one I thought of is light’s exhibiting quantum behavior directly, it being a subset of all that is physical, but all that is physical actually embodying quantum behavior.
A significant confusion is discussing beliefs as if they weren’t probabilistic and actions in some domains as if they ought not be influenced by anything not in a category of true belief “scientifically established”. Bayesianism explains why this is a useful approximation of how one should actually act and thereby permits one to deviate from it without having to claim something like “science doesn’t work”.
Thoughts?