It seems to me that what you’re saying is that our theories of rationality are ultimately based on a process of reflection that starts with pre-theoretical judgments about what we think is rational, about what dispositions help agents achieve their goals, etc., and that this means that we should take quite seriously our pre-theoretical judgments when they strongly conflict with our theories of rationality.
This is right in principle (as well as in the Pascal’s Mugging case), but I think you’re too conservative. For example, should I heavily discount arguments (such as made by Eliezer) that Science is an often-flawed institutionally practical approximation of Bayesianism (which is superior when you can use it), based on the fact that Science has much more to show for itself than Bayesianism? I would say no, because it seems reasonable to believe that Bayesianism can explain much of the success of science and thus can claim a lot of science’s successes for itself. Similarly, if our best theories of rationality (together with our empirical knowledge about psychology and such) can explain why we (wrongly) hold certain pre-theoretical judgments, it seems fine to let our theories of rationality trump our intuitions (as Eliezer tries to do with the “repugnant conclusion” somewhere). So I would not draw arbitrary lines that I commit to never ever cross. I don’t hold my unwashed intuitions in that high a regard.
...because it seems reasonable to believe that Bayesianism can explain much of the success of science and thus can claim a lot of science’s successes for itself.
True, but science was invented because people are really bad at judging evidence. I am troubled by the prospect of people using the core principles of science, e.g. Bayesianism, and applying them loosely and informally to vaguely understood conjectures and follow through on the implied actions.
Prediction, experimentation, peer review and the demand of empirical evidence is what makes science strong. If you think that you can use your rationality in a combination with Bayesianism and run with it then you confound your puny human brain with that of the hypothetical superintelligence that you dreamed up.
What I am arguing for is to be more conservative when it comes to the theoretically superior heuristics being discussed within this community.
It seems to me that what you’re saying is that our theories of rationality are ultimately based on a process of reflection that starts with pre-theoretical judgments about what we think is rational, about what dispositions help agents achieve their goals, etc., and that this means that we should take quite seriously our pre-theoretical judgments when they strongly conflict with our theories of rationality.
This is right in principle (as well as in the Pascal’s Mugging case), but I think you’re too conservative. For example, should I heavily discount arguments (such as made by Eliezer) that Science is an often-flawed institutionally practical approximation of Bayesianism (which is superior when you can use it), based on the fact that Science has much more to show for itself than Bayesianism? I would say no, because it seems reasonable to believe that Bayesianism can explain much of the success of science and thus can claim a lot of science’s successes for itself. Similarly, if our best theories of rationality (together with our empirical knowledge about psychology and such) can explain why we (wrongly) hold certain pre-theoretical judgments, it seems fine to let our theories of rationality trump our intuitions (as Eliezer tries to do with the “repugnant conclusion” somewhere). So I would not draw arbitrary lines that I commit to never ever cross. I don’t hold my unwashed intuitions in that high a regard.
True, but science was invented because people are really bad at judging evidence. I am troubled by the prospect of people using the core principles of science, e.g. Bayesianism, and applying them loosely and informally to vaguely understood conjectures and follow through on the implied actions.
Prediction, experimentation, peer review and the demand of empirical evidence is what makes science strong. If you think that you can use your rationality in a combination with Bayesianism and run with it then you confound your puny human brain with that of the hypothetical superintelligence that you dreamed up.
What I am arguing for is to be more conservative when it comes to the theoretically superior heuristics being discussed within this community.
Most of the science we’re impressed with was done before peer review, and I don’t think there’s much evidence that peer review is helpful on net.