But doesn’t it seem that if you decompartmentalized with correct beliefs you should do way better?
Maybe; there are all sorts of caveats to that. But that aside, more directly on the question of tests:
Possibly in a testable way?
You still run into the problem that the outcome depends greatly on context and phrasing. There is the question with turning over cards to test a hypothesis, on which people’s performance dramatically improves when you rephrase it as an isomorphic question about social rules. There are the trolley questions and the specks versus torture question and the ninety-seven percent versus one hundred percent question, on which the right answer depends entirely on whether you treat it as a mathematical question that happens to be expressed in English syntax or a question about what you should do if you believed yourself to really be in that situation. There are questions about uncertain loss isomorphic to questions about uncertain gain where people nonetheless give different answers, which is irrational if considered as a material problem, but rational in the more likely and actual situation where the only thing at stake is social status, which sometimes does depend on how the question was phrased. Etc.
That’s why I called the testing problem ill posed; it’s not just that it’s hard to figure out the solution, it’s hard to see what would be the criteria of a good solution in the first place.
Those examples are good evidence for us not being able to test coherently yet, but I don’t think they are good evidence that the question is ill-posed.
If the question is “how can we test rationality?”, and the only answers we’ve come up with are limited in scope and subject to all kinds of misinterpretation, I don’t think that means we can’t come up with broad tests that measure progress. I am reminded of a quote: “what you are saying amounts to ‘if it is possible, it ought to be easy’”
I think the place to find good tests will be instead of looking at how well people do against particular biases, look at what it is we think rationality is good for, and measure something related to that.
Ill posed does not necessarily mean impossible. Most of the problems we deal with in real life are ill posed, but we still usually manage to come up with solutions that are good enough for the particular contexts at hand. What it does mean is that we shouldn’t expect the problem in question to be definitely solved once and for all. I’m not arguing against attempting to test rationality. I’m arguing against the position some posters have taken that there’s no point even trying to make progress on rationality until the problem of testing it has been definitely solved.
Maybe; there are all sorts of caveats to that. But that aside, more directly on the question of tests:
You still run into the problem that the outcome depends greatly on context and phrasing. There is the question with turning over cards to test a hypothesis, on which people’s performance dramatically improves when you rephrase it as an isomorphic question about social rules. There are the trolley questions and the specks versus torture question and the ninety-seven percent versus one hundred percent question, on which the right answer depends entirely on whether you treat it as a mathematical question that happens to be expressed in English syntax or a question about what you should do if you believed yourself to really be in that situation. There are questions about uncertain loss isomorphic to questions about uncertain gain where people nonetheless give different answers, which is irrational if considered as a material problem, but rational in the more likely and actual situation where the only thing at stake is social status, which sometimes does depend on how the question was phrased. Etc.
That’s why I called the testing problem ill posed; it’s not just that it’s hard to figure out the solution, it’s hard to see what would be the criteria of a good solution in the first place.
Those examples are good evidence for us not being able to test coherently yet, but I don’t think they are good evidence that the question is ill-posed.
If the question is “how can we test rationality?”, and the only answers we’ve come up with are limited in scope and subject to all kinds of misinterpretation, I don’t think that means we can’t come up with broad tests that measure progress. I am reminded of a quote: “what you are saying amounts to ‘if it is possible, it ought to be easy’”
I think the place to find good tests will be instead of looking at how well people do against particular biases, look at what it is we think rationality is good for, and measure something related to that.
Ill posed does not necessarily mean impossible. Most of the problems we deal with in real life are ill posed, but we still usually manage to come up with solutions that are good enough for the particular contexts at hand. What it does mean is that we shouldn’t expect the problem in question to be definitely solved once and for all. I’m not arguing against attempting to test rationality. I’m arguing against the position some posters have taken that there’s no point even trying to make progress on rationality until the problem of testing it has been definitely solved.
Ok, that’s reasonable. I was taking ill-posed to mean like a confused question. Or something like that.