Testing rationality is something of an ill posed problem, in part because the result depends greatly on context. People spout all kinds of nonsense in a social context where it’s just words, but usually manage to compartmentalize the nonsense in a material context where they will be affected by the results of their actions. (This is a feature! Given that evolution wasn’t able to come up with minds that infallibly distinguish true beliefs from false ones, it’s good that at least it came up with a way to reduce the harm from false beliefs.) I’m not sure how to create an accurate test in the face of that.
Your martial arts analogy isn’t a bad one. The outcome of a karate contest is often not the same as the outcome of a street fight between the same participants. There are any number of cases of a black belt karateka with ten years training getting into a fight with a scrawny untrained criminal, and getting his ass kicked in three seconds flat. Martial arts practitioners have had this testing problem for centuries and still don’t seem close to solving it, which doesn’t make for optimism about our prospects of solving the rationality testing problem this century. Given that, proceeding as best we can in the absence of a comprehensive and accurate test seems reasonable.
People spout all kinds of nonsense in a social context where it’s just words, but usually manage to compartmentalize the nonsense in a material context where they will be affected by the results of their actions.
But doesn’t it seem that if you decompartmentalized with correct beliefs you should do way better? Possibly in a testable way?
Martial arts practitioners have had this testing problem for centuries and still don’t seem close to solving it, which doesn’t make for optimism about our prospects of solving the rationality testing problem this century.
See MMA. There is still a problem of whether being a good fighter is as important or related to being good at self-defense, but martial arts are now measured at least relative to all fighting styles.
But doesn’t it seem that if you decompartmentalized with correct beliefs you should do way better?
Maybe; there are all sorts of caveats to that. But that aside, more directly on the question of tests:
Possibly in a testable way?
You still run into the problem that the outcome depends greatly on context and phrasing. There is the question with turning over cards to test a hypothesis, on which people’s performance dramatically improves when you rephrase it as an isomorphic question about social rules. There are the trolley questions and the specks versus torture question and the ninety-seven percent versus one hundred percent question, on which the right answer depends entirely on whether you treat it as a mathematical question that happens to be expressed in English syntax or a question about what you should do if you believed yourself to really be in that situation. There are questions about uncertain loss isomorphic to questions about uncertain gain where people nonetheless give different answers, which is irrational if considered as a material problem, but rational in the more likely and actual situation where the only thing at stake is social status, which sometimes does depend on how the question was phrased. Etc.
That’s why I called the testing problem ill posed; it’s not just that it’s hard to figure out the solution, it’s hard to see what would be the criteria of a good solution in the first place.
Those examples are good evidence for us not being able to test coherently yet, but I don’t think they are good evidence that the question is ill-posed.
If the question is “how can we test rationality?”, and the only answers we’ve come up with are limited in scope and subject to all kinds of misinterpretation, I don’t think that means we can’t come up with broad tests that measure progress. I am reminded of a quote: “what you are saying amounts to ‘if it is possible, it ought to be easy’”
I think the place to find good tests will be instead of looking at how well people do against particular biases, look at what it is we think rationality is good for, and measure something related to that.
Ill posed does not necessarily mean impossible. Most of the problems we deal with in real life are ill posed, but we still usually manage to come up with solutions that are good enough for the particular contexts at hand. What it does mean is that we shouldn’t expect the problem in question to be definitely solved once and for all. I’m not arguing against attempting to test rationality. I’m arguing against the position some posters have taken that there’s no point even trying to make progress on rationality until the problem of testing it has been definitely solved.
Testing rationality is something of an ill posed problem, in part because the result depends greatly on context. People spout all kinds of nonsense in a social context where it’s just words, but usually manage to compartmentalize the nonsense in a material context where they will be affected by the results of their actions. (This is a feature! Given that evolution wasn’t able to come up with minds that infallibly distinguish true beliefs from false ones, it’s good that at least it came up with a way to reduce the harm from false beliefs.) I’m not sure how to create an accurate test in the face of that.
Your martial arts analogy isn’t a bad one. The outcome of a karate contest is often not the same as the outcome of a street fight between the same participants. There are any number of cases of a black belt karateka with ten years training getting into a fight with a scrawny untrained criminal, and getting his ass kicked in three seconds flat. Martial arts practitioners have had this testing problem for centuries and still don’t seem close to solving it, which doesn’t make for optimism about our prospects of solving the rationality testing problem this century. Given that, proceeding as best we can in the absence of a comprehensive and accurate test seems reasonable.
But doesn’t it seem that if you decompartmentalized with correct beliefs you should do way better? Possibly in a testable way?
See MMA. There is still a problem of whether being a good fighter is as important or related to being good at self-defense, but martial arts are now measured at least relative to all fighting styles.
Maybe; there are all sorts of caveats to that. But that aside, more directly on the question of tests:
You still run into the problem that the outcome depends greatly on context and phrasing. There is the question with turning over cards to test a hypothesis, on which people’s performance dramatically improves when you rephrase it as an isomorphic question about social rules. There are the trolley questions and the specks versus torture question and the ninety-seven percent versus one hundred percent question, on which the right answer depends entirely on whether you treat it as a mathematical question that happens to be expressed in English syntax or a question about what you should do if you believed yourself to really be in that situation. There are questions about uncertain loss isomorphic to questions about uncertain gain where people nonetheless give different answers, which is irrational if considered as a material problem, but rational in the more likely and actual situation where the only thing at stake is social status, which sometimes does depend on how the question was phrased. Etc.
That’s why I called the testing problem ill posed; it’s not just that it’s hard to figure out the solution, it’s hard to see what would be the criteria of a good solution in the first place.
Those examples are good evidence for us not being able to test coherently yet, but I don’t think they are good evidence that the question is ill-posed.
If the question is “how can we test rationality?”, and the only answers we’ve come up with are limited in scope and subject to all kinds of misinterpretation, I don’t think that means we can’t come up with broad tests that measure progress. I am reminded of a quote: “what you are saying amounts to ‘if it is possible, it ought to be easy’”
I think the place to find good tests will be instead of looking at how well people do against particular biases, look at what it is we think rationality is good for, and measure something related to that.
Ill posed does not necessarily mean impossible. Most of the problems we deal with in real life are ill posed, but we still usually manage to come up with solutions that are good enough for the particular contexts at hand. What it does mean is that we shouldn’t expect the problem in question to be definitely solved once and for all. I’m not arguing against attempting to test rationality. I’m arguing against the position some posters have taken that there’s no point even trying to make progress on rationality until the problem of testing it has been definitely solved.
Ok, that’s reasonable. I was taking ill-posed to mean like a confused question. Or something like that.