Alternate Explanations for LW’s Calibration Atrociousness:
Maybe a lot of the untrained people simply looked up the answer to the question. If you did not rule that out with your study methods, then consider seeing whether a suspiciously large number of them entered the exact right year?
Maybe LWers were suffering from something slightly different from the overconfidence bias you’re hoping to detect: difficulty admitting that they have no idea when Thomas Bayes was born because they feel they should really know that.
The mean was 1768, the median 1780, and the mode 1800. Only 169 of 1006 people who answered the question got an answer within 20 years of 1701. Moreover, the three people that admitted to looking it up (and therefore didn’t give a calibration) all gave incorrect answers: 1750, 1759, and 1850. So it seems like your first explanation can’t be right.
After trying a bunch of modifications to the data, it seems like the best explanation is that the poor calibration happened because people didn’t think about the error margin carefully enough. If we change the error margin to 80 years instead of 20, then the responses seem to look roughly like the untrained example from the graph in Yvain’s analysis.
Another observation is that after we drop the 45 people who gave confidence levels >85% (and in fact, 89% of them were right), the remaining data is absolutely abysmal: the remaining answers are essentially uncorrelated with the confidence levels.
This suggests that there were a few pretty knowledgeable people who got the answer right and that was that. Everyone else just guessed and didn’t know how to calibrate; this may correspond to your second explanation.
Another thing I have noticed is that I tend to pigeonhole stuff into centuries; for example, once in a TV quiz there was a question “which of these pairs of people could have met” (i.e. their lives overlapped), I immediately thought “It can’t be Picasso and van Gogh: Picasso lived in the 20th century, whereas van Gogh lived in the 19th century.” I was wrong. Picasso was born in 1881 and van Gogh died in 1890. If other people also have this bias, this can help explain why so many more people answered 17xx than 16xx, thereby causing the median answer to be much later than the correct answer.
I hate the nth century convention because it doesn’t match up with the numbers used for the dates, so I always refer to the dates.… but that actually tends to confuse people.
I was going to say “the 1700s”, but that’s ambiguous as in principle it could refer either to a century or to its first decade. (OTOH, it would be more accurate, as my mental pigeonholes lump the year 1700 together with the following 99 years, not with the previous.)
Good points, Kindly, thank you. New alternate explanation idea:
When these people encounter this question, they’re slogging through this huge survey. They’re not doing an IQ test. This is more casual. They’re being asked stuff like “How many partners do you have?” By the time they get down to that question, they’re probably in a casual answering mode, and they’re probably a little tired and looking for an efficient way to finish. When they see the Bayes question, they’re probably not thinking “This question is so important! They’re going to be gauging LessWrong’s rationality progress with it! I had better really think about this!” They’re probably like “Output answer, next question.”
If we really want to test them, we need to make it clear that we’re testing them. And if we want them to be serious about it, we have to make it clear that it’s important. I hypothesize that if we were to do a test (not a survey) and explain that it’s serious because we’re gauging LessWrong’s progress, and also make it short so that the person can focus a lot of attention onto each question, we’d see less atrocious results.
In hindsight, I wonder why I didn’t think about the effects of context before. Yvain didn’t seem to either; he thought something might be wrong with the question. This seems like one of those things that is right in front of our faces but is hard to see.
If we really want to test them, we need to make it clear that we’re testing them. And if we want them to be serious about it, we have to make it clear that it’s important.
Uh, what? The point of LessWrong is to make people better all the time, not just better when they think “ah, now it’s time to turn on my rationality skills.” If people aren’t applying those skills when they don’t know they’re being tested, that’s a very serious problem, because it means the skills aren’t actually ingrained on the deep and fundamental level that we want.
You know that, Katydee, but do all the people who are taking the survey think that way? The majority of them haven’t even finished the sequences. I agree with you that it’s ideal for us to be good rationalists all the time, but mental stamina is a big factor.
Being rational takes more energy than being irrational. You have to put thought into it. Some people have a lot of mental energy. To refer to something less vague and more scientific: there are different levels of intelligence and different levels of intellectual supersensitivity (A term from Dabrowski that refers to how excitable certain aspects of your nervous system are.) Long story short: Some people cannot analyze constantly because it’s too difficult for them to do so. They run out of juice. Perhaps you are one of those rare people who has such high stamina for analysis that you rarely run into your limit. If that’s the case, it probably seems strange to you that anybody wouldn’t attempt to maintain a state of constant analysis. Most people with unusual intellectual stamina seem to view others as lazy when they observe that those other people aren’t doing intellectual things all the time. It frequently does not occur to them to consider that there may be an intellectual difference. The sad truth is that most people have much lower limits on how much intellectual activity they can do in a day than “constant”. If you want to see evidence of this, you can look at Ford’s studies where he shows that 40 hours a week is the optimum number of hours for his employees to work. Presumably, they were just doing factory work assembling car parts, which (if it fits the stereotype of factory work being repetitive) was probably pretty low on the scale for what’s intellectually demanding, but he found that if they tried to work 60 hours for two weeks in a row, their output would dip below the amount he’d normally get from 40 hours. This is because of mistakes. You’d think that the average human brain could do repetitive tasks constantly but evidently, even that tires the brain.
So in reality, the vast majority of people are not capable of the kind of constant meta-cognitive analysis that is required to be rational all the time. You use the word “ingrained” and I have seen Eliezer talk about how patterns of behavior can become habits (I assume he means that the thoughts are cached) and I think this kind of habit / ingrained response works beautifully when no decision-making is required and you can simply do the same thing that you usually do. But whenever one is trying to figure something out (like for instance working out the answers to questions on a survey) they’re going to need to put additional brainpower into that.
I had an experience where, due to unexpected circumstances, I developed some vitamin deficiencies. I would run out of mental energy very quickly if I tried to think much. I had, perhaps, a half an hour of analysis available to me in a day. This is very unusual for me because I’m used to having a brain that loves analysis and seems to want to do it constantly (I hadn’t tested the actual number of hours for which I was able to analyze, but I would feel bored if I wasn’t doing something like psychoanalysis or problem-solving for the majority of the day). When I was deficient, I began to ration my brainpower. That sounds terrible, but that is what I did. I needed to protect my ability to analyze to make sure I had enough left over to be able to do all the tasks I needed to do each day. I could feel that slipping away while I was working on problems and I could observe what happened to me after I fatigued my brain. (Vegetable like state.)
As I used my brainpower rationing strategies, it dawned on me that others ration brainpower, too. I see it all the time. Suddenly, I understood what they were doing. I understood why they kept telling me things like “You think too much!” They needed to change the subject so they wouldn’t become mentally fatigued. :/
Even if the average IQ at LessWrong is in the gifted range, that doesn’t give everyone the exact same abilities, and doesn’t mean that everyone has the stamina to analyze constantly. Human abilities vary wildly from person to person. Everyone has a limit when it comes to how much thinking they can do in a day. I have no way of knowing exactly what LessWrong’s average limit is, but I would not be surprised if most of them use strategies for rationing brainpower and have to do things like prioritize answering survey questions lower on their list of things to “give it their all” on, especially when there are a lot of them, and they’re getting tired.
Long story short: Some people cannot analyze constantly because it’s too difficult for them to do so. They run out of juice. Perhaps you are one of those rare people who has such high stamina for analysis that you rarely run into your limit.
Fascinating!
It’s making me realize why my summer project, which was to read Eat That Frog by Brian Tracey, was such a failure. The book is meant to be applied to work, preferably in an office environment–i.e. during your 40 productive work-hours. I was already working 40 hours a week at my extremely stimulating job as a nurse’s aid at the hospital, where I had barely any time to sit down and think about anything, and I certainly didn’t have procrastination problems. Then I would get home, exhausted with my brain about to explode from all the new interesting stuff I’d been seeing and doing all day, and try to apply Brian Tracey’s productivity methods to the personal interest projects I was doing in my spare time.
This was a very efficient way to make these things not fun, make me feel guilty about being a procrastinator, etc. It gave me an aversion to starting projects, because the part of my brain that likes and needs to do something easy and fun after work knew it would be roped into doing something mentally tiring, and that it would be made to feel guilty over not wanting to do it.
I’m hoping that once I’m graduated and work as a nurse for a year or two, so that I have a chance to get accustomed to a given unit and don’t have to spend so much mental effort, I’ll have more left over for outside interests and can start reading about physics and programming for fun again. (Used to be able to do this in first and second year, definitely can’t now.)
I’m glad you seem to have benefited from my explanation. If you want to do mentally draining reading, maybe weekends or later on in the evenings after you’ve rested would be a good time for that? If you’ve rested first, you might be able to scrape up a little extra juice.
Of course everyone has their own mental stamina limit, so nobody can tell you whether you do or don’t have enough stamina to do additional intellectual activities after work. And it may vary day to day, as work is not likely to demand the exact same amount of brainpower every day.
An interesting experiment would be to see if there’s anything that restores your stamina like a bath, a 20 minute nap after work, meditation, watching TV, or playing a fun game. Simply laying down in a dark quiet place does wonders for me if I am stressed out or fatigued. I would love to see someone log their mental stamina over time and correlate that to different activities that might restore stamina.
There are also stress reduction techniques that may help prevent you from losing stamina in the first place that could be interesting to experiment with.
And if you’re not taking 15 minute breaks every 90 minutes during work, you might be “over-training” your brain. Over-training might result in an amplification of fatigue. “The Power of Full Engagement: Manage Energy Not Time” is likely to be of interest.
If you decide to do mental stamina experiments, definitely let me know!
An interesting experiment would be to see if there’s anything that restores your stamina like a bath, a 20 minute nap after work, meditation, watching TV, or playing a fun game.
I’ve also found that pouring lots of cold water on my face helps me squeeze out the last drops of stamina I have left, and allow me to work twenty more minutes or so. (It doesn’t actually restore stamina, so it doesn’t work if I do that more than a couple times in a row.)
Hmmm. That might be one or a combination of the following:
Taking a five minute break.
Enjoying physical sensation. (Enjoyment seems to restore stamina for me, perhaps that’s because the brain uses neurotransmitters for processing, and triggering pleasure involves increasing the amount of certain neurotransmitters.)
Fifteen minute breaks are supposed to be optima, and if you maximized pleasure during your break, I wonder what amount of stamina that would restore?
Re the problem of having to think all the time: a good start is to develop a habit of rejecting certainty about judgments and beliefs that you haven’t examined sufficiently (that is, if your intuition shouts at you that something is quite clear, but you haven’t thought about that for a few minutes, ignore (and mark as a potential bug) that intuition unless you understand a reliable reason to not ignore it in that case). If you don’t have the stamina or incentives to examine such beliefs/judgments in more detail, that’s all right, as long as you remain correspondingly uncertain, and realize that the decisions you make might be suboptimal for that reason (which should suitably adjust your incentives for thinking harder, depending on the importance of the decisions).
The process of choosing a probability is not quite that simple. You’re not just making a boolean decision about whether you know enough to know, you’re actually taking the time to distinguish between 10 different amounts of confidence (10%, 20%, 30%, etc), and then making ten more tiny distinctions (30%, 31%, 32% for instance)… at least that’s the way that I do it. (More efficient than making enough distinctions to choose between 100 different options.) When you are wondering exactly how likely you are to know something in order to choose a percentage, that’s when you have to start analyzing things. In order to answer the question, my thought process looked like this:
Bayes. I have to remember who that is. Okay, that’s the guy that came up with Bayesian probability. (This was instant, but that doesn’t mean it took zero mental work.)
Do I have his birthday in here? Nothing comes to mind.
Digs further: Do I have any reason to have read about his birthday at any point? No. Do I remember seeing a page about him? I can’t remember anything I read about his birthday.
Considers whether I should just go “I don’t know” and put a random year with a 0% probability. Decides that this would be copping out and I should try to actually figure this out.
When was Bayesian probability invented? Let’s see… at what point in history would that have occurred?
Try to brainstorm events that may have required Bayesian probability, or that would have suggested it didn’t exist yet.
Try to remember the time periods for when these events happened.
Defines a vague section of time in history.
Considers whether there might be some method of double-checking it.
Considers the meaning of “within 20 years either way” and what that means for the probability that I’m right.
Figures out where in my vague section of time the 40 year range should be fit.
Figures out which year is in the middle of the 40 year range and types it in.
Consider how many years Bayes would likely have to have lived for before giving his theorems to the world and adjust the year to that.
Considers whether it was at all possible for Bayesian probability to have existed before or after each event.
If possible, consider how likely it was that Baye’s probability existed before/after each event.
Calculate how many 40-year ranges there are in the vague section of time between the events where Bayes could not have been born.
Calculate the chance that I chose the correct 40-year section out of all the possible sections, if odds are equal.
Compare this to my probabilities regarding how likely it was for Bayes theorem to have existed before and after certain events.
Adjust my probability figure to take all that into account.
My answer to this question took at least twenty steps, and that doesn’t even count all the steps I went through for each event, nor does it count all the sub steps I went through for things that I sort of hand-waved like “Adjust my probability figure to take all that into account”.
If you think figuring out stuff is instant, you underestimate the number of steps your brain does in order to figure things out. I highly recommend doing meditation to improve your meta-cognition. Meta-cognition is awesome.
you’re actually taking the time to distinguish between 10 different amounts of confidence (10%, 20%, 30%, etc), and then making ten more tiny distinctions (30%, 31%, 32% for instance)… at least that’s the way that I do it
The straightforward interpretation of your words evaluates as a falsity, as you can’t estimate informal beliefs to within 1%.
I’d put it more in terms of decibels of log-odds than percentages of probability. Telling 98% from 99% (i.e. +17 dB from +20 dB) sounds easier to me than telling 50% from 56% (i.e. 0 dB from +1 dB).
No, I’m pretty certain you can’t. You can’t even formulate truth conditions for correctness of such an evaluation. Only in very special circumstances getting to that point would be plausible (when a conclusion is mostly determined by data that is received in an explicit form or if you work with a formalizable specification of a situation, as in probability theory problems; this is not what I meant by “informal beliefs”).
If you think figuring out stuff is instant, you underestimate the number of steps your brain does in order to figure things out.
(I was commenting on a skill/habit that might be useful in the situations where you don’t/can’t make the effort of explicitly reasoning about things. Don’t fight the hypothetical.)
(I was commenting on a skill that might be useful in the situations where you don’t/can’t make the effort of explicitly reasoning about things. Don’t fight the hypothetical.)
Is it your position that there is a thinking skill that is actually accurate for figuring stuff out without thinking about it?
I expect you can improve accuracy in the sense of improving calibration, by reducing estimated precision, avoiding unwarranted overconfidence, even when you are not considering questions in detail, if your intuitive estimation has an overconfidence problem, which seems to be common (more annoying in the form of an “The solution is S!” for some promptly confabulated arbitrary S, when quantifying uncertainty isn’t even on the agenda).
(I feel the language of there being “positions” has epistemically unhealthy connotations of encouraging status quo bias with respect to beliefs, although it’s clear what you mean.)
Being rational takes more energy than being irrational. You have to put thought into it. Some people have a lot of mental energy. To refer to something less vague and more scientific: there are different levels of intelligence and different levels of intellectual supersensitivity (A term from Dabrowski that refers to how excitable certain aspects of your nervous system are.) Long story short: Some people cannot analyze constantly because it’s too difficult for them to do so. They run out of juice. Perhaps you are one of those rare people who has such high stamina for analysis that you rarely run into your limit. If that’s the case, it probably seems strange to you that anybody wouldn’t attempt to maintain a state of constant analysis.
The point is to make these things automatic so that one doesn’t have to analyze all the time. I definitely don’t feel like I “maintain a state of constant analysis,” even when applying purportedly advanced rationality techniques. It basically feels the same as thinking about things normally, except that I am right more often.
So in reality, the vast majority of people are not capable of the kind of constant meta-cognitive analysis that is required to be rational all the time. You use the word “ingrained” and I have seen Eliezer talk about how patterns of behavior can become habits (I assume he means that the thoughts are cached) and I think this kind of habit / ingrained response works beautifully when no decision-making is required and you can simply do the same thing that you usually do. But whenever one is trying to figure something out (like for instance working out the answers to questions on a survey) they’re going to need to put additional brainpower into that.
I don’t believe that your claim is true, but if it is I think LessWrong is doomed as a concept. I frankly do not think people will be able to accurately evaluate when they need to apply thinking skills to their decisions, so if we cannot teach skills on this level—teach habits, as you say—I do not think LessWrong will ever accomplish anything of real worth.
One example of a skill that I have taken on on this level is reference class forecasting. If I need to estimate how long something will take, my go-to method is to take the outside view. I am so used to this that it is now the automatic response to questions of estimating times.
As I used my brainpower rationing strategies, it dawned on me that others ration brainpower, too. I see it all the time. Suddenly, I understood what they were doing. I understood why they kept telling me things like “You think too much!” They needed to change the subject so they wouldn’t become mentally fatigued. :/
I don’t use “brainpower rationing” because I frankly have never felt the need to do so. I have told people that they “think too much” under certain circumstances (most notably when thinking is impeding action), and the thought of “brainpower rationing” has never come to mind until I saw this post.
Maybe I misinterpreted here but it sounds like you’re saying you don’t believe in mental stamina limits? Maybe you mean that you don’t think rationality requires much brainpower?
but if it is I think LessWrong is doomed as a concept.
I don’t think we’d be doomed, and there are a few reasons for that:
There are people in existence who really can analyze pretty much constantly. THOSE people would theoretically have a pretty good chance of being rational all the time.
People who cannot analyze anywhere near constantly can simply choose their battles. If they’re aware of their mental stamina limits, they can work with them. Realizing you don’t know stuff and that you don’t have enough mental stamina to figure it out right now is kind of sad but it is still perfectly rational, so perhaps rationalists with low mental stamina can still be good rationalists that way.
There are things that decrease mental fatigue. For instance, taking 15 minute breaks every 90 minutes (The book “The power of full engagement: manage energy not time” talks about this). We could do experiments on ourselves to find out what other things reduce or prevent mental fatigue. There may be low-hanging apples we’re totally unaware of.
One example of a skill that I have taken on on this level is reference class forecasting. If I need to estimate how long something will take, my go-to method is to take the outside view. I am so used to this that it is now the automatic response to questions of estimating times.
Okay, so you’ve learned to instantly go to a certain method. I can believe that this does not take much brainpower. However, how much brainpower does it take to execute the outside view method, on average, for the types of things you use it for? How many times can you execute the outside view in a day? Have you ever tried to reach your mental stamina limit?
I don’t use “brainpower rationing” because I frankly have never felt the need to do so.
Do you ever get home from work and feel relieved that you can relax now, and then do something that’s not mentally taxing? Do you ever find that you’re starting to hate an activity, and notice you’re making more and more mistakes? Do you ever feel lazy and can’t be bothered to do anything useful? I bet you do experience mental fatigue but don’t recognize it as such. A lot of people just berate themselves for being unproductive, and don’t consciously recognize that they’ve hit a real limit.
therefore he must have been born in the early 1700s.
20 year margin means I don’t have to be precise.
answer: 1700 or 1705 (can’t remember which I put)
get question right
The more difficult part was the probability estimate. But using the heuristics taught to me by this book, this took only a few calculations. And the more I do these types of calculations, the faster and more calibrated I become. Eventually I hope to make them automatic at the 8 + 4 = 12 level.
If I were doing the calculation “for real” and not on a survey my algorithm would be much easier:
see name
copy/paste name into Google
look at Wikipedia
look at other sources to confirm (if important)
Maybe I misinterpreted here but it sounds like you’re saying you don’t believe in mental stamina limits?
I know they exist on some level thanks to my experience with dual n-back, but I’ve yet to encounter any practical situation that imposes them (aside from “getting tired, which is different), and if I did I’m sure I could train my way out, just as I trained my way out of certain physical stamina limits. For example, it was once hard for me to maintain my endurance throughout a full fencing bout, but following some training I can do several in a row without becoming seriously fatigued. I’m sure better fencers than me can do even more.
LessWrong and CFAR, in my view, should provide the mental equivalent of that training if it is indeed necessary for the practice of rationality. I’m not, however, convinced that it is.
However, how much brainpower does it take to execute the outside view method, on average, for the types of things you use it for? How many times can you execute the outside view in a day? Have you ever tried to reach your mental stamina limit?
Immeasurably small (no perceived effort and takes less time than the alternative)/indeterminate/not in this respect. Most of the effort was involved in correctly identifying situations in which the method was useful, not in actually executing the method, but once the method became sufficiently ingrained that too went away.
Do you ever get home from work and feel relieved that you can relax now, and then do something that’s not mentally taxing?
No. My work is generally fun.
Do you ever find that you’re starting to hate an activity, and notice you’re making more and more mistakes?
Not really. Sometimes I get bored, does that count?
Do you ever feel lazy and can’t be bothered to do anything useful?
Nancy, there is already a term for this. It’s “intellectual overexcitability” or “intellectual supersensitivity”. These are terms from Dabrowski. Look up the “Theory of Positive Disintegration” to learn more.
Those terms seem like pathologizing—which is not surprising, considering that Dabrowski puts emphasis on the difficulties of the path. I was thinking more of the idea that some people like thinking more than others, just as some people like moving around more than others, which is something much less intense.
I was wondering whether Dabrowski was influenced by Gurdjieff, and it turns out that he was.
I’m not sure I can reliably recognize what mental fatigue feels like. I’d like to be able to diagnose it in myself (because I suspect that I have less mental energy than I used to), so do you know of any reasonably quick way to induce something that feels like mental fatigue, e.g. alcohol?
Whatever your worst subject is, do a whole bunch of exercises in it until you start making so many mistakes it is not worth continuing. No need for alcohol, might as well wear out your brain.
It would be interesting to see if you’d get different types of fatigue from doing different kinds of activities. For instance, if I do three hours of math problems, I have trouble speaking after that—it’s like my symbol manipulation circuitry is fried. (I have dyslexia, so that’s probably related.) If I wear out my verbal processor (something that I think only started happening to me after I developed some unexpected vitamin deficiencies) this results in irritation. I can’t explain myself very well, so people jump on me for mistakes, and it’s really hard to tell them what I meant instead, so I get frustrated.
So, exercising each area of mental abilities might yeild different fatigue symptoms.
If you decide to experiment on yourself I’m definitely curious about your results!
If I’ve been reading/studying too long, I find much harder to concentrate and am more easily distracted by stray thoughts.
If I’ve been writing computer code/doing maths too long, I make the kind of trivial mistakes that screw up the results but are hard to locate way more often.
How much can you do of each activity before becoming fatigued?
It depends—usually between 20 minutes and 3 hours.
When these people encounter this question, they’re slogging through this huge survey. They’re not doing an IQ test. This is more casual. They’re being asked stuff like “How many partners do you have?” By the time they get down to that question, they’re probably in a casual answering mode, and they’re probably a little tired and looking for an efficient way to finish… If we really want to test them, we need to make it clear that we’re testing them.
My first thought about this is that people’s rationality ‘in real life’ totally is determined by how likely they are to notice a Bayes question in an informal setting, where they may be tired and feeling mentally lazy. In Keith Stanovich’s terms, rationality is mostly about the reflective mind: it’s someone’s capacity and habits to re-compute a problem’s answer, using the algorithmic mind, rather than accept the intuitive default answer that their autonomous mind spits out.
IQ tests tend to be formal; it’s very obvious that you’re being tested. They don’t measure rationality in the sense that most LWers mean it; the ability to apply thinking techniques to real life in order to do better.
It might still be valuable to know how LWers do on a more formal test of probability-related knowledge; after all, most people in the general public don’t know Bayes’ theorem, so it’d be neat to see how good LW is at increasing “rationality literacy”. But that’s not the ultimate goal. There are reasons why you might want to measure a group’s ability to pick out unexpected rationality-related problems and activate the correct mindware. If your Bayesian superpowers only activate when you’re being formally tested, they’re not all that useful as superpowers.
I can see why you’d criticize someone for saying “the problem is that the setting wasn’t formal enough” but that’s not exactly what I was getting at. What I was getting at is that there’s a limit to how much thinking that one can do in a day, everyone’s limit is different, and a lot of people do things to ration their brainpower so they avoid running out of it. This comment on mental stamina explains more.
My point was, more clearly worded: It would be a very rare person who possesses enough mental stamina to be rational in literally every single situation. That’s a wonderful ideal, but the reality is that most people are going to ration brainpower. If your expectation is that rationalists should never ration brainpower and should be rational constantly, this is an unrealistic expectation. A more realistic expectation is that people should identify the things they need to think extra hard about, and correctly use rational thinking skills at those times. Therefore, testing for the skills when they’re trying is probably the only way to detect a difference. There are inevitably going to be times when they’re not trying very hard, and if you catch them at one of those times, well, you’re not going to see rational thinking skills. It may be that some of these things can be ingrained in ways that don’t use up a person’s mental stamina, but to expect that rationality can be learned in such a way that it is applied constantly strikes me as an unreasoned assumption.
Now I wonder if the entire difference between the control groups results and LessWrong’s results was that Yvain asked the control group only one question, whereas LessWrong had answered 14 pages of questions prior to that.
Agreed that rationality is mentally tiring...I went back and read your comment, too. However:
A more realistic expectation is that people should identify the things they need to think extra hard about, and correctly use rational thinking skills at those times.
To me, rationality is mostly the ability to notice that “whew, this is a problem that wasn’t in the problem-set of the ancestral environment, therefore my intuitions probably won’t be useful and I need to think”. The only way a rationalist would have to be analytical all the time is if they were very BAD at doing this, and had to assume that every situation and problem required intense thought. Most situations don’t. In order to be an efficient rationalist, you have to be able to notice which situations do.
Any question on a written test isn’t a great measure of real-life rationality performance, but there are plenty of situations in everyday life when people have to make decisions based on some unknown quantities, and would benefit from being able to calibrate exactly how much they do know. Some people might answer better on the written test than if faced with a similar problem in real life, but I think it’s unlikely that anyone would do worse on the test than in real life.
Re having to think all the time: a good start is to develop a habit of rejecting certainty about judgments and beliefs that you haven’t examined sufficiently (that is, if your intuition shouts at you that something is quite clear, but you haven’t thought about that for a few minutes, ignore that intuition unless you have a reliable reason to not ignore it in that case). If you don’t have stamina or incentives to examine such beliefs/judgments in more detail, that’s all right, as long as you remain correspondingly uncertain, and realize that the decisions you make might be suboptimal for that reason (which should suitably adjust your incentives for thinking harder, depending on the importance of the decisions).
someone’s capacity and habits to re-compute a problem’s answer, using the algorithmic mind, rather than accept the intuitive default answer that their autonomous mind spits out.
I don’t think you could really apply any ‘algorithmic’ method to that question (other than looking it up, but that would be cheating). It was a test on how much confidence you put in your heuristics. (BTW, It seems that I’ve underestimated mine, or I’ve been lucky, since I’ve got the date off by one year but estimated my confidence at 50% IIRC). Still, it was a valuable test, since most of human reasoning is necessarily heuristic.
most people in the general public don’t know Bayes’ theorem
Really? What probability do you assign to that statement being true? :D
I’m under the impression that Bayes’ theorem is included in the high school math programs of most developed countries, and I’m certain it is included in any science and engineering college program.
most people in the general public don’t know Bayes’ theorem
Really? What probability do you assign to that statement being true? :D
I assign about 80% probability to less than 25% of adults knowing Bayes theorem and how to use it. I took physics and calculus and other such advanced courses in high school, and graduated never having heard of Bayes’ Theorem. I didn’t learn about it in university, either–granted, I was in ‘Statistics for Nursing’, it’s possible that the ‘Statistics for Engineering’ syllabus included it.
Even bumping the 30% up to the 56% who have “some college” and using the 44% for a estimate of the true ratio of possible-Bayes’-knowledge, that’s only just 25% of the US adult population.
(I’ve no idea how this extends to the rest of the world, the US data was easiest to find.)
You did your research and earned your confidence level. I didn’t look anything up, just based an estimate on anecdotal evidence (the fact that I didn’t learn it in school despite taking lots of sciences). Knowing what you just told me, I would update my confidence level a little–I’m probably 90% sure that less than 25% of adults know Bayes Theorem. (I should clarify that=adults living in the US, Canada, Britain, and other countries with similar school systems. The percentage for the whole world is likely significantly lower.)
I have heard that the US system is particularly bad for an advanced country.
In terms of outcomes, the US does pretty terribly when considered 1 country, but when split into several countries it appears at the top of each class. Really, the EU is cheating by considering itself multiple countries.
Actually it is quite good (even for an “advanced country”) if you compare the test scores of, say, Swedes and Swedish-Americans rather than Swedes and Americans as a whole.
If you choose maths as one of your A-levels, there’s a good chance you will cover stats 1 which includes the formula for Bayes’ Theorem and how to apply it to calculate medical test false positives/false negatives (and equivalent problems). However it isn’t named and the significance to science/rationality is not explained, so it’s just seen as “one more formula to learn”.
Offhand, 1⁄2 young people do A levels, 1⁄4 of those do maths, and 2⁄3 of those do stats, giving us 1⁄12 of young people. I don’t think any of these numbers are off by enough to push the fraction over 25%
As far as I know, it’s been formally demonstrated to be the absolutely mathematically-optimal method of achieving maximal hypothesis accuracy in an environment with obscured, limited or unreliable information.
That’s basically saying: “There is no possible way to do better than this using mathematics, and as far as we know there doesn’t yet exist anything more powerful than mathematics.”
What more could you want? A theorem proving that any optimal decision theory must necessarily use Bayesian updating? ETA: It has been pointed out that there already exists such a theorem. I could’ve found that out by looking it up. Oops.
What more could you want? A theorem proving that any optimal decision theory must necessarily use Bayesian updating?
There already is such a theorem. From Wikipedia:
A decision-theoretic justification of the use of Bayesian inference was given by Abraham Wald, who proved that every Bayesian procedure is admissible. Conversely, every admissible statistical procedure is either a Bayesian procedure or a limit of Bayesian procedures.
As far as I can tell from wikipedia’s description of admissibility, it makes the same assumptions as CDT: That the outcome depends only on your action and the state of the environment, and not on any other properties of your algorithm. This assumption fails in multi-player games.
So your quote actually means: If you’re going to use CDT then Bayes is the optimal way to derive your probabilities.
And the list of notable problems that have been solved using Bayes is...? Bayes doesn’t tell you how to make your informaton more copious or accuate, although there are plenty of techniques for doing that. Bayes also doesn’t tell you how to formulate novel hypotheses. It also doens’t tell you how to deal with conceptual problems that are not yet suitable for nnumber crucnhing. It looks to me like Bayes is actually a rather small part of the picture.
And the list of notable problems that have been solved using Bayes is...?
Half of statistics these days is Bayesian. Do you want to defend the claim that statistics solves no notable problems?
PS: -T-w-o- Three downvotes, and not a shred of counteargument. Typical.
As usual, I add my downvote to whining about downvotes. Since you think it’s ‘typical’ and this vindicates your claims, I’m sure you’ll be pleased that I’m helping prove you right.
Great. Then the UK education sytem is exactly right in teaching Bayes as part of statitistics, but not as a general-prupose solution to everything. ETA: But surely the LW take on Bayes is that it is much more than something useful in statistics.
Do you want to defend the claim that statistics solves no notable problems?
No, I want to defend the claims that Bayes is not as a general-prupose solution to everything, is not a substitute for other congnitive disciplines, is of no benefit to many people and is of no use in many contexts.
As usual, I add my downvote to whining about downvotes
Please inform me of the correct way to indicate that the karma system is being misused.
Great. Then the UK education sytem is exactly right in teaching Bayes as part of statitistics, but not as a general-prupose solution to everything. ETA: But surely the LW take on Bayes is that it is much more than something useful in statistics.
Now you’re just backing off your claim. What happened to your list?
Then the UK education sytem is exactly right in teaching Bayes as part of statitistics, but not as a general-prupose solution to everything.
First point: if Bayesian statistics is half of statistics, the description of the UK course is of it as being way way less than half the course. Therefore the UK system is very far from being ‘exactly right’.
Second point: The optimistic meta-induction is that Bayesian statistics has gone from being used by a literal handful of statisticians to being widespread and possibly a majority now or in the near future; therefore, it will continue spreading and eating more of statistics in general, and the course will get wronger and wronger, and your claims less and less right.
No, I want to defend the claims that Bayes is not as a general-prupose solution to everything, is not a substitute for other congnitive disciplines, is of no benefit to many people and is of no use in many contexts.
So you’re just splashing around a lot of bullshit and distractions when you demand lists and talk about the UK course being exactly right, since those aren’t what you are actually trying to claim. Good to know!
Please inform me of the correct way to indicate that the karma system is being misused.
What’s the point of indicating when it’s not being misused?
What’s the point of indicating when it’s not being misused?
You have your opinion, on that, I have mine. You can state your opinion, I can’t state mine. I can’t discuss the censorship, because discussions of censorship are censored.
It’s in a downvoted thread.So it isn’t visible.If negative karma doesn’t do anything regarding the visibility of comments, why have the button? Sheesh.
And so begins another goal-shifting, like the list or like the claim of ‘censorship’, this time to defining karma systems. Pardon me if I don’t care to continue this game.
I took physics and calculus and other such advanced courses in high school, and graduated never having heard of Bayes’ Theorem.
Must be a problem of the American school system, I suppose.
I didn’t learn about it in university, either–granted, I was in ‘Statistics for Nursing’, it’s possible that the ‘Statistics for Engineering’ syllabus included it.
Did they teach you about conditional probability? Usually Bayes’ theorem is introduced right after the definition of conditional probability.
most people in the general public don’t know Bayes’ theorem
Really? What probability do you assign to that statement being true? :D
There are national and international surveys of quantitative literacy in adults. The U.S. does reasonably well in these, but in general the level of knowledge is appalling to math teachers. See this pdf (page 118 of the pdf, the in-text page number is “Section III, 93”) for the quantitative literacy questions, and the percentage of the general population attaining each level of skill. less than a fifth of the population can handle basic arithmetic operations to perform tasks like this:
One task in this level, with a difficulty value of 332, asks the reader to
estimate, based on information in a news article, how many miles per day a
driver covered in a sled-dog race. The respondent must know that to calculate
a “ per day” rate requires the use of division.
A more difficult task (355) requires the reader to select from two unit
price labels to estimate the cost per ounce of creamy peanut butter. To perform
this task successfully, readers may have to draw some information from prior
knowledge.
People who haven’t learned and retained basic arithmetic are not going to have a grasp of Bayes’ theorem.
I’m under the impression that Bayes’ theorem is included in the high school math programs of most developed countries, and I’m certain it is included in any science and engineering college program.
It was in my high school curriculum (in Italy, in the mid-2000s), but the teacher spent probably only 5 minutes on it, so I would be surprised if a nontrivial number of my classmates who haven’t also heard of it somewhere else remember it from there. IIRC it was also briefly mentioned in the part about probability and statistics of my “introduction to physics” course in my first year of university, but that’s it. I wouldn’t be surprised if more than 50% of physics graduates remember hardly anything about it other than its name.
most people in the general public don’t know Bayes’ theorem
Really? What probability do you assign to that statement being true? :D
I’m under the impression that Bayes’ theorem is included in the high school math programs of most developed countries, and I’m certain it is included in any science and engineering college program.
I’m pretty sure Ireland doesn’t have it on our curriculum, not sure how typical we are.
If you’ll excuse the expression, I’m suspicious of your sudden epiphany. That is, I accept your suggestion as a possible explanation (although I’m not convinced, mainly because this doesn’t describe the way I answered the question; I don’t know about anyone else). But I think saying “Oh gosh! The true answer has been staring us in the face all along!” is premature.
I am not sure why you took “a new explanation” so seriously. I guess I have to be really careful on LessWrong to distinguish ideas from actual beliefs. I do not think it’s “The True Answer”. I just think it’s a rather obvious alternate explanation that should have occurred to me immediately, and didn’t, and I’m surprised about that, and about the fact that it didn’t seem to occur to Yvain either. I reworded some things to make it more obvious that I am not trying to present this as “The True Answer” but just as an idea.
Would you mind trying to avoid jumping to the conclusion that I’m acting stupid in the future, Kindly? I definitely don’t mind being told “Your statement could be interpreted as such-and-such stupid behavior, so you may want to change it.” but it’s a little frustrating when people speak to me as if they really believe I am as confused as your “The True Answer” interpretation would imply.
I’m not sure why you’re accusing me of this. I often disagree with people, but I usually don’t assume the people I disagree with are stupid. This is especially true when we disagree due to a misunderstanding.
(I don’t intend to continue this line of conversation.)
Well it’s my perception that it would be pretty stupid to jump to the conclusion that I had it all figured out just out of nowhere up there. If that’s not your perception, too, then it’s not—but that would be unexpected to me who holds the perception that it would be a kind of stupid thing to do. I don’t know what wording to use other than “Please try not to jump to the conclusion that I’m doing stupid things.” but just substitute “stupid” for whatever word you would use, and then please try not to jump to conclusions that I am doing whatever it is that you call that, okay?
Alternate Explanations for LW’s Calibration Atrociousness:
Maybe a lot of the untrained people simply looked up the answer to the question. If you did not rule that out with your study methods, then consider seeing whether a suspiciously large number of them entered the exact right year?
Maybe LWers were suffering from something slightly different from the overconfidence bias you’re hoping to detect: difficulty admitting that they have no idea when Thomas Bayes was born because they feel they should really know that.
The mean was 1768, the median 1780, and the mode 1800. Only 169 of 1006 people who answered the question got an answer within 20 years of 1701. Moreover, the three people that admitted to looking it up (and therefore didn’t give a calibration) all gave incorrect answers: 1750, 1759, and 1850. So it seems like your first explanation can’t be right.
After trying a bunch of modifications to the data, it seems like the best explanation is that the poor calibration happened because people didn’t think about the error margin carefully enough. If we change the error margin to 80 years instead of 20, then the responses seem to look roughly like the untrained example from the graph in Yvain’s analysis.
Another observation is that after we drop the 45 people who gave confidence levels >85% (and in fact, 89% of them were right), the remaining data is absolutely abysmal: the remaining answers are essentially uncorrelated with the confidence levels.
This suggests that there were a few pretty knowledgeable people who got the answer right and that was that. Everyone else just guessed and didn’t know how to calibrate; this may correspond to your second explanation.
Another thing I have noticed is that I tend to pigeonhole stuff into centuries; for example, once in a TV quiz there was a question “which of these pairs of people could have met” (i.e. their lives overlapped), I immediately thought “It can’t be Picasso and van Gogh: Picasso lived in the 20th century, whereas van Gogh lived in the 19th century.” I was wrong. Picasso was born in 1881 and van Gogh died in 1890. If other people also have this bias, this can help explain why so many more people answered 17xx than 16xx, thereby causing the median answer to be much later than the correct answer.
I hate the nth century convention because it doesn’t match up with the numbers used for the dates, so I always refer to the dates.… but that actually tends to confuse people.
I was going to say “the 1700s”, but that’s ambiguous as in principle it could refer either to a century or to its first decade. (OTOH, it would be more accurate, as my mental pigeonholes lump the year 1700 together with the following 99 years, not with the previous.)
Good points, Kindly, thank you. New alternate explanation idea:
When these people encounter this question, they’re slogging through this huge survey. They’re not doing an IQ test. This is more casual. They’re being asked stuff like “How many partners do you have?” By the time they get down to that question, they’re probably in a casual answering mode, and they’re probably a little tired and looking for an efficient way to finish. When they see the Bayes question, they’re probably not thinking “This question is so important! They’re going to be gauging LessWrong’s rationality progress with it! I had better really think about this!” They’re probably like “Output answer, next question.”
If we really want to test them, we need to make it clear that we’re testing them. And if we want them to be serious about it, we have to make it clear that it’s important. I hypothesize that if we were to do a test (not a survey) and explain that it’s serious because we’re gauging LessWrong’s progress, and also make it short so that the person can focus a lot of attention onto each question, we’d see less atrocious results.
In hindsight, I wonder why I didn’t think about the effects of context before. Yvain didn’t seem to either; he thought something might be wrong with the question. This seems like one of those things that is right in front of our faces but is hard to see.
I think that people may be rationing their mental stamina, and may not be going through all the steps it takes to answer this type of question.
Uh, what? The point of LessWrong is to make people better all the time, not just better when they think “ah, now it’s time to turn on my rationality skills.” If people aren’t applying those skills when they don’t know they’re being tested, that’s a very serious problem, because it means the skills aren’t actually ingrained on the deep and fundamental level that we want.
You know that, Katydee, but do all the people who are taking the survey think that way? The majority of them haven’t even finished the sequences. I agree with you that it’s ideal for us to be good rationalists all the time, but mental stamina is a big factor.
Being rational takes more energy than being irrational. You have to put thought into it. Some people have a lot of mental energy. To refer to something less vague and more scientific: there are different levels of intelligence and different levels of intellectual supersensitivity (A term from Dabrowski that refers to how excitable certain aspects of your nervous system are.) Long story short: Some people cannot analyze constantly because it’s too difficult for them to do so. They run out of juice. Perhaps you are one of those rare people who has such high stamina for analysis that you rarely run into your limit. If that’s the case, it probably seems strange to you that anybody wouldn’t attempt to maintain a state of constant analysis. Most people with unusual intellectual stamina seem to view others as lazy when they observe that those other people aren’t doing intellectual things all the time. It frequently does not occur to them to consider that there may be an intellectual difference. The sad truth is that most people have much lower limits on how much intellectual activity they can do in a day than “constant”. If you want to see evidence of this, you can look at Ford’s studies where he shows that 40 hours a week is the optimum number of hours for his employees to work. Presumably, they were just doing factory work assembling car parts, which (if it fits the stereotype of factory work being repetitive) was probably pretty low on the scale for what’s intellectually demanding, but he found that if they tried to work 60 hours for two weeks in a row, their output would dip below the amount he’d normally get from 40 hours. This is because of mistakes. You’d think that the average human brain could do repetitive tasks constantly but evidently, even that tires the brain.
So in reality, the vast majority of people are not capable of the kind of constant meta-cognitive analysis that is required to be rational all the time. You use the word “ingrained” and I have seen Eliezer talk about how patterns of behavior can become habits (I assume he means that the thoughts are cached) and I think this kind of habit / ingrained response works beautifully when no decision-making is required and you can simply do the same thing that you usually do. But whenever one is trying to figure something out (like for instance working out the answers to questions on a survey) they’re going to need to put additional brainpower into that.
I had an experience where, due to unexpected circumstances, I developed some vitamin deficiencies. I would run out of mental energy very quickly if I tried to think much. I had, perhaps, a half an hour of analysis available to me in a day. This is very unusual for me because I’m used to having a brain that loves analysis and seems to want to do it constantly (I hadn’t tested the actual number of hours for which I was able to analyze, but I would feel bored if I wasn’t doing something like psychoanalysis or problem-solving for the majority of the day). When I was deficient, I began to ration my brainpower. That sounds terrible, but that is what I did. I needed to protect my ability to analyze to make sure I had enough left over to be able to do all the tasks I needed to do each day. I could feel that slipping away while I was working on problems and I could observe what happened to me after I fatigued my brain. (Vegetable like state.)
As I used my brainpower rationing strategies, it dawned on me that others ration brainpower, too. I see it all the time. Suddenly, I understood what they were doing. I understood why they kept telling me things like “You think too much!” They needed to change the subject so they wouldn’t become mentally fatigued. :/
Even if the average IQ at LessWrong is in the gifted range, that doesn’t give everyone the exact same abilities, and doesn’t mean that everyone has the stamina to analyze constantly. Human abilities vary wildly from person to person. Everyone has a limit when it comes to how much thinking they can do in a day. I have no way of knowing exactly what LessWrong’s average limit is, but I would not be surprised if most of them use strategies for rationing brainpower and have to do things like prioritize answering survey questions lower on their list of things to “give it their all” on, especially when there are a lot of them, and they’re getting tired.
Fascinating!
It’s making me realize why my summer project, which was to read Eat That Frog by Brian Tracey, was such a failure. The book is meant to be applied to work, preferably in an office environment–i.e. during your 40 productive work-hours. I was already working 40 hours a week at my extremely stimulating job as a nurse’s aid at the hospital, where I had barely any time to sit down and think about anything, and I certainly didn’t have procrastination problems. Then I would get home, exhausted with my brain about to explode from all the new interesting stuff I’d been seeing and doing all day, and try to apply Brian Tracey’s productivity methods to the personal interest projects I was doing in my spare time.
This was a very efficient way to make these things not fun, make me feel guilty about being a procrastinator, etc. It gave me an aversion to starting projects, because the part of my brain that likes and needs to do something easy and fun after work knew it would be roped into doing something mentally tiring, and that it would be made to feel guilty over not wanting to do it.
I’m hoping that once I’m graduated and work as a nurse for a year or two, so that I have a chance to get accustomed to a given unit and don’t have to spend so much mental effort, I’ll have more left over for outside interests and can start reading about physics and programming for fun again. (Used to be able to do this in first and second year, definitely can’t now.)
I’m glad you seem to have benefited from my explanation. If you want to do mentally draining reading, maybe weekends or later on in the evenings after you’ve rested would be a good time for that? If you’ve rested first, you might be able to scrape up a little extra juice.
Of course everyone has their own mental stamina limit, so nobody can tell you whether you do or don’t have enough stamina to do additional intellectual activities after work. And it may vary day to day, as work is not likely to demand the exact same amount of brainpower every day.
An interesting experiment would be to see if there’s anything that restores your stamina like a bath, a 20 minute nap after work, meditation, watching TV, or playing a fun game. Simply laying down in a dark quiet place does wonders for me if I am stressed out or fatigued. I would love to see someone log their mental stamina over time and correlate that to different activities that might restore stamina.
There are also stress reduction techniques that may help prevent you from losing stamina in the first place that could be interesting to experiment with.
And if you’re not taking 15 minute breaks every 90 minutes during work, you might be “over-training” your brain. Over-training might result in an amplification of fatigue. “The Power of Full Engagement: Manage Energy Not Time” is likely to be of interest.
If you decide to do mental stamina experiments, definitely let me know!
I hadn’t actually thought of that before...but it’s an awesome idea! I will let you know if I get around to it.
Woo-hoo! (:
I’ve also found that pouring lots of cold water on my face helps me squeeze out the last drops of stamina I have left, and allow me to work twenty more minutes or so. (It doesn’t actually restore stamina, so it doesn’t work if I do that more than a couple times in a row.)
Hmmm. That might be one or a combination of the following:
Taking a five minute break.
Enjoying physical sensation. (Enjoyment seems to restore stamina for me, perhaps that’s because the brain uses neurotransmitters for processing, and triggering pleasure involves increasing the amount of certain neurotransmitters.)
Fifteen minute breaks are supposed to be optima, and if you maximized pleasure during your break, I wonder what amount of stamina that would restore?
Probably 2. -- the break actually lasts about one minute.
Re the problem of having to think all the time: a good start is to develop a habit of rejecting certainty about judgments and beliefs that you haven’t examined sufficiently (that is, if your intuition shouts at you that something is quite clear, but you haven’t thought about that for a few minutes, ignore (and mark as a potential bug) that intuition unless you understand a reliable reason to not ignore it in that case). If you don’t have the stamina or incentives to examine such beliefs/judgments in more detail, that’s all right, as long as you remain correspondingly uncertain, and realize that the decisions you make might be suboptimal for that reason (which should suitably adjust your incentives for thinking harder, depending on the importance of the decisions).
The process of choosing a probability is not quite that simple. You’re not just making a boolean decision about whether you know enough to know, you’re actually taking the time to distinguish between 10 different amounts of confidence (10%, 20%, 30%, etc), and then making ten more tiny distinctions (30%, 31%, 32% for instance)… at least that’s the way that I do it. (More efficient than making enough distinctions to choose between 100 different options.) When you are wondering exactly how likely you are to know something in order to choose a percentage, that’s when you have to start analyzing things. In order to answer the question, my thought process looked like this:
Bayes. I have to remember who that is. Okay, that’s the guy that came up with Bayesian probability. (This was instant, but that doesn’t mean it took zero mental work.)
Do I have his birthday in here? Nothing comes to mind.
Digs further: Do I have any reason to have read about his birthday at any point? No. Do I remember seeing a page about him? I can’t remember anything I read about his birthday.
Considers whether I should just go “I don’t know” and put a random year with a 0% probability. Decides that this would be copping out and I should try to actually figure this out.
When was Bayesian probability invented? Let’s see… at what point in history would that have occurred?
Try to brainstorm events that may have required Bayesian probability, or that would have suggested it didn’t exist yet.
Try to remember the time periods for when these events happened.
Defines a vague section of time in history.
Considers whether there might be some method of double-checking it.
Considers the meaning of “within 20 years either way” and what that means for the probability that I’m right.
Figures out where in my vague section of time the 40 year range should be fit.
Figures out which year is in the middle of the 40 year range and types it in.
Consider how many years Bayes would likely have to have lived for before giving his theorems to the world and adjust the year to that.
Considers whether it was at all possible for Bayesian probability to have existed before or after each event.
If possible, consider how likely it was that Baye’s probability existed before/after each event.
Calculate how many 40-year ranges there are in the vague section of time between the events where Bayes could not have been born.
Calculate the chance that I chose the correct 40-year section out of all the possible sections, if odds are equal.
Compare this to my probabilities regarding how likely it was for Bayes theorem to have existed before and after certain events.
Adjust my probability figure to take all that into account.
My answer to this question took at least twenty steps, and that doesn’t even count all the steps I went through for each event, nor does it count all the sub steps I went through for things that I sort of hand-waved like “Adjust my probability figure to take all that into account”.
If you think figuring out stuff is instant, you underestimate the number of steps your brain does in order to figure things out. I highly recommend doing meditation to improve your meta-cognition. Meta-cognition is awesome.
The straightforward interpretation of your words evaluates as a falsity, as you can’t estimate informal beliefs to within 1%.
I’d put it more in terms of decibels of log-odds than percentages of probability. Telling 98% from 99% (i.e. +17 dB from +20 dB) sounds easier to me than telling 50% from 56% (i.e. 0 dB from +1 dB).
Well, you can, but it would be a waste of time.
No, I’m pretty certain you can’t. You can’t even formulate truth conditions for correctness of such an evaluation. Only in very special circumstances getting to that point would be plausible (when a conclusion is mostly determined by data that is received in an explicit form or if you work with a formalizable specification of a situation, as in probability theory problems; this is not what I meant by “informal beliefs”).
(I was commenting on a skill/habit that might be useful in the situations where you don’t/can’t make the effort of explicitly reasoning about things. Don’t fight the hypothetical.)
Is it your position that there is a thinking skill that is actually accurate for figuring stuff out without thinking about it?
I expect you can improve accuracy in the sense of improving calibration, by reducing estimated precision, avoiding unwarranted overconfidence, even when you are not considering questions in detail, if your intuitive estimation has an overconfidence problem, which seems to be common (more annoying in the form of an “The solution is S!” for some promptly confabulated arbitrary S, when quantifying uncertainty isn’t even on the agenda).
(I feel the language of there being “positions” has epistemically unhealthy connotations of encouraging status quo bias with respect to beliefs, although it’s clear what you mean.)
The point is to make these things automatic so that one doesn’t have to analyze all the time. I definitely don’t feel like I “maintain a state of constant analysis,” even when applying purportedly advanced rationality techniques. It basically feels the same as thinking about things normally, except that I am right more often.
I don’t believe that your claim is true, but if it is I think LessWrong is doomed as a concept. I frankly do not think people will be able to accurately evaluate when they need to apply thinking skills to their decisions, so if we cannot teach skills on this level—teach habits, as you say—I do not think LessWrong will ever accomplish anything of real worth.
One example of a skill that I have taken on on this level is reference class forecasting. If I need to estimate how long something will take, my go-to method is to take the outside view. I am so used to this that it is now the automatic response to questions of estimating times.
I don’t use “brainpower rationing” because I frankly have never felt the need to do so. I have told people that they “think too much” under certain circumstances (most notably when thinking is impeding action), and the thought of “brainpower rationing” has never come to mind until I saw this post.
What do you make of this?
Maybe I misinterpreted here but it sounds like you’re saying you don’t believe in mental stamina limits? Maybe you mean that you don’t think rationality requires much brainpower?
I don’t think we’d be doomed, and there are a few reasons for that:
There are people in existence who really can analyze pretty much constantly. THOSE people would theoretically have a pretty good chance of being rational all the time.
People who cannot analyze anywhere near constantly can simply choose their battles. If they’re aware of their mental stamina limits, they can work with them. Realizing you don’t know stuff and that you don’t have enough mental stamina to figure it out right now is kind of sad but it is still perfectly rational, so perhaps rationalists with low mental stamina can still be good rationalists that way.
There are things that decrease mental fatigue. For instance, taking 15 minute breaks every 90 minutes (The book “The power of full engagement: manage energy not time” talks about this). We could do experiments on ourselves to find out what other things reduce or prevent mental fatigue. There may be low-hanging apples we’re totally unaware of.
Okay, so you’ve learned to instantly go to a certain method. I can believe that this does not take much brainpower. However, how much brainpower does it take to execute the outside view method, on average, for the types of things you use it for? How many times can you execute the outside view in a day? Have you ever tried to reach your mental stamina limit?
Do you ever get home from work and feel relieved that you can relax now, and then do something that’s not mentally taxing? Do you ever find that you’re starting to hate an activity, and notice you’re making more and more mistakes? Do you ever feel lazy and can’t be bothered to do anything useful? I bet you do experience mental fatigue but don’t recognize it as such. A lot of people just berate themselves for being unproductive, and don’t consciously recognize that they’ve hit a real limit.
My method of doing the same calculation was:
see name
remember “mid-1700s”
therefore he must have been born in the early 1700s.
20 year margin means I don’t have to be precise.
answer: 1700 or 1705 (can’t remember which I put)
get question right
The more difficult part was the probability estimate. But using the heuristics taught to me by this book, this took only a few calculations. And the more I do these types of calculations, the faster and more calibrated I become. Eventually I hope to make them automatic at the 8 + 4 = 12 level.
If I were doing the calculation “for real” and not on a survey my algorithm would be much easier:
see name
copy/paste name into Google
look at Wikipedia
look at other sources to confirm (if important)
I know they exist on some level thanks to my experience with dual n-back, but I’ve yet to encounter any practical situation that imposes them (aside from “getting tired, which is different), and if I did I’m sure I could train my way out, just as I trained my way out of certain physical stamina limits. For example, it was once hard for me to maintain my endurance throughout a full fencing bout, but following some training I can do several in a row without becoming seriously fatigued. I’m sure better fencers than me can do even more.
LessWrong and CFAR, in my view, should provide the mental equivalent of that training if it is indeed necessary for the practice of rationality. I’m not, however, convinced that it is.
Immeasurably small (no perceived effort and takes less time than the alternative)/indeterminate/not in this respect. Most of the effort was involved in correctly identifying situations in which the method was useful, not in actually executing the method, but once the method became sufficiently ingrained that too went away.
No. My work is generally fun.
Not really. Sometimes I get bored, does that count?
Negative.
I think mental stamina is an important concept.
I’ll add mental exuberance (not an ideally clear word, but I don’t have a better one)-- how much people feel an impulse to think.
Nancy, there is already a term for this. It’s “intellectual overexcitability” or “intellectual supersensitivity”. These are terms from Dabrowski. Look up the “Theory of Positive Disintegration” to learn more.
Those terms seem like pathologizing—which is not surprising, considering that Dabrowski puts emphasis on the difficulties of the path. I was thinking more of the idea that some people like thinking more than others, just as some people like moving around more than others, which is something much less intense.
I was wondering whether Dabrowski was influenced by Gurdjieff, and it turns out that he was.
Thanks for the details. If I remember correctly, I was running out of the ability to care by the time I got to the Bayes question.
What were the vitamin deficiencies?
I’m not sure I can reliably recognize what mental fatigue feels like. I’d like to be able to diagnose it in myself (because I suspect that I have less mental energy than I used to), so do you know of any reasonably quick way to induce something that feels like mental fatigue, e.g. alcohol?
Alcohol doesn’t induce mental fatigue in me; high temperatures and dehydration do. YMMV.
EDIT: So does not eating enough sugars.
Whatever your worst subject is, do a whole bunch of exercises in it until you start making so many mistakes it is not worth continuing. No need for alcohol, might as well wear out your brain.
It would be interesting to see if you’d get different types of fatigue from doing different kinds of activities. For instance, if I do three hours of math problems, I have trouble speaking after that—it’s like my symbol manipulation circuitry is fried. (I have dyslexia, so that’s probably related.) If I wear out my verbal processor (something that I think only started happening to me after I developed some unexpected vitamin deficiencies) this results in irritation. I can’t explain myself very well, so people jump on me for mistakes, and it’s really hard to tell them what I meant instead, so I get frustrated.
So, exercising each area of mental abilities might yeild different fatigue symptoms.
If you decide to experiment on yourself I’m definitely curious about your results!
That happens to me, too.
What are your fatigue symptoms? How much can you do of each activity before becoming fatigued?
If I’ve been reading/studying too long, I find much harder to concentrate and am more easily distracted by stray thoughts.
If I’ve been writing computer code/doing maths too long, I make the kind of trivial mistakes that screw up the results but are hard to locate way more often.
It depends—usually between 20 minutes and 3 hours.
My first thought about this is that people’s rationality ‘in real life’ totally is determined by how likely they are to notice a Bayes question in an informal setting, where they may be tired and feeling mentally lazy. In Keith Stanovich’s terms, rationality is mostly about the reflective mind: it’s someone’s capacity and habits to re-compute a problem’s answer, using the algorithmic mind, rather than accept the intuitive default answer that their autonomous mind spits out.
IQ tests tend to be formal; it’s very obvious that you’re being tested. They don’t measure rationality in the sense that most LWers mean it; the ability to apply thinking techniques to real life in order to do better.
It might still be valuable to know how LWers do on a more formal test of probability-related knowledge; after all, most people in the general public don’t know Bayes’ theorem, so it’d be neat to see how good LW is at increasing “rationality literacy”. But that’s not the ultimate goal. There are reasons why you might want to measure a group’s ability to pick out unexpected rationality-related problems and activate the correct mindware. If your Bayesian superpowers only activate when you’re being formally tested, they’re not all that useful as superpowers.
I can see why you’d criticize someone for saying “the problem is that the setting wasn’t formal enough” but that’s not exactly what I was getting at. What I was getting at is that there’s a limit to how much thinking that one can do in a day, everyone’s limit is different, and a lot of people do things to ration their brainpower so they avoid running out of it. This comment on mental stamina explains more.
My point was, more clearly worded: It would be a very rare person who possesses enough mental stamina to be rational in literally every single situation. That’s a wonderful ideal, but the reality is that most people are going to ration brainpower. If your expectation is that rationalists should never ration brainpower and should be rational constantly, this is an unrealistic expectation. A more realistic expectation is that people should identify the things they need to think extra hard about, and correctly use rational thinking skills at those times. Therefore, testing for the skills when they’re trying is probably the only way to detect a difference. There are inevitably going to be times when they’re not trying very hard, and if you catch them at one of those times, well, you’re not going to see rational thinking skills. It may be that some of these things can be ingrained in ways that don’t use up a person’s mental stamina, but to expect that rationality can be learned in such a way that it is applied constantly strikes me as an unreasoned assumption.
Now I wonder if the entire difference between the control groups results and LessWrong’s results was that Yvain asked the control group only one question, whereas LessWrong had answered 14 pages of questions prior to that.
Agreed that rationality is mentally tiring...I went back and read your comment, too. However:
To me, rationality is mostly the ability to notice that “whew, this is a problem that wasn’t in the problem-set of the ancestral environment, therefore my intuitions probably won’t be useful and I need to think”. The only way a rationalist would have to be analytical all the time is if they were very BAD at doing this, and had to assume that every situation and problem required intense thought. Most situations don’t. In order to be an efficient rationalist, you have to be able to notice which situations do.
Any question on a written test isn’t a great measure of real-life rationality performance, but there are plenty of situations in everyday life when people have to make decisions based on some unknown quantities, and would benefit from being able to calibrate exactly how much they do know. Some people might answer better on the written test than if faced with a similar problem in real life, but I think it’s unlikely that anyone would do worse on the test than in real life.
Re having to think all the time: a good start is to develop a habit of rejecting certainty about judgments and beliefs that you haven’t examined sufficiently (that is, if your intuition shouts at you that something is quite clear, but you haven’t thought about that for a few minutes, ignore that intuition unless you have a reliable reason to not ignore it in that case). If you don’t have stamina or incentives to examine such beliefs/judgments in more detail, that’s all right, as long as you remain correspondingly uncertain, and realize that the decisions you make might be suboptimal for that reason (which should suitably adjust your incentives for thinking harder, depending on the importance of the decisions).
I don’t think you could really apply any ‘algorithmic’ method to that question (other than looking it up, but that would be cheating). It was a test on how much confidence you put in your heuristics. (BTW, It seems that I’ve underestimated mine, or I’ve been lucky, since I’ve got the date off by one year but estimated my confidence at 50% IIRC). Still, it was a valuable test, since most of human reasoning is necessarily heuristic.
Really? What probability do you assign to that statement being true? :D
I’m under the impression that Bayes’ theorem is included in the high school math programs of most developed countries, and I’m certain it is included in any science and engineering college program.
I assign about 80% probability to less than 25% of adults knowing Bayes theorem and how to use it. I took physics and calculus and other such advanced courses in high school, and graduated never having heard of Bayes’ Theorem. I didn’t learn about it in university, either–granted, I was in ‘Statistics for Nursing’, it’s possible that the ‘Statistics for Engineering’ syllabus included it.
Only 80%?
In the USA, about 30% of adults have a bachelor’s degree or higher, and about 44% of those have done a degree where I can slightly conceive that they might possibly meet Bayes’ theorem (those in the science & engineering and science- & engineering-related categories (includes economics), p. 3), i.e. as a very loose bound 13% of US adults may have met Bayes’ theorem.
Even bumping the 30% up to the 56% who have “some college” and using the 44% for a estimate of the true ratio of possible-Bayes’-knowledge, that’s only just 25% of the US adult population.
(I’ve no idea how this extends to the rest of the world, the US data was easiest to find.)
You did your research and earned your confidence level. I didn’t look anything up, just based an estimate on anecdotal evidence (the fact that I didn’t learn it in school despite taking lots of sciences). Knowing what you just told me, I would update my confidence level a little–I’m probably 90% sure that less than 25% of adults know Bayes Theorem. (I should clarify that=adults living in the US, Canada, Britain, and other countries with similar school systems. The percentage for the whole world is likely significantly lower.)
I hear Britain’s school system is much better than the US’s.
Once you control for demographics, the US public school system actually performs relatively well.
Good point.
It’s not great by international standards, but I have heard that the US system is particularly bad for an advanced country.
In terms of outcomes, the US does pretty terribly when considered 1 country, but when split into several countries it appears at the top of each class. Really, the EU is cheating by considering itself multiple countries.
The EU arguably is more heterogeneous than the US. But then, India is even more so.
How’s it being split?
I actually thought someone would dig up and provide the relevant link by now. I’ll have to find it.
You mean comparing poorer states to poorer countries?
Actually it is quite good (even for an “advanced country”) if you compare the test scores of, say, Swedes and Swedish-Americans rather than Swedes and Americans as a whole.
I wonder what that’s controlling for? Cultural tendencies to have different levels of work ethic?
Hmmm. So it’s “good” but people with the wrong genes are spoiling the average somehow.
The UK high school system does not cover Bayes Theorem.
If you choose maths as one of your A-levels, there’s a good chance you will cover stats 1 which includes the formula for Bayes’ Theorem and how to apply it to calculate medical test false positives/false negatives (and equivalent problems). However it isn’t named and the significance to science/rationality is not explained, so it’s just seen as “one more formula to learn”.
Offhand, 1⁄2 young people do A levels, 1⁄4 of those do maths, and 2⁄3 of those do stats, giving us 1⁄12 of young people. I don’t think any of these numbers are off by enough to push the fraction over 25%
Maybe you guys could solve that problem by publishing some results demonstrating its exteme significance
As far as I know, it’s been formally demonstrated to be the absolutely mathematically-optimal method of achieving maximal hypothesis accuracy in an environment with obscured, limited or unreliable information.
That’s basically saying: “There is no possible way to do better than this using mathematics, and as far as we know there doesn’t yet exist anything more powerful than mathematics.”
What more could you want? A theorem proving that any optimal decision theory must necessarily use Bayesian updating? ETA: It has been pointed out that there already exists such a theorem. I could’ve found that out by looking it up. Oops.
There already is such a theorem. From Wikipedia:
As far as I can tell from wikipedia’s description of admissibility, it makes the same assumptions as CDT: That the outcome depends only on your action and the state of the environment, and not on any other properties of your algorithm. This assumption fails in multi-player games.
So your quote actually means: If you’re going to use CDT then Bayes is the optimal way to derive your probabilities.
And the list of notable problems that have been solved using Bayes is...? Bayes doesn’t tell you how to make your informaton more copious or accuate, although there are plenty of techniques for doing that. Bayes also doesn’t tell you how to formulate novel hypotheses. It also doens’t tell you how to deal with conceptual problems that are not yet suitable for nnumber crucnhing. It looks to me like Bayes is actually a rather small part of the picture.
ETA;
A similar point is cogently argiued by RichardKennaway here
PS: -T-w-o- Three downvotes, and not a shred of counteargument. Typical.
Half of statistics these days is Bayesian. Do you want to defend the claim that statistics solves no notable problems?
As usual, I add my downvote to whining about downvotes. Since you think it’s ‘typical’ and this vindicates your claims, I’m sure you’ll be pleased that I’m helping prove you right.
Great. Then the UK education sytem is exactly right in teaching Bayes as part of statitistics, but not as a general-prupose solution to everything. ETA: But surely the LW take on Bayes is that it is much more than something useful in statistics.
No, I want to defend the claims that Bayes is not as a general-prupose solution to everything, is not a substitute for other congnitive disciplines, is of no benefit to many people and is of no use in many contexts.
Please inform me of the correct way to indicate that the karma system is being misused.
Now you’re just backing off your claim. What happened to your list?
First point: if Bayesian statistics is half of statistics, the description of the UK course is of it as being way way less than half the course. Therefore the UK system is very far from being ‘exactly right’.
Second point: The optimistic meta-induction is that Bayesian statistics has gone from being used by a literal handful of statisticians to being widespread and possibly a majority now or in the near future; therefore, it will continue spreading and eating more of statistics in general, and the course will get wronger and wronger, and your claims less and less right.
So you’re just splashing around a lot of bullshit and distractions when you demand lists and talk about the UK course being exactly right, since those aren’t what you are actually trying to claim. Good to know!
What’s the point of indicating when it’s not being misused?
I am not going to give a full response, because your comments are obstreporous, but See RichardKennaway’s discussion for LW’s oeverarching hopes for Bayes, and its limitations
You have your opinion, on that, I have mine. You can state your opinion, I can’t state mine. I can’t discuss the censorship, because discussions of censorship are censored.
You’re stating it right now. Oh the ironing.
It’s in a downvoted thread.So it isn’t visible.If negative karma doesn’t do anything regarding the visibility of comments, why have the button? Sheesh.
And so begins the equivocation on ‘people have to click a button to see it’ with ‘censorship’.
And so I ask you a second time: what is the button for?
And so begins another goal-shifting, like the list or like the claim of ‘censorship’, this time to defining karma systems. Pardon me if I don’t care to continue this game.
OK. You cannot give an answer that will not embarass yourself. Got that.
Must be a problem of the American school system, I suppose.
Did they teach you about conditional probability? Usually Bayes’ theorem is introduced right after the definition of conditional probability.
There are national and international surveys of quantitative literacy in adults. The U.S. does reasonably well in these, but in general the level of knowledge is appalling to math teachers. See this pdf (page 118 of the pdf, the in-text page number is “Section III, 93”) for the quantitative literacy questions, and the percentage of the general population attaining each level of skill. less than a fifth of the population can handle basic arithmetic operations to perform tasks like this:
People who haven’t learned and retained basic arithmetic are not going to have a grasp of Bayes’ theorem.
It was in my high school curriculum (in Italy, in the mid-2000s), but the teacher spent probably only 5 minutes on it, so I would be surprised if a nontrivial number of my classmates who haven’t also heard of it somewhere else remember it from there. IIRC it was also briefly mentioned in the part about probability and statistics of my “introduction to physics” course in my first year of university, but that’s it. I wouldn’t be surprised if more than 50% of physics graduates remember hardly anything about it other than its name.
I’m pretty sure Ireland doesn’t have it on our curriculum, not sure how typical we are.
Well, it’s certainly not included in the US high school curriculum.
If you’ll excuse the expression, I’m suspicious of your sudden epiphany. That is, I accept your suggestion as a possible explanation (although I’m not convinced, mainly because this doesn’t describe the way I answered the question; I don’t know about anyone else). But I think saying “Oh gosh! The true answer has been staring us in the face all along!” is premature.
I am not sure why you took “a new explanation” so seriously. I guess I have to be really careful on LessWrong to distinguish ideas from actual beliefs. I do not think it’s “The True Answer”. I just think it’s a rather obvious alternate explanation that should have occurred to me immediately, and didn’t, and I’m surprised about that, and about the fact that it didn’t seem to occur to Yvain either. I reworded some things to make it more obvious that I am not trying to present this as “The True Answer” but just as an idea.
Thank you, I appreciate that.
Would you mind trying to avoid jumping to the conclusion that I’m acting stupid in the future, Kindly? I definitely don’t mind being told “Your statement could be interpreted as such-and-such stupid behavior, so you may want to change it.” but it’s a little frustrating when people speak to me as if they really believe I am as confused as your “The True Answer” interpretation would imply.
I’m not sure why you’re accusing me of this. I often disagree with people, but I usually don’t assume the people I disagree with are stupid. This is especially true when we disagree due to a misunderstanding.
(I don’t intend to continue this line of conversation.)
Well it’s my perception that it would be pretty stupid to jump to the conclusion that I had it all figured out just out of nowhere up there. If that’s not your perception, too, then it’s not—but that would be unexpected to me who holds the perception that it would be a kind of stupid thing to do. I don’t know what wording to use other than “Please try not to jump to the conclusion that I’m doing stupid things.” but just substitute “stupid” for whatever word you would use, and then please try not to jump to conclusions that I am doing whatever it is that you call that, okay?