I wonder if the decline of apprenticeships has made overconfidence and underconfidence more common and more severe.
I’m not a history expert, but it seems to me that a blacksmith’s apprentice 700 years ago wouldn’t have had to worry about over/underconfidence in his skill. (Gender-neutral pronouns intentionally not used here!) He would have known exactly how skilled he was by comparing himself to his master every day, and his master’s skill would have been a known quantity, since his master had been accepted by a guild of mutually recognized masters.
Nowadays, because of several factors, calibrating your judgement of your skill seems to be a lot harder. Our education system is completely different, and regardless of whatever else it does, it doesn’t seem to be very good at providing reliable feedback to its students, who properly understand the importance of the feedback and respond accordingly. Our blacksmith’s apprentice (let’s call him John) knows when he’s screwed up—the sword or whatever that he’s made breaks, or his master points out how it’s flawed. And John knows why this is important—if he doesn’t fix the problem, he’s not going to be able to earn a living.
Whereas a modern schoolkid (let’s call him Jaden) may be absolutely unprepared to deal with math, but he doesn’t know exactly how many years he’s behind (it’s hard enough to get this information in aggregate, and it seems to be rarely provided to the students themselves on an individual basis—no one is told “you are 3 years behind where you ought to be”). And Jaden has absolutely no clue why that matters, since the link between math and his future employment isn’t obvious to him, and no one’s explaining it to him. (School isn’t for learning; as Paul Graham has explained, “Officially the purpose of schools is to teach kids. In fact their primary purpose is to keep kids locked up in one place for a big chunk of the day so adults can get things done. And I have no problem with this: in a specialized industrial society, it would be a disaster to have kids running around loose.”)
Another modern schoolkid (let’s call her Jaina) may be really skilled at math, but testing won’t indicate this strongly enough (it works both ways; tests saturate at the high end—especially if they’re targeting a low level of achievement for the rest of the class—and “you are 3 years ahead of everyone else in this room” is not feedback that is commonly given). And there’s a good chance it won’t be obvious to her how important this is, and how important becoming even more skilled is. And if she ends up being underconfident in her ability, and the feedback loop (“I know how skilled I am, I know why becoming stronger is important, and I know what I need to do”) isn’t established, then instead of learning plasma physics and working on ITER or DEMO, she goes into marketing or something. Maybe doing worthy things, but not being as awesome as she could have been.
My point, after this wondering, is that I agree with this post, and want to elaborate: structuring what you do so that you test yourself in the process of doing it is a good way to establish a feedback loop that increases your skill and the accuracy of your confidence in it. I find nothing wrong with the debating example in this post, but I worry that it makes self-testing sound like something that you should go out and do, separate from your everyday work. (Part of this, I think, is due to Eliezer’s very unusual occupation.) My usual self-testing example is something like “can I write this program correctly on the very first try?”. That’s a hard challenge, integrated into my everyday work. Successfully completing it, or coming close, has allowed me to build up my skill (“the compiler in my head”) and avoid the danger of underconfidence.
A friend of mine, normal in most ways, has exceptionally good mental imagery, such that one time she visited my house and saw a somewhat complex 3-piece metalwork puzzle in my living room and thought about it later that evening after she had left, and was able to solve it within moments of picking it up when she visited a second time. At first I was amazed at this, but I soon became more amazed that she didn’t find this odd, and that no one had ever realized she had any particular affinity for this kind of thing in all the time she’d been in school. I’m curious as to how many cognitive skills like this there are to excel at and if many people are actually particularly good at one or many of them without realizing it due to a lack of good tests for various kinds of cognition.
My usual self-testing example is something like “can I write this program correctly on the very first try?”. That’s a hard challenge, integrated into my everyday work.
I should try to remember to try this the next time I have a short piece of code to write. Furthermore, it’s the sort of thing that makes me slightly uncomfortable and is therefore easy to forget, so I should try harder to remember it.
In general, this sort of thing seems like a very useful technique if you can do it without endangering your work. Modded parent up.
If your practice isn’t making you feel scared and uncomfortable, it’s not helping. Imagine training for a running race without any workouts that raise your heart rate and make you breathe hard.
Feeling out of your comfort zone and at risk of failure is something everybody should seek out on a regular basis.
My usual self-testing example is something like “can I write this program correctly on the very first try?”.
I never thought of that as a thing you could do. I think when my code compiles on the first try, it’s more often then not a sign of something very wrong. For example, the last time it happened was because I forgot to add the file I was working on to the makefile.
Perhaps I should try to learn to code more precisely.
I wonder if the decline of apprenticeships has made overconfidence and underconfidence more common and more severe.
I’m not a history expert, but it seems to me that a blacksmith’s apprentice 700 years ago wouldn’t have had to worry about over/underconfidence in his skill. (Gender-neutral pronouns intentionally not used here!) He would have known exactly how skilled he was by comparing himself to his master every day, and his master’s skill would have been a known quantity, since his master had been accepted by a guild of mutually recognized masters.
Nowadays, because of several factors, calibrating your judgement of your skill seems to be a lot harder. Our education system is completely different, and regardless of whatever else it does, it doesn’t seem to be very good at providing reliable feedback to its students, who properly understand the importance of the feedback and respond accordingly. Our blacksmith’s apprentice (let’s call him John) knows when he’s screwed up—the sword or whatever that he’s made breaks, or his master points out how it’s flawed. And John knows why this is important—if he doesn’t fix the problem, he’s not going to be able to earn a living.
Whereas a modern schoolkid (let’s call him Jaden) may be absolutely unprepared to deal with math, but he doesn’t know exactly how many years he’s behind (it’s hard enough to get this information in aggregate, and it seems to be rarely provided to the students themselves on an individual basis—no one is told “you are 3 years behind where you ought to be”). And Jaden has absolutely no clue why that matters, since the link between math and his future employment isn’t obvious to him, and no one’s explaining it to him. (School isn’t for learning; as Paul Graham has explained, “Officially the purpose of schools is to teach kids. In fact their primary purpose is to keep kids locked up in one place for a big chunk of the day so adults can get things done. And I have no problem with this: in a specialized industrial society, it would be a disaster to have kids running around loose.”)
Another modern schoolkid (let’s call her Jaina) may be really skilled at math, but testing won’t indicate this strongly enough (it works both ways; tests saturate at the high end—especially if they’re targeting a low level of achievement for the rest of the class—and “you are 3 years ahead of everyone else in this room” is not feedback that is commonly given). And there’s a good chance it won’t be obvious to her how important this is, and how important becoming even more skilled is. And if she ends up being underconfident in her ability, and the feedback loop (“I know how skilled I am, I know why becoming stronger is important, and I know what I need to do”) isn’t established, then instead of learning plasma physics and working on ITER or DEMO, she goes into marketing or something. Maybe doing worthy things, but not being as awesome as she could have been.
My point, after this wondering, is that I agree with this post, and want to elaborate: structuring what you do so that you test yourself in the process of doing it is a good way to establish a feedback loop that increases your skill and the accuracy of your confidence in it. I find nothing wrong with the debating example in this post, but I worry that it makes self-testing sound like something that you should go out and do, separate from your everyday work. (Part of this, I think, is due to Eliezer’s very unusual occupation.) My usual self-testing example is something like “can I write this program correctly on the very first try?”. That’s a hard challenge, integrated into my everyday work. Successfully completing it, or coming close, has allowed me to build up my skill (“the compiler in my head”) and avoid the danger of underconfidence.
A friend of mine, normal in most ways, has exceptionally good mental imagery, such that one time she visited my house and saw a somewhat complex 3-piece metalwork puzzle in my living room and thought about it later that evening after she had left, and was able to solve it within moments of picking it up when she visited a second time. At first I was amazed at this, but I soon became more amazed that she didn’t find this odd, and that no one had ever realized she had any particular affinity for this kind of thing in all the time she’d been in school. I’m curious as to how many cognitive skills like this there are to excel at and if many people are actually particularly good at one or many of them without realizing it due to a lack of good tests for various kinds of cognition.
I should try to remember to try this the next time I have a short piece of code to write. Furthermore, it’s the sort of thing that makes me slightly uncomfortable and is therefore easy to forget, so I should try harder to remember it.
In general, this sort of thing seems like a very useful technique if you can do it without endangering your work. Modded parent up.
Without risk, there is no growth.
If your practice isn’t making you feel scared and uncomfortable, it’s not helping. Imagine training for a running race without any workouts that raise your heart rate and make you breathe hard.
Feeling out of your comfort zone and at risk of failure is something everybody should seek out on a regular basis.
I never thought of that as a thing you could do. I think when my code compiles on the first try, it’s more often then not a sign of something very wrong. For example, the last time it happened was because I forgot to add the file I was working on to the makefile.
Perhaps I should try to learn to code more precisely.
Heh. (You should use makefiles that automatically build new files, and automatically sense dependencies for rebuild.)
As I recall, Eliezer said somewhere that I’m too tired to Google—there is no limit to the amount of intelligence that you can use while programming.