I thought I’d pose an informal poll, possibly to become a top-level, in preparation for my article about How to Explain.
The question: on all the topics you consider yourself an “expert” or “very knowledgeable about”, do you believe you understand them at least at Level 2? That is, do you believe you are aware of the inferential connections between your expertise and layperson-level knowledge?
Or, to put it another way, do you think that, given enough time, but using only your present knowledge, you could teach a reasonably-intelligent layperson, one-on-one, to understand complex topics in your expertise, teaching them every intermediate topic necessary for grounding the hardest level?
Edit: Per DanArmak’s query, anything you can re-derive or infer from your present knowledge counts as part of your present knowledge for purposes of answering this question.
I’ll save my answer for later—though I suspect many of you already know it!
I have a (I suspect unusual) tendency to look at basic concepts and try to see them in as many ways as possible. For example, here are seven equations, all of which could be referred to as Bayes’ Theorem:
However, each one is different, and forces a different intuitive understanding of Bayes’ Theorem. The fourth one down is my favourite, as it makes obvious that the update depends only on the ratio of likelihoods. It also gives us our motivation for taking odds, since this clears up the 1/(1+x)ness of the equation.
Because of this way of understanding things, I find explanations easy, because if one method isn’t working, another one will.
ETA: I’d love to see more versions of Bayes’ Theorem, if anyone has any more to post.
Very well said, and doubles as a reply to the last part of my comment here. (When I read your comment in my inbox, I thought it was actually a reply to that one! Needless to say, I my favorite versions of the theorem are the last two you listed.)
This strikes me as an un-lifelike assumption. If I had to explain things in this way, I would expect to encounter some things that I don’t explicitly know (and other that I knew and have forgotten), and to have to (re)derive them. But I expect that I would be able to rederive almost all of them.
Refining my own understanding is a natural part of building a complex explanation-story to tell to others, and will happen unless I’ve already built this precise story before and remember it.
I think I know a fair amount about doing calligraphy, but I’m dubious that someone could get a comparable level of knowledge without doing a good bit of calligraphy themselves.
If I were doing a serious job of teaching, I would be learning more about how to teach as I was doing it.
I consider myself to be a good but not expert explainer.
But in some limited areas explanation is completely adequate.
I taught co-worker how to do sudoku puzzles. After teaching him the human-accessible algorithms and allowing time for practice, I was still consistently beating his time. I knew why, and he didn’t. After I explained the difference in mental state I was using, he began beating my time on regular basis. {Instead of checking the list of 1-9 for each box or line, allow your brain to subconsciously spot the missing number and then verify its absence.} He is more motivated and has more focus, while I do puzzles to kill time when waiting.
In another job where I believe I had a thorough understanding of the subject, I was never able to teach any of my (~20) trainees to produce vector graphic maps with the speed and accuracy I obtained because I was unable to impart a mathematical intuition for the approximation of curves. I let them go home with full pay when they completed their work, so they definitely had motivation. But they also had editors who were highly detail oriented.
I mean to suggest that there is a continuum of subjective ability comparing different skills. Sudoku is highly procedural, once familiar all that is required is concentration. Yoga, in the sense mentioned above, is also procedural, proscriptive; the joints allow a limited number of degrees of freedom. Calligraphy strives for an ideal, but depending on the tradition, there is a degree of interpretation allowed for aesthetic considerations. Mapping, particularly in vector graphics, has many ways to be adequate and no way to be perfect.
The number of acceptable outcomes and the degree of variation in useful paths determines the teach-ability of a skillset. The procedural skills can be taught more easily than the subjective, and practice is useful to accomplish mastery of procedural skills. Deeper understanding of a field allows more of the skill’s domain to be expressed procedurally rather than subjectively.
“do you believe you are aware of the inferential connections between your expertise and layperson-level knowledge?”,
since I am not so familiar with the formalism of a “Level 2” understanding.
My uninteresting, simple answer is: yes.
My philosophical answer is that I find the entire question to be very interesting and strange. That is, the relationship between teaching and understanding is quite strange IMO. There are many people who are poor teachers but who excel in their discipline. It seems to be a contradiction because high-level teaching skill seems to be a sufficient, and possibly necessary condition for masterful understanding.
Personally I resolve this contradiction in the following way. I feel like my own limitations make it to where I am forced to learn a subject by progressing at it in very simplistic strokes. By the time I have reached a mastery, I feel very capable of teaching it to others, since I have been forced to understand it myself in the most simplistic way possible.
Other people, who are possibly quite brilliant, are able to master some subjects without having to transmute the information into a simpler level. Consequentially, they are unable to make the sort of connections that you describe as being necessary for teaching.
Personally I feel that the latter category of people must be missing something, but I am unable to make a convincing argument for this point.
A lot of the questions you pose, including the definition of the Level 2 formalism, are addressed in the article I linked (and wrote).
I classify those who can do something well but not explain or understand the connections from the inputs and outputs to the rest of the world, to be at a Level 1 understanding. It’s certainly an accomplishment, but I agree with you that it’s missing something: the ability to recognize where it fits in with the rest of reality (Level 2) and the command of a reliable truth-detecting procedure that can “repair” gaps in knowledge as they arise (Level 3).
“Level 1 savants” are certainly doing something very well, but that something is not a deep understanding. Rather, they are in the position of a computer that can transform inputs into the right outputs, but do nothing more with them. Or a cat, which can fall from great heights without injury, but not know why its method works.
(Yes, this comment seems a bit internally repetitive.)
Ah, OK, I read your article. I think that’s an admirable task to try to classify or identify the levels of understanding. However, I’m not sure I am convinced by your categorization. It seems to me that many of these “Level 1 savants” as you call them are quite capable of fitting their understanding with the rest of reality. Actually it seems like the claim of “Level 1 understanding” basically trivializes that understanding. Yet many of these people who are bad teachers have a very nontrivial understanding—else I don’t think this would be such a common phenomena, for example, in academia. I would argue that these people have some further complications or issues which are not recognized in the 1-2-3 hierarchy.
That being said, you have to start somewhere, and the 0-1-2-3 hierarchy looks like a good place to start. I’d definitely be interested in hearing more about this analysis.
Thanks for reading it and giving me feedback. I’m interested in your claim:
It seems to me that many of these “Level 1 savants” as you call them are quite capable of fitting their understanding with the rest of reality.
Well, they can fit it in the sense that they (over a typical problem set) can match inputs with (what reality deems) the right outputs. But, as I’ve defined the level, they don’t know how those inputs and outputs relate to more distantly-connected aspects of reality.
Yet many of these people who are bad teachers have a very nontrivial understanding—else I don’t think this would be such a common phenomena, for example, in academia.
I had a discussion with others about this point recently. My take is basically: if their understanding is so deep, why exactly is their teaching skill so brittle that no one can follow the inferential paths they trace out? Why can’t they switch to the infinite other paths that a Level 2 understanding enables them to see? If they can’t, that would suggest a lack of depth to their understanding.
And regarding the archetypal “deep understanding, poor teacher” you have in mind, do you envision that they could, say, trace out all the assumptions that could account for an anomalous result, starting with the most tenuous, and continuing outside their subfield? If not, I would call that falling short of Level 2.
My take is basically: if their understanding is so deep, why exactly is their teaching skill so brittle that no one can follow the inferential paths they trace out? Why can’t they switch to the infinite other paths that a Level 2 understanding enables them to see? If they can’t, that would suggest a lack of depth to their understanding.
I would LOVE to agree with this statement, as it justifies my criticism of poor teachers who IMO are (not usually maliciously) putting their students through hell. However, I don’t think it’s obvious, or I think maybe you just have to take it as an axiom of your system. It seems there is some notion of individualism or personal difference which is missing from the system. If someone is just terrible at learning, can you really expect to succeed in explaining, for example? Realistically I think it’s probably impossible to classify the massive concept of understanding by merely three levels, and these problems are just a symptom of that fact.
As another example, in order to understand something, it’s clearly necessary to be able to explain it to yourself. In your system, you are additionally requiring that your understanding means you must be able to explain things to other people. In order to explain things to others, you have to understand them, as has been discussed. Therefore you have to be able to explain other people to yourself. Why should an explanation of other individuals behavior be necessary for understanding some random area of expertise, say, mathematics? It’s not clear to me.
And regarding the archetypal “deep understanding, poor teacher” you have in mind, do you envision that they could, say, trace out all the assumptions that could account for an anomalous result, starting with the most tenuous, and continuing outside their subfield?
It certainly seems like someone with a deep understanding of their subject should be able to identify the validity or uncertainty in their assumptions about the subject. If they are a poor teacher, I think I would still believe this to be true.
I’ve thought about this some, and I think I see your point now. I would phrase it this way: It’s possible for a “Level 3 savant” to exist. A Level 3 savant, let’s posit, has a very deeply connected model of reality, and their excellent truth-detecting procedure allows them to internally repair loss of knowledge (perhaps below the level of their conscious awareness).
Like an expert (under the popular definition), and like a Level 1 savant, they perform well within their field. But this person differs in that they can also peform well in tracing out where its grounding assumptions go wrong—except that they “just have all the answers” but can’t explain, and don’t know, where the answers came from.
So here’s what it would look like: Any problem you pose in the field (like an anomalous result), they immediately say, “look at factor X”, and it’s usually correct. They even tell you to check critical aspects of sensors, or identify circularity in the literature that grounds the field (i.e. sources which generate false knowledge by excessively citing each other), even though most in the field might not even think about or know how all those sensors work.
All they can tell you is, “I don’t know, you told me X, and I immediately figured it had to be a problem with Y misinterpreting Z. I don’t know how Z relates to W, or if W directly relates to X, I just know that Y and Z were the problem.”
I would agree that there’s no contradiction in the existence of such a person. I would just say that in order to get this level of skill you have to accomplish so many subgoals that it’s very unlikely, just as it’s hard to make something act and look like a human without also making it conscious. (Obvious disclaimer: I don’t think my case is as solid as the one against P-zombies.)
I think that the “teaching” benchmark you claim here is actually a bit weaker than a Level 2 understanding. To successfully teach a topic, you don’t need to know lots of connections between your topic and everything else; you only need to know enough such connections to convey the idea. I really think this lies somewhere between Level 1 and Level 2.
I’ll claim to have Level 2 understanding on the core topics of my graduate research, some mathematics, and some core algorithmic reasoning. I’m sure I don’t have all of the connections between these things and the rest of my world model, but I do have many, and they pervade my understanding.
I think that the “teaching” benchmark you claim here is actually a bit weaker than a Level 2 understanding. To successfully teach a topic, you don’t need to know lots of connections between your topic and everything else; you only need to know enough such connections to convey the idea. I really think this lies somewhere between Level 1 and Level 2.
I agree in the sense that full completion of Level 2 isn’t necessary to do what I’ve described, as that implies a very deeply-connected set of models, truly pervading everything you know about.
But at the same time, I don’t think you appreciate some of the hurdles to the teaching task I described: remember, the only assumption is that the student has lay knowledge and is reasonably intelligent. Therefore, you do not get to assume that they find any particular chain of inference easy, or that they already know any particular domain above the lay level. This means you would have to be able to generate alternate inferential paths, and fall back to more basic levels “on the fly”, which requires healthy progress into Level 2 in order to achieve—enough that it’s fair to say you “round to” Level 2.
I’ll claim to have Level 2 understanding on the core topics of my graduate research, some mathematics, and some core algorithmic reasoning. I’m sure I don’t have all of the connections between these things and the rest of my world model, but I do have many, and they pervade my understanding.
If so, I deeply respect you and find that you are the exception and not the rule. Do you find yourself critical of how people in the field (i.e. through textbooks, for example) present it to newcomers (who have undergrad prerequisites), present it to laypeople, and use excessive or unintuitive jargon?
Therefore, you do not get to assume that they find any particular chain of inference easy, or that they already know any particular domain above the lay level. This means you would have to be able to generate alternate inferential paths, and fall back to more basic levels “on the fly”, which requires healthy progress into Level 2 in order to achieve—enough that it’s fair to say you “round to” Level 2.
I agree that the teaching task does require a thick bundle of connections, and not just a single chain of inferences. So much so, actually, that I’ve found that teaching, and preparing to teach, is a pretty good way to learn new connections between my Level 1 knowledge and my world model. That this “rounds” to Level 2 depends, I suppose, on how intelligent you assume the student is.
If so, I deeply respect you and find that you are the exception and not the rule. Do you find yourself critical of how people in the field (i.e. through textbooks) present it to newcomers (who have undergrad prerequisites), present it to laypeople, and use excessive or unintuitive jargon?
Yes, constantly. Frequently, I’m frustrated by such presentations to the point of anger at the author’s apparent disregard for the reader, even when I understand what they’re saying.
I think I have level 2 understanding of many areas of Biology but of course not all of it. It is too large a field. But there are gray areas around my high points of understanding where I am not sure how deep my understanding would go unless it was put to the test. And around the gray areas surrounding the level 2 areas there is a sea of superficial understanding. I have some small areas of computer science at level 2 but they are fewer and smaller, ditto chemistry and geology.
I think your question overlooks the nature of teaching skills. I am pretty good at teaching (verbally and one/few to one) and did it often for years. There is a real knack in finding the right place to start and the right analogies to use with a particular person. Someone could have more understanding than me and not be able to transfer that understanding to someone else. And others could have less understanding and transfer it better.
Finally I like your use of the word ‘understanding’ rather than ‘knowledge’. It implies the connectedness with other areas required to relate to lay people.
Perhaps the reason experts aren’t always good teachers is because their thought processes / problem solving algorithms operate at a level of abstraction that is inaccessible to a beginner.
I have some trouble answering your question, chiefly because my definition of “expert” is approximately synonymous with your definition of “Level 2″.
Or, to put it another way, do you think that, given enough time, but using only your present knowledge, you could teach a reasonably-intelligent layperson, one-on-one, to understand complex topics in your expertise, teaching them every intermediate topic necessary for grounding the hardest level?
“Enough time” would be quite a long period of time. One problem is that there are a lot of textbook results that I would have to use in intermediate steps that would take me a long time to derive. Another is that there are a lot of experimental parameters that I haven’t memorized and would have to look up. But I think I could teach arithmetic, algebra, geometry, calculus, differential equations, and Newtonian physics enough that I could teach them proper engineering analysis.
Criminal Law: Yes to Level 2. Yes to teaching a layperson. It would take a while, for sure, but it’s doable. Some of the work requires an understanding of a different lifestyle; if you can’t see the potential issues with prosecuting a robbery by a prostitute and her armed male friend, or you can’t predict that a domestic violence victim will have a non-credible recantation, you’ll need some other education.
I’ve done a lot of instruction in this field. It is common for instruction not to take until there’s other experience in the field which helps things join up.
Bridge: Yes to Level 2. Possibly to teaching a layperson. The ability to play bridge well is correlated heavily to intelligence, but it also correlates to a certain zeal for winning. I have taught one person to play very well indeed, but that may not be replicable, and took years. (On an aside, I am very likely the world’s foremost expert on online bridge cheating; teaching cheating prevention would require teaching bridge first.)
Teaching requires more than reasonable intelligence on the part of the teachee. Some people who are very intelligent are ineducable. (Many of these are violators of my 40% rule: You are allowed to think you are 40% smarter/faster/stronger/better than you are. After that, it’s obnoxious.) Some people are not interested in learning a given subject. Some people will not overcome preset biases. Some people have high aptitudes in some areas and little aptitude in others (though intelligence strongly tends to spill over.)
Anyway, I’m interested in the article. My penultimate effort to explain something to many people—Bayes’ Theorem to lawyers—was a moderate failure; my last effort to explain something less mathy to a crowd was a substantial success. (My last experience in explaining something, with assistance, to 12 people was a complete failure.)
It’s non-arbitrary, but neither is it precise. 100% is clearly too high, and 10% is clearly too low.
And since I started calling it The 40% Rule fifteen years ago or thereabout, a number of my friends and acquaintances have embraced the rule in this incarnation. Obviously, some things are unquantifiable and the specific number has rather limited application. But people like it at this number. That counts for something—and it gets the message across in a way that other formulations don’t.
Some are nonplussed by the rule, but the vigor of support by some supporters gives me some thought that I picked a number people like. Since I never tried another number, I could be wrong—but I don’t think I am.
Some of the work requires an understanding of a different lifestyle; if you can’t see the potential issues with prosecuting a robbery by a prostitute and her armed male friend, or you can’t predict that a domestic violence victim will have a non-credible recantation, you’ll need some other education.
“The people who buy the services of a prostitute generally don’t want to go on record saying so, which they would have to do at some point to prosecute such a robbery. This is either because they’re married, or the shame associated with using one.”
“Victims of domestic violence have a lot invested in the relationship, and, no matter how much they feel hurt by the abuse, they will not want to tear apart the family and cripple their spouse with a felony conviction. This inner conflict will be present when the victim tries to recant their testimony.”
Did that really require passing the learner off for some other education? Or did I get the explanation wrong?
Anyway, I’m interested in the article. My penultimate effort to explain something to many people—Bayes’ Theorem to lawyers—was a moderate failure; my last effort to explain something less mathy to a crowd was a substantial success. (My last experience in explaining something, with assistance, to 12 people was a complete failure.)
I’d actually tried teaching information theory to my mom a week ago, which involved starting with the Bayes Theorem (my preferred phrasing [1]). She’s a professional engineer, and found it very interesting (to the point where she kept prodding me for the next lesson), saying that it made much more sense of statistics. In about 1.5-2 hours total, I covered the Theorem, application to a car alarm situation, aggregating independent pieces of evidence, the use of log-odds, and some stuff on Bayes nets and using dependent pieces of evidence.
[1] O(H|E) = O(H) * L(E|H) = O(H) * P(E|H) / P(E|~H) = “On observing evidence, amplify the odds you assign to a belief by the probability of seeing the evidence if the belief were true, relative to if it were false.”
Expansion on the explanation about domestic violence victims—the victim may also be afraid that the government will not protect them from the abuser, and the abuser will be angrier because of the attempt at prosecution.
“That is, do you believe you are aware of the inferential connections between your expertise and layperson-level knowledge?”
This is related to an idea that has been brewing at the back of my mind for a while now:
Experts aren’t always good teachers because their problem solving algorithms may operate at a level of abstraction that is inaccessible to a beginner.
Hmm… I’m not sure if I think of myself as an expert at anything, other than when people ask. But I’m pretty sure I have about the best understanding of logic I can hope to have, and could explain virtually all of it to an attentive small child given sufficient time.
And I might be an expert at some sort of computer programming, though I can think of people much better at any bit of it that I can think of; at any rate, I am also confident I could teach that to anyone, or at least anyone who passes a basic test
Computer programming: I’m not sure if I am at Level 2 or not on this.
In favor of being at Level 2: I regularly think about non-computer-related topics with a CS-like approach (i.e. using information theory ideas when playing the inference game Zendo).
Also, I strongly associate my knowledge of “folk psychology” and “folk science” to computer science ideas, and these insights work in both directions. For example, the “learned helplessness” phenomenon, where inexperienced users become so uncomfortable with a system that they prefer to cling to their inexperienced status than to risk failure in an attempt to understand the system better, appears in many areas of life having nothing directly to do with computers.
Evidence against being at Level 2: I do not have the necessary computer engineering knowledge to connect my understanding of computer programming to my understanding of physics. And, although I have not tried this very often, my experiments in attempting to teach computer programming to laypeople have been middling at best.
My assessment at this point is that I am probably near to Level 2 in computer programming, but not quite there yet.
Can you teach a talented, untrained person a skill so that they exceed your own ability? Can you then identify why they are superior? If you have deep level knowledge of your area of expertise that you can impart to others, you ought to be able to evaluate and train a replacement based on “raw talent.”
Considering that intellectual or artistic endeavors may have a variety of details hidden even from the expert, perhaps a clearer example may be found in sports coaches.
Perhaps a clearer example may be found in sports coaches.
The main reason that coaches are important (not just in sports) is because of blind spots—i.e., things that are outside of a person’s direct perceptual awareness.
Think of the Dunning-Kreuger effect: if you can’t perceive it, you can’t improve it.
(This is also why publications have editors; if a writer could perceive the errors in their work, they could fix them themselves.)
I thought I’d pose an informal poll, possibly to become a top-level, in preparation for my article about How to Explain.
The question: on all the topics you consider yourself an “expert” or “very knowledgeable about”, do you believe you understand them at least at Level 2? That is, do you believe you are aware of the inferential connections between your expertise and layperson-level knowledge?
Or, to put it another way, do you think that, given enough time, but using only your present knowledge, you could teach a reasonably-intelligent layperson, one-on-one, to understand complex topics in your expertise, teaching them every intermediate topic necessary for grounding the hardest level?
Edit: Per DanArmak’s query, anything you can re-derive or infer from your present knowledge counts as part of your present knowledge for purposes of answering this question.
I’ll save my answer for later—though I suspect many of you already know it!
I have a (I suspect unusual) tendency to look at basic concepts and try to see them in as many ways as possible. For example, here are seven equations, all of which could be referred to as Bayes’ Theorem:
However, each one is different, and forces a different intuitive understanding of Bayes’ Theorem. The fourth one down is my favourite, as it makes obvious that the update depends only on the ratio of likelihoods. It also gives us our motivation for taking odds, since this clears up the 1/(1+x)ness of the equation.
Because of this way of understanding things, I find explanations easy, because if one method isn’t working, another one will.
ETA: I’d love to see more versions of Bayes’ Theorem, if anyone has any more to post.
P (H|E) = P (H and E) / P(E)
which tends to be how conditional probability is defined, and actually the first version of Bayes that I recall seeing.
Very well said, and doubles as a reply to the last part of my comment here. (When I read your comment in my inbox, I thought it was actually a reply to that one! Needless to say, I my favorite versions of the theorem are the last two you listed.)
This strikes me as an un-lifelike assumption. If I had to explain things in this way, I would expect to encounter some things that I don’t explicitly know (and other that I knew and have forgotten), and to have to (re)derive them. But I expect that I would be able to rederive almost all of them.
Refining my own understanding is a natural part of building a complex explanation-story to tell to others, and will happen unless I’ve already built this precise story before and remember it.
For purposes of this question, things you can rederive from your present knowledge count as part of your present knowledge.
I think I know a fair amount about doing calligraphy, but I’m dubious that someone could get a comparable level of knowledge without doing a good bit of calligraphy themselves.
If I were doing a serious job of teaching, I would be learning more about how to teach as I was doing it.
I consider myself to be a good but not expert explainer.
Possibly of interest: The 10-Minute Rejuvenation Plan: T5T: The Revolutionary Exercise Program That Restores Your Body and Mind : a book about an exercise system which involves 5 yoga moves. It’s by a woman who’d taught 700 people how to do the system, and shows an extensive knowledge of the possible mistakes students can make and adaptations needed to make the moves feasible for a wide variety of people.
My point is that explanation isn’t an abstract perfectible process existing simply in the mind of a teacher.
But in some limited areas explanation is completely adequate.
I taught co-worker how to do sudoku puzzles. After teaching him the human-accessible algorithms and allowing time for practice, I was still consistently beating his time. I knew why, and he didn’t. After I explained the difference in mental state I was using, he began beating my time on regular basis. {Instead of checking the list of 1-9 for each box or line, allow your brain to subconsciously spot the missing number and then verify its absence.} He is more motivated and has more focus, while I do puzzles to kill time when waiting.
In another job where I believe I had a thorough understanding of the subject, I was never able to teach any of my (~20) trainees to produce vector graphic maps with the speed and accuracy I obtained because I was unable to impart a mathematical intuition for the approximation of curves. I let them go home with full pay when they completed their work, so they definitely had motivation. But they also had editors who were highly detail oriented.
I mean to suggest that there is a continuum of subjective ability comparing different skills. Sudoku is highly procedural, once familiar all that is required is concentration. Yoga, in the sense mentioned above, is also procedural, proscriptive; the joints allow a limited number of degrees of freedom. Calligraphy strives for an ideal, but depending on the tradition, there is a degree of interpretation allowed for aesthetic considerations. Mapping, particularly in vector graphics, has many ways to be adequate and no way to be perfect.
The number of acceptable outcomes and the degree of variation in useful paths determines the teach-ability of a skillset. The procedural skills can be taught more easily than the subjective, and practice is useful to accomplish mastery of procedural skills. Deeper understanding of a field allows more of the skill’s domain to be expressed procedurally rather than subjectively.
I’m in general agreement, but I think you’re underestimating yoga—a big piece of it is improving access to your body’s ability to self-organize.
I like “many ways to be adequate and no way to be perfect”. I think most of life is like that, though I’ll add “many ways to be excellent”.
No slight to yoga intended. I only wanted to address the starting point of yoga. I know it is a quite comprehensive field.
I will reply to this in the sense of
since I am not so familiar with the formalism of a “Level 2” understanding.
My uninteresting, simple answer is: yes.
My philosophical answer is that I find the entire question to be very interesting and strange. That is, the relationship between teaching and understanding is quite strange IMO. There are many people who are poor teachers but who excel in their discipline. It seems to be a contradiction because high-level teaching skill seems to be a sufficient, and possibly necessary condition for masterful understanding.
Personally I resolve this contradiction in the following way. I feel like my own limitations make it to where I am forced to learn a subject by progressing at it in very simplistic strokes. By the time I have reached a mastery, I feel very capable of teaching it to others, since I have been forced to understand it myself in the most simplistic way possible.
Other people, who are possibly quite brilliant, are able to master some subjects without having to transmute the information into a simpler level. Consequentially, they are unable to make the sort of connections that you describe as being necessary for teaching.
Personally I feel that the latter category of people must be missing something, but I am unable to make a convincing argument for this point.
A lot of the questions you pose, including the definition of the Level 2 formalism, are addressed in the article I linked (and wrote).
I classify those who can do something well but not explain or understand the connections from the inputs and outputs to the rest of the world, to be at a Level 1 understanding. It’s certainly an accomplishment, but I agree with you that it’s missing something: the ability to recognize where it fits in with the rest of reality (Level 2) and the command of a reliable truth-detecting procedure that can “repair” gaps in knowledge as they arise (Level 3).
“Level 1 savants” are certainly doing something very well, but that something is not a deep understanding. Rather, they are in the position of a computer that can transform inputs into the right outputs, but do nothing more with them. Or a cat, which can fall from great heights without injury, but not know why its method works.
(Yes, this comment seems a bit internally repetitive.)
Ah, OK, I read your article. I think that’s an admirable task to try to classify or identify the levels of understanding. However, I’m not sure I am convinced by your categorization. It seems to me that many of these “Level 1 savants” as you call them are quite capable of fitting their understanding with the rest of reality. Actually it seems like the claim of “Level 1 understanding” basically trivializes that understanding. Yet many of these people who are bad teachers have a very nontrivial understanding—else I don’t think this would be such a common phenomena, for example, in academia. I would argue that these people have some further complications or issues which are not recognized in the 1-2-3 hierarchy.
That being said, you have to start somewhere, and the 0-1-2-3 hierarchy looks like a good place to start. I’d definitely be interested in hearing more about this analysis.
Thanks for reading it and giving me feedback. I’m interested in your claim:
Well, they can fit it in the sense that they (over a typical problem set) can match inputs with (what reality deems) the right outputs. But, as I’ve defined the level, they don’t know how those inputs and outputs relate to more distantly-connected aspects of reality.
I had a discussion with others about this point recently. My take is basically: if their understanding is so deep, why exactly is their teaching skill so brittle that no one can follow the inferential paths they trace out? Why can’t they switch to the infinite other paths that a Level 2 understanding enables them to see? If they can’t, that would suggest a lack of depth to their understanding.
And regarding the archetypal “deep understanding, poor teacher” you have in mind, do you envision that they could, say, trace out all the assumptions that could account for an anomalous result, starting with the most tenuous, and continuing outside their subfield? If not, I would call that falling short of Level 2.
I would LOVE to agree with this statement, as it justifies my criticism of poor teachers who IMO are (not usually maliciously) putting their students through hell. However, I don’t think it’s obvious, or I think maybe you just have to take it as an axiom of your system. It seems there is some notion of individualism or personal difference which is missing from the system. If someone is just terrible at learning, can you really expect to succeed in explaining, for example? Realistically I think it’s probably impossible to classify the massive concept of understanding by merely three levels, and these problems are just a symptom of that fact.
As another example, in order to understand something, it’s clearly necessary to be able to explain it to yourself. In your system, you are additionally requiring that your understanding means you must be able to explain things to other people. In order to explain things to others, you have to understand them, as has been discussed. Therefore you have to be able to explain other people to yourself. Why should an explanation of other individuals behavior be necessary for understanding some random area of expertise, say, mathematics? It’s not clear to me.
It certainly seems like someone with a deep understanding of their subject should be able to identify the validity or uncertainty in their assumptions about the subject. If they are a poor teacher, I think I would still believe this to be true.
I’ve thought about this some, and I think I see your point now. I would phrase it this way: It’s possible for a “Level 3 savant” to exist. A Level 3 savant, let’s posit, has a very deeply connected model of reality, and their excellent truth-detecting procedure allows them to internally repair loss of knowledge (perhaps below the level of their conscious awareness).
Like an expert (under the popular definition), and like a Level 1 savant, they perform well within their field. But this person differs in that they can also peform well in tracing out where its grounding assumptions go wrong—except that they “just have all the answers” but can’t explain, and don’t know, where the answers came from.
So here’s what it would look like: Any problem you pose in the field (like an anomalous result), they immediately say, “look at factor X”, and it’s usually correct. They even tell you to check critical aspects of sensors, or identify circularity in the literature that grounds the field (i.e. sources which generate false knowledge by excessively citing each other), even though most in the field might not even think about or know how all those sensors work.
All they can tell you is, “I don’t know, you told me X, and I immediately figured it had to be a problem with Y misinterpreting Z. I don’t know how Z relates to W, or if W directly relates to X, I just know that Y and Z were the problem.”
I would agree that there’s no contradiction in the existence of such a person. I would just say that in order to get this level of skill you have to accomplish so many subgoals that it’s very unlikely, just as it’s hard to make something act and look like a human without also making it conscious. (Obvious disclaimer: I don’t think my case is as solid as the one against P-zombies.)
I think that the “teaching” benchmark you claim here is actually a bit weaker than a Level 2 understanding. To successfully teach a topic, you don’t need to know lots of connections between your topic and everything else; you only need to know enough such connections to convey the idea. I really think this lies somewhere between Level 1 and Level 2.
I’ll claim to have Level 2 understanding on the core topics of my graduate research, some mathematics, and some core algorithmic reasoning. I’m sure I don’t have all of the connections between these things and the rest of my world model, but I do have many, and they pervade my understanding.
I agree in the sense that full completion of Level 2 isn’t necessary to do what I’ve described, as that implies a very deeply-connected set of models, truly pervading everything you know about.
But at the same time, I don’t think you appreciate some of the hurdles to the teaching task I described: remember, the only assumption is that the student has lay knowledge and is reasonably intelligent. Therefore, you do not get to assume that they find any particular chain of inference easy, or that they already know any particular domain above the lay level. This means you would have to be able to generate alternate inferential paths, and fall back to more basic levels “on the fly”, which requires healthy progress into Level 2 in order to achieve—enough that it’s fair to say you “round to” Level 2.
If so, I deeply respect you and find that you are the exception and not the rule. Do you find yourself critical of how people in the field (i.e. through textbooks, for example) present it to newcomers (who have undergrad prerequisites), present it to laypeople, and use excessive or unintuitive jargon?
I agree that the teaching task does require a thick bundle of connections, and not just a single chain of inferences. So much so, actually, that I’ve found that teaching, and preparing to teach, is a pretty good way to learn new connections between my Level 1 knowledge and my world model. That this “rounds” to Level 2 depends, I suppose, on how intelligent you assume the student is.
Yes, constantly. Frequently, I’m frustrated by such presentations to the point of anger at the author’s apparent disregard for the reader, even when I understand what they’re saying.
I think I have level 2 understanding of many areas of Biology but of course not all of it. It is too large a field. But there are gray areas around my high points of understanding where I am not sure how deep my understanding would go unless it was put to the test. And around the gray areas surrounding the level 2 areas there is a sea of superficial understanding. I have some small areas of computer science at level 2 but they are fewer and smaller, ditto chemistry and geology. I think your question overlooks the nature of teaching skills. I am pretty good at teaching (verbally and one/few to one) and did it often for years. There is a real knack in finding the right place to start and the right analogies to use with a particular person. Someone could have more understanding than me and not be able to transfer that understanding to someone else. And others could have less understanding and transfer it better. Finally I like your use of the word ‘understanding’ rather than ‘knowledge’. It implies the connectedness with other areas required to relate to lay people.
Perhaps the reason experts aren’t always good teachers is because their thought processes / problem solving algorithms operate at a level of abstraction that is inaccessible to a beginner.
I have some trouble answering your question, chiefly because my definition of “expert” is approximately synonymous with your definition of “Level 2″.
“Enough time” would be quite a long period of time. One problem is that there are a lot of textbook results that I would have to use in intermediate steps that would take me a long time to derive. Another is that there are a lot of experimental parameters that I haven’t memorized and would have to look up. But I think I could teach arithmetic, algebra, geometry, calculus, differential equations, and Newtonian physics enough that I could teach them proper engineering analysis.
Criminal Law: Yes to Level 2. Yes to teaching a layperson. It would take a while, for sure, but it’s doable. Some of the work requires an understanding of a different lifestyle; if you can’t see the potential issues with prosecuting a robbery by a prostitute and her armed male friend, or you can’t predict that a domestic violence victim will have a non-credible recantation, you’ll need some other education.
I’ve done a lot of instruction in this field. It is common for instruction not to take until there’s other experience in the field which helps things join up.
Bridge: Yes to Level 2. Possibly to teaching a layperson. The ability to play bridge well is correlated heavily to intelligence, but it also correlates to a certain zeal for winning. I have taught one person to play very well indeed, but that may not be replicable, and took years. (On an aside, I am very likely the world’s foremost expert on online bridge cheating; teaching cheating prevention would require teaching bridge first.)
Teaching requires more than reasonable intelligence on the part of the teachee. Some people who are very intelligent are ineducable. (Many of these are violators of my 40% rule: You are allowed to think you are 40% smarter/faster/stronger/better than you are. After that, it’s obnoxious.) Some people are not interested in learning a given subject. Some people will not overcome preset biases. Some people have high aptitudes in some areas and little aptitude in others (though intelligence strongly tends to spill over.)
Anyway, I’m interested in the article. My penultimate effort to explain something to many people—Bayes’ Theorem to lawyers—was a moderate failure; my last effort to explain something less mathy to a crowd was a substantial success. (My last experience in explaining something, with assistance, to 12 people was a complete failure.)
--JRM
I’m curious, why did you chose 40% for your “40% rule”?
It’s non-arbitrary, but neither is it precise. 100% is clearly too high, and 10% is clearly too low.
And since I started calling it The 40% Rule fifteen years ago or thereabout, a number of my friends and acquaintances have embraced the rule in this incarnation. Obviously, some things are unquantifiable and the specific number has rather limited application. But people like it at this number. That counts for something—and it gets the message across in a way that other formulations don’t.
Some are nonplussed by the rule, but the vigor of support by some supporters gives me some thought that I picked a number people like. Since I never tried another number, I could be wrong—but I don’t think I am.
--JRM
“The people who buy the services of a prostitute generally don’t want to go on record saying so, which they would have to do at some point to prosecute such a robbery. This is either because they’re married, or the shame associated with using one.”
“Victims of domestic violence have a lot invested in the relationship, and, no matter how much they feel hurt by the abuse, they will not want to tear apart the family and cripple their spouse with a felony conviction. This inner conflict will be present when the victim tries to recant their testimony.”
Did that really require passing the learner off for some other education? Or did I get the explanation wrong?
I’d actually tried teaching information theory to my mom a week ago, which involved starting with the Bayes Theorem (my preferred phrasing [1]). She’s a professional engineer, and found it very interesting (to the point where she kept prodding me for the next lesson), saying that it made much more sense of statistics. In about 1.5-2 hours total, I covered the Theorem, application to a car alarm situation, aggregating independent pieces of evidence, the use of log-odds, and some stuff on Bayes nets and using dependent pieces of evidence.
[1] O(H|E) = O(H) * L(E|H) = O(H) * P(E|H) / P(E|~H) = “On observing evidence, amplify the odds you assign to a belief by the probability of seeing the evidence if the belief were true, relative to if it were false.”
Expansion on the explanation about domestic violence victims—the victim may also be afraid that the government will not protect them from the abuser, and the abuser will be angrier because of the attempt at prosecution.
This is related to an idea that has been brewing at the back of my mind for a while now: Experts aren’t always good teachers because their problem solving algorithms may operate at a level of abstraction that is inaccessible to a beginner.
Hmm… I’m not sure if I think of myself as an expert at anything, other than when people ask. But I’m pretty sure I have about the best understanding of logic I can hope to have, and could explain virtually all of it to an attentive small child given sufficient time.
And I might be an expert at some sort of computer programming, though I can think of people much better at any bit of it that I can think of; at any rate, I am also confident I could teach that to anyone, or at least anyone who passes a basic test
Computer programming: I’m not sure if I am at Level 2 or not on this.
In favor of being at Level 2: I regularly think about non-computer-related topics with a CS-like approach (i.e. using information theory ideas when playing the inference game Zendo).
Also, I strongly associate my knowledge of “folk psychology” and “folk science” to computer science ideas, and these insights work in both directions. For example, the “learned helplessness” phenomenon, where inexperienced users become so uncomfortable with a system that they prefer to cling to their inexperienced status than to risk failure in an attempt to understand the system better, appears in many areas of life having nothing directly to do with computers.
Evidence against being at Level 2: I do not have the necessary computer engineering knowledge to connect my understanding of computer programming to my understanding of physics. And, although I have not tried this very often, my experiments in attempting to teach computer programming to laypeople have been middling at best.
My assessment at this point is that I am probably near to Level 2 in computer programming, but not quite there yet.
Can you teach a talented, untrained person a skill so that they exceed your own ability? Can you then identify why they are superior? If you have deep level knowledge of your area of expertise that you can impart to others, you ought to be able to evaluate and train a replacement based on “raw talent.”
Considering that intellectual or artistic endeavors may have a variety of details hidden even from the expert, perhaps a clearer example may be found in sports coaches.
The main reason that coaches are important (not just in sports) is because of blind spots—i.e., things that are outside of a person’s direct perceptual awareness.
Think of the Dunning-Kreuger effect: if you can’t perceive it, you can’t improve it.
(This is also why publications have editors; if a writer could perceive the errors in their work, they could fix them themselves.)