Yup, it seems to me quite correct that general expertise in pedagogy, and specifically pedagogy of technical subjects (math, physics etc) is highly relevant. I do believe that one of the founders of CFAR has a PhD in Math/Science Education, so I think that Eliezer+Anna agreed with you and sought such expertise pretty early on.
(In general from my experience of CFAR workshops, high school, and university, CFAR seems to have excelled in incorporating general lessons of pedagogy, and I think as a whole have been very strong on the due diligence of learning from experts in fields that have valuable insights for their work such as cognitive psychology and teaching. Their lessons often include case studies of pedagogical mistakes they learned from various literature reviews, and they teach how to avoid these mistakes when trying yourself to learn a skill.)
I’m curious if we in fact have a disagreement? Might you want to defend the stronger claim that not just ought they have looked into such fields early on (as they did) but in fact not attempted to test an improvised class whatsoever until they had concluded an in-depth literature review and discussed it with people with more experience and a better track record in pedagogy?
Is Said claiming anything about there being an object-level consensus on these issues? If there’s an academic consensus among education specialists saying something like ‘always improvise classes as a test run as early as possible, even if you don’t have prior experience and don’t have a lesson plan or exercises worked out yet’, then that certainly seems relevant to this example. Ditto to a lesser extent if script-reading and student-led classes are known to be a dead end.
I don’t think Eliezer and the modest would actually disagree that if there’s an academic consensus like that, then that’s an important input to the decisionmaking process. I think that treating this as the crux would not be steel-manning modest epistemology, and would be straw-manning the civilizational inadequacy viewpoint.
But it’s still relevant to the object-level dispute, and the object-level disputes all matter because (in aggregate) they help us judge how these different reasoning heuristics actually perform in the real world.
(This is one reason I’ve been happy to see how often these LW discussions have ended up focusing in on specific object-level details and examples. Eliezer’s book is in some sense an argument for “actually, let’s just talk more about object-level stuff,” so this seems totally appropriate to me.)
I don’t know whether there’s an object-level consensus; I am not an expert in education, after all. (I’ll ask my mother about it at some point, perhaps; it is an interesting point on its own, certainly.)
But this post is fairly clearly making a meta-level point, which is what I am addressing.
Is there an academic[1] consensus? Well, perhaps the thing to ask is, why don’t we, the readers, already know the answer to that? This should be discussed in Eliezer’s post! At the point in his story when the question arises, Eliezer should say—“and then, instead of spending any serious length of time considering or debating which of us was right, we immediately sought the counsel of experts; and delved into the literature; and here is what we found…” He doesn’t say any such thing, because (it seems) nothing of the sort ever happened, and instead he and Anna argued the matter, and then went as far as actually spending two months implementing one plan, and then some more time implementing another plan, etc., without ever seeking expert guidance.
In short, the fact that there is still, among us readers, a question of what the expert consensus is, is itself damning!
[1] By the way, it’s noteworthy that you speak of “academic consensus”, whereas I spoke of expert competence, of domain knowledge, of professional expertise. This is the old dichotomy of an “expert on” vs. an “expert at”. A working educator, even one with a Ph.D., is an expert at, whereas someone who studies the field and does academic research on it is an expert on.
Now, I do not say that in such cases you ought not consult experts on the field of your endeavor, and indeed your expert consultants should have at least some expertise on the field in question; but expertise at the field is at least as important.
There’s a core assumption in CFAR that the way the academics currently go about inventing debiasing interventions doesn’t seem to work well and that it’s worth to try a different strategy.
A while ago I tried to read up on education research. The problem in which I ran into, was that most papers are full of postmodern jargon. The class that was taught at our university was full of such paper assignements. Postmodern fields generally don’t have much consensus either.
Bill Gates presented statistics in one of his Ted talks about what determines teacher performance. According to his numbers students of teacher who have a masters of education don’t do better than when the teacher has only a bachlor degree.
This means that the knowledge that’s taught seems to be worthless for getting students to learn the material they are supposed to learn better and in addition the teaching quality of the education professors is so bad that they don’t manage to teach actual teaching skills.
Yes, that is precisely why I suggested that the right way to go is to speak to a working educator, an expert at, and not merely speaking to academics (experts on), and certainly not to simply read some papers!
There’s a core assumption in CFAR…
And is it an axiom, that CFAR is right about these things? Isn’t that what we’re here to discuss?
This means that the knowledge that’s taught seems to be worthless for getting students to learn the material they are supposed to learn better and in addition the teaching quality of the education professors is so bad that they don’t manage to teach actual teaching skills.
This does not match my experience. In any case, what you would want to do (obviously, or so I thought) is to find an educator who is known (via recommendation, reputation, etc.) to be capable.
It’s hard to teach implicit unconscious knowledge; the best pianists aren’t also the best piano teachers. Just because someone can teach math, doesn’t mean they can teach teaching!
It seems like they could still demonstrate the procedure they’d use for teaching, if only for an unrelated topic, and some of that could be transferred. I expect it would be more efficient than trying to reconstruct the syllabus-building techniques (&c) separately.
There are also teachers who train other teachers, presumably, so you could employ the help of one of them.
I do believe that one of the founders of CFAR has a PhD in Math/Science Education, so I think that Eliezer+Anna agreed with you and sought such expertise pretty early on.
Did they? Then what on earth is this post about…?
Might you want to defend the stronger claim that not just ought they have looked into such fields early on (as they did) but in fact not attempted to test an improvised class whatsoever until they had concluded an in-depth literature review and discussed it with people with more experience and a better track record in pedagogy?
Absolutely—although I would amend, perhaps, the ‘literature review’ bit—not that it would be inadvisable, just that I’d seek out an expert to speak with first, and then, as part of (or in parallel with) following that expert’s advice, review the literature. (Or, heck, do it immediately, why not. Depends how easy it is to deploy one a team member to do a lit review, vs. how easy it is to get hold of a suitable expert.)
More fundamentally, what I am saying is that Eliezer’s ruminations about whether to trust himself or Anna on this matter are simply irrelevant, because both of the possible answers that he proffers are wrong. If it costs nothing, or very little, to test an improvised class, sure, do it. Heck, do whatever you want, at any time and for any reason. But if you think that what you’re doing matters; if success is important, if failure is bad, if time and effort spent on the attempt are valuable; then the answer to “do I trust myself (a non-expert) or my friend (also a non-expert)” is “neither; immediately find an expert and consult them”.
Rule M has a stronger version of the same problem....it assumes methodological solipsisn. And the EY’s preferred alternative assumes non-solipsism, ie interacting with people and things. But non-solipsism isn’t immodesty.
Yup, it seems to me quite correct that general expertise in pedagogy, and specifically pedagogy of technical subjects (math, physics etc) is highly relevant. I do believe that one of the founders of CFAR has a PhD in Math/Science Education, so I think that Eliezer+Anna agreed with you and sought such expertise pretty early on.
(In general from my experience of CFAR workshops, high school, and university, CFAR seems to have excelled in incorporating general lessons of pedagogy, and I think as a whole have been very strong on the due diligence of learning from experts in fields that have valuable insights for their work such as cognitive psychology and teaching. Their lessons often include case studies of pedagogical mistakes they learned from various literature reviews, and they teach how to avoid these mistakes when trying yourself to learn a skill.)
I’m curious if we in fact have a disagreement? Might you want to defend the stronger claim that not just ought they have looked into such fields early on (as they did) but in fact not attempted to test an improvised class whatsoever until they had concluded an in-depth literature review and discussed it with people with more experience and a better track record in pedagogy?
Is Said claiming anything about there being an object-level consensus on these issues? If there’s an academic consensus among education specialists saying something like ‘always improvise classes as a test run as early as possible, even if you don’t have prior experience and don’t have a lesson plan or exercises worked out yet’, then that certainly seems relevant to this example. Ditto to a lesser extent if script-reading and student-led classes are known to be a dead end.
I don’t think Eliezer and the modest would actually disagree that if there’s an academic consensus like that, then that’s an important input to the decisionmaking process. I think that treating this as the crux would not be steel-manning modest epistemology, and would be straw-manning the civilizational inadequacy viewpoint.
But it’s still relevant to the object-level dispute, and the object-level disputes all matter because (in aggregate) they help us judge how these different reasoning heuristics actually perform in the real world.
(This is one reason I’ve been happy to see how often these LW discussions have ended up focusing in on specific object-level details and examples. Eliezer’s book is in some sense an argument for “actually, let’s just talk more about object-level stuff,” so this seems totally appropriate to me.)
I don’t know whether there’s an object-level consensus; I am not an expert in education, after all. (I’ll ask my mother about it at some point, perhaps; it is an interesting point on its own, certainly.)
But this post is fairly clearly making a meta-level point, which is what I am addressing.
Is there an academic[1] consensus? Well, perhaps the thing to ask is, why don’t we, the readers, already know the answer to that? This should be discussed in Eliezer’s post! At the point in his story when the question arises, Eliezer should say—“and then, instead of spending any serious length of time considering or debating which of us was right, we immediately sought the counsel of experts; and delved into the literature; and here is what we found…” He doesn’t say any such thing, because (it seems) nothing of the sort ever happened, and instead he and Anna argued the matter, and then went as far as actually spending two months implementing one plan, and then some more time implementing another plan, etc., without ever seeking expert guidance.
In short, the fact that there is still, among us readers, a question of what the expert consensus is, is itself damning!
[1] By the way, it’s noteworthy that you speak of “academic consensus”, whereas I spoke of expert competence, of domain knowledge, of professional expertise. This is the old dichotomy of an “expert on” vs. an “expert at”. A working educator, even one with a Ph.D., is an expert at, whereas someone who studies the field and does academic research on it is an expert on.
Now, I do not say that in such cases you ought not consult experts on the field of your endeavor, and indeed your expert consultants should have at least some expertise on the field in question; but expertise at the field is at least as important.
There’s a core assumption in CFAR that the way the academics currently go about inventing debiasing interventions doesn’t seem to work well and that it’s worth to try a different strategy.
A while ago I tried to read up on education research. The problem in which I ran into, was that most papers are full of postmodern jargon. The class that was taught at our university was full of such paper assignements. Postmodern fields generally don’t have much consensus either.
Bill Gates presented statistics in one of his Ted talks about what determines teacher performance. According to his numbers students of teacher who have a masters of education don’t do better than when the teacher has only a bachlor degree.
This means that the knowledge that’s taught seems to be worthless for getting students to learn the material they are supposed to learn better and in addition the teaching quality of the education professors is so bad that they don’t manage to teach actual teaching skills.
Yes, that is precisely why I suggested that the right way to go is to speak to a working educator, an expert at, and not merely speaking to academics (experts on), and certainly not to simply read some papers!
And is it an axiom, that CFAR is right about these things? Isn’t that what we’re here to discuss?
This does not match my experience. In any case, what you would want to do (obviously, or so I thought) is to find an educator who is known (via recommendation, reputation, etc.) to be capable.
It’s hard to teach implicit unconscious knowledge; the best pianists aren’t also the best piano teachers. Just because someone can teach math, doesn’t mean they can teach teaching!
It seems like they could still demonstrate the procedure they’d use for teaching, if only for an unrelated topic, and some of that could be transferred. I expect it would be more efficient than trying to reconstruct the syllabus-building techniques (&c) separately.
There are also teachers who train other teachers, presumably, so you could employ the help of one of them.
Did they? Then what on earth is this post about…?
Absolutely—although I would amend, perhaps, the ‘literature review’ bit—not that it would be inadvisable, just that I’d seek out an expert to speak with first, and then, as part of (or in parallel with) following that expert’s advice, review the literature. (Or, heck, do it immediately, why not. Depends how easy it is to deploy one a team member to do a lit review, vs. how easy it is to get hold of a suitable expert.)
More fundamentally, what I am saying is that Eliezer’s ruminations about whether to trust himself or Anna on this matter are simply irrelevant, because both of the possible answers that he proffers are wrong. If it costs nothing, or very little, to test an improvised class, sure, do it. Heck, do whatever you want, at any time and for any reason. But if you think that what you’re doing matters; if success is important, if failure is bad, if time and effort spent on the attempt are valuable; then the answer to “do I trust myself (a non-expert) or my friend (also a non-expert)” is “neither; immediately find an expert and consult them”.
Rule M has a stronger version of the same problem....it assumes methodological solipsisn. And the EY’s preferred alternative assumes non-solipsism, ie interacting with people and things. But non-solipsism isn’t immodesty.