A civilization ran by amateurs
I
When I was a child, I remember thinking: Where do houses come from? They are huge! Building one would take forever! Yet there are so many of them!
Having become a boring adult, I no longer have the same blue-eyed wonder about houses, but humanity does have an accomplishment or two I’m still impressed by.
When going to the airport, the metal boulders really stay up in the air without crashing. Usually they leave at the time they told me two weeks earlier, taking me to the right destination at close to the speed of sound.
There are these boxes with buttons that you can press to send information near-instantly anywhere. They are able to perform billions of operations a second. And you can just buy them at a store!
And okay, I admit that big houses—skyscrapers—still light up some of that child-like marvel in me.
II
Some time ago I watched the Eurovision song contest. For those who haven’t seen it, it looks something like this:
It’s a big contest, and the whole physical infrastructure—huge hall, the stage, stage effects, massive led walls, camera work—is quite impressive. But there’s an objectively less impressive thing I want to focus on here: the hosts.
I basically couldn’t notice the hosts making any errors. They articulate themselves clearly, they don’t stutter or stumble on their words, their gestures and facial expressions are just what they are supposed to be, they pause their speech at the right moments for the right lengths, they could fluently speak some non-English languages as well, …
And, sure, this is not one-in-a-billion talent—there are plenty of competent hosts in all kinds of shows—but they clearly are professionals and much more competent than your average folk.
(I don’t know about you, but when I’ve given talks to small groups of people, I’ve started my sentences without knowing how they’ll end, talked too fast, stumbled in my speech, and my facial expressions probably haven’t been ideal. If the Eurovision hosts get nervous when talking to a hundred million people, it doesn’t show up.)
III
I think many modern big-budget movies are pretty darn good.
I’m particularly thinking of Oppenheimer and the Dune series here (don’t judge my movie taste), but the point is more general. The production quality of big movies is extremely high. Like, you really see that these are not amateur projects filmed in someone’s backyard, but there’s an actual effort to make a good movie.
There’s, of course, a written script that the actors follow. This script has been produced by one or multiple people who have previously demonstrated their competence. The actors are professionals who, too, have been selected for competence. If they screw up, someone tells them. A scene is shot again until they get it right. The actors practice so that they can get it right. The movie is, obviously, filmed scene-by-scene. There are the cuts and sounds and lighting. Editing is used to fix some errors—or maybe even to basically create the whole scene. Movie-making technology improves and the new technology is used in practice, and the whole process builds on several decades of experience.
Imagine an alternative universe where this is not how movies were made. There is no script, but rather the actors improvise from a rough sketch—and by “actors” I don’t mean competent Eurovision-grade hosts, I mean average folk paid to be filmed. No one really gives them feedback on how they are doing, nor do they really “practice” acting on top of simply doing their job. The whole movie is shot in one big session with no cuts or editing. People don’t really use new technology for movies, but instead stick to mid-to-late-1900s era cameras and techniques. Overall movies look largely the same as they have looked for the last few decades.
Obviously the movies would be way worse in quality there than here, and if people there wanted better movies, they would start to do things our way.
IV
This is how I feel about education and teaching.
In our world, teaching is far too often improvisation by amateurs.
Sometimes in the very literal meaning of these words. For example, for years I’ve been involved in the training of Finland’s mathematically most talented high schoolers, both as a student and a teacher. I started teaching fresh out of high school. I was an amateur in most every meaning of the word, and—don’t tell anyone—I didn’t always have a fully planned-out script for my lessons.
(You don’t have to believe me, but even in retrospect I don’t think I was a particularly bad teacher either; this is just what the training looks like.[1])
And almost always, teaching is practically improvisation by amateurs, regardless of whether they have a pedagogy degree or not, or years of work experience or not, or are paid to do it or not.[2] The teacher goes through the materials in one run. If the teacher screws something up, there’s no one to tell them (and too bad for the students). Preparation involves maybe looking at the materials beforehand, but rarely there is a full practice run before the real lesson. Teaching methods, content and the classroom look largely the same as a few decades ago.[3]
Oh, and remember the alternative universe with amateur movies? They don’t actually film a movie by shooting it on a camera, but instead any time someone wants to watch the movie, the actors have to play it live. That is, they do theater. As a result, there are quite a lot of shows, requiring quite a lot of actors. (Of course, they can’t then put much effort into any single show.) They spend a huge amount of resources to keep the shows running year after year. Changes and improvements happen very slowly—you can’t make just expect actors to adapt to new acts just like that! - and there are few people working on them.
So what would non-amateur education look like, then? Here’s one vision:
Education builds largely on high-quality education videos, produced by similar methods as big-budget movies: scripts written carefully by professionals, shot scene-by-scene to get it right, using cuts, visual effects and editing to focus attention and communicate information, employing new techniques to constantly improve the videos. These videos are used at scale: once you have a good video, the cost of showing it to every student in the country is approximately zero, and production becomes cost-effective. You measure what the students learn (and what they don’t) and collect feedback—the education version of box-office revenue and movie reviews—to make the next videos even better, and well working methods developed by some are invented by all.
Of course, I’m not saying that all of education needs to be video-based, any more than current-day education only consists of a teacher lecturing. I also acknowledge that many current educational videos are poor and not very captivating, but don’t think this is indicative of the potential of educational videos. I just flat out don’t believe that a civilization that can make movies like Dune: Part Two couldn’t make educational videos that outperform usual classroom improvisations.[4]
V
There are other places where amateurism pops up (though perhaps to a lesser extent).
In politics, representatives have to deal with a vast variety of potential issues, decisions and trade-offs, and it is a stretch to say that a single person can be qualified to assess all of them. Follow-up on how good the decisions have been, learning from mistakes and training of decisions making skills seem to be scarce to non-existent.
In academic research, there is surprisingly little education on how to do good research in practice. I’ve attended courses such as Academic writing and Ethics of academic research, but none on Choosing good research problems, Historical scientific mistakes, Elements of societally valuable research, or Case studies of scientific breakthroughs.
(I have some other guesses as well, but which I’m less familiar with and thus don’t feel comfortable criticizing—you know, I’m something of an amateur myself.)
There’s certainly a non-zero amount professionalism in these fields. Delegation and deferring to domain experts does happen in politics, and PhD students pick up research skills from their advisors, and this is great. But this is far from where one could hone things, similarly as educating future teachers in university pedagogy courses doesn’t completely solve education.
VI
What makes up for non-amateurism?
One key ingredient is professional specialization. A person has one job, focuses on one domain, rather than half-assing seven unrelated responsibilities. Instead of having a single middle school teacher handle distinct tasks of lecturing and personal guidance (not to mention maintaining order and other non-pedagogic aspects), the different roles are filled by different people who do that one thing well.
Another ingredient is iteration. You collect data and measure what works well and what doesn’t, and then improve. If a profession involves a lot of accidents, you keep track of what caused them and fix them. You have written instructions and warnings. You make doctors wash their hands. You do more of what works. You build on the previous things that have already worked.
A third ingredient is economies of scale. If a lot of people use a thing a lot, invest on making that thing better. Improving a textbook used annually by a hundred thousand students for ten years is quite possibly worth it.
Finally, there’s the crucial ingredient of incentives. There should be some pressure to do a good job. The movie industry has this one checked relatively well—make a good movie and you get a lot of money and fame.
These ingredients of course feed into each other: Once you are doing things at scale, it makes sense to have people working on it full-time. They can iterate on it, and have the incentive to improve things. Things (hopefully) improve, so it makes sense to invest more. The machine starts running.
VII
There’s a common sentiment that goes something like “when I was a child, I thought adults had it all figured out, but now that I’m an adult, I’ve realized that no one has any idea what they are doing”.
I do think that there is a lot of amateur hour stuff going on, and a lot of things lay on the shoulders of amateurs. This is regrettable. Economies of scale, building on previous expertise, professional specialization, education, and aligning incentives simply haven’t yet fully succeeded on all issues critical to our civilization.
I don’t accept stronger versions of the sentiment, though: I’m still impressed by things like Eurovision and modern movies, nevermind things like ubiquitous big buildings, computers, airports, and overall the last couple of centuries of immense material and technological development. This does display a level of competence worthy of the title “not amateur”.
We have got the non-amateurism machine partially running and working—it just needs some fixing to work reliably on the important issues.
- ^
Obligatory note: None of this is a personal insult to other teachers in the training system, who generously do unpaid volunteering while managing day jobs elsewhere. (This is precisely my point: they are not professionals in the literal meaning of the word, where this is the sole thing they are responsible for, but rather they have many other responsibilities as well—and of course this limits the quality of their lessons!)
- ^
I’m using the word “amateur” in the sense of unqualified, not as “isn’t paid to do this”.
- ^
Certainly there are some differences in teaching in 1974 and 2024, with digitalization and its friends being a central one. I still argue that the difference in movie-making from 1974 and 2024 is quite a bit larger. (I also make the claim that it is bad that teaching isn’t changing faster than it is.)
- ^
Indeed, one can already find quite high-quality educational videos from YouTube. 3Blue1Brown has received near-universal acclaim (at least in my circles), and sets a lower bound for how good videos one can make. (I also bet that, unlike for many Hollywood movies, the budget for 3Blue1Brown videos is less than $10 million per hour.)
Sorry for the noncentral point...
I actually don’t think 3Blue1Brown is all that great an example here. How many people, after watching his essence of calculus videos, could find a derivative or an integral of a reasonably complicated function? How many, after watching his linear algebra series, could find the eigenvectors & values of a 3x3 matrix? 3Blue1Brown seems very much like good supplementary material to me, or good as a first high-level approach to a math area.
I’d say a better example for “pedagogy done extraordinarily well at the high-school math level” is Khan Academy. At least, it was 7-8 years ago, and I expect even if it has returned a bit to the mean, LLMs are vastly improving the experience, which I know they’ve been using, and is a big step up over the alternative. Someone whose gone through the Khan Academy lessons on calculus or linear algebra has a far higher chance of correctly performing a derivative or an integral, or finding the eigenvectors & values of a 3x3 matrix.
ETA in the undergraduate & beginning graduate math level, textbooks become the things which are professionalized (but not those recommending textbooks to you), and in the advanced graduate level of course there is no professionalization in pedagogy.
The part about airports reminds me of “If All Stories were Written Like Science Fiction Stories” by Mark Rosenfelder:
https://www.bzpower.com/blogs/entry/58514-if-all-stories-were-written-like-science-fiction-stories/
No one else has mentioned The Case Against Education by Bryan Caplan. He says that after reading and arithmetic, schooling is mostly for signaling employable traits like conscientiousness, not for learning. I think Zvi Mowshowitz and Noah Smith had some interesting discussion about this years ago. Scott Alexander supposes that another secret purpose of school is daycare. Whatever the real purposes are, they will tend to be locked into place by laws. Richard Hanania has written a bit about what he thinks families might choose instead of standard schooling if the laws were relaxed.
I think the idea is really interesting. As someone who spent 5 years creating student video resources, I appreciate the impact they can have, and I have at times tried to convince my father—a life-long maths teacher to collaborate with me on replicating his course… but the fool didn’t take me up on the offer.
I feel like the cost-effectiveness argument is valid but might run into issues. To begin with, as you have in one of your comments pointed out, video resources with a teacher who can respond dynamically, adds much more than a video alone. So, this means there is no cost saving in terms of teachers time—which I think is a good thing (I’ll put a pin in that for later) and then video production on top of that is not at all cheap. One thing that was consistent, in my experience creating educational resources, was the need to constantly update the resources (there was a team of us working full time to just maintain one course).
So, while the cycle of feedback and constant improvement of the resources is a vital part of the process, it makes what seems like a one-off expense into a perpetual expense.
Furthermore teachers are already underpaid, relative to other professions requiring similar skills, so the additional funding for these new resources would need to result from an unprecedented increase in education funding (which could have gone to teachers) or would have to be taken from the budget at the expense of teachers.
Unless of course you leave it to the private sector in which case you have to worry about advertising, special interests and competition leading optimisation for what is appealing to students rather than what is necessarily effective—Hollywood, after all only has the mandate to entertain, they don’t have to also educate.
To get back to that pin: If we did manage to create a resource perhaps incorporating generative AI that can present ideas in an engaging way and provide dynamic feedback, making teachers unnecessary we run into another issue. There’s something to be said for having well-rounded educators in society, learning in a non-specialised way is enriching for people in general. One negative side of chat GPT is this way drastic drop-off in activity on forums like Stack Overflow, because people don’t need other people any more.
There’s something about the person to person trading of ideas that I think contributes to a robust community, in the same way that international trade helps to curb international conflicts—we might find that making human interaction unnecessary to education whether in schools or on forums might lead to a fragmentation of the social fabric. Personally I really like the idea of lots of amateurs sharing ideas—like on LessWrong and other forums, there’s something uniquely human about learning from sharing, with benefits for the teacher also (à la the Feynman Technique).
But, I think you make a good case. Thanks for sharing the idea.
Thanks for the interesting comment.
This is a fair point; I might be underestimating the amount of revision needed. On the other hand, I can’t help but think that surely the economies of scale still make sense here.
Yeah, I agree optimizing for learning is genuinely a harder task than optimizing for vague “I liked this movie” sentiment, and measuring and setting the incentives right is indeed tricky. I do think that setting the equivalent of Hollywood box-office revenue is actually hard. At the same time, I also think that there’s marginal value to be gained by moving into the more professionalized / specialized / scalable / “serious” direction.
Hmm, I suspect that you think I’m proposing something more radical than what I am. (I might be sympathetic to more extreme versions of what I propose, but what I’m actually putting forth is not very extreme, I’d say.) I had a brief point about this in my post, “Of course, I’m not saying that all of education needs to be video-based, any more than current-day education only consists of a teacher lecturing”
To illustrate, what I’m saying is more like “in a 45 min class, have half of your classes begin with a 15 minute well-made educational video explaining the topic, with the rest being essentially the status quo” rather than “replace 80% of your classes with 45 minute videos”. (Again, I wouldn’t necessarily oppose the latter one, but I do think that there are more things one should think through there.) And this would leave plenty of time for non-scripted, natural conversations, at the level that is currently being satisfied.
Another point I want to make: I think we should go much more towards “school is critical infrastructure that we run professionally” than where we currently are. In that, school is not the place where you want to have your improvised amateur hours at, and your authentic human connections could happen sometime else than when you learn new stuff (e.g. hobbies, or classes more designed with that in mind). Obviously if you can pick both you pick both, it’s important that students actually like going to school, with younger children learning the curriculum is far from the only goal, etc.
I think a key challenge in this particular case is that children (and adults to a lesser degree) respond differently to someone who’s physically present—they’re more engaged, they’re following the teacher’s social cues more closely, etc. And the teacher is able to also rapidly pick up cues from children—noticing if a kid is staring blankly, or when another kid looks really excited by what the teacher’s just said. All those aspects would be missing if kids are just watching videos.
I haven’t looked at the research here; these things just seem pretty straightforwardly clear to me. I’m prepared to believe I’m wrong on some or all of it if someone has dug more deeply.
I’ve not dug deep in the research literature either. A quick google search gave me this 2021 meta-analysis: Video Improves Learning in Higher Education: A Systematic Review. From the abstract: “Swapping video for existing teaching methods led to small improvements in student learning (g = 0.28). Adding video to existing teaching led to strong learning benefits (g = 0.80).”
(Huh, that’s better than I expected, apparently videos are already now very useful.)
When thinking of the arguments on “people respond differently to videos”, videos also have some points in favor of them: Videos can actually be quite captivating (there’s undeniably a lot of variance in both teachers and videos), you can insert visual cues to guide attention, it’s easier to iterate to alleviate these issues, …
Stronger effect size than I would have expected also! But unsurprising that the effect size would be much larger when video is added to existing teaching; it seems like that could give you the best of both.
But yeah, absent further info or a closer look at the literature, your argument seems more plausible to me than it did before your comment. Thanks!