Interesting critique of British education by outgoing advisor (warning: some politics)
The soon-to-be-resigning Dominic Cummings, advisor to the Education Secretary of the Coalition government, has released a 250-page manifesto describing the problems of the British educational establishment (“the blob” in Whitehall parlance) and offering solutions. I post this here because both his analysis and recommendations are likely to be interesting to LW, in particular an increased emphasis on STEM, broader knowledge of the limits of human reasoning and how they relate to managing complex systems, an appreciation for “agenty”-ness in organizational leadership, whole-brain emulation, intelligence enhancement, recursive self-improving AGI, analysis of human interactions on a firm evolutionary-psychological basis, and a rejection of fashionable pseudoscientific theories of psychology and society. Relevant extracts:
This essay is aimed mainly at ~15-25 year-olds and those interested in more ambitious education and training for them. Not only are most of them forced into mediocre education but they are also then forced into dysfunctional institutions where many face awful choices: either conform to the patterns set by middle-aged mediocrities (don’t pursue excellence, don’t challenge bosses’ errors, and so on) or soon be despised and unemployed. Some of the ideas sketched here may help some to shape their own education and create their own institutions. As people such as Linus Torvald (Linux) or Mark Zuckerberg (Facebook) have shown, the young are capable of much more than the powerful and middle-aged, who control so many institutions, like to admit.2 A significant change in education and training could also help us partly ameliorate the grim cycle of predictable political dysfunction.
Although it is generally psychologically unpleasant to focus on our problems and admit the great weaknesses of our institutions, including what we in politics and our leaders do not understand, it is only through honest analysis that progress is possible. As Maxwell said, ‘Thoroughly conscious ignorance is a prelude to every real advance in knowledge.’ Reliable knowledge about what works (how do people learn, what teaching methods work, how to use technology) should be built cumulatively on the foundation of empirical tests as suggested by physicist Carl Wieman and others (cf. Section 6 and Endnote). New systems and curricula would work in different ways for a school, a University, or an equivalent to a boxing gym for politicians if such a thing existed.
In particular, it is suggested that we need an ‘Odyssean’ education so that a substantial fraction of teenagers, students and adults might understand something of our biggest intellectual and practical problems, and be trained to take effective action.
The Nobel-winning physicist, Murray Gell Mann, one of the architects of the Standard Model of particle physics and namer of the ‘quark’,3 has described a scientific and political need for an ‘Odyssean’ philosophy that can synthesise a) maths and the natural sciences, b) the social sciences, and c) the humanities and arts, into necessarily crude, trans-disciplinary, integrative thinking about complex systems.
’Today the network of relationships linking the human race to itself and to the rest of the biosphere is so complex that all aspects affect all others to an extraordinary degree. Someone should be studying the whole system, however crudely that has to be done, because no gluing together of partial studies of a complex nonlinear system can give a good idea of the behavior of the whole...
’Those who study complex adaptive systems are beginning to find some general principles that underlie all such systems, and seeking out those principles requires intensive discussions and collaborations among specialists in a great many fields. Of course the careful and inspired study of each specialty remains as vital as ever. But integration of those specialities is urgently needed as well. Important contributions are made by the handful of scholars and scientists who are transforming themselves from specialists into students of simplicity and complexity or of complex adaptive systems in general…
‘[There is] the distinction (made famous by Nietzsche) between “Apollonians”, who favor logic, the analytical approach, and a dispassionate weighing of the evidence, and “Dionysians”’, who lean more toward intuition, synthesis, and passion…[4] But some of us seem to belong to another category: the “Odysseans”, who combine the two predilections in their quest for connections among ideas… We need to celebrate the contribution of those who dare take what I call “a crude look at the whole”…
’… broadly integrative thinking is relegated to cocktail parties. In academic life, in bureaucracies, and elsewhere, the task of integration is insufficiently respected. Yet anyone at the top of an organization … has to make decisions as if all aspects of a situation, along with the interaction among those aspects, were being taken into account. Is it reasonable for the leader, reaching down into the organization for help, to encounter specialists and for integrative thinking to take place only when he or she makes the final intuitive judgements?
’[A] multiplicity of crude but integrative policy studies, involving not just linear projection but evolution and highly nonlinear simulation and gaming, may provide some modest help in generating a collective foresight function for the human race…
’Given the immense complexity of the numerous interlocking issues facing humanity, foresight demands the ability to identify and gather great quantities of relevant information; the ability to catch glimpses, using that information, of the choices offered by the branching alternative histories of the future, and the wisdom to select simplifications and approximations that do not sacrifice the representation of critical qualitative issues, especially issues of values…
’Computers … can serve us both by learning or adapting themselves and by modelling or simulating systems in the real world that learn or adapt or evolve. .. Powerful computers are essential for assistance in looking into the future, but we must not allow their use to bias the formulation of problems toward the quantifiable and analyzable at the expense of the important.’5
One of the things that ‘synthesizers’ need to learn is the way that many themes cut across subjects and generate new subjects. Trans-disciplinary studies of complex systems have been profoundly affected by connected intellectual revolutions in physics, maths, logic, computation, and biology, though these connections have so far barely been integrated in school or university curricula. Ideas about ‘information’ cut across physics (thermodynamics and entropy),7 computation (bits, qubits, and ‘artificial agents’), economics (‘economic agents’, Hayek’s description of prices as an ‘information discovery process’), evolution (genetic networks),8 the brain (neuronal networks), ‘intelligence failures’9 and other subjects. Physics is inseparable from old philosophical questions (it is ‘experimental metaphysics’) and it is merging with computer science to produce quantum computation and ‘quantum information theory’.10 Evolutionary ideas took hold in economics (Hume, Smith) and biology (Darwin), and they have now been incorporated into computer science (e.g. ‘genetic algorithms’) in order to ‘evolve’ solutions to problems in large search spaces, and they have suggested ideas for engineering solutions. (For example, it is suggested that dangers such as bioterrorism or pandemics should be defended by developing ‘artificial immune systems’ in which defences operate according to the evolutionary principles of i) generating lots of solutions with random variation, and ii) differential replication of the most effective agents, instead of reliance on traditional centralised institutions that make similar mistakes repeatedly.) Machine intelligence (or ‘automated reasoning’) has been shaped by, and in turn is reshaping, ideas about the mind and is now used to design computers, including those that are used to investigate the mind. Behavioural genetics, evolutionary psychology, cognitive science and neuroscience are reshaping not only economics (e.g. fMRI scans of people playing financial games) and history (e.g. the influence of evolved antipathy for out-groups) but also how we design institutions (e.g. how we evolved to succeed in the sexual politics of small, primitive collectivist tribes hence many of the standard features of internal politics). As will be discussed, there are various developments in education that seek to reflect these trans-disciplinary themes: e.g. the Nobel-winning neuroscientist, Eric Kandel, plans a new PhD programme combining neuroscience, psychology and art history as part of the ‘Mind, Brain, Behaviour Project’ at Columbia.11
Neuroscience, cognitive science, behavioural genetics, and evolutionary biology have developed our understanding of the mind. They and other disciplines have combined with Moore’s Law and scanning technology to provide increasingly accurate maps and quantitative models of the brain107 (which Obama promised federal support for in 2013) and rapidly improving brain-computer interfaces.108 We can look inside our minds with tools that our minds have given us and watch the brain watching itself. These developments have undermined the basis for Descartes’ Ghost in the Machine, Locke’s Blank Slate, and Rousseau’s Noble Savage (pace the current, sometimes misguided, backlash).109
The brain consists of ~80 billion (8x1010) neurons and ~100 trillion (1014) synapses. Operating on ~20 watts it performs ~1017 floating point computations per second. As well as the brain-computer interfaces already underway, various projects plan to map the brain completely (e.g the Connectome Project), simulate the brain (e.g Markram’s project), and build new computer architectures modeled on the brain and performing similarly to the brain.
For example, the most successful government technology developer, DARPA, has made robotics and machine intelligence a priority with projects such as the SyNAPSE Project (led by IBM’s Modha) to create ‘a brain inspired electronic “chip” that mimics that function, size, and power consumption of a biological cortex.’110 They have recently announced progress in building this new architecture, fundamentally different to the standard ‘von Neumann architecture’ of all normal computers.111
The Human Brain Project is aiming to model a human brain and in 2012 was awarded €1 billion by the EU. In 2005, a single neuron was simulated; in 2008, a cortical column (104 neurons); in 2011, 100 columns (106 neurons). Markram plans for a full rodent brain simulation in 2014 and a human brain simulation 2020-25.
However, so far SyNAPSE and The Human Brain Project have not demonstrated how simulations connect to observable behaviours. In November 2012, Eliasmith et al (Science, 30/12/2012) published details of a large-scale computational model of the brain (Semantic Pointer Architecture Unified Network, or ‘Spaun’) intended to bridge ‘the brain-behaviour gap’ by simulating complex behaviour. Spaun is a ‘spiking neuron model’ of 2.5m simulated neurons organised into subsystems resembling different brain areas. All inputs are images of characters; all outputs are movements of a physically modeled arm. Incoming visual images are compressed by successive layers of the network that extract increasingly complex information, and simple internally generated commands (e.g. ‘draw a number’) are ‘expanded’ into complex mechanical movements. Spaun simulates the working memory and an ‘action selection system’. It performs eight tasks such as image recognition, copy drawing, reinforcement learning, and a fluid reasoning task ‘isomorphic to the induction problems from the Raven’s Progressive Matrices (RPM) test for fluid intelligence’. Spaun managed to pass some basic aspects of an IQ test. Spaun is not task-specific so the model could be extended to other tasks and scaled-up in other ways.
In 2013, Alex Wissner-Gross, a Harvard computer scientist, published a paper in Physical Review in ‘an attempt to describe intelligence as a fundamentally thermodynamic process’, proposing that intelligence can spontaneously emerge from the attempt to maximise freedom of action in the future. He built a software programme, ‘ENTROPICA’, designed to maximise the production of long-term entropy of any system it finds itself in. ENTROPICA then solved various problems including intelligence tests, playing games, social cooperation, trading financial instruments, and ‘balancing’ a physical system and so on.
‘We were actually able to successfully reproduce standard intelligence tests and other cognitive behaviors, all without assigning any explicit goals… ’
Think of games like chess or Go in which good players try to preserve as much freedom of action as possible. When the best computer programs play Go, they rely on a principle in which the best move is the one which preserves the greatest fraction of possible wins. When computers are equipped with this simple strategy—along with some pruning for efficiency—they begin to approach the level of Go grandmasters…
’Our causal entropy maximization theory predicts that AIs may be fundamentally antithetical to being boxed. If intelligence is a phenomenon that spontaneously emerges through causal entropy maximization, then it might mean that you could effectively reframe the entire definition of Artificial General Intelligence to be a physical effect resulting from a process that tries to avoid being boxed...
’The conventional storyline has been that we would first build a really intelligent machine, and then it would spontaneously decide to take over the world…We may have gotten the order of dependence all wrong. Intelligence and superintelligence may actually emerge from the effort of trying to take control of the world—and specifically, all possible futures—rather than taking control of the world being a behavior that spontaneously emerges from having superhuman machine intelligence…
’The recursive self-improving of an AI can be seen as implicitly inducing a flow over the entire space of possible AI programs. In that context, if you look at that flow over AI program space, it is conceivable that causal entropy maximization might represent a fixed point and that a recursively self-improving AI will tend to self-modify so as to do a better and better job of maximizing its future possibilities.
‘In the problem solving example, I show that cooperation can emerge as a means for the systems to maximize their causal entropy, so it doesn’t always have to be competition. If more future possibilities are gained through cooperation rather than competition, then cooperation by itself should spontaneously emerge, speaking to the potential for friendliness.’ (Interview.)
The education of the majority even in rich countries is between awful and mediocre. A tiny number, less than 1 percent, are educated in the basics of how the ‘unreasonable effectiveness of mathematics’ provides the ‘language of nature’ and a foundation for our scientific civilisation116 and only a small subset of that <1% then study trans-disciplinary issues concerning the understanding, prediction and control of complex nonlinear systems. Unavoidably, the level of one’s mathematical understanding imposes limits on the depth to which one can explore many subjects. For example, it is impossible to follow academic debates about IQ unless one knows roughly what ‘normal distribution’ and ‘standard deviation’ mean, and many political decisions, concerning issues such as risk, cannot be wisely taken without at least knowing of the existence of mathematical tools such as conditional probability. Only a few aspects of this problem will be mentioned.
There is widespread dishonesty about standards in English schools,117 low aspiration even for the brightest children,118 and a common view that only a small fraction of the population, a subset of the most able, should be given a reasonably advanced mathematical and scientific education, while many other able pupils leave school with little more than basic numeracy and some scattered, soon-forgotten facts. A reasonable overall conclusion from international comparisons, many studies, and how universities have behaved, is that overall standards have roughly stagnated over the past thirty years (at best), there are fewer awful schools, the sharp rises in GCSE results reflect easier exams rather than real educational improvements, and the skills expected of the top 20 percent of the ability range studying core A Level subjects significantly declined (while private schools continued to teach beyond A Levels), hence private schools have continued to dominate Oxbridge entry while even the best universities have had to change degree courses substantially
There is hostility to treating education as a field for objective scientific research to identify what different methods and resources might achieve for different sorts of pupils. The quality of much education research is poor. Randomised control trials (RCTs) are rarely used to evaluate programmes costing huge amounts of money. They were resisted by the medical community for decades (‘don’t challenge my expertise with data’)119 and this attitude still pervades education. There are many ‘studies’ that one cannot rely on and which have not been replicated. Methods are often based on technological constraints of centuries ago, such as lectures. Square wheels are repeatedly reinvented despite the availability of exceptional materials and subject experts are routinely ignored by professional ‘educationalists’.120 There is approximately zero connection between a) debates in Westminster and the media about education and b) relevant science, and little desire to make such connections or build the systems necessary; almost everybody prefers the current approach despite occasional talk of ‘evidence-based policy’ (this problem is one of the reasons we asked Ben Goldacre to review the DfE’s analysis division). The political implications of discussing the effects of evolutionary influences on the variance of various characteristics (such as intelligence (‘g’) and conscientiousness) and the gaps between work done by natural scientists and much ‘social science’ commentary have also prevented rational public discussion (cf. Endnote on IQ).121
Westminster and Whitehall have distorted incentives to learn and improve,122 have simultaneously taken control of curricula and exams and undermined the credibility of both, and have then blamed universities for the failures of state schools123 and put enormous pressure on Universities and academics not to speak publicly about problems with exams, which has made rational discussion of exams impossible. Most people with power in the education system are more worried about being accused of ‘elitism’ (and ‘dividing children into sheep and goats’) than they are about problems caused by poor teaching and exams and they would rather live with those problems than deal with those accusations.124
[124 E.g. Almost everybody the DfE consulted 2011-13 about curriculum and exam reform was much more concerned about accusations of elitism than about the lack of ambition for the top 20%. Although they would not put it like this, most prominent people in the education world tacitly accept that failing to develop the talents of the most able is a price worth paying to be able to pose as defenders of ‘equality’. The insistence that ~95% of pupils be able to take the same exam at 16 means (if one assumes symmetrical exclusions) that the exam must embrace plus and minus two standard deviations on the cognitive ability range: i.e. they exclude only the bottom 2.5% (i.e. an IQ of <~70) and top 2.5% (i.e an IQ of >~130, which is the average Physics PhD).]
There is huge variation in school performance (on exams that are sub-optimal) among schools with the poorest children. About a quarter of primaries have over a quarter of their pupils leave each year who are not properly prepared for basic secondary studies and few such pupils enjoy a turnaround at secondary;125 other primaries (including in the poorest areas) have <5% in such a desperate situation. Consider a basic benchmark: getting four-fifths of pupils to at least a ‘C’ in existing English and Maths GCSE. A small minority of state schools achieve this, while others with similar funding and similarly impoverished pupils struggle to get two-fifths to this level. This wide variety in performance combined with severe limits on direct parent choice means the system is partly rationed by house price.126
This wide variety in performance also strongly suggests that the block to achieving this basic benchmark is the management and quality of teaching in the school; the block is not poverty,127 IQ, money,128 lack of innovation, or a lack of understanding about how to teach basics. Making a transition to a school system in which ~4/5 meet this basic level is therefore an issue of doing things we already know how to do; the obstacles are political and bureaucratic (such as replacing management and bad teachers despite political resistance and legal complexity), although this must not blind us to the fact that most variation in performance is due to within school factors (including genetics) rather than between school factors (see below).
There are various problems with maths and science education…
The Royal Society estimates that ~300,000 per year need some sort of post-GCSE Maths course but only ~100,000 do one now. About 6⁄10 now get at least a C in English and Maths GCSE; most never do any more maths after GCSE.129 There is no widely respected ‘maths for non-maths specialists’ 16-18 course (see below).130 About 70-80,000 (~1/10 of the cohort) do Maths A Level each year (of these ~⅓ come from private schools and grammars)131 and ~1-2% also do Further Maths. In the last year for which we have data, ~0.5% (3,580 pupils) went on to get A or A* in each of A Level Maths, Further Maths, and Physics.132 Further, many universities only demand GCSE Maths as a condition of entry even for scientific degrees, so ~20% of HE Engineering entrants, ~40% of Chemistry and Economics entrants, and ~60-70% of Biology and Computer Science entrants do not have A Level Maths. Less than10% of undergraduate bioscience degree courses demand A Level Maths.
Because of how courses have been devised, ~4/5 pupils leave England’s schools without basic knowledge of subjects like logarithms and exponential functions which are fundamental to many theoretical and practical problems (such as compound interest and interpreting a simple chart on a log scale), and unaware of the maths and physics of Newton (basic calculus and mechanics). Less than one in ten has a grasp of the maths of probability developed in the 19th Century such as ‘normal distributions’ and the Central Limit Theorem (‘bell curves’) and conditional probability.133 Only the 1-2% doing Further Maths study complex numbers, matrices and basic linear algebra. Basic logic and set theory (developed c. 1850-1940) do not feature in Maths or Further Maths A levels, so almost nobody leaves school with even a vague idea of the modern axiomatic approach to maths unless they go to a very unusual school or teach themselves.
133 Gigerenzer’s ‘Reckoning With Risk’ has terrifying stats on the inability of trained medical professionals making life and death decisions to understand the basics of conditional probability, which is not covered in the pre-16 curriculum (cf. Endnote). Current A Level modules have conditional probability and normal distributions in S1 and S2 (not compulsory), so one could have an A* in A Level Maths and Further Maths without knowing what these are. Data on who does which modules is not published by exam boards.
(...)
The education world generally resists fiercely the idea that a large fraction of children can or should be introduced to advanced ideas but we could substantially raise expectations without embracing ‘Ender’s Game’. It is almost never asked: how could we explore rigorously how ambitious it is realistic to be? If you ask, including in the Royal Society, ‘what proportion of kids with an IQ of X could master integration given a great teacher?’, you will get only blank looks and ‘I don’t think anyone has researched that’. Given the lack of empirical research into what pupils with different levels of cognitive ability are capable of with good teachers, research that obviously should be undertaken, and given excellent schools (private or state) show high performance is possible, it is important to err on the side of over-ambition rather than continue the current low expectations. Programmes in America have shown that ‘adolescents scoring 500 or higher on SAT-M or SAT-V by age 13 (top 1 in 200), can assimilate a full high school course (e.g., chemistry, English, and mathematics) in three weeks at summer residential programs for intellectually precocious youth; yet, those scoring 700 or more (top 1 in 10,000), can assimilate at least twice this amount…’ (Lubinski, 2010). (See Endnote.)
It would be nice to have a summary of the ideas mentioned in this long text, so we could discuss them individually. In this form, it feels overwhelming.
I like the parts about an importance of math, and about a lack of (and resistance against) scientific research in education.
TL;DR:
Inter-disciplinary thought and research is increasingly valuable, and often depends on mathematics.
UK high school standards are low and declining, especially in math and stats.
The high variation in school performance even in the same socio-economic group suggests that the problem is teaching quality (and low standards), not money.
There ought to be action to get rid of bad teachers and bad school managers.
There ought to be research to find out what the highest realistic standards can be (eg what fraction of students are inherently capable of learning integrals).
As another data point, I hit my internet blog reading limit about a third of the way through.
Summaries that provide an itinerary of the article help to motivate readers to persevere.
I see a lot of internal contradictions in this, but I can’t tell if that’s because I’m misreading it or not. For example, when I read this:
My reaction hinges on the numerical interpretation of “substantial” (whose emphasis is mine). Is it 5%, compared to a current <1%? Then yeah, that seems doable. Is it a 20%? Then no, that doesn’t seem doable, and seeking to write a class on integrative systems thinking that the 80th percentile student can survive is probably going to make it a worthless class.
To the extent that this report is an attack on the doomed quest for equality and a proposal to educate students according to their talents, I am entirely in favor.
Agreed that this research should obviously be undertaken. But I think wording this as “err on the side of over-ambition” is a mistake; the current philosophy of equality errs on the side of over-ambition, and in some sense that is its root problem! If you have the “ambitious” goal of putting every child through university, the only result is the debasement of the university degree to something of no value. The necessary change is reducing ambitions down to something realistic- educate each child as far as it makes sense for them to go- and once you accept that individuals have different limits, then it becomes possible to make the full use of each child.
Yep. But again, note the fractions here- students that are 1 in 200! That hardly seems like a substantial fraction to me.
I think your reading is being uncharitable; ‘substantial’ doesn’t refer to any particular amount, just that the amounts are meaningful, which will be context-specific. (I also like the term because it isn’t deceptively conflated with a technical meaning, like ‘significant’ is—significant could mean either substantial/import/large or meets an arbitrary p-value).
The way I see it, he’s stating that basically 0% of students are trained the way he would like them (surely true), that 0% is not optimal (also surely true but I’m not sure what the optimal percentage is), and that right now, we don’t even have good ideas of what is possible for the various qualities of student because the research simply hasn’t been done aside from some suggestive anecdotes (true).
There’s something like 15 million high school students in the USA; that’s 75,000 students. How many of them are in gifted summer programs?
Overall, this manifesto was quite a read. Reading it gave me a weird sort of feeling—I didn’t learn very much that was new, but it’s a little shocking to see someone cover in a single place so many topics of interest to me in a competent manner and with an attitude & writing style similar to my own.
I hoped so too, but didn’t have time to read the whole work (which might have settled my concerns).
The range 5,000 to 25,000 seems like a decent 50% interval to me. So yeah, it could be higher. The real issue, in my mind, is not summer but the rest of the year; it seems odd to me that we don’t have boarding schools for students that are 1 in 10,000, where they can learn about a high school course a week, or do things like this.
The impression I get from his discussion of the French & Russian elite schools is that the summer courses may be better considered part of the filter for admission into those sort of schools; specifically, footnote #150:
This makes sense since cognitive testing has known measurement error, which while very low, is still problematic if you’re trying to select the top 1% (or higher) of students* for these expensive schools (and because intelligence itself changes over time, so a single test means regression to the mean), and it may either spark enthusiasm for going to the advanced schools or test the child’s attitudes toward it—not much point in enrolling a kid who doesn’t want to be there, right?
* for example, apparently the Termite study missed out on two Nobelists because they were under the filter by some tiny amount on the test; if Terman had been able to afford a larger sample or had a slightly better test, his results would have been much more impressive, especially since Nobelists are one in millions.
Agreed that seeing someone in class is useful information, and that this is a good feeder system for elite schools. But to the best of my knowledge there is not a quality national network of elite math and science schools in the US, just a collection of local magnet schools.
It also depends on what was meant by “something of” (emphasis mine).
In my opinion the best blog about education “Scenes From The Battleground” is written by a British teacher.
A good starting point is A Guide To Scenes From The Battleground. It’s like Sequences about education.
I am familiar with Scenes from the Battleground. Another favorite of mine is Educationrealist.
I am reading the blog now… and I guess the main idea there is that the problems with education are caused by IQ differences.
This doesn’t match my experience. I was teaching at a high school for gifted children, where all students must have IQ 130 or more. But I have seen there some of the problems that this blogger attributes to low IQ. (Specifically: learning something, and then forgetting it completely.) Because of this, I think the author’s analysis is wrong. Although I admit that with lower IQ, those problems can be much greater than what I saw.
Specifically, for the “learn and forget” learning style, I think the cause is not enough repetition. Repeating is essential for remembering. But the modern trend for teachers is doing creative things and discussions at classroom, not giving homework, not giving too much tests. In other words, reducing repetition as much as possible. Because the repetition is boring, and there is no worse sin for a teacher than being boring. But this is how our brains are built; remove the repetition and you remove the remembering. -- If your parents are smart and talk with you a lot, you get some of the information repeated informally at home. If they make you learn at home, you repeat the information formally.
EDIT: This is made even worse by many people saying that doing homework should not be reflected in your grades; only what you know. I agree with the argument… but the unfortunate consequence is that if you say homework is optional, most students won’t bother. And then, predictably, they will not remember the lessons. (And then, predicably, if this happens to most of the class, it will be considered the teacher’s problem. But the teacher will often not be allowed to fix it by giving a lot of homework and making it mandatory. Instead, they are required to do some miracle. And judging by the general situation in education, most teachers are pretty bad at making miracles.)
On the other hand, the article about Asians cheating on exams was very interesting. I just propose an alternative hypothesis that it’s not just Asians, but pretty much everyone except for Americans. I would bet that immigrants from Slovakia would be cheating as much as possible… and if not, then the reason wouldn’t be honesty, but merely lack of strategic coordination. So perhaps “not being (culturally) an American” and “being good at cooperation” are the two necessary ingredients, and the children of Asian immigrants are the most visible example of both.
Okay, reading some more stuff from the blog, my impression improved. He seems to be a great teacher (example 1, example 2). It’s just the IQ hypothesis that I disagree with, but I guess for many readers that one is the most interesting. I am tempted to propose a compromise hypothesis, that at the lower end of the IQ scale IQ is the hard limit of ability to learn, but at the higher end of the IQ scale the most limiting factors are elsewhere.
Some quotes I liked:
Sure, magic skills are a requirement for a teacher. On the higher end of the IQ scale, the delusion is that a spoiled kid with behavioral problems who never had to work or behave at school and right now is acting even worse than usual because their parents are divorcing… can be transformed into a hard-working and well-socialized student by saying the right magic words.
(Source.) This reminds me of “The Naughty Boy” and “The Disruptive Girl” from Scenes from the Battleground. The difference on the higher end of the IQ scale is that each of those misbehaving kids is a certified special snowflake, and their parents often have good lawyers or sponsor the school or both, so you just can’t kick those kids out of the classroom when you need to.
Cummings retrospective on the essay: https://dominiccummings.wordpress.com/the-odyssean-project-2/
I started actually reading the whole thing and am about halfway through it now. This looks really good. It’s not just a rant about education, he seems to have come up with a whole guidebook to the 21st century style thing here, detailing possible coming trends in science, technology, decisionmaking and leadership in addition to education. And the breadth of stuff is huge, he goes from modeling chaotic systems to modern materials science to quantum computing and advances in AI to the progress in genetic sequencing to heuristic and biases research and a long list of improvements to try to make leadership structures more efficient.
I agree that trans-disciplinary, integrative, complex systems thinking needs to become an important and respectable field in its own right. Nexialism anyone?
I think a better question is to ask: “How do you teach integration in a way that a lot of children can understand it for a reasonable time investment.
Not to mention, I strongly suspect that within the current educational system, the limiting factor is conscientiousness, not IQ. (At least, for high school and most undergraduate systems).
It’s true, few people really get integration, but I know plenty of people who are capable of memorizing integration rules sufficiently well to succeed academically—when you get down to it, integral(ax^n)dx=[a/(n+1)]*x^(n+1) shouldn’t be hard for a sufficiently conscientious person to memorize and apply. Actually understanding what’s going on and being able to derive that from scratch is just an obscure bonus accomplishment.
Why do I bring this up? Memorized integration probably isn’t worth the time investment outside of academic success, while an intuitive understanding of integration probably is worth the time investment. If you don’t design the indicators of learning carefully, you’ll end up optimizing for the wrong thing.
Not really. I think a lot of us can attest that it is pretty easy to skate by on IQ alone, and even get decent grades while doing it.
My anecdotal evidence runs opposite yours, and I suspect the people you are thinking of are either both smart *and conscientious, or happen to be in one of those rare academic setting which favor intelligence over conscientiousness. Or maybe your perception of peoples intelligence is influenced by how academically successful you know them to be. Who knows how perceptions of intelligence map on to IQ and GPA conscientiousness when humans use funny metrics like facial symmetry, clothing, facial expressions, eye contact and other forms of nonverbal body language?
My opinion isn’t just based on anecdote, though:
Various measures of self discipline twice as predictive of GPA as IQ in adolescents—and you’ll notice the importance of IQ really drops off after the middle quintile.
Self-Reported Conscientiousness predicts college GPA better than both SAT & High School GPA
Summarizing from my general impression of things I’ve read, IQ is generally more predictive than self-reported conscientiousness for grades …but the difference is way closer then you’d expect for a self report going up against a cognitive test.
Hmm...so you’re drawing from the LW demographic? That’s gonna be a higher IQ group, but they’ll also have other unusual traits...I’m kind of curious now. Good candidate question for the next census...
Edit: Here’s a poll with lots of relevant data (on GPA , IQ, and procrastination of Lesswrongers). Analysis not geared towards our specific question. Raw data is available. Eyeballing it, LW GPA’s tend to be 3.3-3.9, IQ’s range 130-160. Apparently IQ correlates with absolutely none of these things...which is actually not that unexpected, since everyone is in the higher range and they probably didn’t all use the same test. But still, it doesn’t seem like LWers are skating through school...just hovering above average. And this is just the out of the people that volunteered the self-report GPA. Self reported procrastination in childhood did correlate with self reported low high school GPA,
http://lesswrong.com/lw/7s4/poll_results_lw_probably_doesnt_cause_akrasia/
Quite possibly true, but “a lot of LessWrongers” does not an argument about the mass of the population make.
Actually academic achievement is something IQ tests excel at predicting quite well. We also, on average, see clear differences in average intelligence between people with differences in academic achievement, if one was tempted to dismiss the greatest achievement of psychometrics out of hand based on this. This is not to say that conscientiousness isn’t another thing the educational system selects for, it is the second best predictor, but it is just that, second best. Formal education taken as a whole from primary school to grad seems to be primarily g loaded.
I dislike the ideological side effects of the current educational system, but it seems far from obvious that formal education should value raw intelligence even more over conscientiousness. Neither is something people have control over, so the question is what is the economic value of both and which kind of virtues, and by extension the people who undeserving hold them, do we wish to celebrate.
I don’t deny that. I’m not one of those IQ-doesn’t-measure-anything people.
Okay, so: When you measure a correlation, you aren’t just measuring how two things are related. Construct validity plays a huge role.
If you just asked people “How smart are you” and correlated it with grades, you’d likely see a positive correlation. But if you give them Raven’s Progressive Matrices, you’d see a much stronger correlation with grades.
The correlation reflects not just the relationships between underlying phenomenon, but the degree to which you have successfully measured the underlying phenomenon. Unless you’re measuring opinions or something, self-reports suffer from all sorts of issues with validity that cognitive tests do not.
So when you compare simple self-reported conscientiousness to IQ (which is, as you said, the greatest achievement in psycho-metrics), you’re pitting a mouse against a lion.
The study I cited further down the thread, which says that willpower is more important than IQ, was able to get that result because they put a lot more effort into measuring willpower than other studies. The willpower variable was a composite of several self reports, teacher reports, parent reports, and a behavioral delay of gratification task. This composite willpower score will have more validity than any of its individual components.
(It’s the same with IQ: a composite IQ test, with verbal tests, visuospatial tests, reaction time, etc will be more g-loaded (against a separate test battery,) than any individual measurement, and will probably predict grades better too. Excuse my glossing over things—see here for an example of how CCFT’s low question diversity results in lower g)
Don’t be too quick to downplay the importance of conscientiousness—a lot of the weaker correlations can be chalked up to the difficulty of measuring the underlying thing.
That’s not self-evident to me, if only because people’s self-perception of smartness is likely to be (partially) driven precisely by their grades.
I agree that its very plausible that grades affect self perception. However, I have low confidence that it would be more predictive than IQ.
Same could be said for conscientiousness, but thus far it seems like the behavioral testing and the reports from grade-naive “homeroom” teachers seem to increase the correlation rather than decrease it compared to self-report alone.
I don’t have strong opinions on the subject, but I wouldn’t necessarily expect uniform results across the IQ spectrum. It might well be that different things are more predictive at different ends of the IQ curve.
In particular with respect to IQ and conscientiousness, it seems to me that at high IQ levels the “necessary-for-A+” IQ tops out and conscientiousness starts to dominate, while it’s the reverse with low IQ—if you’re just not smart enough, conscientiousness won’t help much.
I low-confidence-agree with you. That seems gut-level correct, and fig 1 supports this notion but I don’t know the extent that trend should be trusted and there are other possibilities.
If I had to guess I’d rephrase it as ” both have diminishing returns, but the diminishing returns on IQ are both more dramatic than those on conscientiousness.”
actually, looking at figure 1 it doesn’t look like self discipline has diminishing returns at all, whereas it looks like IQ does. Just eyeballing, the “IQ test ceiling” for GPA is the 3rd quintile of this population’s OLSAT7 lv G scorers whereas Self Discipline doesn’t even hit a ceiling. But conclusions via eyeballing would be criminally non-rigorous :)
You are right, instead of focusing on teaching integration better, focusing on teaching conscientiousness might be the better goal.
Assuming that it can be taught. It’s possible that the key components that link conscientiousness to academic success are prefrontal/ventral tegmental stuff such as different reward mechanisms (motivation, willpower) and differences in executive function (attention and inhibitory control), and we don’t know to what extent teaching can modify that. Executive function, at least, seems difficult to alter via training.
Plus, we’d have to separate out the components of conscientiousness though—it’s possible that conscientiousness is not entirely positive. Conscientiousness has a rather controversial relationship with fluid intelligence and creativity, (as usual, most stereotypes carry at least some distorted truth) though it’s too soon to make definitive statements.
I do mostly agree with you—these are just rather important qualifiers.
When we talk about training executive function through principles such as Dual ‘n’ Back we are talking about relatively little time investment.
A child spends years in school much longer than your average psychology study runs.
One of the most interesting parts of Science and Sanity (Alfred Korzybski, Map is Not the Territory originator) was the analysis of mathematics and physics, and in particular “On the Semantics of the Differential Calculus”.
A better and more intuitive way to talk about Calculus, IMO.
I imagine that things like this could be experimentally measured using Khan Academy style education.
Ask teams of teachers to create different video lessons explaining X. Take a lot of students, assign them randomly to different lessons. After lessons, give them tests. Measure how good they were at tests. Choose the best lessons and reward the authors. Use those lessons for education, and once in a while announce a new competition.
Unfortunately, teaching in person cannot be replicated as well as video teaching. It is difficult to copy a teacher, so even if one teacher has a lot of success using some specific method, it does not mean others will have the same success when trying the same thing.
That accepts basic premises about education that I don’t think make sense. I don’t think that there any basis to believe that video explaining something are a time effective way of learning something. The same goes for straight lecturing of information.
Children don’t learn their native language because their parents explain them how it works.
On the level of making education policy the easist thing would be to simple get rid of the curriculum and let every school teach what they consider to make sense. Additionally you facillitate lots of knowledge exchange between teachers.
That would allow innovation. It might even allow for children spending more time in front of Khan Academy.
That reminds me of Aikido grandmaster Koichi Tohei who made the point that the really important skill is to teach people who teach teachers effectively.
Even if video teaching is (today?) not the best way, I think it would be nice to create a feedback loop, because feedback is what’s missing in education now. Once you have a system “here is a video, students see it, students take tests”, you can experiment with various changes and see whether those changes improved the results. The same thing could be done with books, of course. The important part is to allow the education to replicate, and measure its results. Then a way to gradual improvement (instead of random drift) is opened.
These days, education is typically done like this: random people create new educational theories mostly based on pseudoscience, teachers are taught these theories, teachers do random things in classrooms, nobody really evaluates what’s going on as long as nothing extreme happens. -- In this situation if you ask questions like “what are the different known ways to teach integrals, and how efficient is each of them”, no one really knows, because no one ever measured that in any meaningful way. The only answer you could get is the latest fashion in pseudoscience, for example “you should support multiple learning styles and, uhm, make it funny”, which, even if you’d happen to agree with it, is not specific enough to give measurable results.
It is certainly not the only way to teach, but it is a way that could be measured. We should at least try it experimentally, so see what kinds of results it could produce.
And measure the outcomes, and reward those who have the best ones. The rewards can be financial, but also the prestige. (If you have a private school, and the state makes everyone known that you are the best school in the country, you are going to get a lot of money even without the state giving it to you.)
Going more meta! But the question of measuring is still here.
That’s the huge advantage of online learning. Performance and behavioral metrics can be as detailed as you like.
That leaves the question about which outcomes you measure. I think it’s okay to have a world with some school that run like KIPP where there a lot of measurement and others that run like Sudbury Valley with has feedback principle like internal elections and reviews how many of it’s student succeed at college.
When it comes to the specific example of teaching integration I think that will be done best via some computer tool.
I guess that 20 hours of time investment into practing a well developed Anki deck on integration would leave most students with more knowledge of integration afterwards.