Epistemologist specialized in the difficulties of alignment and how to solve AI X-Risks. Currently at Conjecture.
Blogging at For Methods.
adamShimi
Karma: 6,646
I did not particularly intend to do a book review per say, and I don’t claim to be an expert on the topic. So completely fine with tagging this in some way as “non-expert” if you wish.
Not planning to change how I wrote my posts based on this feedback, as I have no interest in following some arbitrary standard of epistemic expertise for a fun little blog post that will be read by 10 people max.
I agree that it is in the text. If it wasn’t clear, my message was trying to reverse engineer why I bounced off, which is more about my experience of reading than fully about the text.
I remember reading this post, and really disliking it.
Then today, as I was reflecting on things, I recalled that this existed, and went back to read it. And this time, my reaction was instead “yep, that’s pointing to the mental move that I’ve lost and that I’m now trying to relearn”.
Which is interesting. Because that means a year or two ago, up till now, I was the kind of people who would benefit from this post; yet I couldn’t get the juice out of it. I think a big reason is that while the description of the play/fun mental move is good and clear, the description of the opposite mental move, the one short-circuiting play/fun, felt very caricatural and fake.
My conjecture (though beware mind fallacy), is that it’s because you emphasize “naive deference” to others, which looks obviously wrong to me and obviously not what most people I know who suffer from this tend to do (but might be representative of the people you actually met).
Instead, the mental move that I know intimately is what I call “instrumentalization” (or to be more memey, “tyranny of whys”). It’s a move that doesn’t require another or a social context (though it often includes internalized social judgements from others, aka superego); it only requires caring deeply about a goal (the goal doesn’t actually matter that much), and being invested in it, somewhat neurotically.
Then, the move is that whenever a new, curious, fun, unexpected idea pop up, it hits almost instantly a filter: is this useful to reach the goal?
Obviously this filter removes almost all ideas, but even the ones it lets through don’t survive unharmed: they get trimmed, twisted, simplified to fit the goal, to actually sound like they’re going to help with the goal. And then in my personal case, all ideas start feeling like should, like weight and responsibility and obligations.
Anyway, I do like this post now, and I am trying to relearn how to use the “play” mental move without instrumentalizing everything away.
Typo addressed in the latest patch!
Now addressed in the latest patch!
Now addressed in the latest patch!
Now addressed in the latest patch!
Thanks for the comment!
We have indeed gotten the feedback by multiple people that this part didn’t feel detailed enough (although we got this much more from very technical readers than from non-technical ones), and are working at improving the arguments.
Thanks for the comment!
We’ll correct the typo in the next patch/bug fix.
As for the more direct adversarial tone of the prologue, it is an explicit choice (and is contrasted by the rest of the document). For the moment, we’re waiting to get more feedback on the doc to see if it really turns people off or not.
Yep, I think you’re correct.
Will correct in the next minor update. Thanks!
Thanks for the comment!
We’ll consider this point for future releases, but personally, I would say that this kind of hedging also has a lot of downsides: it makes you sound far more uncertain and defensive than you really want to.
This document tries to be both grounded and to the point, and so we by default don’t want to put ourselves in a defensive position when arguing things that we think make sense and are supported by the evidence.
Thanks for the comment!
We have gotten this feedback by a handful of people, so we want to reread the links and the whole literature about o1 and its evaluation to check whether we’ve indeed gotten the right point, or if we mischaracterized the situation.
We will probably change the phrasing (either to make our criticism clearer or to correct it) in the next minor update.
Good catch, I think we are indeed mixing the sizes here.
As you say, the point still stands, but we will change it in the next minor update to either compare the same size or make the difference in size explicit.
Thanks for the comment!
We want to check the maths, but if you’re indeed correct we will update the numbers (and reasoning) in the next minor version.
I guess it depends on if you’re pivoting based on things that you’ve learned, versus grass-is-greener.
Yeah, I didn’t mean “iterative thoughtful processes”, I meant “compulsion that unfold at the level of days”. If you arbitrarily change your job every couple of days/weeks, not based on new significant information but because you feel this other one is the one, this is bad.
So there is a vibe here that I maybe didn’t convey well, about the time frame and the auto-generated part of the loops I’m pointing at: it happens often enough that your friends and family can notice, and it happens in reaction to events that no one around you agree would lead to such a drastic change (highlighting that the events are not so much the cause as the post-hoc rationalization).
Recently found a new link: Annual Reviews
It sounds like a place that centralizes many different review articles across a lot of disciplines. Only checked a few for the moment, but definitely sounds worth a try!
@Elizabeth suggested that I share here the quick tips I gave her for finding cool history and philosophy of science books, so let’s do it.
I like using awards as starting points. They’re not exhaustive, but often they point to particularly good references in a field that I don’t know about.
For philosophy of science, often with a decent dose of history, there is the Lakatos Award.
For history of science, there is the Sarton Medal, which is given to individuals, not works
Same with book reviews by journals focused on the topic
My favorite are from the British Journal for The Philosophy of Science reviews
Knowing the terminology helps. I find that “History and Philosophy of X” is often a good google query
I recently discovered https://hiphilangsci.net/ on linguistics that way!
Obviously, follow the citations: cool books tend to reference cool books. (And terrible ones, but let’s not mention that)
Also known, but just in case: going to https://scholar.google.com/ and searching for the most cited books that cite a book you liked often leads to great reading material.
Yeah, I agree with the general point (don’t have strong opinion about chaos theory at the moment).
First, negative results are really, really important. Mostly because they let you not lose your time trying to do something impossible, and sometimes they actually point you toward an answer. In general, conservation laws in physics have this role. And knowing what is undecidable is really important in formal methods, where the trick is generally to simplify what you want or the expressive power of your programs in order to sidestep it.
Then, they are indeed quite hard to prove, at least in non-trivial cases. Conservations laws are the results of literally centuries of reframing of classical mechanics and reduction, leading to seeing the importance of energy and potential in unifying everything in physics. Undecidability is the result of 60 years of metamathetical work trying to clean formalisms enough to be able to study these kind of properties.
Is there any empirical question the phlogiston theorists got right that compositional chemistry did not? AFAIK, no, but it’s a real question and I’d like to know if I’m wrong here.
Although I haven’t digged into the historical literature that much, I think there are two main candidates here: explaining the behavior of metals, and potential chemical energy.
On explaining the behavior of metal, this is Chang (Is Water H2O? p.43)
Phlogistonists explained the common properties of metals by saying that all metals were rich in phlogiston; this explanation was lost through the Chemical Revolution, as it does not work if we make the familiar substitution of phlogiston with the absence of oxygen (or, as Lavoisier had it, a strong affinity for oxygen). As Paul Hoyningen-Huene puts it (2008, 110): “Only after more than a 100 years could the explanatory potential of the phlogiston theory be regained in modern chemistry. One had to wait until the advent of the electron theory of metals”.
(Is Water H2O? p.21)
One salient case was the explanation of why metals (which were compounds for phlogistonists) had a set of common properties (Kuhn 1970 , 148). Actually by the onset of the Chemical Revolution this was no longer a research problem in the phlogiston paradigm, as it was accepted almost as common sense that metals had their common metallic properties (including shininess, malleability, ductility, electrical conductivity) because of the phlogiston they contained. The oxygenist side seems to have rejected not so much this answer as the question itself; chemistry reclaimed this stretch of territory only in the twentieth century.
And on potential chemical energy, here are the quotes from Chang again
(Is Water H2O? p.46)
William Odling made the same point in a most interesting paper from 1871. Although not a household name today, Odling was one of the leading theoretical chemists of Victorian Britain, and at that time the Fullerian Professor of Chemistry at the Royal Institution. According to Odling (1871, 319), the major insight from the phlogistonists was that “combustible bodies possess in common a power or energy capable of being elicited and used”, and that “the energy pertaining to combustible bodies is the same in all of them, and capable of being transferred from the combustible body which has it to an incombustible body which has it not”. Lavoisier had got this wrong by locating the energy in the oxygen gas in the form of caloric, without a convincing account of why caloric contained in other gases would not have the ability to cause combustion.
(Is Water H2O? p.47)
Although phlogiston was clearly not exactly chemical potential energy as understood in 1871, Odling (p. 325) argued that “the phlogistians had, in their time, possession of a real truth in nature which, altogether lost sight of in the intermediate period, has since crystallized out in a definite form.” He ended his discourse by quoting Becher: “I trust that I have got hold of my pitcher by the right handle.” And that pitcher (or Becher, cup?), the doctrine of energy, was of course “the grandest generalization in science that has ever yet been established.”
As a summary, let’s quote Chang one last time. (Is Water H2O? p.47-48)
All in all, I think it is quite clear that killing phlogiston off had two adverse effects: one was to discard certain valuable scientific problems and solutions; the other was to close off certain theoretical and experimental avenues for future scientific work. Perhaps it’s all fine from where we sit, since I think the frustrated potential of the phlogistonist system was quite fully realized eventually, by some very circuitous routes. But it seems to me quite clear that the premature death of phlogiston retarded scientific progress in quite tangible ways. If it had been left to develop, I think the concept of phlogiston would have split into two. On the one hand, by the early nineteenth century someone might well have hit upon energy conservation, puzzling over this imponderable entity which seemed to have an elusive sort of reality which could be passed from one ponderable substance to another.
In that parallel universe, we would be talking about the conservation of phlogiston, and how phlogiston turned out to have all sorts of different forms, but all interconvertible with each other. This would be no more awkward than what we have in our actual universe, in which we still talk about the role of “oxygen” (acid-generator, Sauerstoff ) in supporting combustion, and the “oxidation” number of ions. On the other hand, the phlogiston concept could have led to a study of electrons without passing through such a categorical and over-simplified atomic theory as Dalton’s. Chemists might have skipped right over from phlogiston to elementary particles, or at least found an alternative path of development that did not pass through the false simplicity of the atom–molecule–bulk matter hierarchy. Keeping the phlogiston theory would have led chemists to pay more attention to the “fourth state of matter”, starting with flames, and served as a reminder that the durability of compositionist chemical building-blocks may only be an appearance. Keeping phlogiston alive could have challenged the easy Daltonian assumption that chemical atoms were physically unbreakable units. The survival of phlogiston into the nineteenth century would have sustained a vigorous alternative tradition in chemistry and physics, which would have allowed scientists to recognize with more ease the wonderful fluidity of matter, and to come to grips sooner with the nature of ions, solutions, metals, plasmas, cathode rays, and perhaps even radioactivity.
One point evoked by other comments, which I’ve realized only after leaving France and living in the UK, is that there is still a massive prestige for engineering. ENS is not technically an engineering school, but it benefits from this prestige by being lumped with them, and by being accessed mainly from the national contests at the end of Prepas.
As always with these kind of cultural phenomena, I didn’t really notice them until I left France for the UK. There is a sense in France (more when I was a student, but still there) that the most prestigious jobs are engineering ones. Going to engineering school is considered one of the top options (with medecine), and it is considered a given that any good student with a knack for maths, physics, science, will go to prepa and engineering school.[1] It’s almost free (and in practice is free if your parents don’t make more than a certain amount), and it is guaranteed to lead to a good future.
This means that the vast majority of mathematical talent studies the equivalent of a undergraduate degree in maths, compressed in the span of 2 years. In addition of giving the standard french engineer much more of a mathematical training, it shows to the potential mathematicians, by default, a lot of what they could do. And if they decide to go to ENS (or Polytechnique, which is the best engineering school but still quite researchy if you want to), this is actually one of the most prestigious options you could take.
Similarly, the prestige of engineering (and science to some extent) impacts what people decide to do after their degrees. I remember that in my good prepa and my good engineering school, the cool ones were those going to build planes and bridges. The ones who went into consulting and finance were pitied and mocked as the failures, not the impressive successes to emulate. Yet what my UK friends tell me is that this is the exact opposite of what happens even in great universities in the UK.
This has become less true, as more private schools open, and the whole elitist system is wormed out by software engineering startups (which generally doesn’t ask you for an engineering degree, as opposed to the older big french companies).