Epistemologist specialized in the difficulties of alignment and how to solve AI X-Risks. Currently at Conjecture.
Blogging at For Methods.
adamShimi
Karma: 6,594
Now addressed in the latest patch!
Now addressed in the latest patch!
Now addressed in the latest patch!
Thanks for the comment!
We have indeed gotten the feedback by multiple people that this part didn’t feel detailed enough (although we got this much more from very technical readers than from non-technical ones), and are working at improving the arguments.
Thanks for the comment!
We’ll correct the typo in the next patch/bug fix.
As for the more direct adversarial tone of the prologue, it is an explicit choice (and is contrasted by the rest of the document). For the moment, we’re waiting to get more feedback on the doc to see if it really turns people off or not.
Yep, I think you’re correct.
Will correct in the next minor update. Thanks!
Thanks for the comment!
We’ll consider this point for future releases, but personally, I would say that this kind of hedging also has a lot of downsides: it makes you sound far more uncertain and defensive than you really want to.
This document tries to be both grounded and to the point, and so we by default don’t want to put ourselves in a defensive position when arguing things that we think make sense and are supported by the evidence.
Thanks for the comment!
We have gotten this feedback by a handful of people, so we want to reread the links and the whole literature about o1 and its evaluation to check whether we’ve indeed gotten the right point, or if we mischaracterized the situation.
We will probably change the phrasing (either to make our criticism clearer or to correct it) in the next minor update.
Good catch, I think we are indeed mixing the sizes here.
As you say, the point still stands, but we will change it in the next minor update to either compare the same size or make the difference in size explicit.
Thanks for the comment!
We want to check the maths, but if you’re indeed correct we will update the numbers (and reasoning) in the next minor version.
I guess it depends on if you’re pivoting based on things that you’ve learned, versus grass-is-greener.
Yeah, I didn’t mean “iterative thoughtful processes”, I meant “compulsion that unfold at the level of days”. If you arbitrarily change your job every couple of days/weeks, not based on new significant information but because you feel this other one is the one, this is bad.
So there is a vibe here that I maybe didn’t convey well, about the time frame and the auto-generated part of the loops I’m pointing at: it happens often enough that your friends and family can notice, and it happens in reaction to events that no one around you agree would lead to such a drastic change (highlighting that the events are not so much the cause as the post-hoc rationalization).
Recently found a new link: Annual Reviews
It sounds like a place that centralizes many different review articles across a lot of disciplines. Only checked a few for the moment, but definitely sounds worth a try!
@Elizabeth suggested that I share here the quick tips I gave her for finding cool history and philosophy of science books, so let’s do it.
I like using awards as starting points. They’re not exhaustive, but often they point to particularly good references in a field that I don’t know about.
For philosophy of science, often with a decent dose of history, there is the Lakatos Award.
For history of science, there is the Sarton Medal, which is given to individuals, not works
Same with book reviews by journals focused on the topic
My favorite are from the British Journal for The Philosophy of Science reviews
Knowing the terminology helps. I find that “History and Philosophy of X” is often a good google query
I recently discovered https://hiphilangsci.net/ on linguistics that way!
Obviously, follow the citations: cool books tend to reference cool books. (And terrible ones, but let’s not mention that)
Also known, but just in case: going to https://scholar.google.com/ and searching for the most cited books that cite a book you liked often leads to great reading material.
Yeah, I agree with the general point (don’t have strong opinion about chaos theory at the moment).
First, negative results are really, really important. Mostly because they let you not lose your time trying to do something impossible, and sometimes they actually point you toward an answer. In general, conservation laws in physics have this role. And knowing what is undecidable is really important in formal methods, where the trick is generally to simplify what you want or the expressive power of your programs in order to sidestep it.
Then, they are indeed quite hard to prove, at least in non-trivial cases. Conservations laws are the results of literally centuries of reframing of classical mechanics and reduction, leading to seeing the importance of energy and potential in unifying everything in physics. Undecidability is the result of 60 years of metamathetical work trying to clean formalisms enough to be able to study these kind of properties.
Is there any empirical question the phlogiston theorists got right that compositional chemistry did not? AFAIK, no, but it’s a real question and I’d like to know if I’m wrong here.
Although I haven’t digged into the historical literature that much, I think there are two main candidates here: explaining the behavior of metals, and potential chemical energy.
On explaining the behavior of metal, this is Chang (Is Water H2O? p.43)
Phlogistonists explained the common properties of metals by saying that all metals were rich in phlogiston; this explanation was lost through the Chemical Revolution, as it does not work if we make the familiar substitution of phlogiston with the absence of oxygen (or, as Lavoisier had it, a strong affinity for oxygen). As Paul Hoyningen-Huene puts it (2008, 110): “Only after more than a 100 years could the explanatory potential of the phlogiston theory be regained in modern chemistry. One had to wait until the advent of the electron theory of metals”.
(Is Water H2O? p.21)
One salient case was the explanation of why metals (which were compounds for phlogistonists) had a set of common properties (Kuhn 1970 , 148). Actually by the onset of the Chemical Revolution this was no longer a research problem in the phlogiston paradigm, as it was accepted almost as common sense that metals had their common metallic properties (including shininess, malleability, ductility, electrical conductivity) because of the phlogiston they contained. The oxygenist side seems to have rejected not so much this answer as the question itself; chemistry reclaimed this stretch of territory only in the twentieth century.
And on potential chemical energy, here are the quotes from Chang again
(Is Water H2O? p.46)
William Odling made the same point in a most interesting paper from 1871. Although not a household name today, Odling was one of the leading theoretical chemists of Victorian Britain, and at that time the Fullerian Professor of Chemistry at the Royal Institution. According to Odling (1871, 319), the major insight from the phlogistonists was that “combustible bodies possess in common a power or energy capable of being elicited and used”, and that “the energy pertaining to combustible bodies is the same in all of them, and capable of being transferred from the combustible body which has it to an incombustible body which has it not”. Lavoisier had got this wrong by locating the energy in the oxygen gas in the form of caloric, without a convincing account of why caloric contained in other gases would not have the ability to cause combustion.
(Is Water H2O? p.47)
Although phlogiston was clearly not exactly chemical potential energy as understood in 1871, Odling (p. 325) argued that “the phlogistians had, in their time, possession of a real truth in nature which, altogether lost sight of in the intermediate period, has since crystallized out in a definite form.” He ended his discourse by quoting Becher: “I trust that I have got hold of my pitcher by the right handle.” And that pitcher (or Becher, cup?), the doctrine of energy, was of course “the grandest generalization in science that has ever yet been established.”
As a summary, let’s quote Chang one last time. (Is Water H2O? p.47-48)
All in all, I think it is quite clear that killing phlogiston off had two adverse effects: one was to discard certain valuable scientific problems and solutions; the other was to close off certain theoretical and experimental avenues for future scientific work. Perhaps it’s all fine from where we sit, since I think the frustrated potential of the phlogistonist system was quite fully realized eventually, by some very circuitous routes. But it seems to me quite clear that the premature death of phlogiston retarded scientific progress in quite tangible ways. If it had been left to develop, I think the concept of phlogiston would have split into two. On the one hand, by the early nineteenth century someone might well have hit upon energy conservation, puzzling over this imponderable entity which seemed to have an elusive sort of reality which could be passed from one ponderable substance to another.
In that parallel universe, we would be talking about the conservation of phlogiston, and how phlogiston turned out to have all sorts of different forms, but all interconvertible with each other. This would be no more awkward than what we have in our actual universe, in which we still talk about the role of “oxygen” (acid-generator, Sauerstoff ) in supporting combustion, and the “oxidation” number of ions. On the other hand, the phlogiston concept could have led to a study of electrons without passing through such a categorical and over-simplified atomic theory as Dalton’s. Chemists might have skipped right over from phlogiston to elementary particles, or at least found an alternative path of development that did not pass through the false simplicity of the atom–molecule–bulk matter hierarchy. Keeping the phlogiston theory would have led chemists to pay more attention to the “fourth state of matter”, starting with flames, and served as a reminder that the durability of compositionist chemical building-blocks may only be an appearance. Keeping phlogiston alive could have challenged the easy Daltonian assumption that chemical atoms were physically unbreakable units. The survival of phlogiston into the nineteenth century would have sustained a vigorous alternative tradition in chemistry and physics, which would have allowed scientists to recognize with more ease the wonderful fluidity of matter, and to come to grips sooner with the nature of ions, solutions, metals, plasmas, cathode rays, and perhaps even radioactivity.
Apparently people want some clarification on what I mean by anti-library. It’s a Nassim Taleb term which refers to books you own but haven’t read, whose main value is to remind you and keep in mind what you don’t know and where to find it if you want to expand that knowledge.
If the point you’re trying to make is: “the way we go from preparadigmatic to paradigmatic is by solving some hard problems, not by communicating initial frames and idea”, I think this points to an important point indeed.
Still, two caveats:
First, Kuhn’s concept of paradigm is quite an oversimplification of what actually happens in the history of science (and the history of most fields). More recent works that go through history in much more detail realize that at any point in fields there are often many different pieces of paradigms, or some strong paradigm for a key “solved” part of the field and then a lot of debated alternative for more concrete specific details.
Generally, I think the discourse on history and philosophy of science on LW would improve a lot if it didn’t mostly rely on one (influential) book published in the 60s, before much of the strong effort to really understand history of science and practices.
Second, to steelman John’s point, I don’t think he means that you should only communicate your frame. He’s the first to actively try to apply his frames to some concrete problems, and to argue for their impressiveness. Instead, I read him as pointing to a bunch of different needs in a preparadigmatic field (which maybe he could separate better ¯\_(ツ)_/¯)
That in a preparadigmatic field, there is no accepted way of tackling the problems/phenomena. So if you want anyone else to understand you, you need to bridge a bigger inferential distance than in a paradigmatic field (or even a partially paradigmatic field), because you don’t even see the problem in the same way, at a fundamental level.
That if your goal is to create a paradigm, almost by definition you need to explain and communicate your paradigm. There is a part of propaganda in defending any proposed paradigm, especially when the initial frame is alien to most people, and even the impressiveness require some level of interpretation.
That one way (not the only way) by which a paradigm emerges is by taking different insights from different clunky frames, and unifying them (for a classic example, Newton relied on many previous basic frames, from Kepler’s laws to Galileo’s interpretation of force as causing acceleration). But this requires that the clunky frames are at least communicated clearly.
Curated. I’ve heard this book suggested a few times over the years, and feels like it’s a sort of unofficial canon among people studying how preparadigmatic science happens. This review finally compelled me to get the book.
There’s something quite funny in that I discovered this book in January 2022, during the couple of days I spent at Lightcone offices. It was in someone’s office, and I was curious about it. Now, we’re back full circle. ^^
I do think this review would be a lot better if it actually distilled the messy-bits-that-you-need-to-experientially-stew-over into a something that was (probably) much longer than this post, but, much shorter than the book. But that does seem legitimately hard.
Agreed.
But as I said in the post, I think it’s much more important to get the feel from this book than just the big ideas. I believe that there’s a way to write a really good blog post that shares that feel and compresses it, but that was not what I had the intention or energy (or mastery) to write.
It sounds cool, though also intuitively temperature seems like one of the easiest attributes to measure because literally everything is kind of a thermometer in the sense that everything equillibrates in temperature.
Can’t guarantee that you would benefit from it, but this sentence makes me think you have a much cleaner and simplified idea of how one develops even simple measuring device than what the history shows (especially when you don’t have any good theory of temperature or thermodynamics).
So would say you might benefit from reading it. ;)
Typo addressed in the latest patch!