Epistemologist specialized in the difficulties of alignment and how to solve AI X-Risks. Currently at Conjecture.
Blogging at For Methods.
adamShimi
Karma: 6,591
Now addressed in the latest patch!
Now addressed in the latest patch!
Now addressed in the latest patch!
Thanks for the comment!
We have indeed gotten the feedback by multiple people that this part didn’t feel detailed enough (although we got this much more from very technical readers than from non-technical ones), and are working at improving the arguments.
Thanks for the comment!
We’ll correct the typo in the next patch/bug fix.
As for the more direct adversarial tone of the prologue, it is an explicit choice (and is contrasted by the rest of the document). For the moment, we’re waiting to get more feedback on the doc to see if it really turns people off or not.
Yep, I think you’re correct.
Will correct in the next minor update. Thanks!
Thanks for the comment!
We’ll consider this point for future releases, but personally, I would say that this kind of hedging also has a lot of downsides: it makes you sound far more uncertain and defensive than you really want to.
This document tries to be both grounded and to the point, and so we by default don’t want to put ourselves in a defensive position when arguing things that we think make sense and are supported by the evidence.
Thanks for the comment!
We have gotten this feedback by a handful of people, so we want to reread the links and the whole literature about o1 and its evaluation to check whether we’ve indeed gotten the right point, or if we mischaracterized the situation.
We will probably change the phrasing (either to make our criticism clearer or to correct it) in the next minor update.
Good catch, I think we are indeed mixing the sizes here.
As you say, the point still stands, but we will change it in the next minor update to either compare the same size or make the difference in size explicit.
Thanks for the comment!
We want to check the maths, but if you’re indeed correct we will update the numbers (and reasoning) in the next minor version.
Fragile, Robust, and Antifragile Preference Satisfaction
The Compendium, A full argument about extinction risk from AGI
Triangulating My Interpretation of Methods: Black Boxes by Marco J. Nathan
I guess it depends on if you’re pivoting based on things that you’ve learned, versus grass-is-greener.
Yeah, I didn’t mean “iterative thoughtful processes”, I meant “compulsion that unfold at the level of days”. If you arbitrarily change your job every couple of days/weeks, not based on new significant information but because you feel this other one is the one, this is bad.
So there is a vibe here that I maybe didn’t convey well, about the time frame and the auto-generated part of the loops I’m pointing at: it happens often enough that your friends and family can notice, and it happens in reaction to events that no one around you agree would lead to such a drastic change (highlighting that the events are not so much the cause as the post-hoc rationalization).
Self-Help Corner: Loop Detection
My Methodological Turn
Recently found a new link: Annual Reviews
It sounds like a place that centralizes many different review articles across a lot of disciplines. Only checked a few for the moment, but definitely sounds worth a try!
@Elizabeth suggested that I share here the quick tips I gave her for finding cool history and philosophy of science books, so let’s do it.
I like using awards as starting points. They’re not exhaustive, but often they point to particularly good references in a field that I don’t know about.
For philosophy of science, often with a decent dose of history, there is the Lakatos Award.
For history of science, there is the Sarton Medal, which is given to individuals, not works
Same with book reviews by journals focused on the topic
My favorite are from the British Journal for The Philosophy of Science reviews
Knowing the terminology helps. I find that “History and Philosophy of X” is often a good google query
I recently discovered https://hiphilangsci.net/ on linguistics that way!
Obviously, follow the citations: cool books tend to reference cool books. (And terrible ones, but let’s not mention that)
Also known, but just in case: going to https://scholar.google.com/ and searching for the most cited books that cite a book you liked often leads to great reading material.
Yeah, I agree with the general point (don’t have strong opinion about chaos theory at the moment).
First, negative results are really, really important. Mostly because they let you not lose your time trying to do something impossible, and sometimes they actually point you toward an answer. In general, conservation laws in physics have this role. And knowing what is undecidable is really important in formal methods, where the trick is generally to simplify what you want or the expressive power of your programs in order to sidestep it.
Then, they are indeed quite hard to prove, at least in non-trivial cases. Conservations laws are the results of literally centuries of reframing of classical mechanics and reduction, leading to seeing the importance of energy and potential in unifying everything in physics. Undecidability is the result of 60 years of metamathetical work trying to clean formalisms enough to be able to study these kind of properties.
Typo addressed in the latest patch!