Book Launch: The Engines of Cognition

[Ben Pace]

The Engines of Cognition is a newly published collection of essays by the LessWrong Community. They’re currently available with Prime at Amazon US, where they’ll arrive in time for Christmas, and you can read on to learn more about them.

About

In The Engines of Cognition, the writers set out to understand key elements of the art of rationality. Starting with the simple epistemic question of when and how to trust different sources of information, the essays in this collection move through understanding the lens of incentives, an exploration of when and why complex systems become modular, and finally into a discussions of failure, both personal and civilizational.

This book set is for people who want to read the most interesting ideas LessWrong has recently explored. It’s for the people who read best away from screens, away from distractions. It’s for people who do not check the site regularly, but would still like to get the ideas within. For such people, this is intended to be the ideal way to read LessWrong.

Essays in this book set take a variety of forms, from thought experiments to literature reviews, as well as book reviews, interviews, personal stories, microeconomic arguments, mathematical explanations, research advice, philosophical musings, published papers, disagreements-with-Robin-Hanson, forecasts for the future, survey data, cartoons, and more.

Authors featured include Eliezer Yudkowsky, Scott Alexander, Zvi Mowshowitz, and over 30 more LessWrong writers. The essays were originally published to LessWrong in 2019, and for the first time are available with editing and illustration in print form. In addition, given recent advances in machine learning art, each essay has a unique piece of ML-generated art based on the essay title, some of which are shown below.

Order now on Amazon Prime US ($30) and it will arrive in time for Christmas.

Book Spreads

Table of Contents

Trust
Rule Thinkers In, Not Out	Scott Alexander
Gears vs Behavior	John S. Wentworth
Book Review: The Secret Of Our Success	Scott Alexander
Reason isn’t magic	Ben Hoffman
“Other people are wrong” vs “I am right”	Buck Shlegeris
In My Culture	Duncan Sabien
Chris Olah’s views on AGI safety	Evan Hubinger
Understanding “Deep Double Descent”	Evan Hubinger
How to Ignore Your Emotions (while also thinking you’re awesome at emotions)	Hazard
Paper-Reading for Gears	John S. Wentworth
Book summary: Unlocking the Emotional Brain	Kaj Sotala
Noticing Frame Differences	Raymond Arnold
Propagating Facts into Aesthetics	Raymond Arnold
Do you fear the rock or the hard place?	Ruben Bloom
Mental Mountains	Scott Alexander
Steelmanning Divination	Vaniver
Modularity
Book Review: Design Principles of Biological Circuits	John S. Wentworth
Reframing Superintelligence: Comprehensive AI Services as General Intelligence	Rohin M. Shah
Building up to an Internal Family Systems model	Kaj Sotala
Being the (Pareto) Best in the World	John S. Wentworth
The Schelling Choice is “Rabbit”, not “Stag”	Raymond Arnold
Literature Review: Distributed Teams	Elizabeth Van Nostrand
Gears-Level Models are Capital Investments	John S. Wentworth
Evolution of Modularity	John S. Wentworth
You Have About Five Words	Raymond Arnold
Coherent decisions imply consistent utilities	Eliezer Yudkowsky
Alignment Research Field Guide	Abram Demski
Forum participation as a research strategy	Wei Dai
The Credit Assignment Problem	Abram Demski
Selection vs Control	Abram Demski
Incentives
Asymmetric Justice	Zvi Mowshowitz
The Copenhagen Interpretation of Ethics	Jai Dhyani
Unconscious Economics	Jacob Lagerros
Power Buys You Distance From The Crime	Elizabeth Van Nostrand
Seeking Power is Often Convergently Instrumental in MDPs	Alexander Turner & Logan Smith
Yes Requires the Possibility of No	Scott Garrabrant
Mistakes with Conservation of Expected Evidence	Abram Demski
Heads I Win,Tails?—Never Heard of Her; Or, Selective Reporting and the Tragedy of the Green Rationalists	Zack M. Davis
Excerpts from a larger discussion about simulacra	Ben Hoffman
Moloch Hasn’t Won	Zvi Mowshowitz
Integrity and accountability are core parts of rationality	Oliver Habryka
The Real Rules Have No Exceptions	Said Achmiz
Simple Rules of Law	Zvi Mowshowitz
The Amish, and Strategic Norms around Technology	Raymond Arnold
Risks from Learned Optimization: Introduction	Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, & Scott Garrabrant
Gradient hacking	Evan Hubinger
Failure
The Parable of Predict-O-Matic	Abram Demski
Blackmail	Zvi Mowshowitz
Bioinfohazards	Megan Crawford, Finan Adamson, & Jeffrey Ladish
What failure looks like	Paul Christiano
Seeking Power is Often Convergently Instrumental in MDPs	Alexander Turner & Logan Smith
AI Safety “Success Stories”	Wei Dai
Reframing Impact	Alexander Turner
The strategy-stealing assumption	Paul Christiano
Is Rationalist Self-Improvement Real?	Jacob Falkovich
The Curse Of The Counterfactual	P.J. Eby
human psycholinguists: a critical appraisal	Nostalgebraist
Why wasn’t science invented in China?	Ruben Bloom
Make more land	Jeff Kaufman
Rest Days vs Zombie Days	Lauren Lee

FAQ

How were the essays in the books selected?

Each year LessWrong holds an annual review, where members of the community review and vote on the best content of the year prior. These essays are chosen from the most highly upvoted of the last review, when in 2020 we reviewed essays from 2019.

There are also a few small additional essays, such as referenced essays included for context or new postscripts by authors.

How are the books made?

Ben Pace on the LessWrong team led the project. Asimov Collective did most of the hard work of putting the books together, and Hemlock Printers in Canada printed the books.

What size are the books?

Each book is 5 x 7.75 inches, notably larger than the last set of LessWrong books released.

How was the art generated?

Using VQGAN+CLIP, a system that takes in a base image and a written prompt, and attempts to transform the base image into what the GAN expects the image to look like given the associated text. This ML system is available in many public notebooks. For each essay the LessWrong and Asimov Collective teams generated ~60 images using slight variations in prompts, from which each essay’s image was selected.

Here is an approximate recreation of how the cover images were generated.

To give a sense of what prompts were used, here was the prompt used for the “Trust” volume:

The Engines of Cognition by Alex Hillkurtz | System of Gears | Aquarelle | Greek Architecture | Blue on White Color Palette | Trending on Artstation

Does this book assume I have read other LessWrong content, like The Sequences?

No. It’s largely standalone, and does not require reading other content on the site, although it will be enhanced by having engaged with those ideas. A few essays will assume certain sorts of background context, but the vast majority will not.

How can I get the books?

We have books in store in Amazon US, and we will be shipping books to Amazon UK in the coming month. You can sign up below to be notified when they arrive in Amazon UK. Sending books to other regions is a lot of work so we won’t do it this year. Note that both Amazon US and UK can do international shipping, and you can currently order from Amazon to the UK.

These will be the only ways to buy the books, though if you are planning to buy in bulk (as few as 20+) feel free to contact me at [e-mail] to see if I can help.

When will the books be on Amazon UK?

In a couple of weeks. You can sign up below to be notified by me (Ben) when the books are in stock.

Notify me when the UK books are available

Email Address

I have a question not answered here?

There’s a comment box right below.

Remind me again, how can I order it in the US?

Order now on Amazon Prime US ($30) and it will arrive in time for Christmas.

[Yoav Ravid]

The covers are beautiful!

[Ben Pace]

Glad you think so :)

[Alex Vermillion]

I don’t have a copy yet, so this could be written somewhere in them, but is there any way we could learn what the prompt for all the art/​covers are? I found it pretty fun to learn what one of them was.

[Ben Pace]

I agree, sharing all the prompts is fun. I’d like to share them at some point.

[Pattern]

Only physical books, no ebooks?

[holomanga]

Would me compiling the Lesswrong posts as they are on the website into an .epub and uploading it provide a significant amount of the value of an ebook, or does enough of it come from other sources that one can’t do at home like the pretty new diagrams and being on the Kindle store?

[Ben Pace]

25% of the value I reckon? You wouldn’t recreate the art or layout or additions content. Would be nice if you did it, I’d certainly learn a new fact based on number of downloads if that was measurable.

[Vanessa Kosoy]

+1, also I wish there was an ebook version of “A Map that Reflects the Territory”

[Ben Pace]

Yep, at this point. May change later.

[yagudin]

I would appreciate it if the ToC linked to the web versions of the essay.

[yagudin]

Trust
Rule Thinkers In, Not Out	Scott Alexander
Gears vs Behavior	John S. Wentworth
Book Review: The Secret Of Our Success	Scott Alexander
Reason isn’t magic	Ben Hoffman
“Other people are wrong” vs “I am right”	Buck Shlegeris
In My Culture	Duncan Sabien
Chris Olah’s views on AGI safety	Evan Hubinger
Understanding “Deep Double Descent”	Evan Hubinger
How to Ignore Your Emotions (while also thinking you’re awesome at emotions)	Hazard
Paper-Reading for Gears	John S. Wentworth
Book summary: Unlocking the Emotional Brain	Kaj Sotala
Noticing Frame Differences	Raymond Arnold
Propagating Facts into Aesthetics	Raymond Arnold
Do you fear the rock or the hard place?	Ruben Bloom
Mental Mountains	Scott Alexander
Steelmanning Divination	Vaniver
Modularity
Book Review: Design Principles of Biological Circuits	John S. Wentworth
Reframing Superintelligence: Comprehensive AI Services as General Intelligence	Rohin M. Shah
Building up to an Internal Family Systems model	Kaj Sotala
Being the (Pareto) Best in the World	John S. Wentworth
The Schelling Choice is “Rabbit”, not “Stag”	Raymond Arnold
Literature Review: Distributed Teams	Elizabeth Van Nostrand
Gears-Level Models are Capital Investments	John S. Wentworth
Evolution of Modularity	John S. Wentworth
You Have About Five Words	Raymond Arnold
Coherent decisions imply consistent utilities	Eliezer Yudkowsky
Alignment Research Field Guide	Abram Demski
Forum participation as a research strategy	Wei Dai
The Credit Assignment Problem	Abram Demski
Selection vs Control	Abram Demski
Incentives
Asymmetric Justice	Zvi Mowshowitz
The Copenhagen Interpretation of Ethics	Jai Dhyani
Unconscious Economics	Jacob Lagerros
Power Buys You Distance From The Crime	Elizabeth Van Nostrand
Seeking Power is Often Convergently Instrumental in MDPs	Alexander Turner & Logan Smith
Yes Requires the Possibility of No	Scott Garrabrant
Mistakes with Conservation of Expected Evidence	Abram Demski
Heads I Win,Tails?—Never Heard of Her; Or, Selective Reporting and the Tragedy of the Green Rationalists	Zack M. Davis
Excerpts from a larger discussion about simulacra	Ben Hoffman
Moloch Hasn’t Won	Zvi Mowshowitz
Integrity and accountability are core parts of rationality	Oliver Habryka
The Real Rules Have No Exceptions	Said Achmiz
Simple Rules of Law	Zvi Mowshowitz
The Amish, and Strategic Norms around Technology	Raymond Arnold
Risks from Learned Optimization: Introduction	Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, & Scott Garrabrant
Gradient hacking	Evan Hubinger
Failure
The Parable of Predict-O-Matic	Abram Demski
Blackmail	Zvi Mowshowitz
Bioinfohazards	Megan Crawford, Finan Adamson, & Jeffrey Ladish
What failure looks like	Paul Christiano
Seeking Power is Often Convergently Instrumental in MDPs	Alexander Turner & Logan Smith
AI Safety “Success Stories”	Wei Dai
Reframing Impact	Alexander Turner
The strategy-stealing assumption	Paul Christiano
Is Rationalist Self-Improvement Real?	Jacob Falkovich
The Curse Of The Counterfactual	P.J. Eby
human psycholinguists: a critical appraisal	Nostalgebraist
Why wasn’t science invented in China?	Ruben Bloom
Make more land	Jeff Kaufman
Rest Days vs Zombie Days	Lauren Lee

Here is a google sheet.

[Pattern]

As would I.

Perhaps that can be fixed with a spreadsheet:

Title Author Formula Base Link

[etc.] [etc.] =concat(A4, A2, A3) https://​​www.lesswrong.com/​​search?terms=

Then it’s just a matter of copying columns 3 and 4 down, (after pasting in the TOCs), opening a bunch of tabs, then copying the results over to column 5. Then a formula for column 6 could spit out the markdown formatting, which can be copied and pasted* into a comment with the appropriate formatting selected.

*Copied as a column for speed.

Any ideas for ways to point to, or retrieve*, the first search result by default?

*Might already have the formatting (title and link).

[null]

5 x 7.75 inches

fwiw that’s 12.7 x 19.7 cm

[holomanga]

There is a listing already on Amazon UK, automatically generated by Amazon reselling from Amazon US, which as of the time of writing claims to deliver by January 5th. For me it was slightly cheaper to order it there than from the US site with international delivery directly :).

[Ben Pace]

Wow, great to hear!

[Alex_Altair]

Whooo mine just arrived! First thoughts;

• I mean, they’re amazing, that’s the first 15 things I have to say

• Sad that they’re much larger, but I can’t blame you, and they’re still on the small side, and you warned me

• The covers are more matte, whereas the previous ones are more glossy. This is fine by me aesthetically, though it does make me feel like maybe they’d be more damaged by water.

• Amazon delivered it in one of those plastic bubble wrap bags rather than in a box. It seems like this caused the edges to get a bit dinged up, which is unfortunate. Not sure if there’s anything you can do about that.

[Ben Pace]

It’s very cool that people get the books so quickly this time! Thanks for the comments :)

[Tomas Piala]

Its a shame Amazon is sending them this way, mine came with corners completely bruised off, so I will be probably returning them :(

[lumenwrites]

Looks amazing!

I’d love to buy an ebook version though. Or even better—an audiobook.

[Christopher Galias]

Will the previous LW books be made available again?

[Ben Pace]

I’d be surprised if we don’t get around to it sometime, but no concrete plans right now.

[m0sys]

Just the fact that they’re AI generated images based on the content of each essay makes the books beautifully sweet. Can’t wait till they arrive!

[Alex_Altair]

Me: Hm, I notice that all the swirly images have the same basic structure. I wonder what base image they used for it...
Me: *scrolls* oh my goood it’s the Mississippi river map

[Ben Pace]

Yeah, last year’s books evolved into this year’s books :)

One idea we’ve been playing with it to have a tradition every year of using the last year’s covers as the base for the new covers.

[JanGoergens]

I would like to have these books in audio form. Are there any plans to record an audiobook on audible (or other platforms)?

[MikkW]

Is there anywhere the art for all the essays can be viewed online?

[Ben Pace]

No, it’s just in the books.

[niplav]

Power Buys You Distance From The Crime was by Elizabeth, not Ben Hoffman.

[Ben Pace]

Fixed.

[__nobody]

It says the UK Amazon doesn’t ship to Germany [at least for the auto-generated listing], and from the US it’d be ~$45 incl. shipping + taxes… =/​

And since it’s above the magic number of 1kg (around 1.2kg), even a bulk order with local distribution would have to add about €5 for the last leg of shipping, which (adding packaging etc.) makes that just not worth it.

Since Amazon UK is happy to ship other books, I subscribed to the UK availability notification—maybe it’ll work once it’s “really” there. I’ll update this once the notification comes and I have time to check.

(If that doesn’t fix it: If it were available at some endpoint within the EU, Amazon’s shipping would probably be around €6 (based on looking at comparable book boxes from ES or FR.) So if UK refuses to ship and it’s not too terrible (or would add extra costs that make it useless), then I’d appreciate having some EU endpoint next year again… but first let’s see what’ll happen.)

[__nobody]

Update: Amazon Germany now also has the books listed, for €36 (which is fine.) Since I haven’t received the “Notify me when the UK books are available” mail yet, I assume this is further downstream propagation from the Amazon US listing.

If that is accurate, then there should be no need at all to manually ship books to other regions?! I guess that’s very good news for future books!

[Ben Pace]

Indeed the books are not yet in-stock in Amazon UK.

[MondSemmel]

Thanks for writing this; I didn’t even think to check Amazon Germany since it wasn’t listed here.

That said, a quick search for Engines of Cognition on some websites that purportedly compare product prices on all Amazon EU stores did only list prices for UK and DE. Since the product is imported from the US in either case, and the non-US store pages were not created manually, I don’t really understand why it wouldn’t be offered on all Amazon storefronts.

EDIT: Apparently the product is available on Amazon DE (though probably only within Germany) for “free worldwide shipping” from the US: the checkout page lists a shipping fee of 11.14€, which it then discounts to zero. This is the same procedure they use for nationwide free shipping, which is nominally priced at 3~4€ IIRC.

[Steven Byrnes]

“Seeking Power is Often Convergently Instrumental in MDPs” is listed as being present in two books…

[Ben Pace]

I guess I’ll go open the books to double check it’s just a typo...

[Mr. Mar]

Are there any plans to print more copies of last year’s set?

[Ben Pace]

Not right now, but that was probably a mistake. I had some edits I wanted to do for last year’s books that I didn’t get around to yet for a second print run.

[__nobody]

Design feedback: Alignment is hard, even when it’s just printing. Consider bumping up the running footer by 1-2mm next time, it ended up uncomfortably close to the bottom edge at times. (Also the chapter end note /​ references pages were a mess.) More details:

variance: For reference, in the books that I have, the width of the colored bars along the page edge at each chapter (they’re easy to measure) varies between ~4.25mm and ~0.75mm, and sometimes there’s a ~2mm width difference between top and bottom. (No complaints here. The thin /​ rotated ones look a bit awkward if you really look at them, but you’ll likely be distracted by the nice art on the facing page anyway. So who cares, and they do their job.)

footers: Technically, the footer was always at least 2mm away from the edge (so it didn’t really run the risk of getting cut off), but occasionally it felt so close that it was hard not to notice. That distracted from reading, and made those pages feel uncomfortable… giving it just 1 or 2mm more should take out the tension. (While I didn’t experiment with it, my gut feeling says the text block probably won’t have to move to make more space.)

end notes/​references: These just looked weird to me. Rambling train of thought style notes:

• Choice of a different font that looked thin and spindly compared to the main text, worsened by the all-italic choice. (That also made it look fidgety/​restless, thanks to constant kerning weirdness – the font is clearly not meant to be used this way, worst in URLs like Incentives p. 77 item 8.)

• Usual citation formats are something like ‘Author (Year). “Title.” More stuff.’ or ‘Author (Year). Title. More stuff.‘, so there’s either sufficient punctuation to create noticeable points, or the emphasis creates a texture difference. The chosen ‘Author. Title. Year’ just runs together with no strongly noticeable points, making it hard to pick out the individual fields. It seems to be close to the Bluebook style, but that really relies on the texture contrast in the “Author, Title. Year” to function.

• The parenthesized item numbering (23) also looked weird in that font, maybe dotted 23. or 23: would have looked better; combined with the numbers being the only thing not italicized, it looked more like a mistake than intention to me.

• Also lots of typos /​ inconsistencies, e.g. occasional missing years/​dates, sometimes it’s just “Wikipedia. Article name.” (good IMHO) then sometimes it’s “Article. Wikipedia” (why is the “author” not first here?) and sometimes it’s a badly-formatted link (whyyy), occasionally it looks like there’s two spaces instead of one (or just more kerning weirdness?), and e.g. Incentives p. 206 has items 4 + 5 running into each other.

• Some indication for where the text can be found would be helpful. (Also whether it’s a text or something else.) Most seem to be on LW, some can be guessed, but for others you have to search. Shorthands like e.g. a ° after the title to mark “this is on LW” or maybe a small ˣ for “can be found in arXiv” would be enough for the usual sources. But try finding Incentives p 133 item (10) by searching! The only way to find it is to look up the original article, locate the link, and then facepalm hard at the incomplete information.

• It all seemed more like an afterthought than a planned part of the book, and the look of the whole really encouraged me to quickly turn to the next page, instead of looking for more things to read.

• (Also: Consider reducing the line height and item spacing? It’s not text that you read continuously, so less leading both reduces the required space and makes it stand apart from the main text without using questionable fonts.)

Apart from that, I loved the design! Thanks to everyone involved for making the books, they’re lovely! <3

[Ben Pace]

Thanks for the feedback! Agree with your overall paragraph. Main constraint you’re not modeling is how increasing margin size increases total pages and thus cost. Seems plausible I should cut one or two essays to accommodate, but I do love all the essays, and actually the real answer is just that the paragraph spacing is way too big.

I didn’t notice that about the footers, I will go back and take a look.

The end notes were time-crunched for a lot of reasons, I wish they had been better.

[__nobody]

Main constraint you’re not modeling is how increasing margin size increases total pages and thus cost.

That’s why I’m saying it probably won’t need that for the footers. There’s ~10mm between running footer and text block, if that’s reduced to ~8 or 9mm and those 1-2mm go below the footer instead, that’s still plenty of space to clearly separate the two, while greatly reducing the “falling off the page” feeling. (And the colored bars that mark chapters are fine, no need to touch those.)

[Ben Pace]

I see, I misread, yup that makes sense.

[PointlessOne]

The books are unavailable anywhere. Can we expect more anytime soon?

[gekaklam]

Are there any updates about when the books would be available on Amazon UK or even better Amazon DE /​ FR to make it easier for people based in EU countries to get them?

Thanks for making the books, I’m really looking forward to get them! :-D

[MarcelloV]

Was “Rest Days vs Zombie Days” removed from LessWrong, or at the very least available online anywhere else? I can’t find it either via Google search or LessWrong’s site search. All I see is “Rest Days vs. Recovery Days” by Unreal from 2019.

[Ben Pace]

Same post! Author changed the name for the book. I’ll probs check in with author and then change it on LW too.

[Yoav Ravid]

Shouldn’t the order of Asymmetric Justice and The Copenhagen Interpretation of Ethics be flipped? (the former references the latter)

[Ben Pace]

The former essay makes sense without it, and then reading it as a follow-up gives more depth. Mostly I didn’t want to open with an essay that wasn’t in the 2019 Review.

[Yoav Ravid]

Fair enough.

[Samuel Clamons]

Late to the party, I know, but I just got my copies and really looking forward to reading them! I really miss the tiny form factor of “Maps that Reflect the Territory” though, they really felt elegant in a way the more recent ones don’t.

[Orome]

I missed these somehow, and they’re no longer available on Amazon. Any chance of a reprint?

[Dan Meltz]

I missed the print run. any chance of a new printing?

[Kenny]

The layouts look impressively well done!

[Alex Vermillion]

For what it’s worth, https://​​www.lesswrong.com/​​books/​​ still points to https://​​www.lesswrong.com/​​books/​​2018, not https://​​www.lesswrong.com/​​books/​​2019, which is where the new books are. I noticed only because I checked both /​​books and /​​books/​​2019 in order since I didn’t know which link would update, but luckily found these are out!

[Ben Pace]

Lol, I didn’t finish the 2019 page and intentionally didn’t change the link to point to it.

[lc]

Fantastic work on the books by the way.

[Ben Pace]

snort Cheers!