RSS

Moloch

TagLast edit: Dec 18, 2023, 6:53 AM by Yoav Ravid

Moloch is the personification of the forces that coerce competing individuals to take actions which, although locally optimal, ultimately lead to situations where everyone is worse off. Moreover, no individual is able to unilaterally break out of the dynamic. The situation is a bad Nash equilibrium. A trap.

It happens when “In some competition optimizing for X, the opportunity arises to throw some other value under the bus for improved X. Those who take it prosper. Those who don’t take it die out. Eventually, everyone’s relative status is about the same as before, but everyone’s absolute status is worse than before. The process continues until all other values that can be traded off have been – in other words, until human ingenuity cannot possibly figure out a way to make things any worse.”—Scott Alexander

One example of a Molochian dynamic is a Red Queen race between scientists who must continually spend more time writing grant applications just to keep up with their peers doing the same. Through unavoidable competition, they have all lost time while not ending up with any more grant money. And any scientist who unilaterally tried to not engage in the competition would soon be replaced by one who still does. If they all promised to cap their grant writing time, everyone would face an incentive to defect.

The topic of Moloch receives a formal treatment in the sequence Inadequate Equilibria, particularly in the chapter Moloch’s Toolbox.

Origin

Scott Alexander linked the name to the concept in his eponymous post, Meditations on Moloch. The post intersperses lines of Allan Ginsberg’s poem, Howl, with multiples examples of the dynamic including: the Prisoner’s Dilemma, dollar auctions, fish farming story, Malthusian trap, capitalism, two-income trap, agriculture, arms races, races to the bottom, education system, science, and government corruption and corporate welfare.

From Allan Ginsberg’s Howl:

What sphinx of cement and aluminum bashed open their skulls and ate up their brains and imagination?
Moloch! Solitude! Filth! Ugliness! Ashcans and unobtainable dollars! Children screaming under the stairways! Boys sobbing in armies! Old men weeping in the parks!
Moloch! Moloch! Nightmare of Moloch! Moloch the loveless! Mental Moloch! Moloch the heavy judger of men!
Moloch the incomprehensible prison! Moloch the crossbone soulless jailhouse and Congress of sorrows! Moloch whose buildings are judgment! Moloch the vast stone of war! Moloch the stunned governments!
Moloch whose mind is pure machinery! Moloch whose blood is running money! Moloch whose fingers are ten armies! Moloch whose breast is a cannibal dynamo! Moloch whose ear is a smoking tomb!
Moloch whose eyes are a thousand blind windows! Moloch whose skyscrapers stand in the long streets like endless Jehovahs! Moloch whose factories dream and croak in the fog! Moloch whose smoke-stacks and antennae crown the cities!
Moloch whose love is endless oil and stone! Moloch whose soul is electricity and banks! Moloch whose poverty is the specter of genius! Moloch whose fate is a cloud of sexless hydrogen! Moloch whose name is the Mind!

See also: Eldritch Analogies, Game Theory, Group Rationality, Social Reality

Med­i­ta­tions On Moloch

Scott AlexanderJul 30, 2014, 4:00 AM
196 points
10 comments47 min readLW link

Moloch’s Toolbox (1/​2)

Eliezer YudkowskyNov 4, 2017, 9:46 PM
154 points
70 comments33 min readLW link

Stud­ies On Slack

Scott AlexanderMay 13, 2020, 5:00 AM
165 points
34 comments24 min readLW link1 review
(slatestarcodex.com)

Moloch games

Chris van MerwijkOct 16, 2020, 3:19 PM
80 points
9 comments4 min readLW link

Lost Purposes

Eliezer YudkowskyNov 25, 2007, 9:01 AM
186 points
79 comments6 min readLW link

What Mul­tipo­lar Failure Looks Like, and Ro­bust Agent-Ag­nos­tic Pro­cesses (RAAPs)

Andrew_CritchMar 31, 2021, 11:50 PM
282 points
65 comments22 min readLW link1 review

Mazes Se­quence Summary

ZviMay 23, 2020, 4:40 PM
81 points
2 comments7 min readLW link
(thezvi.wordpress.com)

Moloch’s Toolbox (2/​2)

Eliezer YudkowskyNov 7, 2017, 1:58 AM
112 points
56 comments24 min readLW link

Moloch Hasn’t Won

ZviDec 28, 2019, 4:30 PM
186 points
40 comments7 min readLW link1 review
(thezvi.wordpress.com)

A Taijitu sym­bol for Moloch and Slack

B JacobsMay 25, 2020, 8:03 PM
91 points
3 comments1 min readLW link

The Choice Transition

Nov 18, 2024, 12:30 PM
50 points
4 comments15 min readLW link
(strangecities.substack.com)

Failing to fix a dan­ger­ous intersection

alyssavanceJun 30, 2022, 6:09 PM
110 points
17 comments2 min readLW link

Dar­wi­nian Traps and Ex­is­ten­tial Risks

KristianRonnAug 25, 2024, 10:37 PM
85 points
14 comments10 min readLW link

Moloch in whom I sit alone

KatjaGraceOct 3, 2018, 11:40 PM
55 points
16 comments1 min readLW link
(worldlypositions.tumblr.com)

5 gen­eral vot­ing patholo­gies: lesser names of Moloch

Jameson QuinnApr 13, 2018, 6:38 PM
73 points
16 comments10 min readLW link

Univer­sal Ba­sic In­come and Poverty

Eliezer YudkowskyJul 26, 2024, 7:23 AM
321 points
139 comments9 min readLW link

Pri­son­ers’ Dilemma with Costs to Modeling

Scott GarrabrantJun 5, 2018, 4:51 AM
123 points
20 comments7 min readLW link

Gen­er­al­ized Effi­cient Mar­kets and Academia

johnswentworthApr 30, 2020, 9:59 PM
52 points
29 comments2 min readLW link

Liv Bo­eree Ted Talk Moloch & AI

Neil Nov 10, 2023, 2:04 PM
10 points
2 comments1 min readLW link
(m.youtube.com)

Moloch: op­ti­mi­sa­tion, “and” vs “or”, in­for­ma­tion, and sac­ri­fi­cial ems

Stuart_ArmstrongAug 6, 2014, 3:57 PM
35 points
59 comments3 min readLW link

An Equil­ibrium of No Free Energy

Eliezer YudkowskyOct 31, 2017, 9:27 PM
133 points
50 comments27 min readLW link

Com­ment on SSC’s Re­view of Inad­e­quate Equilibria

Ben PaceDec 1, 2017, 11:46 AM
13 points
5 comments2 min readLW link

[Book Re­view] Destiny Disrupted

lsusrMar 21, 2021, 7:09 AM
58 points
4 comments9 min readLW link

[Question] Look­ing for ideas of pub­lic as­sets (stocks, funds, ETFs) that I can in­vest in to have a chance at prof­it­ing from the mass adop­tion and com­mer­cial­iza­tion of AI technology

AnnapurnaDec 7, 2022, 10:35 PM
15 points
9 comments1 min readLW link

Keep hu­mans in the loop

Apr 19, 2023, 3:34 PM
23 points
1 comment10 min readLW link

Moloch and the sand­pile catastrophe

Eric RaymondApr 2, 2022, 3:35 PM
120 points
25 comments3 min readLW link

The Thing and the Sym­bolic Rep­re­sen­ta­tion of The Thing

ZviJun 30, 2015, 12:15 AM
17 points
0 comments5 min readLW link

The Era of the Divi­d­ual—are we fal­ling apart?

James Stephen BrownApr 12, 2025, 10:35 PM
3 points
2 comments4 min readLW link

Mak­ing a Crow­dac­tion platform

B JacobsMay 16, 2020, 4:08 PM
25 points
9 comments5 min readLW link

The Case Against UBI

Zero ContradictionsJul 27, 2024, 6:36 AM
−1 points
2 comments2 min readLW link
(thewaywardaxolotl.blogspot.com)

Tack­ling Moloch: How YouCongress Offers a Novel Co­or­di­na­tion Mechanism

Hector Perez ArenasMay 15, 2024, 11:13 PM
28 points
9 comments6 min readLW link

The Frag­ility of Life Hy­poth­e­sis and the Evolu­tion of Cooperation

KristianRonnSep 4, 2024, 9:04 PM
50 points
6 comments11 min readLW link

Fac­to­rio, Ac­celerando, Em­pathiz­ing with Em­pires and Moder­ate Takeoffs

RaemonFeb 4, 2018, 2:33 AM
51 points
19 comments4 min readLW link

​​ Open-ended/​Phenom­e­nal ​Ethics ​(TLDR)

Ryo Nov 9, 2023, 4:58 PM
3 points
0 comments1 min readLW link

The Village and the River Mon­sters… Or: Less Fight­ing, More Brainstorming

ExCephOct 3, 2022, 11:01 PM
7 points
29 comments8 min readLW link
(ginnungagapfoundation.wordpress.com)

[Question] Is there an as­surance-con­tract web­site in work?

Yoav RavidFeb 20, 2019, 6:14 AM
18 points
31 comments1 min readLW link

Liv­ing in an Inad­e­quate World

Eliezer YudkowskyNov 9, 2017, 9:23 PM
93 points
47 comments20 min readLW link

Tal Yarkoni: No, it’s not The In­cen­tives—it’s you

Zack_M_DavisJun 11, 2019, 7:09 AM
89 points
119 comments1 min readLW link4 reviews
(www.talyarkoni.org)

Win­ning is for Losers

Jacob FalkovichOct 11, 2017, 4:01 AM
31 points
12 comments18 min readLW link
(putanumonit.com)

The Road to Mazedom

ZviJan 18, 2020, 2:10 PM
97 points
26 comments7 min readLW link2 reviews
(thezvi.wordpress.com)

Mazes Se­quence Roundup: Fi­nal Thoughts and Paths Forward

ZviFeb 6, 2020, 4:10 PM
88 points
28 comments14 min readLW link1 review
(thezvi.wordpress.com)

On the con­struc­tion of beacons

BenquoOct 16, 2017, 9:10 AM
21 points
9 comments10 min readLW link
(benjaminrosshoffman.com)

Timer Toxicities

ZviSep 22, 2019, 12:10 PM
39 points
4 comments15 min readLW link
(thezvi.wordpress.com)

Perfect Competition

ZviDec 29, 2019, 1:30 PM
54 points
29 comments7 min readLW link
(thezvi.wordpress.com)

Franken­stein De­lenda Est

eapacheJul 10, 2020, 2:38 PM
11 points
0 comments4 min readLW link

Coun­ter­fac­tu­als about So­cial Media

ZviApr 22, 2019, 12:20 PM
53 points
23 comments6 min readLW link
(thezvi.wordpress.com)

Re­cur­sive Mid­dle Man­ager Hell: AI Edition

VojtaKovarikMay 4, 2023, 8:08 PM
30 points
11 comments2 min readLW link

The Stable State is Broken

BakkotMar 12, 2012, 6:31 PM
80 points
43 comments3 min readLW link

Moloch and multi-level selection

thoughtfulmadisonAug 11, 2020, 7:08 AM
19 points
3 comments3 min readLW link

[Question] Is there an anal­y­sis of the com­mon con­sid­er­a­tion that split­ting an AI lab into two (e.g. the found­ing of An­thropic) speeds up the de­vel­op­ment of TAI and there­fore in­creases AI x-risk?

tchauvinMar 16, 2023, 2:16 PM
4 points
0 comments1 min readLW link

Sys­temic risk: a moral tale of ten in­surance companies

Stuart_ArmstrongNov 17, 2014, 4:43 PM
37 points
18 comments2 min readLW link

A Pro­posal for Defeat­ing Moloch in the Pri­son In­dus­trial Complex

lululuJun 2, 2015, 10:03 PM
33 points
77 comments4 min readLW link

Op­ti­mized for Some­thing other than Win­ning or: How Cricket Re­sists Moloch and Good­hart’s Law

A.H.Jul 5, 2023, 12:33 PM
53 points
26 comments4 min readLW link

Marginally Zero-Sum Efforts

Eliezer YudkowskyApr 11, 2007, 5:22 AM
32 points
14 comments2 min readLW link

A Brief Chat on World Government

eapacheSep 13, 2020, 6:33 PM
4 points
6 comments3 min readLW link

[Question] Are there non-AI pro­jects fo­cused on defeat­ing Moloch globally?

eapacheSep 14, 2020, 2:13 AM
13 points
19 comments1 min readLW link

The Align­ment Prob­lem No One Is Talk­ing About

James Stephen BrownMay 10, 2024, 6:34 PM
10 points
10 comments2 min readLW link
(nonzerosum.games)

The Ax­iolog­i­cal Treadmill

eapacheSep 15, 2020, 6:36 PM
6 points
13 comments2 min readLW link

Moloch—An Illus­trated Primer

James Stephen BrownMay 26, 2024, 1:04 AM
5 points
0 comments7 min readLW link
(nonzerosum.games)

The Apoca­lypse is Near. Can Hu­man­ity Coex­ist with Ar­tifi­cial Su­per­in­tel­li­gence?

Jakub GrowiecMar 31, 2025, 1:17 PM
4 points
0 comments11 min readLW link

The An­cient God Who Rules High School

lifelonglearnerApr 5, 2017, 6:55 PM
12 points
113 comments1 min readLW link
(medium.com)

[Question] Would more model evals teams be good?

Ryan KiddFeb 25, 2023, 10:01 PM
20 points
4 comments1 min readLW link

Con­trol­ling In­tel­li­gent Agents The Only Way We Know How: Ideal Bureau­cratic Struc­ture (IBS)

Justin BullockMay 24, 2021, 12:53 PM
14 points
15 comments6 min readLW link

God­shat­ter Ver­sus Leg­i­bil­ity: A Fun­da­men­tally Differ­ent Ap­proach To AI Alignment

LukeOnlineApr 9, 2022, 9:43 PM
15 points
14 comments7 min readLW link

Where Utopias Go Wrong, or: The Four Lit­tle Planets

ExCephMay 27, 2022, 1:24 AM
15 points
0 comments11 min readLW link
(ginnungagapfoundation.wordpress.com)

The Great Bootstrap

KristianRonnOct 11, 2024, 7:46 PM
12 points
0 comments15 min readLW link

How worker co-ops can help re­store so­cial trust

B JacobsApr 17, 2025, 2:14 PM
3 points
5 comments1 min readLW link
(bobjacobs.substack.com)

Open-ended ethics of phe­nom­ena (a desider­ata with uni­ver­sal moral­ity)

Ryo Nov 8, 2023, 8:10 PM
1 point
0 comments8 min readLW link

Repli­ca­tors, Gods and Bud­dhist Cosmology

KristianRonnJan 16, 2025, 10:51 AM
15 points
3 comments26 min readLW link

How would you gov­ern Mars?

BernhardDec 31, 2020, 2:32 AM
5 points
14 comments6 min readLW link

Non-Malthu­sian Scenarios

Wei DaiSep 26, 2009, 2:44 AM
24 points
89 comments1 min readLW link

The benev­olence of the butcher

dr_sApr 8, 2023, 4:29 PM
84 points
32 comments6 min readLW link1 review

My ex­pe­rience get­ting fund­ing for my biolog­i­cal research

MetacelsusApr 16, 2023, 10:53 PM
77 points
10 comments5 min readLW link
(denovo.substack.com)
No comments.