RSS

Moloch

TagLast edit: 18 Dec 2023 6:53 UTC by Yoav Ravid

Moloch is the personification of the forces that coerce competing individuals to take actions which, although locally optimal, ultimately lead to situations where everyone is worse off. Moreover, no individual is able to unilaterally break out of the dynamic. The situation is a bad Nash equilibrium. A trap.

It happens when “In some competition optimizing for X, the opportunity arises to throw some other value under the bus for improved X. Those who take it prosper. Those who don’t take it die out. Eventually, everyone’s relative status is about the same as before, but everyone’s absolute status is worse than before. The process continues until all other values that can be traded off have been – in other words, until human ingenuity cannot possibly figure out a way to make things any worse.”—Scott Alexander

One example of a Molochian dynamic is a Red Queen race between scientists who must continually spend more time writing grant applications just to keep up with their peers doing the same. Through unavoidable competition, they have all lost time while not ending up with any more grant money. And any scientist who unilaterally tried to not engage in the competition would soon be replaced by one who still does. If they all promised to cap their grant writing time, everyone would face an incentive to defect.

The topic of Moloch receives a formal treatment in the sequence Inadequate Equilibria, particularly in the chapter Moloch’s Toolbox.

Origin

Scott Alexander linked the name to the concept in his eponymous post, Meditations on Moloch. The post intersperses lines of Allan Ginsberg’s poem, Howl, with multiples examples of the dynamic including: the Prisoner’s Dilemma, dollar auctions, fish farming story, Malthusian trap, capitalism, two-income trap, agriculture, arms races, races to the bottom, education system, science, and government corruption and corporate welfare.

From Allan Ginsberg’s Howl:

What sphinx of cement and aluminum bashed open their skulls and ate up their brains and imagination?
Moloch! Solitude! Filth! Ugliness! Ashcans and unobtainable dollars! Children screaming under the stairways! Boys sobbing in armies! Old men weeping in the parks!
Moloch! Moloch! Nightmare of Moloch! Moloch the loveless! Mental Moloch! Moloch the heavy judger of men!
Moloch the incomprehensible prison! Moloch the crossbone soulless jailhouse and Congress of sorrows! Moloch whose buildings are judgment! Moloch the vast stone of war! Moloch the stunned governments!
Moloch whose mind is pure machinery! Moloch whose blood is running money! Moloch whose fingers are ten armies! Moloch whose breast is a cannibal dynamo! Moloch whose ear is a smoking tomb!
Moloch whose eyes are a thousand blind windows! Moloch whose skyscrapers stand in the long streets like endless Jehovahs! Moloch whose factories dream and croak in the fog! Moloch whose smoke-stacks and antennae crown the cities!
Moloch whose love is endless oil and stone! Moloch whose soul is electricity and banks! Moloch whose poverty is the specter of genius! Moloch whose fate is a cloud of sexless hydrogen! Moloch whose name is the Mind!

See also: Eldritch Analogies, Game Theory, Group Rationality, Social Reality

Med­i­ta­tions On Moloch

Scott Alexander30 Jul 2014 4:00 UTC
187 points
10 comments47 min readLW link

Moloch’s Toolbox (1/​2)

Eliezer Yudkowsky4 Nov 2017 21:46 UTC
154 points
70 comments33 min readLW link

Stud­ies On Slack

Scott Alexander13 May 2020 5:00 UTC
165 points
34 comments24 min readLW link1 review
(slatestarcodex.com)

Lost Purposes

Eliezer Yudkowsky25 Nov 2007 9:01 UTC
183 points
79 comments6 min readLW link

Moloch games

Chris van Merwijk16 Oct 2020 15:19 UTC
80 points
9 comments4 min readLW link

What Mul­tipo­lar Failure Looks Like, and Ro­bust Agent-Ag­nos­tic Pro­cesses (RAAPs)

Andrew_Critch31 Mar 2021 23:50 UTC
278 points
65 comments22 min readLW link1 review

Mazes Se­quence Summary

Zvi23 May 2020 16:40 UTC
81 points
2 comments7 min readLW link
(thezvi.wordpress.com)

Moloch’s Toolbox (2/​2)

Eliezer Yudkowsky7 Nov 2017 1:58 UTC
110 points
56 comments24 min readLW link

Moloch Hasn’t Won

Zvi28 Dec 2019 16:30 UTC
186 points
40 comments7 min readLW link1 review
(thezvi.wordpress.com)

The Choice Transition

18 Nov 2024 12:30 UTC
35 points
4 comments15 min readLW link
(strangecities.substack.com)

A Taijitu sym­bol for Moloch and Slack

B Jacobs25 May 2020 20:03 UTC
91 points
3 comments1 min readLW link

Failing to fix a dan­ger­ous intersection

alyssavance30 Jun 2022 18:09 UTC
110 points
17 comments2 min readLW link

[Book Re­view] Destiny Disrupted

lsusr21 Mar 2021 7:09 UTC
58 points
4 comments9 min readLW link

Gen­er­al­ized Effi­cient Mar­kets and Academia

johnswentworth30 Apr 2020 21:59 UTC
52 points
29 comments2 min readLW link

Dar­wi­nian Traps and Ex­is­ten­tial Risks

KristianRonn25 Aug 2024 22:37 UTC
76 points
14 comments10 min readLW link

Com­ment on SSC’s Re­view of Inad­e­quate Equilibria

Ben Pace1 Dec 2017 11:46 UTC
13 points
5 comments2 min readLW link

Moloch in whom I sit alone

KatjaGrace3 Oct 2018 23:40 UTC
55 points
16 comments1 min readLW link
(worldlypositions.tumblr.com)

Liv Bo­eree Ted Talk Moloch & AI

Neil 10 Nov 2023 14:04 UTC
10 points
2 comments1 min readLW link
(m.youtube.com)

5 gen­eral vot­ing patholo­gies: lesser names of Moloch

Jameson Quinn13 Apr 2018 18:38 UTC
73 points
16 comments10 min readLW link

Univer­sal Ba­sic In­come and Poverty

Eliezer Yudkowsky26 Jul 2024 7:23 UTC
281 points
131 comments9 min readLW link

Pri­son­ers’ Dilemma with Costs to Modeling

Scott Garrabrant5 Jun 2018 4:51 UTC
123 points
20 comments7 min readLW link

An Equil­ibrium of No Free Energy

Eliezer Yudkowsky31 Oct 2017 21:27 UTC
129 points
48 comments27 min readLW link

[Question] Look­ing for ideas of pub­lic as­sets (stocks, funds, ETFs) that I can in­vest in to have a chance at prof­it­ing from the mass adop­tion and com­mer­cial­iza­tion of AI technology

Annapurna7 Dec 2022 22:35 UTC
15 points
9 comments1 min readLW link

Moloch: op­ti­mi­sa­tion, “and” vs “or”, in­for­ma­tion, and sac­ri­fi­cial ems

Stuart_Armstrong6 Aug 2014 15:57 UTC
35 points
59 comments3 min readLW link

Re­cur­sive Mid­dle Man­ager Hell: AI Edition

VojtaKovarik4 May 2023 20:08 UTC
30 points
11 comments2 min readLW link

The Stable State is Broken

Bakkot12 Mar 2012 18:31 UTC
80 points
43 comments3 min readLW link

Moloch and multi-level selection

thoughtfulmadison11 Aug 2020 7:08 UTC
19 points
3 comments3 min readLW link

Sys­temic risk: a moral tale of ten in­surance companies

Stuart_Armstrong17 Nov 2014 16:43 UTC
37 points
18 comments2 min readLW link

A Pro­posal for Defeat­ing Moloch in the Pri­son In­dus­trial Complex

lululu2 Jun 2015 22:03 UTC
33 points
77 comments4 min readLW link

Op­ti­mized for Some­thing other than Win­ning or: How Cricket Re­sists Moloch and Good­hart’s Law

A.H.5 Jul 2023 12:33 UTC
53 points
26 comments4 min readLW link

Marginally Zero-Sum Efforts

Eliezer Yudkowsky11 Apr 2007 5:22 UTC
32 points
14 comments2 min readLW link

A Brief Chat on World Government

eapache13 Sep 2020 18:33 UTC
4 points
6 comments3 min readLW link

[Question] Are there non-AI pro­jects fo­cused on defeat­ing Moloch globally?

eapache14 Sep 2020 2:13 UTC
13 points
19 comments1 min readLW link

The Ax­iolog­i­cal Treadmill

eapache15 Sep 2020 18:36 UTC
6 points
13 comments2 min readLW link

The An­cient God Who Rules High School

lifelonglearner5 Apr 2017 18:55 UTC
12 points
113 comments1 min readLW link
(medium.com)

Con­trol­ling In­tel­li­gent Agents The Only Way We Know How: Ideal Bureau­cratic Struc­ture (IBS)

Justin Bullock24 May 2021 12:53 UTC
14 points
15 comments6 min readLW link

God­shat­ter Ver­sus Leg­i­bil­ity: A Fun­da­men­tally Differ­ent Ap­proach To AI Alignment

LukeOnline9 Apr 2022 21:43 UTC
15 points
14 comments7 min readLW link

Where Utopias Go Wrong, or: The Four Lit­tle Planets

ExCeph27 May 2022 1:24 UTC
15 points
0 comments11 min readLW link
(ginnungagapfoundation.wordpress.com)

Moloch and the sand­pile catastrophe

Eric Raymond2 Apr 2022 15:35 UTC
120 points
25 comments3 min readLW link

Fac­to­rio, Ac­celerando, Em­pathiz­ing with Em­pires and Moder­ate Takeoffs

Raemon4 Feb 2018 2:33 UTC
51 points
19 comments4 min readLW link

The Village and the River Mon­sters… Or: Less Fight­ing, More Brainstorming

ExCeph3 Oct 2022 23:01 UTC
7 points
29 comments8 min readLW link
(ginnungagapfoundation.wordpress.com)

[Question] Is there an as­surance-con­tract web­site in work?

Yoav Ravid20 Feb 2019 6:14 UTC
18 points
31 comments1 min readLW link

[Question] Would more model evals teams be good?

Ryan Kidd25 Feb 2023 22:01 UTC
20 points
4 comments1 min readLW link

[Question] Is there an anal­y­sis of the com­mon con­sid­er­a­tion that split­ting an AI lab into two (e.g. the found­ing of An­thropic) speeds up the de­vel­op­ment of TAI and there­fore in­creases AI x-risk?

tchauvin16 Mar 2023 14:16 UTC
4 points
0 comments1 min readLW link

The Thing and the Sym­bolic Rep­re­sen­ta­tion of The Thing

Zvi30 Jun 2015 0:15 UTC
16 points
0 comments5 min readLW link

Moloch—An Illus­trated Primer

James Stephen Brown26 May 2024 1:04 UTC
5 points
0 comments7 min readLW link
(nonzerosum.games)

The Align­ment Prob­lem No One Is Talk­ing About

James Stephen Brown10 May 2024 18:34 UTC
10 points
10 comments2 min readLW link
(nonzerosum.games)

The Frag­ility of Life Hy­poth­e­sis and the Evolu­tion of Cooperation

KristianRonn4 Sep 2024 21:04 UTC
49 points
6 comments11 min readLW link

Tack­ling Moloch: How YouCongress Offers a Novel Co­or­di­na­tion Mechanism

Hector Perez Arenas15 May 2024 23:13 UTC
28 points
9 comments6 min readLW link

The Case Against UBI

Zero Contradictions27 Jul 2024 6:36 UTC
−1 points
2 comments2 min readLW link
(thewaywardaxolotl.blogspot.com)

The Great Bootstrap

KristianRonn11 Oct 2024 19:46 UTC
7 points
0 comments15 min readLW link

Open-ended ethics of phe­nom­ena (a desider­ata with uni­ver­sal moral­ity)

Ryo 8 Nov 2023 20:10 UTC
1 point
0 comments8 min readLW link

How would you gov­ern Mars?

Bernhard31 Dec 2020 2:32 UTC
5 points
14 comments6 min readLW link

Non-Malthu­sian Scenarios

Wei Dai26 Sep 2009 2:44 UTC
24 points
89 comments1 min readLW link

The benev­olence of the butcher

dr_s8 Apr 2023 16:29 UTC
72 points
30 comments6 min readLW link

My ex­pe­rience get­ting fund­ing for my biolog­i­cal research

Metacelsus16 Apr 2023 22:53 UTC
75 points
10 comments5 min readLW link
(denovo.substack.com)

Keep hu­mans in the loop

19 Apr 2023 15:34 UTC
23 points
1 comment10 min readLW link

Mak­ing a Crow­dac­tion platform

B Jacobs16 May 2020 16:08 UTC
24 points
9 comments5 min readLW link

​​ Open-ended/​Phenom­e­nal ​Ethics ​(TLDR)

Ryo 9 Nov 2023 16:58 UTC
3 points
0 comments1 min readLW link

Liv­ing in an Inad­e­quate World

Eliezer Yudkowsky9 Nov 2017 21:23 UTC
93 points
47 comments20 min readLW link

Tal Yarkoni: No, it’s not The In­cen­tives—it’s you

Zack_M_Davis11 Jun 2019 7:09 UTC
89 points
119 comments1 min readLW link4 reviews
(www.talyarkoni.org)

Win­ning is for Losers

Jacob Falkovich11 Oct 2017 4:01 UTC
31 points
12 comments18 min readLW link
(putanumonit.com)

The Road to Mazedom

Zvi18 Jan 2020 14:10 UTC
97 points
26 comments7 min readLW link2 reviews
(thezvi.wordpress.com)

Mazes Se­quence Roundup: Fi­nal Thoughts and Paths Forward

Zvi6 Feb 2020 16:10 UTC
88 points
28 comments14 min readLW link1 review
(thezvi.wordpress.com)

On the con­struc­tion of beacons

Benquo16 Oct 2017 9:10 UTC
21 points
9 comments10 min readLW link
(benjaminrosshoffman.com)

Timer Toxicities

Zvi22 Sep 2019 12:10 UTC
39 points
4 comments15 min readLW link
(thezvi.wordpress.com)

Perfect Competition

Zvi29 Dec 2019 13:30 UTC
54 points
29 comments7 min readLW link
(thezvi.wordpress.com)

Franken­stein De­lenda Est

eapache10 Jul 2020 14:38 UTC
11 points
0 comments4 min readLW link

Coun­ter­fac­tu­als about So­cial Media

Zvi22 Apr 2019 12:20 UTC
53 points
23 comments6 min readLW link
(thezvi.wordpress.com)
No comments.