RSS

Co­or­di­na­tion /​ Cooperation

TagLast edit: Aug 5, 2020, 12:40 PM by Yoav Ravid

Coordination is the challenge of distinct actors being able to jointly choose their actions to achieve a favorable outcome. An example of easy coordination is coordinating to drive on the same side of the road. A hard example is coordinating everyone to move a superior yet different social media platform. Many failures of civilization are failures of coordination. A closely related concept is that of cooperation – multiple actors choosing their actions in ways that maximize collective value despite the temptation of greater short-term individual gain by acting to the detriment of the group/​other actors. The Prisoner’s Dilemma is the canonical cooperation/​defection game, but the term is used in other games too, e.g. Stag Hunt.

Coordination and cooperation are fundamental concepts in Game Theory. LessWrong discussion has long been interested in overcoming the gnarly coordination and cooperation challenges that prevent many improvements to optimizing the world. There is also interest because failures to coordinate/​cooperate are likely causes of existential risks such as AI Risk or nuclear war.

See also: Game Theory, Moloch

Another related term in this space is collective action problem.

Related Sequences: Kickstarter for Coordinated Action

Co­or­di­na­tion as a Scarce Resource

johnswentworthJan 25, 2020, 11:32 PM
261 points
22 comments4 min readLW link2 reviews

Con­scious­ness as a con­fla­tion­ary al­li­ance term for in­trin­si­cally val­ued in­ter­nal experiences

Andrew_CritchJul 10, 2023, 8:09 AM
212 points
54 comments11 min readLW link2 reviews

The Costly Co­or­di­na­tion Mechanism of Com­mon Knowledge

Ben PaceMar 15, 2018, 8:20 PM
201 points
31 comments20 min readLW link2 reviews

Unify­ing Bar­gain­ing No­tions (1/​2)

DiffractorJul 25, 2022, 12:28 AM
210 points
41 comments16 min readLW link

Two Co­or­di­na­tion Styles

abramdemskiFeb 7, 2018, 9:00 AM
40 points
14 comments7 min readLW link

Co­hab­itive Games so Far

mako yassSep 28, 2023, 3:41 PM
131 points
143 comments19 min readLW link2 reviews
(makopool.com)

“Win First” vs “Chill First”

lionhearted (Sebastian Marshall)Sep 28, 2020, 6:48 AM
101 points
20 comments3 min readLW link

Most Pri­soner’s Dilem­mas are Stag Hunts; Most Stag Hunts are Schel­ling Problems

abramdemskiSep 14, 2020, 10:13 PM
176 points
36 comments10 min readLW link3 reviews

Iter­ated Trust Kickstarters

RaemonApr 20, 2021, 3:18 AM
74 points
19 comments10 min readLW link

What Mul­tipo­lar Failure Looks Like, and Ro­bust Agent-Ag­nos­tic Pro­cesses (RAAPs)

Andrew_CritchMar 31, 2021, 11:50 PM
282 points
65 comments22 min readLW link1 review

Three rea­sons to cooperate

paulfchristianoDec 24, 2022, 5:40 PM
86 points
14 comments10 min readLW link
(sideways-view.com)

Lev­er­age points for a pause

RemmeltAug 28, 2024, 9:21 AM
3 points
0 comments1 min readLW link

The Co­or­di­na­tion Fron­tier: Se­quence Intro

RaemonSep 4, 2021, 10:11 PM
163 points
22 comments6 min readLW link1 review

Parable of the Dammed

johnswentworthDec 10, 2020, 12:08 AM
118 points
29 comments2 min readLW link

Jean Mon­net: The Guerilla Bureaucrat

Martin SustrikMar 20, 2021, 10:37 AM
177 points
25 comments18 min readLW link1 review

Your Cheer­ful Price

Eliezer YudkowskyFeb 13, 2021, 5:41 AM
272 points
83 comments17 min readLW link6 reviews

Con­cen­tra­tion of Force

Duncan Sabien (Deactivated)Nov 6, 2021, 8:20 AM
244 points
23 comments12 min readLW link1 review

Co­or­di­na­tion Schemes Are Cap­i­tal Investments

RaemonSep 6, 2021, 11:27 PM
140 points
31 comments7 min readLW link1 review

On De­stroy­ing the World

Chris_LeongSep 28, 2020, 7:38 AM
82 points
86 comments5 min readLW link

Co­or­di­na­tion Sur­veys: why we should sur­vey to or­ga­nize re­spon­si­bil­ities, not just predictions

AcademianMay 7, 2019, 5:43 PM
68 points
12 comments3 min readLW link3 reviews

My Mar­riage Vows

Vanessa KosoyJul 21, 2021, 10:48 AM
84 points
53 comments3 min readLW link

AI co­or­di­na­tion needs clear wins

evhubSep 1, 2022, 11:41 PM
147 points
16 comments2 min readLW link1 review

In­sights from ‘The Strat­egy of Con­flict’

DanielFilanJan 4, 2018, 5:05 AM
46 points
13 comments7 min readLW link

A Key Power of the Pres­i­dent is to Co­or­di­nate the Ex­e­cu­tion of Ex­ist­ing Con­crete Plans

Ben PaceJul 16, 2019, 5:06 AM
152 points
13 comments10 min readLW link

Co­or­di­na­tion Skills I Wish I Had For the Pandemic

RaemonNov 13, 2021, 11:32 PM
96 points
9 comments6 min readLW link1 review

Norm In­no­va­tion and The­ory of Mind

RaemonSep 18, 2021, 9:38 PM
92 points
15 comments7 min readLW link1 review

Some con­struc­tions for proof-based co­op­er­a­tion with­out Löb

James PayorMar 21, 2023, 4:12 PM
43 points
3 comments4 min readLW link

Why Our Kind Can’t Cooperate

Eliezer YudkowskyMar 20, 2009, 8:37 AM
295 points
211 comments9 min readLW link

Failures in Kindness

silentbobMar 26, 2024, 9:30 PM
415 points
60 comments9 min readLW link

Gaia Net­work: a prac­ti­cal, in­cre­men­tal path­way to Open Agency Architecture

Dec 20, 2023, 5:11 PM
22 points
8 comments16 min readLW link

Democ­racy Is in Danger, but Not for the Rea­sons You Think

ExCephNov 6, 2022, 9:15 PM
−7 points
4 comments12 min readLW link
(ginnungagapfoundation.wordpress.com)

Co­op­er­at­ing with agents with differ­ent ideas of fair­ness, while re­sist­ing exploitation

Eliezer YudkowskySep 16, 2013, 8:27 AM
103 points
42 comments4 min readLW link

A tax­on­omy of co­or­di­na­tion problems

damiensnyderAug 22, 2021, 11:10 PM
30 points
1 comment6 min readLW link

Si­mu­lacrum 3 As Stag-Hunt Strategy

johnswentworthJan 26, 2021, 7:40 PM
183 points
37 comments4 min readLW link3 reviews

Your Price for Joining

Eliezer YudkowskyMar 26, 2009, 7:16 AM
89 points
58 comments4 min readLW link

Refine­ment of Ac­tive In­fer­ence agency ontology

Roman LeventovDec 15, 2023, 9:31 AM
16 points
0 comments5 min readLW link
(arxiv.org)

[Question] What does it look like for AI to sig­nifi­cantly im­prove hu­man co­or­di­na­tion, be­fore su­per­in­tel­li­gence?

Bird ConceptJan 15, 2024, 7:22 PM
22 points
2 comments1 min readLW link

Re­view: Conor More­ton’s “Civ­i­liza­tion & Co­op­er­a­tion”

Duncan Sabien (Deactivated)May 26, 2024, 7:32 PM
88 points
8 comments38 min readLW link

In­sti­tu­tional eco­nomics through the lens of scale-free reg­u­la­tive de­vel­op­ment, mor­pho­gen­e­sis, and cog­ni­tive science

Roman LeventovJan 23, 2024, 7:42 PM
8 points
0 comments14 min readLW link

If we solve al­ign­ment, do we die any­way?

Seth HerdAug 23, 2024, 1:13 PM
84 points
129 comments4 min readLW link

Are most per­son­al­ity di­s­or­ders re­ally trust di­s­or­ders?

chaosmageFeb 6, 2024, 12:37 PM
20 points
4 comments1 min readLW link

How it All Went Down: The Puz­zle Hunt that took us way, way Less Online

A*Jun 2, 2024, 8:01 AM
135 points
5 comments5 min readLW link

WSJ: In­side Ama­zon’s Se­cret Oper­a­tion to Gather In­tel on Rivals

trevorApr 23, 2024, 9:33 PM
37 points
5 comments5 min readLW link
(www.wsj.com)

Ag­grega­tive Prin­ci­ples of So­cial Justice

Cleo NardoJun 5, 2024, 1:44 PM
29 points
10 comments37 min readLW link

Se­cond-Order Ra­tion­al­ity, Sys­tem Ra­tion­al­ity, and a fea­ture sug­ges­tion for LessWrong

Mati_RoyJun 5, 2024, 7:20 AM
13 points
2 comments8 min readLW link

Cap­i­tal­is­ing On Trust—A Simulation

James Stephen BrownJun 21, 2024, 4:43 AM
2 points
0 comments1 min readLW link
(nonzerosum.games)

AI de­vel­op­ment is an act of so­cial revolution

artemiocobbJul 3, 2024, 6:00 PM
3 points
0 comments3 min readLW link

Travel Buffer

jefftkJul 6, 2024, 2:20 AM
17 points
3 comments1 min readLW link
(www.jefftk.com)

Re­sponse to Dileep Ge­orge: AGI safety war­rants plan­ning ahead

Steven ByrnesJul 8, 2024, 3:27 PM
27 points
7 comments27 min readLW link

Trust as a bot­tle­neck to grow­ing teams quickly

benkuhnJul 13, 2024, 6:00 PM
44 points
3 comments5 min readLW link
(www.benkuhn.net)

AI Con­sti­tu­tions are a tool to re­duce so­cietal scale risk

Sammy MartinJul 25, 2024, 11:18 AM
30 points
2 comments18 min readLW link

A Nar­row Path: a plan to deal with AI ex­tinc­tion risk

Oct 7, 2024, 1:02 PM
73 points
12 comments2 min readLW link
(www.narrowpath.co)

[Question] A Co­or­di­na­tion Cook­book?

azerganteNov 10, 2024, 11:20 PM
2 points
0 comments1 min readLW link

Preface

Allison DuettmannJan 2, 2025, 6:59 PM
26 points
2 comments7 min readLW link

2. Skim the Man­ual: In­tel­li­gent Vol­un­tary Cooperation

Allison DuettmannJan 2, 2025, 7:02 PM
13 points
3 comments18 min readLW link

3. Im­prove Co­op­er­a­tion: Bet­ter Technologies

Allison DuettmannJan 2, 2025, 7:03 PM
4 points
2 comments23 min readLW link

Six Small Co­hab­itive Games

ScrewtapeJan 15, 2025, 9:59 PM
38 points
7 comments13 min readLW link

Haber­mas Machine

NicholasKeesMar 13, 2025, 6:16 PM
45 points
7 comments6 min readLW link
(mosaic-labs.org)

On Responsibility

silentbobJan 21, 2025, 10:47 AM
9 points
2 comments6 min readLW link

Chaos In­vest­ments v0.31

ScrewtapeFeb 8, 2025, 6:53 AM
15 points
1 comment9 min readLW link

FLI And Eliezer Should Reach Consensus

JenniferRMApr 11, 2023, 4:07 AM
21 points
6 comments23 min readLW link

My idea of sa­cred­ness, di­v­inity, and religion

Kaj_SotalaOct 29, 2023, 12:50 PM
40 points
10 comments4 min readLW link
(kajsotala.fi)

Urg­ing an In­ter­na­tional AI Treaty: An Open Letter

Olli JärviniemiOct 31, 2023, 11:26 AM
48 points
2 comments1 min readLW link
(aitreaty.org)

[Linkpost] Ge­orge Mack’s Razors

trevorNov 27, 2023, 5:53 PM
38 points
8 comments3 min readLW link
(twitter.com)

Pro­posal for im­prov­ing the global on­line dis­course through per­son­al­ised com­ment or­der­ing on all websites

Roman LeventovDec 6, 2023, 6:51 PM
35 points
21 comments6 min readLW link

[Question] Does it be­come eas­ier, or harder, for the world to co­or­di­nate around not build­ing AGI as time goes on?

Eli TyreJul 29, 2019, 10:59 PM
86 points
31 comments3 min readLW link2 reviews

Tac­ti­cal vs. Strate­gic Cooperation

sarahconstantinAug 12, 2018, 4:41 PM
74 points
18 comments5 min readLW link

Ro­bust Co­op­er­a­tion in the Pri­soner’s Dilemma

orthonormalJun 7, 2013, 8:30 AM
124 points
147 comments7 min readLW link

Black­mail, Nukes and the Pri­soner’s Dilemma

Stuart_ArmstrongMar 10, 2010, 2:58 PM
25 points
20 comments2 min readLW link

What counts as defec­tion?

TurnTroutJul 12, 2020, 10:03 PM
81 points
21 comments5 min readLW link1 review

Sec­tions 1 & 2: In­tro­duc­tion, Strat­egy and Governance

JesseCliftonDec 17, 2019, 9:27 PM
35 points
8 comments14 min readLW link

Sec­tions 3 & 4: Cred­i­bil­ity, Peace­ful Bar­gain­ing Mechanisms

JesseCliftonDec 17, 2019, 9:46 PM
20 points
2 comments12 min readLW link

“You can’t pos­si­bly suc­ceed with­out [My Pet Is­sue]”

RaemonDec 19, 2019, 1:12 AM
54 points
14 comments1 min readLW link

Shap­ing eco­nomic in­cen­tives for col­lab­o­ra­tive AGI

Kaj_SotalaJun 29, 2018, 4:26 PM
45 points
15 comments4 min readLW link

Mul­ti­verse-wide Co­op­er­a­tion via Cor­re­lated De­ci­sion Making

Kaj_SotalaAug 20, 2017, 12:01 PM
5 points
2 comments1 min readLW link
(foundational-research.org)

From Per­sonal to Pri­son Gangs: En­forc­ing Proso­cial Behavior

johnswentworthJan 24, 2019, 6:07 PM
153 points
26 comments5 min readLW link2 reviews

Win­ning is for Losers

Jacob FalkovichOct 11, 2017, 4:01 AM
31 points
12 comments18 min readLW link
(putanumonit.com)

Bayesi­ans vs. Barbarians

Eliezer YudkowskyApr 14, 2009, 11:45 PM
105 points
277 comments8 min readLW link

The Im­por­tance of Sidekicks

Swimmer963 (Miranda Dixon-Luinenburg) Jan 8, 2015, 11:21 PM
245 points
209 comments5 min readLW link

Team Co­he­sion and Ex­clu­sion­ary Egalitarianism

lionhearted (Sebastian Marshall)Sep 17, 2018, 4:48 AM
31 points
6 comments7 min readLW link

Healthy Competition

RaemonOct 20, 2019, 8:55 PM
56 points
13 comments2 min readLW link1 review

Reneg­ing proso­cially by Dun­can Sabien

philip_bJun 18, 2019, 6:52 PM
70 points
5 comments1 min readLW link
(medium.com)

Bab­ble on grow­ing trust

qbolecMay 21, 2023, 1:19 PM
13 points
1 comment5 min readLW link

[Question] How could “Kick­starter for Inad­e­quate Equil­ibria” be used for evil or turn out to be net-nega­tive?

RaemonFeb 21, 2019, 9:36 PM
25 points
17 comments1 min readLW link

Schel­ling Reach

ZizJan 7, 2018, 6:18 PM
10 points
3 comments1 min readLW link
(sinceriously.fyi)

Real-World Co­or­di­na­tion Prob­lems are Usu­ally In­for­ma­tion Problems

johnswentworthJun 13, 2019, 6:21 PM
35 points
2 comments4 min readLW link

Vanilla and choco­late and prefer­ence judgements

Swimmer963 (Miranda Dixon-Luinenburg) Apr 18, 2011, 10:14 PM
39 points
32 comments6 min readLW link

The Align­ment-Com­pe­tence Trade-Off, Part 1: Coal­i­tion Size and Sig­nal­ing Costs

GentzelJan 15, 2020, 11:10 PM
30 points
4 comments3 min readLW link
(theconsequentialist.wordpress.com)

Cor­rectly Cal­ibrated Trust

habrykaJun 24, 2023, 7:48 PM
36 points
3 comments11 min readLW link
(forum.effectivealtruism.org)

An at­tempt at a “good enough” solu­tion for hu­man two-party negotiations

Isaac KingSep 16, 2023, 12:38 AM
11 points
22 comments1 min readLW link

Col­lAc­tion his­tory and les­sons learned

RonJan 17, 2021, 3:01 PM
35 points
6 comments8 min readLW link

Google’s Eth­i­cal AI team and AI Safety

magfrumpFeb 20, 2021, 9:42 AM
12 points
16 comments7 min readLW link

Partnership

RubyMar 11, 2021, 5:17 PM
17 points
0 comments3 min readLW link

An Ex­er­cise in Ra­tional Co­op­er­a­tion and Com­mu­ni­ca­tion: Let’s Play Hanabi

ejacobApr 14, 2021, 8:36 AM
14 points
10 comments7 min readLW link

April 15, 2040

NisanMay 4, 2021, 9:18 PM
97 points
25 comments2 min readLW link

[Book Re­view] Blueprint for Revolution

lsusrJun 7, 2021, 6:14 PM
28 points
8 comments2 min readLW link

A Bet­ter Web is Coming

mattoAug 21, 2021, 8:46 PM
34 points
20 comments6 min readLW link

Notes on Courtesy

David GrossDec 16, 2020, 4:08 AM
11 points
0 comments7 min readLW link

AISC5 Ret­ro­spec­tive: Mechanisms for Avoid­ing Tragedy of the Com­mons in Com­mon Pool Re­source Problems

Sep 27, 2021, 4:46 PM
8 points
3 comments7 min readLW link

The role of tribes in achiev­ing last­ing im­pact and how to cre­ate them

Sep 29, 2021, 8:48 PM
14 points
3 comments24 min readLW link

Co­or­di­na­tion Mo­ti­va­tion: The Pandemic

RaemonOct 21, 2021, 9:57 PM
64 points
1 comment3 min readLW link

Post­mod­ern Warfare

lsusrOct 25, 2021, 9:02 AM
64 points
26 comments2 min readLW link

First Strike and Se­cond Strike

lsusrNov 25, 2021, 9:23 AM
28 points
5 comments1 min readLW link

Co­or­di­nat­ing the Unequal Treaties

lsusrNov 25, 2021, 10:47 AM
34 points
4 comments2 min readLW link

Pareto im­prove­ments are rarer than they seem

philhJan 27, 2018, 10:23 PM
27 points
13 comments3 min readLW link
(reasonableapproximation.net)

My Overview of the AI Align­ment Land­scape: A Bird’s Eye View

Neel NandaDec 15, 2021, 11:44 PM
127 points
9 comments15 min readLW link

My Overview of the AI Align­ment Land­scape: Threat Models

Neel NandaDec 25, 2021, 11:07 PM
53 points
3 comments28 min readLW link

Create a Full Alter­na­tive Stack

ZviJan 31, 2020, 5:10 PM
84 points
14 comments6 min readLW link1 review
(thezvi.wordpress.com)

Nar­ra­tive Syncing

AnnaSalamonMay 1, 2022, 1:48 AM
118 points
48 comments7 min readLW link1 review

An­nounc­ing the Align­ment of Com­plex Sys­tems Re­search Group

Jun 4, 2022, 4:10 AM
91 points
20 comments5 min readLW link

Failing to fix a dan­ger­ous intersection

alyssavanceJun 30, 2022, 6:09 PM
110 points
17 comments2 min readLW link

Digi­tal Din­ner Signup

jefftkAug 23, 2022, 2:20 AM
13 points
5 comments1 min readLW link
(www.jefftk.com)

The ethics of re­clin­ing air­plane seats

bracesSep 4, 2022, 5:59 PM
93 points
70 comments1 min readLW link

Ve­hi­cle Pla­toon­ing—a real world ex­am­i­na­tion of the difficul­ties in coordination

M. Y. ZuoOct 13, 2022, 7:33 PM
24 points
6 comments2 min readLW link

Why Aren’t There More Schel­ling Holi­days?

johnswentworthOct 31, 2022, 7:31 PM
63 points
21 comments1 min readLW link

The biolog­i­cal func­tion of love for non-kin is to gain the trust of peo­ple we can­not deceive

chaosmageNov 7, 2022, 8:26 PM
43 points
3 comments8 min readLW link

In­ter­nal­iz­ing the dam­age of bad-act­ing part­ners cre­ates in­cen­tives for due diligence

tailcalledNov 11, 2022, 8:57 PM
17 points
7 comments1 min readLW link

“Ru­de­ness”, a use­ful co­or­di­na­tion mechanic

RaemonNov 11, 2022, 10:27 PM
49 points
20 comments2 min readLW link

Löbian emo­tional pro­cess­ing of emer­gent co­op­er­a­tion: an example

Andrew_CritchJan 17, 2023, 5:59 AM
23 points
0 comments8 min readLW link

Distributed Cooperation

DiffractorMar 18, 2018, 5:46 AM
13 points
2 comments9 min readLW link

[Question] Why should we ex­pect AIs to co­or­di­nate well?

Jonathan PaulsonFeb 14, 2023, 3:50 PM
25 points
9 comments1 min readLW link

Selec­tive, Cor­rec­tive, Struc­tural: Three Ways of Mak­ing So­cial Sys­tems Work

Said AchmizMar 5, 2023, 8:45 AM
99 points
13 comments2 min readLW link

Notes on Loyalty

David GrossNov 15, 2020, 7:30 PM
17 points
2 comments8 min readLW link

Notes on So­cial Responsibility

David GrossMar 19, 2022, 2:44 PM
17 points
4 comments19 min readLW link

Re­lease: Op­ti­mal Weave (P1): A Pro­to­type Co­hab­itive Game

mako yassAug 17, 2024, 2:08 PM
82 points
21 comments7 min readLW link

[Question] If a “Kick­starter for Inad­e­quate Equlibria” was built, do you have a con­crete in­ad­e­quate equil­ibrium to fix?

RaemonFeb 21, 2019, 9:32 PM
56 points
40 comments1 min readLW link

Machines vs Memes Part 3: Imi­ta­tion and Memes

ceru23Jun 1, 2022, 1:36 PM
7 points
0 comments7 min readLW link

Shared re­al­ity: a key driver of hu­man behavior

kdbscottDec 24, 2022, 7:35 PM
126 points
25 comments4 min readLW link

Co­or­di­na­tion by com­mon knowl­edge to pre­vent un­con­trol­lable AI

Karl von WendtMay 14, 2023, 1:37 PM
10 points
2 comments9 min readLW link

So­phie Grouchy on A Build-Break Model of Cooperation

weftMay 25, 2018, 2:55 PM
25 points
6 comments8 min readLW link
(sophiegrouchy.wordpress.com)

Prop­erty as Co­or­di­na­tion Minimization

VaniverAug 4, 2020, 7:24 PM
38 points
35 comments4 min readLW link

Let’s split the cake, length­wise, up­wise and slantwise

Stuart_ArmstrongOct 25, 2010, 1:15 PM
74 points
31 comments4 min readLW link

New co­op­er­a­tion mechanism—quadratic fund­ing with­out a match­ing pool

Filip SondejJun 5, 2022, 1:55 PM
11 points
0 comments5 min readLW link

[Question] Where are peo­ple think­ing and talk­ing about global co­or­di­na­tion for AI safety?

Wei DaiMay 22, 2019, 6:24 AM
112 points
22 comments1 min readLW link

How hard is it for al­tru­ists to dis­cuss go­ing against bad equil­ibria?

abramdemskiJun 22, 2019, 3:42 AM
46 points
6 comments11 min readLW link

AI co­op­er­a­tion is already stud­ied in academia as “pro­gram equil­ibrium”

cousin_itJul 30, 2012, 3:22 PM
60 points
40 comments1 min readLW link

Tomb­stone Men­tal­ity

SuperDaveWhiteAug 13, 2020, 1:46 PM
6 points
0 comments2 min readLW link

Self-Similar­ity Experiment

Dawn DrescherAug 15, 2020, 1:19 PM
12 points
0 comments10 min readLW link

Alle­gory On AI Risk, Game The­ory, and Mithril

James_MillerFeb 13, 2017, 8:41 PM
45 points
57 comments3 min readLW link

Seek­ing (Paid) Case Stud­ies on Standards

HoldenKarnofskyMay 26, 2023, 5:58 PM
69 points
9 comments11 min readLW link

[AN #150]: The sub­types of Co­op­er­a­tive AI research

Rohin ShahMay 12, 2021, 5:20 PM
17 points
0 comments6 min readLW link
(mailchi.mp)

CLR’s re­cent work on multi-agent systems

JesseCliftonMar 9, 2021, 2:28 AM
54 points
2 comments13 min readLW link

Worked Ex­am­ples of Shap­ley Values

lalaithionJun 24, 2022, 5:13 PM
75 points
11 comments8 min readLW link

One Web­site To Rule Them All?

anna_macdonaldJan 11, 2019, 7:14 PM
30 points
23 comments10 min readLW link

Col­lec­tive Apa­thy and the Internet

Eliezer YudkowskyApr 14, 2009, 12:02 AM
53 points
34 comments2 min readLW link

Break­ing Or­a­cles: su­per­ra­tional­ity and acausal trade

Stuart_ArmstrongNov 25, 2019, 10:40 AM
25 points
15 comments1 min readLW link

Quick Sum­maries of Two Papers on Kant and Game Theory

Erich_GrunewaldJun 25, 2022, 10:25 AM
8 points
2 comments4 min readLW link
(www.erichgrunewald.com)

Abadar­ian Trades

David UdellJun 30, 2022, 4:41 PM
17 points
22 comments2 min readLW link

Moloch—An Illus­trated Primer

James Stephen BrownMay 26, 2024, 1:04 AM
5 points
0 comments7 min readLW link
(nonzerosum.games)

[Question] De­gree of du­pli­ca­tion and co­or­di­na­tion in pro­jects that ex­am­ine com­put­ing prices, AI progress, and re­lated top­ics?

riceissaApr 23, 2019, 12:27 PM
26 points
1 comment2 min readLW link

Su­per­in­tel­li­gence 28: Collaboration

KatjaGraceMar 24, 2015, 1:29 AM
13 points
21 comments6 min readLW link

Fun­da­men­tally Fuzzy Con­cepts Can’t Have Crisp Defi­ni­tions: Co­op­er­a­tion and Align­ment vs Math and Physics

VojtaKovarikJul 21, 2023, 9:03 PM
12 points
18 comments3 min readLW link

The Dilemma’s Dilemma

James Stephen BrownFeb 19, 2025, 11:50 PM
7 points
11 comments7 min readLW link
(nonzerosum.games)

Key take­aways from our EA and al­ign­ment re­search sur­veys

May 3, 2024, 6:10 PM
111 points
10 comments21 min readLW link

Acausal Now: We could to­tally acausally bar­gain with aliens at our cur­rent tech level if desired

Christopher KingAug 9, 2023, 12:50 AM
1 point
5 comments4 min readLW link

Knowl­edge Base 6: Con­sen­sus the­ory of truth

iwisNov 3, 2023, 1:56 PM
−8 points
0 comments1 min readLW link

Knowl­edge Base 2: The struc­ture and the method of building

iwisOct 9, 2023, 11:53 AM
2 points
4 comments7 min readLW link

Knowl­edge Base 4: Gen­eral applications

iwisOct 16, 2023, 12:26 PM
3 points
0 comments1 min readLW link

Knowl­edge Base 3: Shop­ping ad­vi­sor and other uses of knowl­edge base about products

iwisOct 9, 2023, 11:53 AM
0 points
0 comments4 min readLW link

Knowl­edge Base 8: The truth as an at­trac­tor in the in­for­ma­tion space

iwisApr 25, 2024, 3:28 PM
−8 points
0 comments2 min readLW link

[Question] Where do (did?) sta­ble, co­op­er­a­tive in­sti­tu­tions come from?

AnnaSalamonNov 3, 2020, 10:14 PM
151 points
72 comments4 min readLW link

Co­op­er­a­tion is for Winners

Jacob FalkovichFeb 15, 2019, 2:58 PM
21 points
6 comments4 min readLW link

Co­op­er­a­tion with and be­tween AGI\’s

PeterMcCluskeyJul 7, 2022, 4:45 PM
10 points
3 comments10 min readLW link
(www.bayesianinvestor.com)

Can Kauff­man’s NK Boolean net­works make hu­mans swarm?

Yori-92May 8, 2024, 2:09 AM
6 points
2 comments20 min readLW link

Spaghetti Towers

eukaryoteDec 22, 2018, 5:29 AM
212 points
38 comments3 min readLW link1 review
(eukaryotewritesblog.com)

Pas­cal’s Mug­ging: The Word Wars

johncroxOct 16, 2023, 5:54 PM
9 points
1 comment6 min readLW link

[Question] How do you con­cisely com­mu­ni­cate & nav­i­gate the poli­tics /​ cul­ture at your job work­ing at a large cor­po­ra­tion or in­sti­tu­tion?

WillaJul 12, 2022, 3:22 AM
10 points
6 comments1 min readLW link

Notes on Schel­ling’s “Strat­egy of Con­flict” (1960)

MichaelAFeb 10, 2021, 2:48 AM
25 points
8 comments8 min readLW link

Prac­ti­cal post on 4 #ProRep meth­ods (Pro­por­tional Rep­re­sen­ta­tion)

Jameson QuinnJan 9, 2021, 2:56 PM
37 points
12 comments11 min readLW link

[Question] What are some al­ter­na­tives to Shap­ley val­ues which drop ad­di­tivity?

eapiAug 9, 2022, 9:16 AM
11 points
6 comments1 min readLW link
(math.stackexchange.com)

The Pro­to­typ­i­cal Ne­go­ti­a­tion Game

johnswentworthFeb 20, 2021, 9:33 PM
99 points
16 comments5 min readLW link1 review

Let’s think about slow­ing down AI

KatjaGraceDec 22, 2022, 5:40 PM
551 points
182 comments38 min readLW link3 reviews
(aiimpacts.org)

Co­op­er­a­tion is op­ti­mal, with weaker agents too  -  tldr

Ryo Apr 18, 2024, 3:03 PM
12 points
22 comments4 min readLW link
(medium.com)

I’m from a par­allel Earth with much higher co­or­di­na­tion: AMA

Apr 5, 2021, 10:09 PM
173 points
35 comments61 min readLW link1 review

The Ex­pand­ing Mo­ral Cine­matic Universe

RaemonAug 28, 2022, 6:42 PM
64 points
9 comments14 min readLW link

How to co­or­di­nate de­spite our bi­ases? - tldr

Ryo Apr 18, 2024, 3:03 PM
3 points
2 comments3 min readLW link
(medium.com)

Un­lock­ing Solu­tions—By Un­der­stand­ing Co­or­di­na­tion Problems

James Stephen BrownJul 27, 2024, 4:52 AM
56 points
4 comments5 min readLW link
(nonzerosum.games)

Knowl­edge Base 1: Could it in­crease in­tel­li­gence and make it safer?

iwisSep 30, 2024, 4:00 PM
−4 points
0 comments4 min readLW link

Should we ab­stain from vot­ing? (In non­de­ter­minis­tic elec­tions)

B JacobsOct 2, 2024, 10:07 AM
5 points
6 comments4 min readLW link
(bobjacobs.substack.com)

Re­view and Com­par­i­son: Onto-Car­tog­ra­phy & Promise Theory

JosephineMay 9, 2021, 12:42 AM
13 points
0 comments5 min readLW link

Un­der­stand­ing Shap­ley Values with Venn Diagrams

Carson LDec 6, 2024, 9:56 PM
213 points
35 comments1 min readLW link
(medium.com)

The Stag Hunt—cul­ti­vat­ing co­op­er­a­tion to reap rewards

James Stephen BrownFeb 25, 2025, 11:45 PM
7 points
0 comments4 min readLW link
(nonzerosum.games)

The non-tribal tribes

PatrickDFarleyFeb 26, 2025, 5:22 PM
24 points
4 comments16 min readLW link

On Shifgrethor

JustisMillsOct 27, 2024, 3:30 PM
66 points
18 comments2 min readLW link
(justismills.substack.com)

LDT (and ev­ery­thing else) can be irrational

Christopher KingNov 6, 2024, 4:05 AM
10 points
13 comments2 min readLW link

In the Name of All That Needs Saving

pleiotrothNov 7, 2024, 3:26 PM
18 points
2 comments22 min readLW link

Con­trol­ling In­tel­li­gent Agents The Only Way We Know How: Ideal Bureau­cratic Struc­ture (IBS)

Justin BullockMay 24, 2021, 12:53 PM
14 points
15 comments6 min readLW link

So­cial Dilem­mas — pub­lic goods, free rid­ers, and exploitation

James Stephen BrownMar 5, 2025, 11:31 PM
6 points
0 comments3 min readLW link
(nonzerosum.games)

Knowl­edge Base 7: Long-tail knowl­edge and col­lec­tive intelligence

iwisApr 18, 2024, 2:21 PM
−6 points
0 comments1 min readLW link

For­mal­iz­ing Ob­jec­tions against Sur­ro­gate Goals

VojtaKovarikSep 2, 2021, 4:24 PM
16 points
23 comments1 min readLW link

Ask­ing For Help

Neel NandaDec 27, 2020, 11:32 AM
23 points
5 comments12 min readLW link
(neelnanda.io)

Ex­pevolu, Part II: Buy­ing land to cre­ate countries

FernandoJan 9, 2025, 9:11 PM
4 points
0 comments20 min readLW link
(expevolu.substack.com)

[Question] How does the ever-in­creas­ing use of AI in the mil­i­tary for the di­rect pur­pose of mur­der­ing peo­ple af­fect your p(doom)?

JustausernameApr 6, 2024, 6:31 AM
19 points
16 comments1 min readLW link

Cy­berE­con­omy. The Limits to Growth

Feb 16, 2025, 9:02 PM
−3 points
0 comments23 min readLW link

Com­par­ing Align­ment to other AGI in­ter­ven­tions: Ba­sic model

Martín SotoMar 20, 2024, 6:17 PM
12 points
4 comments7 min readLW link

Scal­ing Wargam­ing for Global Catas­trophic Risks with AI

Jan 18, 2025, 3:10 PM
38 points
2 comments4 min readLW link
(blog.sentinel-team.org)

be the per­son that makes the meet­ing productive

OldmanrahulJan 18, 2025, 10:32 PM
9 points
0 comments1 min readLW link

Mon­i­tor­ing for de­cep­tive alignment

evhubSep 8, 2022, 11:07 PM
135 points
8 comments9 min readLW link

Tether­ware #1: The case for hu­man­like AI with free will

Jáchym FibírJan 30, 2025, 10:58 AM
5 points
11 comments10 min readLW link
(tetherware.substack.com)

Ad­ver­sar­ial vs Col­lab­o­ra­tive Contexts

jefftkOct 4, 2022, 2:40 AM
31 points
4 comments2 min readLW link
(www.jefftk.com)

[Question] how do the CEOs re­spond to our con­cerns?

KvmanThinkingFeb 11, 2025, 11:39 PM
−12 points
7 comments1 min readLW link

Ex­plor­ing the Pre­cau­tion­ary Prin­ci­ple in AI Devel­op­ment: His­tor­i­cal Analo­gies and Les­sons Learned

Christopher KingMar 21, 2023, 3:53 AM
−1 points
2 comments9 min readLW link

2024 Free­dom Com­mu­ni­ties Events

Tudor IliescuJul 12, 2024, 8:04 AM
−6 points
1 comment1 min readLW link

In­stru­men­tal con­ver­gence in sin­gle-agent systems

Oct 12, 2022, 12:24 PM
33 points
4 comments8 min readLW link
(www.gladstone.ai)

AI Pre­dic­tion Ser­vices and Risks of War

isaduanOct 3, 2021, 10:26 AM
3 points
2 comments10 min readLW link

Keep hu­mans in the loop

Apr 19, 2023, 3:34 PM
23 points
1 comment10 min readLW link

Misal­ign­ment-by-de­fault in multi-agent systems

Oct 13, 2022, 3:38 PM
21 points
8 comments20 min readLW link
(www.gladstone.ai)

Mak­ing AIs less likely to be spiteful

Sep 26, 2023, 2:12 PM
116 points
4 comments10 min readLW link

In­stru­men­tal con­ver­gence: scale and phys­i­cal interactions

Oct 14, 2022, 3:50 PM
22 points
0 comments17 min readLW link
(www.gladstone.ai)

Agents which are EU-max­i­miz­ing as a group are not EU-max­i­miz­ing individually

MlxaDec 4, 2023, 6:49 PM
3 points
2 comments2 min readLW link

Y2K: Suc­cess­ful Prac­tice for AI Alignment

DarmaniNov 5, 2021, 6:09 AM
49 points
5 comments6 min readLW link

Co­op­er­a­tion and Align­ment in Del­e­ga­tion Games: You Need Both!

Aug 3, 2024, 10:16 AM
8 points
0 comments14 min readLW link
(www.oliversourbut.net)

Ex­pevolu, a laissez-faire ap­proach to coun­try creation

FernandoDec 5, 2024, 7:29 PM
4 points
4 comments44 min readLW link
(expevolu.substack.com)

List of Col­lec­tive In­tel­li­gence Projects

ChipmonkJul 2, 2024, 2:10 PM
40 points
9 comments2 min readLW link
(chrislakin.blog)

Speak­ing of Stag Hunts

Duncan Sabien (Deactivated)Nov 6, 2021, 8:20 AM
191 points
373 comments18 min readLW link

AI take­off and nu­clear war

owencbJun 11, 2024, 7:36 PM
80 points
6 comments11 min readLW link
(strangecities.substack.com)

A civ­i­liza­tion ran by amateurs

Olli JärviniemiMay 30, 2024, 5:57 PM
61 points
7 comments6 min readLW link

At­las: Stress-Test­ing ASI Value Learn­ing Through Grand Strat­egy Scenarios

NeilFoxFeb 17, 2025, 11:55 PM
1 point
0 comments2 min readLW link

Align­ment is not enough

Alan ChanJan 12, 2023, 12:33 AM
12 points
6 comments11 min readLW link
(coordination.substack.com)

[Question] Rough Sketch for Product to En­hance Ci­ti­zen Par­ti­ci­pa­tion in Politics

T431Oct 20, 2022, 8:04 PM
13 points
5 comments1 min readLW link

Inad­e­quate Equil­ibria vs. Gover­nance of the Commons

Martin SustrikMay 25, 2018, 1:17 PM
184 points
17 comments14 min readLW link2 reviews

Co­op­er­a­tors are more pow­er­ful than agents

Ivan VendrovOct 21, 2022, 8:02 PM
29 points
7 comments3 min readLW link

Joscha Bach on Syn­thetic In­tel­li­gence [an­no­tated]

Roman LeventovMar 2, 2023, 11:02 AM
9 points
1 comment9 min readLW link
(www.jimruttshow.com)

Melt­ing Gold, and Or­ga­ni­za­tional Capacity

RaemonDec 11, 2017, 7:16 PM
73 points
23 comments5 min readLW link

Ev­i­den­tial Cor­re­la­tions are Sub­jec­tive, and it might be a problem

Martín SotoMar 7, 2024, 6:37 PM
26 points
6 comments14 min readLW link

Ideas for an ac­tion co­or­di­na­tion website

Yoav RavidMar 8, 2019, 1:32 PM
20 points
38 comments5 min readLW link

[Question] Has pri­vate AGI re­search made in­de­pen­dent safety re­search in­effec­tive already? What should we do about this?

Roman LeventovJan 23, 2023, 7:36 AM
43 points
5 comments5 min readLW link

[Question] Is there an as­surance-con­tract web­site in work?

Yoav RavidFeb 20, 2019, 6:14 AM
18 points
31 comments1 min readLW link

Norms of Mem­ber­ship for Vol­un­tary Groups

sarahconstantinDec 11, 2018, 10:10 PM
192 points
10 comments7 min readLW link
(srconstantin.wordpress.com)

What is op­er­a­tions?

Swimmer963 (Miranda Dixon-Luinenburg) Sep 26, 2019, 2:16 PM
112 points
9 comments7 min readLW link

Defect­ing by Ac­ci­dent—A Flaw Com­mon to An­a­lyt­i­cal People

lionhearted (Sebastian Marshall)Dec 1, 2010, 8:25 AM
127 points
433 comments15 min readLW link

Les­sons from the Cold War on In­for­ma­tion Hazards: Why In­ter­nal Com­mu­ni­ca­tion is Critical

GentzelFeb 24, 2018, 11:34 PM
47 points
10 comments4 min readLW link

[Question] Good books about over­com­ing co­or­di­na­tion prob­lems?

Sean AubinFeb 2, 2022, 7:09 PM
22 points
14 comments1 min readLW link

Tol­er­ate Tolerance

Eliezer YudkowskyMar 21, 2009, 7:34 AM
149 points
87 comments2 min readLW link

Lead­ers of Men

ZviOct 29, 2017, 6:10 PM
51 points
16 comments2 min readLW link
(thezvi.wordpress.com)

The Dar­win Results

ZviNov 25, 2017, 1:30 PM
53 points
10 comments5 min readLW link
(thezvi.wordpress.com)

Search­ing for the Root of the Tree of Evil

Ivan VendrovJun 8, 2024, 5:05 PM
36 points
14 comments5 min readLW link
(nothinghuman.substack.com)

I Want To Live In A Baugruppe

AlicornMar 17, 2017, 1:36 AM
109 points
195 comments3 min readLW link

The Dar­win Pregame

ZviNov 21, 2017, 1:10 AM
44 points
6 comments5 min readLW link
(thezvi.wordpress.com)

Buri­dan’s ass in co­or­di­na­tion games

jessicataJul 16, 2018, 2:51 AM
52 points
26 comments10 min readLW link

Re­duc­ing col­lec­tive ra­tio­nal­ity to in­di­vi­d­ual op­ti­miza­tion in com­mon-pay­off games us­ing MCMC

jessicataAug 20, 2018, 12:51 AM
59 points
12 comments6 min readLW link

Coun­ter­fac­tu­als about So­cial Media

ZviApr 22, 2019, 12:20 PM
53 points
23 comments6 min readLW link
(thezvi.wordpress.com)

Some Ways Co­or­di­na­tion is Hard

ZviJun 13, 2019, 1:00 PM
55 points
11 comments10 min readLW link1 review
(thezvi.wordpress.com)

The Frag­ility of Life Hy­poth­e­sis and the Evolu­tion of Cooperation

KristianRonnSep 4, 2024, 9:04 PM
50 points
6 comments11 min readLW link

Can you elimi­nate memetic scarcity, in­stead of fight­ing?

RaemonNov 25, 2019, 2:07 AM
71 points
11 comments4 min readLW link

Dath Ilan vs. Sid Meier’s Alpha Cen­tauri: Pareto Improvements

David UdellApr 28, 2022, 7:26 PM
35 points
16 comments2 min readLW link

AI safety ad­vo­cates should con­sider pro­vid­ing gen­tle push­back fol­low­ing the events at OpenAI

civilsocietyDec 22, 2023, 6:55 PM
16 points
5 comments3 min readLW link
No comments.