RSS

Co­or­di­na­tion /​ Cooperation

TagLast edit: 5 Aug 2020 12:40 UTC by Yoav Ravid

Coordination is the challenge of distinct actors being able to jointly choose their actions to achieve a favorable outcome. An example of easy coordination is coordinating to drive on the same side of the road. A hard example is coordinating everyone to move a superior yet different social media platform. Many failures of civilization are failures of coordination. A closely related concept is that of cooperation – multiple actors choosing their actions in ways that maximize collective value despite the temptation of greater short-term individual gain by acting to the detriment of the group/​other actors. The Prisoner’s Dilemma is the canonical cooperation/​defection game, but the term is used in other games too, e.g. Stag Hunt.

Coordination and cooperation are fundamental concepts in Game Theory. LessWrong discussion has long been interested in overcoming the gnarly coordination and cooperation challenges that prevent many improvements to optimizing the world. There is also interest because failures to coordinate/​cooperate are likely causes of existential risks such as AI Risk or nuclear war.

See also: Game Theory, Moloch

Another related term in this space is collective action problem.

Related Sequences: Kickstarter for Coordinated Action

Co­or­di­na­tion as a Scarce Resource

johnswentworth25 Jan 2020 23:32 UTC
246 points
22 comments4 min readLW link2 reviews

Con­scious­ness as a con­fla­tion­ary al­li­ance term for in­trin­si­cally val­ued in­ter­nal experiences

Andrew_Critch10 Jul 2023 8:09 UTC
193 points
47 comments11 min readLW link

Unify­ing Bar­gain­ing No­tions (1/​2)

Diffractor25 Jul 2022 0:28 UTC
209 points
41 comments16 min readLW link

The Costly Co­or­di­na­tion Mechanism of Com­mon Knowledge

Ben Pace15 Mar 2018 20:20 UTC
198 points
31 comments20 min readLW link2 reviews

Two Co­or­di­na­tion Styles

abramdemski7 Feb 2018 9:00 UTC
40 points
14 comments7 min readLW link

Most Pri­soner’s Dilem­mas are Stag Hunts; Most Stag Hunts are Schel­ling Problems

abramdemski14 Sep 2020 22:13 UTC
177 points
36 comments10 min readLW link3 reviews

What Mul­tipo­lar Failure Looks Like, and Ro­bust Agent-Ag­nos­tic Pro­cesses (RAAPs)

Andrew_Critch31 Mar 2021 23:50 UTC
280 points
65 comments22 min readLW link1 review

Co­hab­itive Games so Far

mako yass28 Sep 2023 15:41 UTC
105 points
129 comments19 min readLW link
(makopool.com)

Iter­ated Trust Kickstarters

Raemon20 Apr 2021 3:18 UTC
74 points
19 comments10 min readLW link

“Win First” vs “Chill First”

lionhearted (Sebastian Marshall)28 Sep 2020 6:48 UTC
101 points
20 comments3 min readLW link

Your Cheer­ful Price

Eliezer Yudkowsky13 Feb 2021 5:41 UTC
267 points
83 comments17 min readLW link6 reviews

Parable of the Dammed

johnswentworth10 Dec 2020 0:08 UTC
118 points
29 comments2 min readLW link

Lev­er­age points for a pause

Remmelt28 Aug 2024 9:21 UTC
3 points
0 comments1 min readLW link

The Co­or­di­na­tion Fron­tier: Se­quence Intro

Raemon4 Sep 2021 22:11 UTC
162 points
22 comments6 min readLW link1 review

Three rea­sons to cooperate

paulfchristiano24 Dec 2022 17:40 UTC
82 points
14 comments10 min readLW link
(sideways-view.com)

Jean Mon­net: The Guerilla Bureaucrat

Martin Sustrik20 Mar 2021 10:37 UTC
177 points
25 comments18 min readLW link1 review

Co­or­di­na­tion Sur­veys: why we should sur­vey to or­ga­nize re­spon­si­bil­ities, not just predictions

Academian7 May 2019 17:43 UTC
68 points
12 comments3 min readLW link3 reviews

Co­or­di­na­tion Schemes Are Cap­i­tal Investments

Raemon6 Sep 2021 23:27 UTC
140 points
31 comments7 min readLW link1 review

A Key Power of the Pres­i­dent is to Co­or­di­nate the Ex­e­cu­tion of Ex­ist­ing Con­crete Plans

Ben Pace16 Jul 2019 5:06 UTC
152 points
13 comments10 min readLW link

My Mar­riage Vows

Vanessa Kosoy21 Jul 2021 10:48 UTC
84 points
53 comments3 min readLW link

Co­or­di­na­tion Skills I Wish I Had For the Pandemic

Raemon13 Nov 2021 23:32 UTC
89 points
9 comments6 min readLW link1 review

On De­stroy­ing the World

Chris_Leong28 Sep 2020 7:38 UTC
82 points
86 comments5 min readLW link

AI co­or­di­na­tion needs clear wins

evhub1 Sep 2022 23:41 UTC
147 points
16 comments2 min readLW link1 review

In­sights from ‘The Strat­egy of Con­flict’

DanielFilan4 Jan 2018 5:05 UTC
46 points
13 comments7 min readLW link

Con­cen­tra­tion of Force

Duncan Sabien (Deactivated)6 Nov 2021 8:20 UTC
240 points
23 comments12 min readLW link1 review

Si­mu­lacrum 3 As Stag-Hunt Strategy

johnswentworth26 Jan 2021 19:40 UTC
183 points
37 comments4 min readLW link3 reviews

Failures in Kindness

silentbob26 Mar 2024 21:30 UTC
401 points
60 comments9 min readLW link

Norm In­no­va­tion and The­ory of Mind

Raemon18 Sep 2021 21:38 UTC
92 points
14 comments7 min readLW link1 review

A tax­on­omy of co­or­di­na­tion problems

damiensnyder22 Aug 2021 23:10 UTC
30 points
1 comment6 min readLW link

Why Our Kind Can’t Cooperate

Eliezer Yudkowsky20 Mar 2009 8:37 UTC
291 points
211 comments9 min readLW link

Some con­struc­tions for proof-based co­op­er­a­tion with­out Löb

James Payor21 Mar 2023 16:12 UTC
43 points
3 comments4 min readLW link

Co­op­er­at­ing with agents with differ­ent ideas of fair­ness, while re­sist­ing exploitation

Eliezer Yudkowsky16 Sep 2013 8:27 UTC
103 points
42 comments4 min readLW link

Democ­racy Is in Danger, but Not for the Rea­sons You Think

ExCeph6 Nov 2022 21:15 UTC
−7 points
4 comments12 min readLW link
(ginnungagapfoundation.wordpress.com)

Gaia Net­work: a prac­ti­cal, in­cre­men­tal path­way to Open Agency Architecture

20 Dec 2023 17:11 UTC
22 points
8 comments16 min readLW link

Your Price for Joining

Eliezer Yudkowsky26 Mar 2009 7:16 UTC
87 points
58 comments4 min readLW link

Refine­ment of Ac­tive In­fer­ence agency ontology

Roman Leventov15 Dec 2023 9:31 UTC
16 points
0 comments5 min readLW link
(arxiv.org)

[Question] What does it look like for AI to sig­nifi­cantly im­prove hu­man co­or­di­na­tion, be­fore su­per­in­tel­li­gence?

jacobjacob15 Jan 2024 19:22 UTC
22 points
2 comments1 min readLW link

Re­view: Conor More­ton’s “Civ­i­liza­tion & Co­op­er­a­tion”

Duncan Sabien (Deactivated)26 May 2024 19:32 UTC
89 points
8 comments38 min readLW link

In­sti­tu­tional eco­nomics through the lens of scale-free reg­u­la­tive de­vel­op­ment, mor­pho­gen­e­sis, and cog­ni­tive science

Roman Leventov23 Jan 2024 19:42 UTC
8 points
0 comments14 min readLW link

If we solve al­ign­ment, do we die any­way?

Seth Herd23 Aug 2024 13:13 UTC
70 points
102 comments4 min readLW link

Don’t sleep on Co­or­di­na­tion Takeoffs

trevor27 Jan 2024 19:55 UTC
62 points
24 comments5 min readLW link

Are most per­son­al­ity di­s­or­ders re­ally trust di­s­or­ders?

chaosmage6 Feb 2024 12:37 UTC
20 points
4 comments1 min readLW link

How it All Went Down: The Puz­zle Hunt that took us way, way Less Online

A*2 Jun 2024 8:01 UTC
134 points
5 comments5 min readLW link

WSJ: In­side Ama­zon’s Se­cret Oper­a­tion to Gather In­tel on Rivals

trevor23 Apr 2024 21:33 UTC
37 points
5 comments5 min readLW link
(www.wsj.com)

Ag­grega­tive Prin­ci­ples of So­cial Justice

Cleo Nardo5 Jun 2024 13:44 UTC
29 points
10 comments37 min readLW link

Se­cond-Order Ra­tion­al­ity, Sys­tem Ra­tion­al­ity, and a fea­ture sug­ges­tion for LessWrong

Mati_Roy5 Jun 2024 7:20 UTC
13 points
2 comments8 min readLW link

Cap­i­tal­is­ing On Trust—A Simulation

James Stephen Brown21 Jun 2024 4:43 UTC
2 points
0 comments1 min readLW link
(nonzerosum.games)

AI de­vel­op­ment is an act of so­cial revolution

artemiocobb3 Jul 2024 18:00 UTC
3 points
0 comments3 min readLW link

Travel Buffer

jefftk6 Jul 2024 2:20 UTC
17 points
3 comments1 min readLW link
(www.jefftk.com)

Re­sponse to Dileep Ge­orge: AGI safety war­rants plan­ning ahead

Steven Byrnes8 Jul 2024 15:27 UTC
27 points
7 comments27 min readLW link

Trust as a bot­tle­neck to grow­ing teams quickly

benkuhn13 Jul 2024 18:00 UTC
42 points
3 comments5 min readLW link
(www.benkuhn.net)

AI Con­sti­tu­tions are a tool to re­duce so­cietal scale risk

Sammy Martin25 Jul 2024 11:18 UTC
30 points
2 comments18 min readLW link

A Nar­row Path: a plan to deal with AI ex­tinc­tion risk

7 Oct 2024 13:02 UTC
73 points
11 comments2 min readLW link
(www.narrowpath.co)

[Question] A Co­or­di­na­tion Cook­book?

azergante10 Nov 2024 23:20 UTC
1 point
0 comments1 min readLW link

FLI And Eliezer Should Reach Consensus

JenniferRM11 Apr 2023 4:07 UTC
19 points
6 comments23 min readLW link

My idea of sa­cred­ness, di­v­inity, and religion

Kaj_Sotala29 Oct 2023 12:50 UTC
40 points
10 comments4 min readLW link
(kajsotala.fi)

Urg­ing an In­ter­na­tional AI Treaty: An Open Letter

Olli Järviniemi31 Oct 2023 11:26 UTC
48 points
2 comments1 min readLW link
(aitreaty.org)

[Linkpost] Ge­orge Mack’s Razors

trevor27 Nov 2023 17:53 UTC
38 points
8 comments3 min readLW link
(twitter.com)

Pro­posal for im­prov­ing the global on­line dis­course through per­son­al­ised com­ment or­der­ing on all websites

Roman Leventov6 Dec 2023 18:51 UTC
35 points
21 comments6 min readLW link

[Question] Does it be­come eas­ier, or harder, for the world to co­or­di­nate around not build­ing AGI as time goes on?

Eli Tyre29 Jul 2019 22:59 UTC
86 points
31 comments3 min readLW link2 reviews

Tac­ti­cal vs. Strate­gic Cooperation

sarahconstantin12 Aug 2018 16:41 UTC
74 points
18 comments5 min readLW link

Ro­bust Co­op­er­a­tion in the Pri­soner’s Dilemma

orthonormal7 Jun 2013 8:30 UTC
120 points
147 comments7 min readLW link

Black­mail, Nukes and the Pri­soner’s Dilemma

Stuart_Armstrong10 Mar 2010 14:58 UTC
25 points
20 comments2 min readLW link

What counts as defec­tion?

TurnTrout12 Jul 2020 22:03 UTC
81 points
21 comments5 min readLW link1 review

Sec­tions 1 & 2: In­tro­duc­tion, Strat­egy and Governance

JesseClifton17 Dec 2019 21:27 UTC
35 points
8 comments14 min readLW link

Sec­tions 3 & 4: Cred­i­bil­ity, Peace­ful Bar­gain­ing Mechanisms

JesseClifton17 Dec 2019 21:46 UTC
20 points
2 comments12 min readLW link

“You can’t pos­si­bly suc­ceed with­out [My Pet Is­sue]”

Raemon19 Dec 2019 1:12 UTC
53 points
14 comments1 min readLW link

Shap­ing eco­nomic in­cen­tives for col­lab­o­ra­tive AGI

Kaj_Sotala29 Jun 2018 16:26 UTC
45 points
15 comments4 min readLW link

Mul­ti­verse-wide Co­op­er­a­tion via Cor­re­lated De­ci­sion Making

Kaj_Sotala20 Aug 2017 12:01 UTC
5 points
2 comments1 min readLW link
(foundational-research.org)

From Per­sonal to Pri­son Gangs: En­forc­ing Proso­cial Behavior

johnswentworth24 Jan 2019 18:07 UTC
153 points
26 comments5 min readLW link2 reviews

Win­ning is for Losers

Jacob Falkovich11 Oct 2017 4:01 UTC
31 points
12 comments18 min readLW link
(putanumonit.com)

Bayesi­ans vs. Barbarians

Eliezer Yudkowsky14 Apr 2009 23:45 UTC
103 points
277 comments8 min readLW link

The Im­por­tance of Sidekicks

Swimmer963 (Miranda Dixon-Luinenburg) 8 Jan 2015 23:21 UTC
232 points
209 comments5 min readLW link

Team Co­he­sion and Ex­clu­sion­ary Egalitarianism

lionhearted (Sebastian Marshall)17 Sep 2018 4:48 UTC
31 points
6 comments7 min readLW link

Healthy Competition

Raemon20 Oct 2019 20:55 UTC
56 points
13 comments2 min readLW link1 review

Reneg­ing proso­cially by Dun­can Sabien

philip_b18 Jun 2019 18:52 UTC
70 points
5 comments1 min readLW link
(medium.com)

Bab­ble on grow­ing trust

qbolec21 May 2023 13:19 UTC
13 points
1 comment5 min readLW link

[Question] How could “Kick­starter for Inad­e­quate Equil­ibria” be used for evil or turn out to be net-nega­tive?

Raemon21 Feb 2019 21:36 UTC
25 points
17 comments1 min readLW link

Schel­ling Reach

Ziz7 Jan 2018 18:18 UTC
10 points
3 comments1 min readLW link
(sinceriously.fyi)

Real-World Co­or­di­na­tion Prob­lems are Usu­ally In­for­ma­tion Problems

johnswentworth13 Jun 2019 18:21 UTC
35 points
2 comments4 min readLW link

Vanilla and choco­late and prefer­ence judgements

Swimmer963 (Miranda Dixon-Luinenburg) 18 Apr 2011 22:14 UTC
39 points
32 comments6 min readLW link

The Align­ment-Com­pe­tence Trade-Off, Part 1: Coal­i­tion Size and Sig­nal­ing Costs

Gentzel15 Jan 2020 23:10 UTC
30 points
4 comments3 min readLW link
(theconsequentialist.wordpress.com)

Cor­rectly Cal­ibrated Trust

habryka24 Jun 2023 19:48 UTC
36 points
3 comments11 min readLW link
(forum.effectivealtruism.org)

An at­tempt at a “good enough” solu­tion for hu­man two-party negotiations

Isaac King16 Sep 2023 0:38 UTC
11 points
22 comments1 min readLW link

Col­lAc­tion his­tory and les­sons learned

Ron17 Jan 2021 15:01 UTC
34 points
6 comments8 min readLW link

Google’s Eth­i­cal AI team and AI Safety

magfrump20 Feb 2021 9:42 UTC
12 points
16 comments7 min readLW link

Partnership

Ruby11 Mar 2021 17:17 UTC
17 points
0 comments3 min readLW link

An Ex­er­cise in Ra­tional Co­op­er­a­tion and Com­mu­ni­ca­tion: Let’s Play Hanabi

ejacob14 Apr 2021 8:36 UTC
14 points
10 comments7 min readLW link

April 15, 2040

Nisan4 May 2021 21:18 UTC
97 points
25 comments2 min readLW link

[Book Re­view] Blueprint for Revolution

lsusr7 Jun 2021 18:14 UTC
28 points
8 comments2 min readLW link

A Bet­ter Web is Coming

matto21 Aug 2021 20:46 UTC
34 points
20 comments6 min readLW link

Notes on Courtesy

David Gross16 Dec 2020 4:08 UTC
11 points
0 comments6 min readLW link

AISC5 Ret­ro­spec­tive: Mechanisms for Avoid­ing Tragedy of the Com­mons in Com­mon Pool Re­source Problems

27 Sep 2021 16:46 UTC
8 points
3 comments7 min readLW link

The role of tribes in achiev­ing last­ing im­pact and how to cre­ate them

29 Sep 2021 20:48 UTC
14 points
3 comments24 min readLW link

Co­or­di­na­tion Mo­ti­va­tion: The Pandemic

Raemon21 Oct 2021 21:57 UTC
64 points
1 comment3 min readLW link

Post­mod­ern Warfare

lsusr25 Oct 2021 9:02 UTC
64 points
25 comments2 min readLW link

First Strike and Se­cond Strike

lsusr25 Nov 2021 9:23 UTC
28 points
5 comments1 min readLW link

Co­or­di­nat­ing the Unequal Treaties

lsusr25 Nov 2021 10:47 UTC
34 points
4 comments2 min readLW link

Pareto im­prove­ments are rarer than they seem

philh27 Jan 2018 22:23 UTC
27 points
13 comments3 min readLW link
(reasonableapproximation.net)

My Overview of the AI Align­ment Land­scape: A Bird’s Eye View

Neel Nanda15 Dec 2021 23:44 UTC
127 points
9 comments15 min readLW link

My Overview of the AI Align­ment Land­scape: Threat Models

Neel Nanda25 Dec 2021 23:07 UTC
52 points
3 comments28 min readLW link

Create a Full Alter­na­tive Stack

Zvi31 Jan 2020 17:10 UTC
84 points
14 comments6 min readLW link1 review
(thezvi.wordpress.com)

Nar­ra­tive Syncing

AnnaSalamon1 May 2022 1:48 UTC
118 points
48 comments7 min readLW link1 review

An­nounc­ing the Align­ment of Com­plex Sys­tems Re­search Group

4 Jun 2022 4:10 UTC
91 points
20 comments5 min readLW link

Failing to fix a dan­ger­ous intersection

alyssavance30 Jun 2022 18:09 UTC
110 points
17 comments2 min readLW link

Digi­tal Din­ner Signup

jefftk23 Aug 2022 2:20 UTC
13 points
5 comments1 min readLW link
(www.jefftk.com)

The ethics of re­clin­ing air­plane seats

braces4 Sep 2022 17:59 UTC
93 points
70 comments1 min readLW link

Ve­hi­cle Pla­toon­ing—a real world ex­am­i­na­tion of the difficul­ties in coordination

M. Y. Zuo13 Oct 2022 19:33 UTC
24 points
6 comments2 min readLW link

Why Aren’t There More Schel­ling Holi­days?

johnswentworth31 Oct 2022 19:31 UTC
63 points
21 comments1 min readLW link

The biolog­i­cal func­tion of love for non-kin is to gain the trust of peo­ple we can­not deceive

chaosmage7 Nov 2022 20:26 UTC
43 points
3 comments8 min readLW link

In­ter­nal­iz­ing the dam­age of bad-act­ing part­ners cre­ates in­cen­tives for due diligence

tailcalled11 Nov 2022 20:57 UTC
17 points
7 comments1 min readLW link

“Ru­de­ness”, a use­ful co­or­di­na­tion mechanic

Raemon11 Nov 2022 22:27 UTC
49 points
20 comments2 min readLW link

Löbian emo­tional pro­cess­ing of emer­gent co­op­er­a­tion: an example

Andrew_Critch17 Jan 2023 5:59 UTC
23 points
0 comments8 min readLW link

Distributed Cooperation

Diffractor18 Mar 2018 5:46 UTC
13 points
2 comments9 min readLW link

[Question] Why should we ex­pect AIs to co­or­di­nate well?

Jonathan Paulson14 Feb 2023 15:50 UTC
25 points
9 comments1 min readLW link

Selec­tive, Cor­rec­tive, Struc­tural: Three Ways of Mak­ing So­cial Sys­tems Work

Said Achmiz5 Mar 2023 8:45 UTC
99 points
13 comments2 min readLW link

Notes on Loyalty

David Gross15 Nov 2020 19:30 UTC
17 points
2 comments8 min readLW link

Notes on So­cial Responsibility

David Gross19 Mar 2022 14:44 UTC
17 points
4 comments19 min readLW link

Re­lease: Op­ti­mal Weave (P1): A Pro­to­type Co­hab­itive Game

mako yass17 Aug 2024 14:08 UTC
82 points
21 comments7 min readLW link

Seek­ing (Paid) Case Stud­ies on Standards

HoldenKarnofsky26 May 2023 17:58 UTC
69 points
9 comments11 min readLW link

[AN #150]: The sub­types of Co­op­er­a­tive AI research

Rohin Shah12 May 2021 17:20 UTC
17 points
0 comments6 min readLW link
(mailchi.mp)

CLR’s re­cent work on multi-agent systems

JesseClifton9 Mar 2021 2:28 UTC
54 points
2 comments13 min readLW link

Worked Ex­am­ples of Shap­ley Values

lalaithion24 Jun 2022 17:13 UTC
72 points
11 comments8 min readLW link

One Web­site To Rule Them All?

anna_macdonald11 Jan 2019 19:14 UTC
30 points
23 comments10 min readLW link

Col­lec­tive Apa­thy and the Internet

Eliezer Yudkowsky14 Apr 2009 0:02 UTC
52 points
34 comments2 min readLW link

Break­ing Or­a­cles: su­per­ra­tional­ity and acausal trade

Stuart_Armstrong25 Nov 2019 10:40 UTC
25 points
15 comments1 min readLW link

Quick Sum­maries of Two Papers on Kant and Game Theory

Erich_Grunewald25 Jun 2022 10:25 UTC
8 points
2 comments4 min readLW link
(www.erichgrunewald.com)

Abadar­ian Trades

David Udell30 Jun 2022 16:41 UTC
17 points
22 comments2 min readLW link

Moloch—An Illus­trated Primer

James Stephen Brown26 May 2024 1:04 UTC
5 points
0 comments7 min readLW link
(nonzerosum.games)

[Question] De­gree of du­pli­ca­tion and co­or­di­na­tion in pro­jects that ex­am­ine com­put­ing prices, AI progress, and re­lated top­ics?

riceissa23 Apr 2019 12:27 UTC
26 points
1 comment2 min readLW link

Su­per­in­tel­li­gence 28: Collaboration

KatjaGrace24 Mar 2015 1:29 UTC
13 points
21 comments6 min readLW link

Fun­da­men­tally Fuzzy Con­cepts Can’t Have Crisp Defi­ni­tions: Co­op­er­a­tion and Align­ment vs Math and Physics

VojtaKovarik21 Jul 2023 21:03 UTC
12 points
18 comments3 min readLW link

AI take­off and nu­clear war

owencb11 Jun 2024 19:36 UTC
76 points
6 comments11 min readLW link
(strangecities.substack.com)

Search­ing for the Root of the Tree of Evil

Ivan Vendrov8 Jun 2024 17:05 UTC
36 points
14 comments5 min readLW link
(nothinghuman.substack.com)

Acausal Now: We could to­tally acausally bar­gain with aliens at our cur­rent tech level if desired

Christopher King9 Aug 2023 0:50 UTC
1 point
5 comments4 min readLW link

Knowl­edge Base 6: Con­sen­sus the­ory of truth

iwis3 Nov 2023 13:56 UTC
−8 points
0 comments1 min readLW link

Knowl­edge Base 2: The struc­ture and the method of building

iwis9 Oct 2023 11:53 UTC
2 points
4 comments7 min readLW link

Knowl­edge Base 4: Gen­eral applications

iwis16 Oct 2023 12:26 UTC
3 points
0 comments1 min readLW link

Knowl­edge Base 3: Shop­ping ad­vi­sor and other uses of knowl­edge base about products

iwis9 Oct 2023 11:53 UTC
0 points
0 comments4 min readLW link

Knowl­edge Base 8: The truth as an at­trac­tor in the in­for­ma­tion space

iwis25 Apr 2024 15:28 UTC
−8 points
0 comments2 min readLW link

[Question] Where do (did?) sta­ble, co­op­er­a­tive in­sti­tu­tions come from?

AnnaSalamon3 Nov 2020 22:14 UTC
151 points
72 comments4 min readLW link

Co­op­er­a­tion is for Winners

Jacob Falkovich15 Feb 2019 14:58 UTC
21 points
6 comments4 min readLW link

Co­op­er­a­tion with and be­tween AGI\’s

PeterMcCluskey7 Jul 2022 16:45 UTC
10 points
3 comments10 min readLW link
(www.bayesianinvestor.com)

The Frag­ility of Life Hy­poth­e­sis and the Evolu­tion of Cooperation

KristianRonn4 Sep 2024 21:04 UTC
49 points
6 comments11 min readLW link

Spaghetti Towers

eukaryote22 Dec 2018 5:29 UTC
209 points
36 comments3 min readLW link1 review
(eukaryotewritesblog.com)

Pas­cal’s Mug­ging: The Word Wars

johncrox16 Oct 2023 17:54 UTC
9 points
1 comment6 min readLW link

[Question] How do you con­cisely com­mu­ni­cate & nav­i­gate the poli­tics /​ cul­ture at your job work­ing at a large cor­po­ra­tion or in­sti­tu­tion?

Willa12 Jul 2022 3:22 UTC
10 points
6 comments1 min readLW link

Notes on Schel­ling’s “Strat­egy of Con­flict” (1960)

MichaelA10 Feb 2021 2:48 UTC
25 points
8 comments8 min readLW link

Key take­aways from our EA and al­ign­ment re­search sur­veys

3 May 2024 18:10 UTC
103 points
10 comments21 min readLW link

[Question] What are some al­ter­na­tives to Shap­ley val­ues which drop ad­di­tivity?

eapi9 Aug 2022 9:16 UTC
11 points
6 comments1 min readLW link
(math.stackexchange.com)

The Pro­to­typ­i­cal Ne­go­ti­a­tion Game

johnswentworth20 Feb 2021 21:33 UTC
97 points
16 comments5 min readLW link1 review

Let’s think about slow­ing down AI

KatjaGrace22 Dec 2022 17:40 UTC
549 points
182 comments38 min readLW link3 reviews
(aiimpacts.org)

Can Kauff­man’s NK Boolean net­works make hu­mans swarm?

Yori-928 May 2024 2:09 UTC
6 points
2 comments20 min readLW link

I’m from a par­allel Earth with much higher co­or­di­na­tion: AMA

5 Apr 2021 22:09 UTC
166 points
35 comments61 min readLW link1 review

The Ex­pand­ing Mo­ral Cine­matic Universe

Raemon28 Aug 2022 18:42 UTC
64 points
9 comments14 min readLW link

Re­view and Com­par­i­son: Onto-Car­tog­ra­phy & Promise Theory

Josephine9 May 2021 0:42 UTC
13 points
0 comments5 min readLW link

A civ­i­liza­tion ran by amateurs

Olli Järviniemi30 May 2024 17:57 UTC
61 points
7 comments6 min readLW link

Con­trol­ling In­tel­li­gent Agents The Only Way We Know How: Ideal Bureau­cratic Struc­ture (IBS)

Justin Bullock24 May 2021 12:53 UTC
14 points
15 comments6 min readLW link

Prac­ti­cal post on 4 #ProRep meth­ods (Pro­por­tional Rep­re­sen­ta­tion)

Jameson Quinn9 Jan 2021 14:56 UTC
37 points
12 comments11 min readLW link

For­mal­iz­ing Ob­jec­tions against Sur­ro­gate Goals

VojtaKovarik2 Sep 2021 16:24 UTC
16 points
23 comments1 min readLW link

Co­op­er­a­tion is op­ti­mal, with weaker agents too  -  tldr

Ryo 18 Apr 2024 15:03 UTC
12 points
22 comments4 min readLW link
(medium.com)

How to co­or­di­nate de­spite our bi­ases? - tldr

Ryo 18 Apr 2024 15:03 UTC
3 points
2 comments3 min readLW link
(medium.com)

Knowl­edge Base 7: Long-tail knowl­edge and col­lec­tive intelligence

iwis18 Apr 2024 14:21 UTC
−6 points
0 comments1 min readLW link

In the Name of All That Needs Saving

pleiotroth7 Nov 2024 15:26 UTC
18 points
2 comments22 min readLW link

Mon­i­tor­ing for de­cep­tive alignment

evhub8 Sep 2022 23:07 UTC
135 points
8 comments9 min readLW link

Ad­ver­sar­ial vs Col­lab­o­ra­tive Contexts

jefftk4 Oct 2022 2:40 UTC
31 points
4 comments2 min readLW link
(www.jefftk.com)

Ask­ing For Help

Neel Nanda27 Dec 2020 11:32 UTC
23 points
5 comments12 min readLW link
(neelnanda.io)

Ex­plor­ing the Pre­cau­tion­ary Prin­ci­ple in AI Devel­op­ment: His­tor­i­cal Analo­gies and Les­sons Learned

Christopher King21 Mar 2023 3:53 UTC
−1 points
2 comments9 min readLW link

LDT (and ev­ery­thing else) can be irrational

Christopher King6 Nov 2024 4:05 UTC
3 points
6 comments2 min readLW link

In­stru­men­tal con­ver­gence in sin­gle-agent systems

12 Oct 2022 12:24 UTC
33 points
4 comments8 min readLW link
(www.gladstone.ai)

AI Pre­dic­tion Ser­vices and Risks of War

isaduan3 Oct 2021 10:26 UTC
3 points
2 comments10 min readLW link

Keep hu­mans in the loop

19 Apr 2023 15:34 UTC
23 points
1 comment10 min readLW link

Misal­ign­ment-by-de­fault in multi-agent systems

13 Oct 2022 15:38 UTC
21 points
8 comments20 min readLW link
(www.gladstone.ai)

Mak­ing AIs less likely to be spiteful

26 Sep 2023 14:12 UTC
116 points
4 comments10 min readLW link

In­stru­men­tal con­ver­gence: scale and phys­i­cal interactions

14 Oct 2022 15:50 UTC
22 points
0 comments17 min readLW link
(www.gladstone.ai)

Agents which are EU-max­i­miz­ing as a group are not EU-max­i­miz­ing individually

Mlxa4 Dec 2023 18:49 UTC
3 points
2 comments2 min readLW link

Y2K: Suc­cess­ful Prac­tice for AI Alignment

Darmani5 Nov 2021 6:09 UTC
49 points
5 comments6 min readLW link

Co­op­er­a­tion and Align­ment in Del­e­ga­tion Games: You Need Both!

3 Aug 2024 10:16 UTC
8 points
0 comments14 min readLW link
(www.oliversourbut.net)

On Shifgrethor

JustisMills27 Oct 2024 15:30 UTC
66 points
18 comments2 min readLW link
(justismills.substack.com)

Should we ab­stain from vot­ing? (In non­de­ter­minis­tic elec­tions)

B Jacobs2 Oct 2024 10:07 UTC
5 points
6 comments4 min readLW link
(bobjacobs.substack.com)

Knowl­edge Base 1: Could it in­crease in­tel­li­gence and make it safer?

iwis30 Sep 2024 16:00 UTC
−4 points
0 comments4 min readLW link

Speak­ing of Stag Hunts

Duncan Sabien (Deactivated)6 Nov 2021 8:20 UTC
191 points
373 comments18 min readLW link

Un­lock­ing Solu­tions—By Un­der­stand­ing Co­or­di­na­tion Problems

James Stephen Brown27 Jul 2024 4:52 UTC
54 points
4 comments5 min readLW link
(nonzerosum.games)

2024 Free­dom Com­mu­ni­ties Events

Tudor Iliescu12 Jul 2024 8:04 UTC
−6 points
1 comment1 min readLW link

[Question] How does the ever-in­creas­ing use of AI in the mil­i­tary for the di­rect pur­pose of mur­der­ing peo­ple af­fect your p(doom)?

Justausername6 Apr 2024 6:31 UTC
19 points
16 comments1 min readLW link

Com­par­ing Align­ment to other AGI in­ter­ven­tions: Ba­sic model

Martín Soto20 Mar 2024 18:17 UTC
12 points
4 comments7 min readLW link

Align­ment is not enough

Alan Chan12 Jan 2023 0:33 UTC
12 points
6 comments11 min readLW link
(coordination.substack.com)

[Question] Rough Sketch for Product to En­hance Ci­ti­zen Par­ti­ci­pa­tion in Politics

Fer32dwt34r3dfsz20 Oct 2022 20:04 UTC
13 points
3 comments1 min readLW link

Inad­e­quate Equil­ibria vs. Gover­nance of the Commons

Martin Sustrik25 May 2018 13:17 UTC
184 points
17 comments14 min readLW link2 reviews

Co­op­er­a­tors are more pow­er­ful than agents

Ivan Vendrov21 Oct 2022 20:02 UTC
22 points
7 comments3 min readLW link

Joscha Bach on Syn­thetic In­tel­li­gence [an­no­tated]

Roman Leventov2 Mar 2023 11:02 UTC
9 points
1 comment9 min readLW link
(www.jimruttshow.com)

Melt­ing Gold, and Or­ga­ni­za­tional Capacity

Raemon11 Dec 2017 19:16 UTC
73 points
23 comments5 min readLW link

Ev­i­den­tial Cor­re­la­tions are Sub­jec­tive, and it might be a problem

Martín Soto7 Mar 2024 18:37 UTC
26 points
6 comments14 min readLW link

Ideas for an ac­tion co­or­di­na­tion website

Yoav Ravid8 Mar 2019 13:32 UTC
20 points
37 comments5 min readLW link

[Question] Has pri­vate AGI re­search made in­de­pen­dent safety re­search in­effec­tive already? What should we do about this?

Roman Leventov23 Jan 2023 7:36 UTC
43 points
5 comments5 min readLW link

[Question] Is there an as­surance-con­tract web­site in work?

Yoav Ravid20 Feb 2019 6:14 UTC
18 points
31 comments1 min readLW link

Norms of Mem­ber­ship for Vol­un­tary Groups

sarahconstantin11 Dec 2018 22:10 UTC
192 points
10 comments7 min readLW link
(srconstantin.wordpress.com)

What is op­er­a­tions?

Swimmer963 (Miranda Dixon-Luinenburg) 26 Sep 2019 14:16 UTC
110 points
9 comments7 min readLW link

Defect­ing by Ac­ci­dent—A Flaw Com­mon to An­a­lyt­i­cal People

lionhearted (Sebastian Marshall)1 Dec 2010 8:25 UTC
125 points
433 comments15 min readLW link

Les­sons from the Cold War on In­for­ma­tion Hazards: Why In­ter­nal Com­mu­ni­ca­tion is Critical

Gentzel24 Feb 2018 23:34 UTC
47 points
10 comments4 min readLW link

[Question] Good books about over­com­ing co­or­di­na­tion prob­lems?

Sean Aubin2 Feb 2022 19:09 UTC
22 points
14 comments1 min readLW link

Tol­er­ate Tolerance

Eliezer Yudkowsky21 Mar 2009 7:34 UTC
146 points
87 comments2 min readLW link

Lead­ers of Men

Zvi29 Oct 2017 18:10 UTC
44 points
16 comments2 min readLW link
(thezvi.wordpress.com)

The Dar­win Results

Zvi25 Nov 2017 13:30 UTC
52 points
10 comments5 min readLW link
(thezvi.wordpress.com)

List of Col­lec­tive In­tel­li­gence Projects

Chipmonk2 Jul 2024 14:10 UTC
39 points
9 comments2 min readLW link
(chrislakin.blog)

I Want To Live In A Baugruppe

Alicorn17 Mar 2017 1:36 UTC
109 points
195 comments3 min readLW link

The Dar­win Pregame

Zvi21 Nov 2017 1:10 UTC
43 points
6 comments5 min readLW link
(thezvi.wordpress.com)

Buri­dan’s ass in co­or­di­na­tion games

jessicata16 Jul 2018 2:51 UTC
52 points
26 comments10 min readLW link

Re­duc­ing col­lec­tive ra­tio­nal­ity to in­di­vi­d­ual op­ti­miza­tion in com­mon-pay­off games us­ing MCMC

jessicata20 Aug 2018 0:51 UTC
59 points
12 comments6 min readLW link

Coun­ter­fac­tu­als about So­cial Media

Zvi22 Apr 2019 12:20 UTC
53 points
23 comments6 min readLW link
(thezvi.wordpress.com)

Some Ways Co­or­di­na­tion is Hard

Zvi13 Jun 2019 13:00 UTC
55 points
11 comments10 min readLW link1 review
(thezvi.wordpress.com)

Can you elimi­nate memetic scarcity, in­stead of fight­ing?

Raemon25 Nov 2019 2:07 UTC
71 points
11 comments4 min readLW link

Dath Ilan vs. Sid Meier’s Alpha Cen­tauri: Pareto Improvements

David Udell28 Apr 2022 19:26 UTC
35 points
16 comments2 min readLW link

AI safety ad­vo­cates should con­sider pro­vid­ing gen­tle push­back fol­low­ing the events at OpenAI

civilsociety22 Dec 2023 18:55 UTC
16 points
5 comments3 min readLW link

[Question] If a “Kick­starter for Inad­e­quate Equlibria” was built, do you have a con­crete in­ad­e­quate equil­ibrium to fix?

Raemon21 Feb 2019 21:32 UTC
55 points
39 comments1 min readLW link

Machines vs Memes Part 3: Imi­ta­tion and Memes

ceru231 Jun 2022 13:36 UTC
7 points
0 comments7 min readLW link

Shared re­al­ity: a key driver of hu­man behavior

kdbscott24 Dec 2022 19:35 UTC
126 points
25 comments4 min readLW link

Co­or­di­na­tion by com­mon knowl­edge to pre­vent un­con­trol­lable AI

Karl von Wendt14 May 2023 13:37 UTC
10 points
2 comments9 min readLW link

So­phie Grouchy on A Build-Break Model of Cooperation

weft25 May 2018 14:55 UTC
25 points
6 comments8 min readLW link
(sophiegrouchy.wordpress.com)

Prop­erty as Co­or­di­na­tion Minimization

Vaniver4 Aug 2020 19:24 UTC
38 points
35 comments4 min readLW link

Let’s split the cake, length­wise, up­wise and slantwise

Stuart_Armstrong25 Oct 2010 13:15 UTC
74 points
31 comments4 min readLW link

New co­op­er­a­tion mechanism—quadratic fund­ing with­out a match­ing pool

Filip Sondej5 Jun 2022 13:55 UTC
11 points
0 comments5 min readLW link

[Question] Where are peo­ple think­ing and talk­ing about global co­or­di­na­tion for AI safety?

Wei Dai22 May 2019 6:24 UTC
112 points
22 comments1 min readLW link

How hard is it for al­tru­ists to dis­cuss go­ing against bad equil­ibria?

abramdemski22 Jun 2019 3:42 UTC
46 points
6 comments11 min readLW link

AI co­op­er­a­tion is already stud­ied in academia as “pro­gram equil­ibrium”

cousin_it30 Jul 2012 15:22 UTC
60 points
40 comments1 min readLW link

Tomb­stone Men­tal­ity

SuperDaveWhite13 Aug 2020 13:46 UTC
6 points
0 comments2 min readLW link

Self-Similar­ity Experiment

Dawn Drescher15 Aug 2020 13:19 UTC
12 points
0 comments10 min readLW link

Alle­gory On AI Risk, Game The­ory, and Mithril

James_Miller13 Feb 2017 20:41 UTC
45 points
57 comments3 min readLW link
No comments.