Possibly the main and original inspiration for Yudkowsky’s various musings on what advanced game theories should do (eg. cooperate in the Prisoner’s Dilemma) is a set of essays penned by Douglas Hofstadter (of Godel, Escher, Bach) 1983. Unfortunately, they were not online and only available as part of a dead-tree collection. This is unfortunate. Fortunately the collection is available through the usual pirates as a scan, and I took the liberty of transcribing by hand the relevant essays with images, correcting errors, annotating with links, etc: http://www.gwern.net/docs/1985-hofstadter
The 3 essays:
discuss the Prisoner’s dilemma, the misfortune of defection, what sort of cooperative reasoning would maximize returns in a souped-up Prisoner’s dilemma, and then offers a public contest
finally, Hofstadter gives an extended parable about cooperation in the face of nuclear warfare; it is fortunate for us that it applies to most existential threats as well
I hope you find them educational. I am not 100% confident of the math transcriptions since the original ebook messed some of them up; if you find any apparent mistakes or typos, please leave comments.
Hofstadter’s Superrationality
Possibly the main and original inspiration for Yudkowsky’s various musings on what advanced game theories should do (eg. cooperate in the Prisoner’s Dilemma) is a set of essays penned by Douglas Hofstadter (of Godel, Escher, Bach) 1983. Unfortunately, they were not online and only available as part of a dead-tree collection. This is unfortunate. Fortunately the collection is available through the usual pirates as a scan, and I took the liberty of transcribing by hand the relevant essays with images, correcting errors, annotating with links, etc: http://www.gwern.net/docs/1985-hofstadter
The 3 essays:
discuss the Prisoner’s dilemma, the misfortune of defection, what sort of cooperative reasoning would maximize returns in a souped-up Prisoner’s dilemma, and then offers a public contest
then we learn the results of the contest, and a discussion of ecology and the tragedy of the commons
finally, Hofstadter gives an extended parable about cooperation in the face of nuclear warfare; it is fortunate for us that it applies to most existential threats as well
I hope you find them educational. I am not 100% confident of the math transcriptions since the original ebook messed some of them up; if you find any apparent mistakes or typos, please leave comments.