(Lately, it’s seemed to me that focusing my time on nearer-term / early but post-AGI futures seems better than spending my time discussing ideas like these on the margin, but this may be more of a fact about myself than it is about other people, I’m not sure.)
I endorse Three worlds collide as a fun and insightful read. It states upfront that it does not feature AGI:
This is a story of an impossible outcome, where AI never worked, molecular nanotechnology never worked, biotechnology only sort-of worked; and yet somehow humanity not only survived, but discovered a way to travel Faster-Than-Light: The past’s Future.
Yet, it’s themes are quite relevant for civilization-scale outer alignment.
Some things I’ve found useful for thinking about what the post-AGI future might look like:
Moore’s Law for Everything
Carl Shulman’s podcasts with Dwarkesh (part 1, part 2)
Carl Shulman’s podcasts with 80000hours
Age of Em (or the much shorter review by Scott Alexander)
More philosophical:
Letter from Utopia by Nick Bostrom
Actually possible: thoughts on Utopia by Joe Carlsmith
Entertainment:
Pantheon (TV show)
Do people have recommendations for things to add to the list?
Some more links from the philosophical side that I’ve found myself returning to a lot:
The fun theory sequence
Three worlds collide
(Lately, it’s seemed to me that focusing my time on nearer-term / early but post-AGI futures seems better than spending my time discussing ideas like these on the margin, but this may be more of a fact about myself than it is about other people, I’m not sure.)
I endorse Three worlds collide as a fun and insightful read. It states upfront that it does not feature AGI:
Yet, it’s themes are quite relevant for civilization-scale outer alignment.
Some Wait But Why links on this topic:
https://waitbutwhy.com/2017/04/neuralink.html
https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html
And some books by Kurzweil:
https://www.amazon.com/gp/aw/d/0670033847?ref_=dbs_m_mng_wam_calw_thcv_0&storeType=ebooks
https://www.amazon.com/dp/B08ZJRMWVS/?bestFormat=true&k=the singularity is nearer by ray kurzweil&ref_=nb_sb_ss_w_scx-ent-pd-bk-m-si_de_k0_1_8&crid=X3GZ8HDDAEPI&sprefix=the sing
Holden Karnosfky published a list of “Utopia links” in his blog Cold Takes back in 2021:
I forcefully endorse Chaser 6. I find myself thinking about it about once a month at a rough guess. The rest I haven’t checked.
The Utopia links were motivated by Karnosfky’s previous writings about Utopia in Cold Takes: Why describing Utopia Goes Badly and Visualizing Utopia.