Extended Interview with Zhukeepa on Religion
Introduction from Ben
Zhukeepa is a LessWronger who I respect and whose views I’m interested in. In 2018 he wrote the first broadly successful explication of Paul Christiano’s research ideas for AI alignment, has spent a lot of time interviewing people in AI about their perspectives, and written some more about neuroscience and agent foundations research. He came first in the 2012 US Math Olympiad, and formerly worked on a startup called AlphaSheets that raised many millions of dollars and then got acquihired by Google.
He has also gone around saying (in my opinion) pretty silly-sounding things like he believes in his steelman of the Second Coming of Christ. He also extols the virtues of various psychedelics, and has done a lot of circling and meditation. As a person who thinks most religions are pretty bad for the world and would like to see them die, and thinks many people trick themselves into false insights with spiritual and psychological practices like those Alex has explored, I was interested in knowing what this meant to him and why he was interested in it, and get a better sense of whether there’s any value here or just distraction.
So we sat down for four 2-hour conversations over the course of four weeks, either written or transcribed, and have published them here as an extended LessWrong dialogue.
I think of this as being more of an interview about Zhukeepa’s perspective, with me learning and poking at various parts of it. While I found it interesting throughout, this is a meandering conversation that many may prefer to skip unless they too are especially curious about Zhukeepa’s perspective or have a particular interest in the topics discussed. You can skim through the table of contents on the left to get a sense of the discussion, and also read Zhu’s introductory thoughts immediately below.
Introduction from Alex
Despite the warnings and admonishments against doing so, I’d decided 5 years ago to venture off to the Dangerous Foreign Land of Religion and Spirituality, after becoming convinced that something in that land was crucial for thinking clearly about AI alignment and AI coordination. Since embarking on that journey, I’ve picked up a lot of customs and perspectives that the locals here on LessWrong are highly suspicious of.
A few months ago, I caught up with my old friend Ben Pace on a walk, who expressed a lot of skepticism toward my views, but nevertheless remained kind, patient, respectful, and curious about understanding where I was coming from. He also seemed to have a lot of natural aptitude in making sense of my views. This gave me hope that:
With Ben’s help, I could clarify a lot of my views, and translate a lot of the core insights I’ve picked up into a LessWrong-compatible a ontology
By dialoguing with Ben, I could form gears-level models for – and learn how to make friends with – the parts of the LessWrong memeplex that think I’ve gone off the deep end
… which is what motivated me to begin an extended series of dialogues with him. Below are a couple of excerpts going into some points that I’m particularly glad surfaced over the course of this dialogue:
There’s a thing where the good scientists all know that science as an institution is broken in a bunch of ways. For example, most published studies fail to replicate. But when the median person thinks about science, they don’t think about the opinions of the good scientists who understand how broken the whole thing is. The experience I had being at the Parliament of World Religions was like finding the religious leaders who are the analogues of the good scientists, who were just like, “Yeah, most mainstream religion is totally broken. It sucks for exactly the reasons the skeptics says it sucks.”
[...]
The proximal thing I’m trying to target is creating a mathematical Rosetta Stone for the world religions [...] I think it would be more like “here’s the correct way to interpret these stories, in contrast to these other interpretations that a majority of people currently used to interpret them”. Like, a lot of people interpret hell as like, “Oh yeah, it’s a place where there’s going to be lots of heat, that people will be stuck in for literally infinity years.” That’s just wrong. That’s not making it into the Rosetta Stone.
[...]
I think my picture is basically that the Schelling coalition / the preference synthesis coalition is going to have minimal Moloch internally within it, and that they become powerful enough to determine how the world runs. This is kind of like killing Moloch, and the process by which this all happens is pretty much how I interpret religious prophecies about the end of the world.
I also managed to clarify a couple of my core beliefs over the course of this dialogue, that I’d like to summarize here:
Religious exclusivism (“my religion is the only way, and any other way is bad”), which is what most people associate with the word “religion”, still seems wrong to me. Religious pluralism (“my religion is one path out of many possible paths up the mountain of religious truth“), is the orientation toward religion that seems most interesting to me.
A solution to agent foundations is equivalent to a technical formulation of the religious metaphysics that I believe are convergent across the mystical traditions of the major world religions (Judaism, Christianity, Islam, Buddhism, Hinduism), à la Perennialism, and would in effect amount to a synthesis of the scientific worldview and the worldviews of each of the world religions. I furthermore think that Chris Langan may have already found such a technical formulation.
A steelman of Arbital, that’s able to facilitate a process that converges toward a synthesis of the beliefs and the preferences of its users – eventually extending to the whole world – is crucial to my conception of a positive AI singularity.
A solution to agent foundations is necessary for an un-gameable implementation of such a steelman of Arbital.
A robust solution to global coordination involves robustly defeating Moloch, via the establishment of a decentralized singleton powered by an un-gameable steelman of Arbital.
This process of defeating Moloch and establishing a decentralized singleton is consistent with pluralist interpretations of religious prophecies about the end of the world (like The Second Coming of Christ).
Conversation 1 — April 6th 2024
Alex and Ben had gone on a walk to discuss religion, and decided to continue the discussion over a LessWrong dialogue.
Alex’s steelman of the Second Coming of Christ
Let’s get into it.
So, I’m not sure if your position is:
Classical religions were doing lots of good and meaningful and important things
Or the same as (1) but with the additional claim that they’re the best sources of these meaningful and important things, and so we should all study/follow them
Much more like (1) than like (2), although I would also want to emphasize the bad and mind-killing things in (1). (Not sure exactly what you mean by “classical religions”.)
Some topics I would be interested in exploring:
how AI plays into the (Distributed) Second Coming of Christ(-Consciousness) picture I’d started fleshing out during our walk
the connections between forgiveness and overcoming Moloch
more details around my epistemic status around religions and spirituality (especially the kinds of claims that they’re commonly associated with)
I think I don’t understand (1) (am up for poking at it). I think (2) sounds interesting. They both seem more specific than (3) so I prefer them.
Well, maybe I’ll ask a few initial q’s about 1?
Yes, that sounds good
I think it’s a broader topic that includes (2) as a subcase anyway.
I’m going to forget all the specific lore of these old religions, so remind me whenever it’s relevant, but… I’ll just open the wikipedia page on the second coming of Christ… the main thing I recall is something like “he returns, and then all wrongs are set right, and all people go to heaven or hell”.
<opening wikipedia page>
Right, so the first thing (which maybe you’re already tracking) is that the thing I mean by “Second Coming” is not Christianity-specific, and is more like the common denominator behind the eschatological prophecies found across world religions (I think Islam has the Mahdi, Buddhism has the Maitreya, Hinduism has the Kalki, Judaism has the non-Jesus Messiah).
Just adding some quotes from wikipedia for context:
The Second Coming (sometimes called the Second Advent or the Parousia) is the Christian belief that Jesus Christ will return to Earth after his ascension to Heaven (which is said to have occurred about two thousand years ago). The idea is based on messianic prophecies and is part of most Christian eschatologies. Other faiths have various interpretations of it.
and
Most English versions of the Nicene Creed include the following statements:[citation needed]
...he ascended into heaven and is seated at the right hand of the Father. He will come again in his glory to judge the living and the dead, and his kingdom will have no end. … We look for the resurrection of the dead, and the life of the world to come.
In Islam, Jesus (or Isa; Arabic: عيسى ʿĪsā) is considered to be a Messenger of God and the masih (messiah) who was sent to guide banī isrā′īl (the Israelites) with a new scripture, the Injīl (Gospel).[85] The belief in Jesus (and all other messengers of God) is required in Islam, and a requirement of being a Muslim. However, Muslims believe that Jesus was never crucified or resurrected, instead ascending directly into heaven. Additionally, they do not recognize Jesus as the Son of God, as they believe God has no equals, but rather that he was a prophet.
and
In the Quran, the second coming of Jesus is heralded in Sura Az-Zukhruf as a sign of the Day of Judgment.
And (Jesus) shall be a Sign (for the coming of) the Hour (of Judgment): therefore have no doubt about the (Hour), but follow ye Me: this is a Straight Way. 43:61
(I have generally taken very little interest in religions and was not aware that Jesus appeared as a character in the Islamic holy books.)
(Makes it seem more like fanfic.)
I am pretty sympathetic to religious myths being compared to fanfiction. (I think I will get some flak from religious leaders for this lol.)
A core aspect of this common denominator (as I interpret it) that I’m tracking: the true, good, meaningful, and important things that all the religions are trying to point at become common knowledge in the world, and the global power structures (e.g. the justice systems, defense systems) get restructed to operate in alignment with this thing that’s pointed at, with a corollary that the world operates in a way that’s actually just and peaceful (perhaps not literally 100% so, but true enough to an extent that would lead the global situation then to feel qualitatively different from the current one—kind of like a “phase transition” in which Moloch is mostly vanquished).
This is too vague, you should say more concrete things about what will change / what will be fixed.
Sure, gimme a sec to copy/paste something I’ve written in a Google Doc:
Mass psychological healing
Mass recognition that insatiable materialistic desires (e.g. for money, fame, power, and status) ultimately ground out in desires for love and belonging
Mass recognition of the ethical and metaphysical truths shared across religions
Mass reassessments of history, including mass acknowledgments of historical injustices, according to the standards of these ethical truths (reminiscent of the Last Judgment)
Fading of attachments to tribal identities (like nation, race, and religion)
A new form of governance involving an upgraded form of democracy, in which individuals’ conflicting preferences are synthesized (as in the parable of the blind men and the elephant) to determine how society should be run, including how resources should be distributed
Phasing out of participation in dysfunctional systems (e.g. factory farming and environmental exploitation), and phasing in of participation in systems more aligned with life
Misguided spiritual leaders (“false prophets”) trying to claim unique spiritual authority, and genuine spiritual leaders (the vanguard of the Distributed Second Coming) who recognize and respect each others’ spiritual authority, collaborate very effectively, acknowledge and accept their own and each others’ personal shortcomings, and show compassion to the misguided spiritual leaders
Idk if this is the sort of concreteness that you’re looking for.
I also want to caveat that the type signature of this sort of thing is less like “something that is preordained to definitely happen” and more like “a self-fulfilling prophecy that might happen if enough of humanity believes in it”.
These seem like 8 good things (probably), we could chat about how real/plausible/attainable they are.
I don’t know why you’re talking about religions. I think a lot of irreligious people could write down a list of big things like this that they’d like to solve.
Why is Alex interested in religions in the first place?
Yeah, the bit about why we’re talking about religions seems really important to hash out. Can you name some of your biggest confusions around that?
Well, okay. I’ll frame them as questions for you to answer that would help me understand your perspective.
Why are you reading stories about Muhammad? Why aren’t you telling me stories about Lord of the Rings or of stories by Dostoyevsky or Heinlein? They seem like better stories in many ways.
Why are you pinning your dreams to a phrase connecting Christianity/Islam? These institutions are totally epistemologically borked and their stories are pretty boring to read. Why don’t you just start afresh?
To be clear I find some of the meaning you find in these stories to be real, I don’t object that they’re empty/meaningless.
What are 3 things you’ve personally gotten from the religious perspective on things that you’ve not gotten elsewhere? I think some of the discussion we had of forgiveness was interesting, and I could be led to believe that you find it more tractable to get this discussion from religious groups than from, I dunno, literary book clubs (that sounds fairly plausible now that I write it). It seems plausible to me more examples of “spiritual growth” (ugh I don’t like that phrase) you’ve gained from exploring the religious texts would be in-retrospect the thing that has the most impact upon me.
I suppose I could also start with my motivation for bringing religion in to the picture, which has two main components:
the first is that I think certain kinds of philosophical insight need to be incorporated for these goals to get realized, and I think these philosophical insights are the underpinnings of each of the major world religions. While I don’t necessarily think that any particular religion is likely to be a particularly good gateway for these insights to a randomly chosen person, I do think these insights can be found in the religions if you look in the right places and interpret them in the right ways.
the second is that I think it is in fact possible to meaningfully coordinate with certain religious people—religious pluralists in particular—who are up for “reclaiming religion” into something less obviously shitty/absurd, and jointly crafting a positive vision for the future that taps into powerful tropes that are already embedded in the collective psyches of most of the world population.
I think these philosophical insights are the underpinnings of each of the major world religions
I want to poke at this but I expect you will say things that are too vague (for me), so I am not actually going to try to engage with this directly yet (I’d rather hack away at the edges more first).
the second is that I think it is in fact possible to meaningfully coordinate with certain religious people—religious pluralists in particular—who are up for “reclaiming religion” into something less obviously shitty/absurd, and jointly crafting a positive vision for the future that taps into powerful tropes that are already embedded in the collective psyches of most of the world population.
I think I want to wait on this for now, seems down the road.
What has Alex gotten from religion?
Looking at your four bullet points. Taking a moment to think about it…
OK, for context, I think my real crux for engaging a bunch with spiritual/religious exemplars is that I somehow started taking seriously the hypothesis that letting go of all my resistances, resentments, judgments, etc. (tanha is the technical Buddhist term) toward everything and anything in reality is (1) actually where my CEV points (2) actually an easier way for me to live. I can maybe go into how I started taking that hypothesis seriously. But living according to this hypothesis is… a pretty different way of living from how most people choose to live (it REALLY goes against what mainstream culture recommends, for example), and insofar as I cared about getting better at living my life in this way, I’ve found myself most interested in reading stories about the historical figures who (according to me) have done the best job at living in this way.
I’ve heard some rationalists talk about how they only want to read rationalist fiction upon discovering rationalist fiction, because it just feels so hard to relate to non-rationalist fiction. What’s going with me here feels vaguely analogous—not in the sense that I find non-religious/spiritual stories uninteresting, but insofar as I find myself obsessed with learning how to be a better person in a very particular way, where it seems like the only other people who care about being better in this way are super into religion/spirituality… well, those are the people whose stories I want to read about.
I think I’m going to elaborate on bullet point 4 now (around things that I get from the religious perspective that I don’t see mentioned much elsewhere), besides what the deal is with forgiveness / grace / turning the other cheek:
this is sort of a corollary to the bit about forgiveness we discussed on our walk, but holding myself to the standard of “What would Jesus do?” in all of my close relationships (family, romantic partner, good friends, close collaborators...) has totally changed my paradigm for how I approach relationship conflicts. In particular, I find that there are way more degrees of freedom for navigating relationship conflicts when I try to be utterly uncompromising about disendorsing any place I might feel judgmental/resentful, even if it’s “technically fair” for me to feel those ways, and even if most of mainstream society would back me up on feeling those ways. I think this is still pretty vague and high-level—I’m happy to elaborate further, but I’d like to move on to other points for now.
I think I’m unusually good at both (1) acknowledging my personal shortcomings, (2) forgiving myself for them, partly because of the high standards of forgiveness I hold myself to (3) calling out others’ personal shortcomings in a forgiving way. (According to me, a common misunderstanding of the point of forgiveness is that you “should do it” because “that makes you a good person”, whereas I think the real point is that forgiving others goes hand-in-hand with forgiving yourself. Verbatim from the Lord’s prayer: “And forgive us our debts, as we also have forgiven our debtors.”) Because I am unusually good at these things, I am also unusually good at earning the trust of people I encounter, from a broad array of backgrounds.
The religious perspective gives me a lot more nuance on the trope that “what people ultimately value is love”. I think there’s a sense in which most modern adults are pretty jaded about romance, and in some sense have layers of heartbreak that disconnect from the original innocent and open-hearted love they’d initially felt as children / young teens. I think there’s something of a generalization of this, wherein babies and toddlers are sort of “in love with” their lives, their families, and to some extent all of reality, that they lose touch with in the course of repeatedly getting “heartbroken” as they discover that their naively optimistic perceptions of reality aren’t calibrated. In one sense, I think the “goal” of the spiritual path is to live in a way that embodies both the innocence, lovingness, and open-heartedness of infancy/toddlerhood, while simultaneously embodying the nuance and discernment of adulthood; and one only comes to reach this place by recognizing that a majority of our judgments and resentments and entitlements are defense mechanisms to cover up underlying senses of vulnerability, from having our “hearts broken”.
I’ve also been updating in the direction of “family values are correct values”. There’s one very abstract sense of this, in which there are decision-theoretic grounds for honoring the processes that made you, like your family / your culture / evolution as a whole—by doing so, you’re acausally “causing more of yourself to exist”. There’s another more tangible sense of this for me, which is related to the previous bullet—the more I’ve been able to forgive my family, the more I’ve been able to authentically connect with them in the sorts of ways that I had as a child, which in turn has gotten me in touch with the innocence and open-heartedness that I’d embodied as a child, and the depth of the unconditional love that my family feels for me, to the point where I actively look forward to calling both my parents every week. It’s kind of hard to verbalize the magnitude of what this has been like for me… around grokking that so much of who I’d thought I was, my values and my ambitions, were actually just defense mechanisms because I’d felt heartbroken by my family, and then reconnecting with my family’s unconditional love, and realizing that the love I was looking for had actually been present all along. I’m aware I’m still being pretty vague and unspecific here, and I’m happy to elaborate later, but I’ll leave it as is for now. I’ll just mention that there’s an Islamic saying “heaven lies at the feet of your mother” and I feel like I now grok the kind of truth it’s trying to point at.
the more I’ve been able to authentically connect with them in the sorts of ways that I had as a child, which in turn has gotten me in touch with the innocence and open-heartedness that I’d embodied as a child, and the depth of the unconditional love that my family feels for me
I’d say the relationship I have with my mother (who raised me alone) is the most intense relationship of my life. (I don’t mean dramatic, I mean intensely felt.)
Mm, I can imagine.
I think I’m unusually good at both (1) acknowledging my personal shortcomings, (2) forgiving myself for them, partly because of the high standards of forgiveness I hold myself to (3) calling out in a forgiving way others’ personal shortcomings.
Are you saying that this is a general property of you or one that you gained from engaging with religious stories and thinking?
I think this is a general property of me that followed from me “taking seriously the hypothesis that letting go of all my resistances, resentments, judgments, etc. toward everything and anything in reality is (1) actually where my CEV points (2) actually an easier way for me to live.” I think religious stories are much more interesting when you interpret them from this lens, and I don’t really know how to find them that interesting if you don’t have this lens.
AFAIU, in theory, every Christian and every Muslim is supposed to take this hypothesis seriously, and live by it uncompromisingly. In practice I think this is not really the case, but I think the mythologized portrayals of Jesus and Muhammad are exemplars of people who do live by it uncompromisingly, which is why I find it interesting to read stories about them.
I’d say the relationship I have with my mother (who raised me alone) is the most intense relationship of my life. (I don’t mean dramatic, I mean intensely felt.)
Literally last text from her (an hour ago) reads:
To be clear, my love for you is stronger than the raging & not unpleasant winds (46.3km per hour) that currently rock this little island where I live.
Literally last text from her (an hour ago) reads:
To be clear,my love for you is stronger than the raging & not unpleasant winds (46.3km per hour) that currently rock this little island where I live.
That really warms my heart to read <3
It’s such a good line. And yeah I really like my relationship with her.
That also really warms my heart to hear <3
(As I’m reflecting on what you wrote, it would help me to know if there’s something you’re wanting from me in response.)
Got it. Nothing immediately jumps to mind—I think you asked good and resaonable questions, and mostly I was just trying to answer them.
Resentment, forgiveness, and acceptance
Can you tell me more about how you try to live up to “What would Jesus do?” in your relationships? I’d like to understand that a bit better.
I try to do what’s good and right. And sometimes I ask myself “How would person X respond?” and that gives me ideas I haven’t thought of. But I don’t know how Jesus would respond.
I feel like he would do something unrealistically good and it would work out unrealistically well (e.g. and then I said to my muggers “go back to your fathers and give them all your gold” and then they did and wept and praised me and were saved) but I don’t think this will work out IRL.
In practice, it looks like asking myself: “Am I judging them or resenting them in any way?” If I am, I do not count that as part of my endorsed behavior, and I will try to let go this judgment / resentment, and relate with them from that place.
I try pretty hard to live without resentment, and to relate to people without judgment/prejudice. I don’t know that I’d endorse “it’s always the wrong call with literally no exceptions”.
That’s a huge part of it, at least. I wouldn’t say that’s the totality. (And, in practice, for me asking myself “What would Jesus do?” and “What would Muhammad do?” and “What would the Buddha do?” don’t ever return conflicting answers, but I do sometimes get clearer answers from one question than others.)
I think I would endorse “it’s always the wrong call with literally no exceptions”, which I think is super counterintuitive!! In some sense, I think this is the crux of Jesus’s crucifixion—he was not resenting or judging his crucifiers, and instead praying for them to be forgiven.
Note that this is still consistent with setting boundaries, or otherwise saying that you’re not OK with some things.
My guess is that it would help (me, at least) if you could tell me a time where you used this approach, though it can be hard to pull up specific instances so it’s nbd if it’s more effort than it’s worth rn.
One particularly salient example was when I was navigating some poly drama that I won’t go into. Let’s just say there was a lot of judgment and resentment bubbling up in me, for reasons that third parties agreed were quite fair…
Another particularly salient example is in relating with my parents, who I had previously thought of as emotionally abusive, for which I’d seriously judged them and resented them. My current stance is “they were emotionally abusive by Western standards, but also they were pretty much normal by Chinese standards—see e.g. Battle Hymn of the Tiger Mother by Amy Chua—which in some sense doesn’t mean their behavior was morally OK, but does sort of lead me to feel like it’s weird to judge them for not having been different, and makes it even feel weird for me to label them abusive”. A crucial prerequisite for connecting deeply with my parents was being able to let go of the “emotionally abusive” label—which Western culture was totally backing me on—and forgiving them for the mistakes that they’d had no idea they were making, because they were just doing their best to parent us based on what they’d implicitly learned from their culture and their parents, who in turn did the same things based on what they’d implicitly learned from their culture and their parents, etc.
(Responding to your first example)
In these situations a common heuristic I use is “How do I act in this situation so as to keep open the possibility of a strong and close relationship in the future?”
This can include:
Not getting worked up over the current conflict, because it could easily be seen (by me) as relatively minor in the scheme of things later
Making sure to be up front and open with them about issues that I have, and stand up for myself, so that later on I’ll be able to honestly move towards them in friendship / alliance
Not say things in private that would cause them to be unable to trust me (if they later learned what I said)
Nonetheless it’s not never that a situation occurs where I’m like “I can see no concrete way forward to have a strong and close relationship in the future, and it’s best to just dissociate / move away.” I wish I could do better and suspect it’s a skill issue. Even then I aim to harbor no resentment or ill-will.
A crucial prereq for connecting deeply with my parents was being able to let go of the “emotionally abusive” label—which Western culture was totally backing me on
Sounds like it was a pretty good move
I also think “I can see no concrete way forward to have a strong and close relationship in the future, and it’s best to just dissociate / move away” can sometimes be the right call. I think the crucial piece is aiming to harbor no resentment or ill-will. (It’s plausible to me that you’re already holding yourself to pretty similar standards to the ones I’m holding myself to, which is partly why I’d remarked that I thought you had a high spiritual IQ :P)
I personally taboo ‘abuse’ and ‘trauma’ and a lot of other words that seem to me overloaded, and too load-bearing for people’s ethical attitudes.
(I keep a list of such words that I aim to always taboo.)
Sounds like it was a pretty good move.
Yes, it was an extremely rewarding one!
(I feel like I’m making too many assumptions too quickly with this next sentence, feel free to say it doesn’t make sense)
I assume the relationship with your parents/family wasn’t “fixed”, and didn’t “become the relationship you had previously wanted”, but it became a good relationship with the people who were actually there? Like, the best version of the relationship that was possible?
That does resonate. I would sort of say that it got “fixed”, but it definitely didn’t turn into the relationship that I’d previously wanted, and did more become a good relationship with the people who were actually there (both me honoring who my parents actually are, and them honoring who I actually am).
Our relationships certainly aren’t perfect now, but I would say that the core resentments that I’d had since a teenager that I’d felt hopeless about ever resolving for most of my post-teenage years have in fact gotten resolved.
Even then I aim to harbor no resentment or ill-will.
One more semi-related note on this: I generally have a low opinion of people who become actively antagonistic with people they were formerly close with. I recall reading Person A text my friend Person B asking them to repair some damage in their relationship, saying something like “well you don’t want to unnecessarily make enemies” and I was kind of appalled that in their world the default thing that happens when friendships breakdown is that you have ‘enemies’ rather than ‘there is no longer any ongoing relationship’.
both me honoring who my parents actually are, and them honoring who I actually am
Sometimes people say this and I kind of get it but I’m not sure what it caches out to. What does it mean to “honor” who your parents actually are?
I generally have a low opinion of people who become actively antagonistic with people they were formerly close with.
Maybe as a datapoint around how I hold myself to my religious / spiritual standards: upon reading this, the thoughts that came up for me were “gee, it must suck to be those people—it must be lonely and scary for them to have those expectations around their friendships” and “I fucking hate that too, screw those people” and “hmm, I appear to harbor some resentment for them, and will probably feel like I am screwing up for doing so… I aim to forgive them, and also to forgive myself for this resentment”. Lol
I agree, it is probably scary to be those people. My guess is that their lives are more dramatic than mine though, which is kind of enjoyable from some perspectives.
(Also it kinda sucks to be around those people!)
Sometimes people say this and I kind of get it but I’m not sure what it caches out to. What does it mean to “honor” who your parents actually are?
Hmm, something like, there’s a way your parents Actually Are, In The Actual Territory, which was super frustrating to me as a child because they were supposed to be Perfect People (TM), not Who They Actually Are… and then being like, “wait, they were never supposed to have been Perfect People (TM) in the first place, that was just a childish fantasy, they’re just normal people like myself, why don’t I start interacting with them based on Who They Actually Are, rather than my projections of them?”
I think there’s also a flip side around parents seeing their children for Who They Actually Are, In The Actual Territory, rather than as a vessel in which for them to project all their hopes and dreams and whatever.
Well, I actually want to argue with this a bit.
There’s a great thing you describe, which is seeing the substance instead of the symbol you believed. “Oh, these people aren’t the personification of safety and love and home that I believed they were when I was 6, they’re actually an English teacher and a plumber who occasionally read books and occasionally fight though love each other.”
At the same time, sometimes people do make claim to be the symbol I see them as, and should be held accountable for failing at it.
...on reflection my guess is that the error typically goes this way consistently with parents, but I think sometimes people are like “I am a politician and will represent you” or “I am a journalist and will speak truth to power” or “I am a scientist and I will publish true results” and they totally suck and don’t do that and then you find out and it’s not quite right to go “Ah, well, I was holding them to my ideals of who a politician/journalist/scientist is, after all they’re just people” and it does make sense to be like “you betrayed these ideals and you are in debt to me/society”.
According to me, in that case, the person that they actually are is someone who is failing to be a person they are claiming to be, who should be held accountable for that. (Part of my improved relationships with my parents has been around having more candid conversations with both of them around their shortcomings, as well as vice-versa.)
Alright, our time’s up, let’s wrap up here.
Seeya next week :D
Yup, seeya. This was great. Also happy to respond async before then if there are further things you’d like to ask.
Conversation 2 – April 13th 2024
At the next meeting we recorded audio and had it transcribed, and then lightly edited. This was 2 hours, and Ben was quite underslept.
What do you want to talk about? What is on this chain train?
I feel like picking up where we left off could be a good place to start.
Yeah, I could open that and re-read, skim it, and jog my memory.
What Alex gets from religious stories
I remembered you asking something like why should we be thinking of any of this in terms of religion? It seems like there are a bunch of nice things that you want that seem good and cool and stuff, but why bring religion into picture? And then we talked about some particular ways I was interpreting religion and how I was interpreting religious stories through that lens and why I found the religious stories interesting. I think that’s pretty much where we left off last time.
Yeah. What do I think? So I think there are two things.
One, you can get lots of things from different stories, lots of different sorts of characters and archetypes and virtues and vices. But something about these stories is optimized for communicating something like, I don’t know, maximal goodness is probably wrong, but is really with the intention of storing the ideal virtues and so on.
And then the second thing is there is a whole sort of institution built around trying to find meaning in life and trying to expect the good out of these stories. And that institution is not built around other stories by a factor of a thousand or a hundred or something.
And so both of these things are attractive about these stories relative to other stories like Lord of the Rings or Heinlein.
That was a pretty quick gloss. I’m interested in how much that felt like it pointed that the things that you think versus how the details wrong or totally missed some things you think or you just rather say-
I think there’s a particular thesis for what the good is that I think I find in religious stories that I tend to not find much elsewhere. And when I do find them elsewhere, it’s often the case that the authors think highly of religion in some particular way. I think I somewhat articulated this thesis for what the good is as endorsing living in a way where you let go of all judgments and resentments and resistances, something in that ballpark, which is not super precise and part of what we could try to do here is make that more precise but...
I mean, I feel like I pretty regularly aspire to this sort of unattachment that the Buddhist stuff and talk about and I feel like came up in the story you were telling about, with our chat about Jesus.
Sorry, can you say that last part again? It came up in the Jesus story too?
I mean just him on the cross just being unwavering in communicate that ‘I forgive you all in my heart’. And you’re saying that’s a level of detachment of unattachment to hatred or adversarialness or so forth that is surprising and rare and interesting.
But I don’t think other stories are like ‘no, all our characters are attached and we think it’s good’. I think lots of other stories also have characters that are aspiring to not hold onto hatred and anger and bitterness. He’s not the only such character in storydom.
I agree that they’re not, the prophets aren’t the only characters who are trying to live up to those standards. I do think they’re interesting as exemplars of people who are successfully living up to these standards, at least when you read them the right ways. I think a lot of fundamentalist Muslims do not read the story of Muhammad in the right way, for example.
Right. It’s plausible we should try to do some comparative literature. Perhaps you can take a story you like and I take a story and we spoke about what yours has got the good things in it and mine doesn’t and then I defend it. But also it’s possible you should say more about the things that you get, the nutrients you’re finding in the religious stories. Maybe just tell me another story you haven’t told me.
I could. But I feel like there’s a crux for our worldview somewhere, and I don’t have the sense that that’s where the crux is going to lie.
Okay.
Orientation toward death
I’m trying to go into my sense of where the crux actually lies.
I think a big part of it has to do around the orientation toward death, where as a special case of resisting nothing, death is something to not resist—which, again, doesn’t mean I think you should just roll over and not give a shit about life, but I think it does imply that the “death is the enemy that must be vanquished” perspective is sort of off-base somehow. I’m still pro-life extension research, and I’m still pro-medicine, but...
I mean, I think most of history I should have been like:
“Okay, I woke up. I’m 18, I’m an adult, I’m going to die. Everyone’s going to die. There’s nothing I can really do about it. It’s time to grieve that now, and then live in such a way that I focus on the positive things I can change in the world that will last and will have longer effects than my life. And not particularly be attached my own legacy or whatever.”
That seems like the approach I would try to take.
There’s dozens of things that I would give my life for that seem worth it. I’m not especially like ‘Oh, but my life it’s so important’, and I think there are probably other characters of fiction who can live this way. I can think of one I think in Worth the Candle who doesn’t give a damn about their own life, just following on of the tradition and goodness. So I think my guess is I think there’s a lot of characters that aspire to be unattached to their fear of death.
Yeah. First thought is, that sounds right to me. Second thought is I think the spiritual traditions and the mystical traditions of the religions aren’t just about aspiring to have that orientation toward death, but actually doing the work to let go of a bunch of the fear.
Yeah. I have fewer threads on how to do the work, so to speak, or have a harder time pointing to like ‘Hey, you’re a person who’s scared of death, here’s the seven things you do, and at the end of it you’ll be freed’. I don’t know how to say that to someone.
And maybe the religious people have got a better handle on that. Probably Jesus would say some things. He would say ‘go into the town and give both your shoes to the first person you see’ or ‘go and tell your mother you love her’. You’ll tell you seven things and at the end of it you’ll be freed from your fear of death. And then the characters in the bible did it and then they were freed, and it was a great story. So maybe he’s better at that than me.
So Buddhist enlightenment is often described as dying before you die. Obtaining enlightenment and overcoming the fear of death by seeing how the fear of death is actually predicated on certain confusions, I think are kind of the same thing. And on my understanding, that’s basically what Buddhism is about. And I think there are analogues of this in the mystical traditions of other religions.
I mean, to be clear, I wouldn’t be like “and I have achieved this”. I’d be like, “I occasionally notice that it would be good to achieve and that I aspire to it and that I don’t endorse not having achieved it”. And I try not to allow myself to be the sort of person that acts afraid of death, but sure I could be doing better on it.
Yeah, cool.
I remember one time a friend of mine was telling me about a great meditation experience they had. I was asking them why they decided to have kids and they told me that they’d gone to a Buddhist retreat and had a meditation on death for half an hour. It was about walking consciously through the experience after you’re dead, of what people do with your body, and putting your body into the ground. And as your body dissipates, I think feeling that in your body. I don’t know, I can’t remember exactly how the conversation went, but after that they said that somehow as a result of that, they were like, oh yeah, there’s just a couple of things I really care about. And one of them was having kids. And that changed it for them. And then I think, she found the recording of the meditation on death, and we listened to it together.
And I think it was a good experience getting in touch with that world. I think I don’t really live in that world where I will be dead very much. I don’t really work for it or work around it or orient into it or think about it very concretely. And I think it was upsetting at the time. I think it made me want to connect out with people I’d not seen in a long time, I think. Yeah, I can’t remember. I felt some different things after it that were helpful. I might just text her and see if I can get a link to it. Pretty interesting. [Yeah, here’s the link to it.]
What useful things do old religious stories have to say about how to live life well today?
Okay, I’m still struggling to find our worldview difference because I feel like it sure exists. It definitely feels like a smaller worldview difference than I have talking with a bunch of other LessWrongers—related to my point about you having a high spiritual IQ—but it still feels like there is some worldview difference somewhere. I’m going to take another stab at it.
Well, I have multiple different thoughts.
The first one came to my mind is a bit rude, but I’ll go ahead anyway. I don’t know exactly what you’ve been up to in the last five years, but it doesn’t seem like it’s directly going to prevent us all from dying. Whereas I’ve read HPMOR and aspired to think about the most important things and work on them, and I’ve been trying to prevent extinction of humanity for the last five years as my primary goal, and building better epistemic infrastructure with that as a key motive. So I don’t know that your stories have helped you more than my stories have helped me. Anyway, that was a rude thing I want to say.
Let me give relatively weaker objections just because I have them and it seems good to mention them. I feel like the stories were written a few thousand years ago and there are people around today who know way more about human nature. You just get to do a bunch of surveys. You’d ask questions like Aella does, like do you cheat on your partner? How many years have you been together? And you just get to see the trend after ~15 years when 30-40% of monogamous relationships start becoming unfaithful. Before then they could be like being faithful just works and it’s good. And broadly had way less of the relevant data and understand. So probably their stories are pretty dumb.
I have so much more information about how humans work today. I also understand evolutionary psychology. The religious leaders might’ve said ‘anger is a demon that is sent to hurt us’. But I think anger often has a good game theoretic justification and it’s often worth letting yourself grow angry when someone screws you over, and acting on it. The religious stories probably had some good stuff in it, but surely it is the case is you could do better now if you tried.
Better on the axis of understanding human CEV?
No, better on the axis of how to live well as a human. I mean morally.
Yeah. My first pass is that the specific concrete recommendations for how to live well, that the religions were built around, I think were solidly good for the time and cultures in which they arose, but then get overgeneralized. And in general if you’re like, hey, it seems like this religion recommends this particular way of relating to food or sex or whatever, and this was the best way of doing it, I would probably not agree with the religion about that.
I don’t know which parts you want to save. I keep being like, well, not this bit. And you’re like, of course not that bit. So I’m like, which bits?
I mean, being good to your family I think is a pretty timeless bit.
Hey, my mum should have cut her family out many years before she did. Some of them mistreated her and stole money from her.
Right. I mean, Jesus also says to cut off your family sometimes. There’s a lot of nuance in the whole thing. I think the thing where Muhammad said “heaven is at the feet of your mother” captures a pretty timeless human psychological thing. And it doesn’t imply that-
Almost until when the ems come, when we become ems, we’ll probably make changes.
Sure.
But I agree that whilst we’re still humans, it is pretty profound.
Cool. I think there is stuff around sex that religions understand pretty well that get overgeneralized and overly rigidified in ways that I think are lame. But I think there is something “sacred” about sex that actually does get honored in religions that don’t get talked about much in modern culture. In particular, I think there’s something “sacred” around having children, and evolution gave us sex to have children, which is what the whole deal with sex was originally. But the associations between sex and the sacredness of having kids, and all the implications of those associations, are rarely emphasized in modern culture.
Well also, we invented contraception and now the connection between the two is meaningfully severed in a bunch of pragmatic ways.
Agreed. And I think a bunch of the cases against sex that were present at the time the religions got formed are much weaker now.
All the diseases and the kid, the child-rearing...
Yeah. I’m with you there. And still… if you’re in a culture where 90% of the associations with sex are totally decoupled from having kids, I think that actually has bad consequences for the culture. I think there’s something that’s just pretty bad about hookup culture, in terms of people’s long-term happiness and fulfillment, and certainly for having families.
That is the kind of thing that I think religions are more tuned into—not that I think the super rigid recommendations from religions are necessarily better than hookup culture.
Yeah, yeah.
But zooming out a bit, I think it’s useful to clarify that when I say religions have figured out how humans should live or something, I don’t mean in the sense of like, yeah, the Jews have these laws, and these laws are human CEV. And the Quran says you should do this at such-and-such time, and blah blah. I’m also like, no, that’s also --
Yeah, yeah, yeah. I just want you to argue to me what’s good about it, and tell me the thing that I’m missing.
For some reason, the thing I’m now wanting to bring up is that I think the thing that’s good about religions is also present in a lot of spiritually developed indigenous cultures and traditions. I don’t think the good thing is uniquely found in religions, and I’m saying this mostly to clarify the type signature of the kinds of things I’m saying in favor of religion.
Nonetheless, would the best place to look for it is in some of the stories about the religious prophets?
I think the prophets are particularly good exemplars for a lot of what the good stuff is. And also, I think the good stuff is meaningfully present in numerous indigenous spiritual traditions as well, even before they were touched by any of the major world religions.
I think I would like… I don’t know. Maybe we should compare one of your prophets to the Comet King from Unsong who’s sort of-
I haven’t read Unsong.
Let me just check. I think I have this blog post. It’s got some relevant quotes.
I bet none of the Unsong characters would forgive their crucifiers while getting tortured to death.
I’d be interested in you, maybe, I don’t think it’s quite again exactly the thing you want, but I think I would be interested in you reading some of the quotes from Unsong, about the Comet King. All right. Go on, ask me a question first.
Forgiving yourself, and devoting your life to something
When we were on our walk, I mentioned something about how part of the deal with Christ is grokking that every aspect of you can actually be accepted and forgiven. Even the parts that you’ve buried so deep inside you because you’re too afraid to acknowledge their existence, because it just feels like there’s no way they could possibly be accepted or forgiven.
By the way, I think one of the things that I find dissatisfying about all the religious exemplars is I feel like the people who are most into it are the people who do nothing else in their life, other than sit and forgive people or whatever, and that seems kind of utterly pointless. I think, probably, as you’re trying to imagine people, pick people who actually do anything in the world using these tools.
For what it’s worth, I think Martin Luther King and Gandhi are examples of people who had done things in the world inspired by religious exemplars.
I would agree with that. I more meant the Buddhist who just meditates their whole life, but-
I see.
Or even the priests, I guess. But maybe they do things, maybe I’m giving them a short shift. Sorry, I interrupted you. Please say that again, I was interested.
I would say that part of the deal with Christ was grokking that every aspect of your being could be forgiven. And I remember you making a remark in the form, like, “Oh, I could imagine why someone might devote their whole life to this, if they understood this fully, or felt this fully.”
Yes.
That feels like a pretty good inroad to worldview-bridging. That thing that someone might devote their whole life to gets at the core of what seems good to me about religion.
I mean, it’d be very good if everyone did that. That would be great.
Everyone did what?
That would be a worthy thing. Everyone came to fully, actually believe in forgiving themselves, and that they could, and that others would, and that it was okay to forgive your own self, all the different parts of it. That would be very different than the current world, and it would be better than the current world, is my guess.
Cool. Before you continue, can I just add another crucial piece of that picture in my head? Forgiving the self and forgiving others go hand in hand for Christianity, and in general.
Yeah. It just seems like… I don’t know, it feels like there’ll be a lot less friction in doing anything good, in making things better. That’s probably it. I don’t really know how to describe it.
Cool. This feels very juicy in my belly, by the way.
What? Which bit? What? I don’t know what...
Oh, just the line of thinking you’re going down, that we’re exploring right now.
Cool. I didn’t quite understand why, but that’s fine. Anyway.
A world in which everyone is actually living in this way gets pretty close to the core of the thing I’m trying to gesture at when I say “Second Coming of Christ”.
Changing the minds of all of humanity / steel-Arbital
Yeah. I don’t know. Go take one person and make them have this experience. Cool.
Coming to me, being like, “Here’s the plan I’m going to make 8 billion people have this experience.”
Eh, the point I’m making is too weak. I’ll have to think of a better argument.
All right, so we’ve engaged with the steelman of this first. I don’t know. I think also… if I was like, “How do I do this?” Step one would be to improve the epistemology, and to make sure that words mean things, and to build LessWrong.
I would still be like, “Look, I need to tell everyone what’s available and communicate to them.” You know what’s messed up? Words don’t mean things anymore, so when you say things, people go, “Ah, you’re on the tribe of those people who say those words.” I’m like, “No, no, no, I meant something important.” And they go, “Oh, you’re the tribe of people who insist that they’re not playing language games—”
Okay, this feels really juicy to me, because I largely agree, actually. My current conception of the good thing that happens with AI is pretty much a steelman of Arbital.
Yeah, Arbital would be great. Maybe you should build it.
I think… Have you heard of UpTrust? Jordan Allen’s attempts to do something like Arbital?
Someone mentioned that word to me yesterday, or today, and I remarked that it would be a funny 3rd axis of LW voting. Upvoting, agree-voting, and up-trusting? Anyway, that was a dumb joke. Someone mentioned UpTrust, but I didn’t know what it was.
Okay. I used to work with him on it. We had a co-founder split, and we’re still friends, but talking with him about the idea did actually help shape my thinking around what a steelman of Arbital might look like. The steelman of what Jordan’s trying to do, and the steelman of Arbital, pretty much point to the same thing in my head.
The thing that Jordan is tracking way more than I think the original Arbital idea was tracking, is that it’s kind of misleading to say there is simply one objective truth, and everything that isn’t that is just wrong. Different perspectives can have different kinds of truths, but that also doesn’t mean that all of them are equally valid.
In other words, there’s a question about how to think about truth in a way that honors perspectivalism, while also not devolving into relativism. And the way Jordan and I were thinking about this, was to have each filter bubble—with their own standards of judgment for what’s true and what’s good—to be fed the best content from the other filter bubbles by the standards from within each filter bubble, rather than the worst content, which is more like what we see with social media today.
And if this were to happen, the end result would basically be a process of coherence of the filter bubbles, and mutual understanding, and common knowledge actually being possible of what’s true and what matters.
I once again want to say that I’ve been trying something remotely at all in this problem area with LessWrong, and it has been very difficult, and not like, you know...
I think a robust solution to this is as hard as the AI alignment problem. I think LessWrong is doing a very good job for a particular filter bubble.
But even within it, I think there are a lot of people with very different standards, and I don’t think we’re doing a great job of… We’re just like, “You’re all in the same room, deal with it.”
So, yes. So, it would be good if people’s perspectives were made coherent, and their standards were made coherent, and they were able to collectively think within those perspectives, and then those perspectives were encouraged to interact with each other in a reciprocal and cooperative way, as opposed to a maximally antagonistic way. As such has happened on places like Reddit and Twitter and Facebook.
Yes.
Those all seem like laudable goals. We could… I forget how that’s relevant.
I was trying to say where I agreed around reason on the internet being possible at all, being pretty central to what I think of as the core issue.
I mean, I think… I like believing in honest communication, as being able to be the key ingredient to… I’m trying to think of the right religious term. A savior or something, or… No, that’s the Bible. Anyway, you know, things working out great.
Yes, yes.
And the thing you’ll do is say to people “By the way, forgive yourselves. It works. Look, I did it. Here’s the story of a guy who did it. You believe him.” And then they’ll go “Ah, wow, I could do that.” Great. That’s a nice idea. I would be up for trying telling people that one if we got to the place where people believed words meant things.
I also have other ideas, like, for God’s sake, it’s all prediction markets. Just legalize them, everything will go better. A few other things like that. But I don’t know. I do think that the religious one-
Can I just name the central place the religion stuff comes into this picture for me?
Okay.
Totally with you about believing in an honest communication. I think there’s trickiness because the word “honest” is pretty loaded. I think the meta-context for the conversational container, in order for the whole thing to actually end up being robust, needs to be forgiving of what seems like dishonest communication.
What did you just say? You said that it won’t seem honest to everyone, so everyone needs to have a high level of forgiveness in those situations?
That’s not a bad summary.
What is the thing? What’s a better summary?
In order for the communication context to stay robust, without just decaying or devolving, people need to forgive what they see as dishonest communication.
They need to be able to.
Under the nuanced version of forgiveness. Yes.
Not expect to be taken advantage of, and to believe that the person is trying to act well, even if they sometimes really screw up and hurt someone? Maybe you disagree with that last clause?
Nope, I don’t disagree with that last clause.
Yeah… something about this conversation’s annoying. I can’t figure out what it is.
Let’s try to figure out what it is.
I don’t know. It’s good of you to offer, but I think I just at least want to note it to myself so I can remember it later. I don’t know. Mostly, probably, just that we’re moving through a lot of big, difficult, and, vague topics. Not vague, but topics that are hard to be precise about quite quickly, and that’s not how I normally interact.
But that’s probably okay… I don’t know. I think I feel a bit worried that somehow I’m… I think I feel a bit like I’m not saying anything worthwhile or interesting, and you’re probably disappointed in the conversation, but that doesn’t seem to match up with the things you said.
Yeah, that’s not my experience of you. I usually find it helpful to talk with people who aren’t in “Alex’s crazyland”, and see which things land and which things don’t.
Good. You have some dreams of how to save humanity. I don’t know, I think I maybe get your things a bit… But actually do think that they would help a lot. Everyone’s lives do seem kind of… Sorry, I’m not quite supposed to say that sentence, but I was going to say, everyone’s lives do seem kind of meaningless, and they could just not be. That’s not quite what I think, but I do feel like everyone could be a bit more full of magic if they wanted to be.
Yeah, and I’ll also reflect back that the thing that would be the “savior” being infrastructure for good discourse, I’m also super on the same wavelength with you about.
So I’m sorry I keep bouncing off and being slow for some reason and maybe it’s because the topic is emotional and difficult. I’m not exactly sure why. So anyway, I have some thoughts, but I don’t know what’s up anyway.
That’s fine.
How Alex thinks about changing the world
So I feel like the worldview crux is less interesting than us talking pragmatically about how to supposedly cause your steelman of the Second Coming of Christ to come about. It seems like you have a thing that you want to do, and I like some parts of it and see enough of it that I...
It’s more relevant to chat about how to do it, but I kind of feel like the structures we have for talking about plans are lame and will suck, but it’s probably worth trying. And then I’ll notice how they suck and then I can change what we’re talking about.
Yeah, that sounds good.
Whenever effective altruists talk about plans for changing the world, I have a feeling like everything they say is a waste of time. That’s not literally true, but it feels true and it kind of is true, anyway, and so I kind of feel like you and I don’t have scripts for talking about how to change the world and so we’re going to try and I’m going to be super triggered, but it’s not quite true, but kind of by all the ways we talk. But we can try and then I can point out.
I like this direction.
Yes.
Okay. First, I don’t think of myself as trying to save the world. I think there are forces in the world that are leading it to try to kill itself, and there are forces in the world that are trying to save the world. And I see myself as trying to do my parts to differentially amplify the forces of the world that are leading it to save itself. But I’m not thinking of myself as a world savior. I’m thinking of myself as a guy doing his part.
Yeah. That’s good.
And so I think what I want to then do is describe what I think it means for me to do my part, with the background context that, in my background models, the way the world gets saved is via enough people doing their parts that the world can actually come together and land in a totally new stable equilibrium.
There’s the technical front and the coordination fronts, which maybe isn’t a good decomposition of how I’m thinking about things. It’s kind of a carryover from how I used to think about the AI problem. There’s a technical AI alignment problem and there’s the AI coordination problem.
The coordination front looks something like reconciling the world religions, and atheism, via a common shared vision of the future that we can all build toward together. And the reason why I mention messianic prophecies is because those are positive visions of the future that are already embedded in large swaths of the global population, which I think also have interpretations that are mutually compatible.
Oh, I think that’s a third reason to be interested in the stories of religion in so far as everyone’s already heard of them to some extent.
What were the first two reasons?
There were written in order to portray people aspiring to maximum virtue and there are institutions built around trying to find meaning and goodness in these stories in particular, which there are not for Unsong and HPMOR and Lord of the Rings.
I see.
But the third one being everyone knows the stories, which is pretty… That might be the most important one.
That feels pretty lumped with the second one in how I think about it.
It doesn’t to me, I think there are cults, there’s probably fandom, maybe I can’t think of a good example. It seems plausible to me that are sub-communities that are really interested in certain stories and finding meaning in them. But the rest of the world’s not got it. So much less worth investing in. But anyway, not important or not very important.
Coordinating with the rest of the world sounds annoying
I do think that if this kind of reconciliation happens, that would be a big deal in the world, and whatever’s at the locus of this reconciliation would be an obvious Schelling direction for what humanity should do.
You’re reminding me of the rest of the world and I do find it annoying how… I’m going to say a bad version of the sentence. I do hate how stupid most people are. That’s not quite right. That’s not very-
Something to forgive!
What did you say?
That’s something I’ve been learning to forgive! How stupid most people are.
Hehe. I think I haven’t really looked at it. I think to forgive it, I first have to look right at it and I think I hadn’t been looking at it for most time. I was like, “Oh, most people are probably fine or smart”, or I just didn’t think about them. And now I’m looking at them more and going, “Oh dear,” and I’m seeing all the problems. I can only forgive after I have really come to understand it.
Yeah, that sounds right.
Otherwise, it’s going to keep hurting me and then I’ll get bitter again. I’d be like, what the hell? And then I won’t expect it and that’s painful. Anyway, just to be clear, I don’t dislike people for being stupid, but it’s very depressing and upsetting when I see how stupid they can be.
It’s very normal.
Actually, sorry, it seems likely to me that most of the smart, nerdy-ish people experience this. I don’t think I’ve chatted with them about it. I’ve been pretty bubbled myself. Just everyone around me is at least a standard deviation above average, if not two or three in terms of IQ and I forget about the rest of the world.
Yeah.
I forget that there’s so many people that are standard deviation below average. I can’t imagine it. It seems uncomfortable. I would not like it.
I know what you mean. Are you having an experience of being like, wait, the median SAT math score score is 500—
Oh, 800.
The median SAT math score out of 200 to 800, is 500.
Yeah.
When I realized this, I felt shocked about my difficulties in empathizing with the median human.
It must really suck to be so disconnected from very simple tools for understanding the world like quick multiplication and order of magnitude estimates.
Yeah. I have also made some friends with some low IQ people, and that’s also been good for my soul because I really liked them and I learned a lot from them. Like how to be more relaxed, and how to savor life.
I’m sorry I cut you off.… You were in the middle of something and I lost the attention. You were saying “I want to assimilate the world religions and atheism into some better vision of the future for everyone.”
Something more like, produce an intellectual foundation that can unite or reconcile at least the world religions and science and articulate a positive vision of the future that’s consistent with the existing positive visions of the future that are present in their eschatologies.
This seems interesting to chat about for a while, how to do that. Before we do that or instead of doing that for now, I don’t know. Anyway, you also were like there’s a technical component. Do you want to just say a sentence about that?
Yeah, which is actually coming up with the intellectual foundation.
And also figuring out how to build tools that actually enable the kind of robustly good discourse that could save the world. I think of this robustly good discourse as a meta-tool that will differentially empower the world-saving forces in the world over the Molochian, suicidal forces of the world, and I think they are very non-trivial technical challenges in building the infrastructure for this discourse.
Yeah. I think, wait, I should say, I’m sorry, I’m tired, so I’m a bit quick to be annoyed or something in a way that’s not about you or anything. Sorry, the second one again was you want to build better tools for good academic discourse and maybe you’ll be able to talk about these ideas.
Oh, no, it was also you wanted to build the-
The intellectual foundation for the coordination, which I think will also be the intellectual foundations for building the tools for the good discourse.
Yeah. I don’t buy that those two are that related.
Yeah, there is a link there. I can make the argument for them being related.
Maybe religious pluralists are sane?
No, I think the thing I’m more interested in, supposedly we should pause and come return to it in a week or something, is… The first thing you said was you would like to unite the religions and atheism and these sort of underlying stories and archetypes and virtues and so on.
Yeah.
I dunno, it kind of feels tasty to just try and do that together for a bit.
Cool.
Or it would be fun to do it or it would be meaningful to try.
Can you tell me something you’ve maybe been thinking about there about how to… any particular thread of the religious stories and atheism that you been thinking about how to reconcile?
I would like to share that a big part of my current thinking around this came from attending the Parliament of World Religions, which is pretty much the main conference about reconciling religions.
Where is it? When’s it ran? Who runs it?
I’m not sure who runs it. I went last August in Chicago. I think it’s once every two years or so these days.
How many people are you talking, a few hundred, a 1000?
Like 10,000-ish I think.
Oh, that’s quite big. Oh, I forgot. This is religions. There are 8 billion people, so it’s quite easy for 10,000 people to show up to a thing.
Yeah. And all the big official events were vanilla and didn’t feel like they were really saying anything. It felt like they were saying “yay! unity of religions!” in a way that seemed kind of lame and empty. But in the back rooms there were lots of religious leaders who I had really, really good conversations with about reconciling religions.
One thing I learned was the concepts of religious exclusivism versus religious inclusivism versus religious pluralism. Exclusivism is like, “my way is the only way and anything that isn’t my way is bad”, which is most Christianity and Islam. I think when most people hear “religion”, they associate it with religious exclusivism, because that is in fact how most religious people think about their religions.
Religious pluralism is the diametric opposite of that, which says “my religion is one path out of many possible paths up the mountain of religious truth”. I talked to a Catholic priest at the Parliament who’d said something like, “there was a girl I know who started out Christian, but had a really bad experience with Christianity, but then found something spiritually appealing in a Hindu temple, and I encouraged her to find God in Hinduism”, or something like that.
I feel like this is nice. I feel like my sense is the religions don’t really endorse this. The Pope won’t say this and the Imams won’t say this.
So I met imams and priests who said this at the Parliament. There was a room for Catholics, and the first thing I saw in their room was a pamphlet which said “Free Quran reading”. And I grilled the people there about whether that was heretical, and they were just very straightforwardly like, “No, this is not heresy. This is not a mainline consensus view among Catholics, but our views here are actually strictly within Catholic doctrine.”
Sorry. By free Quran reading you mean come to a session and we’ll read the Quran to you and teach you about it?
I think so. Yeah. And it was right next to the Muslim room.
I am confused by this. I feel like the institutions of the religions couldn’t put up with this shit. I’m confused by it. All the books don’t they say, “I shall have no other God before me but me,” or whatever?
Yeah. But the God might be the same God across all the religions. There’s a thing where when Jesus is like, “I am the way and the truth and the life”, some people interpret that as “the kind of consciousness that Jesus embodies, which some other people like the Buddha also embody, is the way, the truth, and the life”. And others are like, “No, it’s specifically Jesus himself, and exclusively so, and anyone who says otherwise is wrong.”
I know in so far as this is true, it’s very strange and important, but I don’t currently believe it, but it is.
I can tell you some more facts in this direction that I can just send you links about.
Yeah, that seems good.
A few months ago, the Vatican released a statement calling Jesus and the Buddha great healers.
What is happening? Are these people losing their edge?
A lot of hardline or Catholics think so. They’re like, “This Pope is the antipope.”
Didn’t they want to kill all the Muslims and the Jews? Why are they saying that they’re great healers?
Because religion pluralism is a thing. The pulse I got from talking to some of the pluralists there—it’s definitely a biased perspective—but the pulse I got was like, exclusivism is waning, it’s losing popularity very fast, and pluralism is on the rise.
No, I understand that. I just want to… Is it just the case that all the hardcore Catholics are like exclusivists and all the lukewarm Catholics are pluralists or other hardcore people who are also hardcore pluralists? Is it just a straight correlation with how strongly you’re committed to your-
The Pope seems pretty open to pluralist views, it seems.
Look, it’s very surprising and interesting. I don’t quite know what I make of it yet.
I want to give an articulation of what this really felt like for me.
<bathroom break>
The conversation’s really annoying.
Don’t worry about it.
I don’t want to, I don’t know what’s up with it? I was like, oh no, we’re not still talking about God, are we? This has been going on forever and sucks.
So I was going to give you an analogy. There’s a thing where the good scientists all know that science as an institution is broken in a bunch of ways. For example, most published studies fail to replicate. Most published studies-
Yeah. It’s awful.
Yeah. But when the median person thinks about science, they don’t think about the opinions of the good scientists who understand how broken the whole thing is.
I know, I know.
Yeah. The experience I had being at the Parliament of World Religions was like finding the religious leaders who are the analogues of the good scientists, who were just like, “Yeah, most mainstream religion is totally broken. It sucks for exactly the reasons the skeptics says it sucks.” And I was like, “Oh, okay.” And then suddenly-
That’s a crazy analogy. That’d be pretty interesting if true.
Yes.
I was telling one of the Catholic priests there about my experience of Jesus during an ayahuasca ceremony and he was just like, “I don’t know what ayahuasca is, but the story you told sounds super legit and you are super lucky to have had that experience at such a young age. I’ve only had this experience after decades and decades of going deep into Catholicism and all the rites and rituals. All the doctrines of Catholicism are really about having that kind of experience. And you just had it directly.” And another one who heard it was just like, “Whatever you’re doing, Alex, keep doing it. It sounds like you’re on the right track.” And I was just like, “That feels great.”
And I’ve also befriended an imam who’s spent a decade living in Mecca. He is on the World Council of Muslims for Interfaith Relations, and he knows a bunch of the head imams in the main mosques in Saudi Arabia. And he’d heard about Nick Bostrom, and one time we were just talking about superintelligence and he was like, “Yeah, if an aligned superintelligence gets built that could be the messianic figure of Islamic prophecy, the Mahdi.” He said that to me before I said anything of that sort with him. He is a very smart guy.
Why do you say he is a very smart guy?
Because he is. He has a high IQ.
Right. Cool.
So these are the kinds of people I have in mind when I’m thinking about religious leaders from the world of religion I want to be allied with, in terms of reconciling the religions.
Yeah, it’s a nice idea. As I say I feel very… Anyway, you want to reconcile them. I still, I’m not quite, what would it look like? What’s a property of the world that would be different if you had succeeded in your goal? How do you know if you successfully reconciled them?
A mathematical synthesis of religious metaphysics?
The proximal thing I’m trying to target is creating a mathematical Rosetta Stone for the world religions.
Okay. I guess I thought you wanted to make a new religion that was better, but now it sounds like you’re like, “No, we just should be able to work together.”
What this “new religion” would be is just taking all the good correct parts of all the old ones and making them precise, so I don’t really think of it as a new religion.
Sure. But you’ve got to write new books.
Maybe. I think there are lots of people doing that already.
I don’t understand. I asked if you were saying “Come together imams and priests, quit your current jobs, we’re going to make you priests of a new religion that’s not got a bunch of metaphysical claims in the books that are false, and has stories that are actually appropriate for the current civilization, and so forth.” It sounds like you’re saying “No,” so I just don’t know what you would say “Yes” to.
Let me say one concrete thing I want to say “Yes” to, which is being able to ground the apparently contradictory metaphysical claims across religions into a single mathematical framework.
Look, a bunch of the metaphysical claims are there just to give people stuff to memorize and signal tribal allegiance. Not all are attempting to be deep truths about the human spirit.
Yeah, I think that stuff would largely not end up in the math. But stuff like, is there reincarnation, or is there heaven/hell? I think these concepts are just under-specified, and I think most people’s intuitive conceptions of these are wrong. I think they can be good metaphors for pointing at something deeper and more nuanced and complex, and the thing I want is a formalization of the thing that’s more nuanced and complex.
I don’t know why you need anything formal.
Because I think there’s a there there, and the best way to talk about it is formally. When the religions talk about afterlife stuff, I think there’s a there there.
I understand that… I don’t know, the formal thing feels like a red herring to me. Like, maybe Scott Garrabrant one day will come to us and be like, “Oh yeah, here’s some math, how to think about the death of your agent and the values continuing in the world.” I would go, “Thank you, Scott Garrabrant.” But I didn’t need it to be able to get the right attitude towards the world. It helps, but I didn’t need it.
I don’t know that it’s strictly necessary either, but I think it would help a lot in thinking through things more clearly (similarly to how FDT helps me think about decisions more clearly, for example), and in providing bridges between religions.
But also, I am annoyed again. Is there heaven and hell, or is there reincarnation? These are very different. Concretely, they’re very different stories of the world. They can’t both be the case.
And so my story is that if you ask, is an electron more like a particle, or is it more like a wave? The math kind of just answers this question. And if you don’t have the math, if you just try to debate it in English, particle and wave seem like they imply pretty different views of what’s there. But when we actually get precise about what’s meant by wave or particle, it’s like, “Oh, it’s just both, in this weird way that’s hard to understand when you don’t have the math.” And that’s pretty much my sense of what the deal is with reincarnation vs. heaven/hell.
So, let me point to something. Often I will make a decision that is locally costly in order to signal how I will behave in future counterfactual worlds. I will not take a bad deal, I will instead take no deal, so that people know they can’t take advantage of me and force me to accept bad deals. It hurts to take no deal right now, but then in the future people will evaluate the counterfactuals and see they’ll be better off giving me a fair deal than an unfair deal.
So you have these conversations about what would happen in worlds that are not this world, and that’s why you’re making the choices you are in this world. And it can sound kind of mystical or confusing, to be making choices based on what happens in worlds that don’t exist in our physics and will never happen. And so sometimes you might tell people different made-up stories or approximations that sound a bit mystical in order to get them to make the right decision. You might tell someone that whenever they accept an unfair deal it hurts their mothers’ heart in heaven, and they don’t want to do that, so they shouldn’t accept unfair deals, and they won’t, and that will go well for them, but the reason is just that their counterparties do counterfactual analysis and notice that a bad deal means no deal.
That’s not an amazing example but it gets the point across. There are lots of stories you might tell in order to approximate the math, and your religious stories might be like that. “What we tell everyone is that in the future, Christ will return and all judgments will be set right to wrong.” Or we tell everyone, “You’re going to live a life repeatedly, and each life, it’ll be as good or as bad as you were in the last one.” And this causes you to behave in a way that you would want to, on reflection, but it’s just harder to give the actual reasons.
And then you might tell the people it is actually true, because people are too stupid to take nuanced specific stories. This is what I want you to be saying, and not bullshitting with statements like “Ah, yes, they’re all true in their own way.”
I’m on board with there being different stories that aren’t ultimately true, that are all trying to point to the same core truth thing, that the math can let us talk about precisely.
I would be annoyed if, at the end, your Rosetta Stone says “They’re all true, and here’s how they’re all reflecting some deeper truth or something.” They’re not all true. They’re all false.
I think it would be more like “here’s the correct way to interpret these stories, in contrast to these other interpretations that a majority of people currently used to interpret them”. Like, a lot of people interpret hell as like, “Oh yeah, it’s a place where there’s going to be lots of heat, that people will be stuck in for literally infinity years.” That’s just wrong. That’s not making it into the Rosetta Stone.
So if you worked on this reconciliation for the next year, or you’d made efforts to cause it to happen… I don’t exactly know how you view your role in causing it to happen, or would be something you’d be satisfied with having happened over this next year, as a result of things you did. Or, what things could you do that you’d be satisfied with over the next year?
For this one, for the nearer term, I’m picturing stuff like hosting retreats with AI alignment people and interfaith leaders.
But it doesn’t seem to me like it’s got anything to do in the short term with alignment research, do you mean, I don’t–
Except to the extent that there are alignment researchers who think the right way to think about alignment involves the thing that’s shared behind all the religions.
I don’t really get it. They don’t seem directly connected to me. It seems like it would plausibly be good to reconcile all the religions, and give people truth and virtue and meaning to aspire to correctly with good stories. It seems to me like a different project than the AI alignment project.
I think solving agent foundations is a lot like–
Solving agent foundations would help with everything. I understand. But I still don’t actually think that Scott Garrabrant talking to some imams is going to help him make a better sub-agent category theory.
I do think Scott Garrabrant talking to me, who has talked to imams and Chris Langan, does help him.
That may be true, but I make the relatively strong claim that you are not a central example of the imams and whoever the other guys are.
Yeah. I do just also want to mention that my current model of Chris Langan has a solution to agent foundations in his head.
Okay, sorry, he’s not an imam, he’s a weird, okay. I don’t know enough about him. I didn’t read that post, I’m sorry.
The Rosetta Stone that I’m gesturing at, I think Chris has a version of that in his head. And so–
So how old’s the guy again?
Early 70s.
He’s not fully dead yet.
I hope he stays alive for as long as possible.
Scott Sumner and Russ Roberts about 69. It’s quite old. I didn’t know how old they were. I know Russ Roberts stays sharp for a man of that age.
Chris is definitely still very sharp.
Where does he live in the world? What does he do with his time?
Missouri? I’m not sure what he does with his time, but one of the things I’m also working on is helping more people understand what Chris is trying to say.
Okay. Maybe we should start coming up for air. We’re scheduled to wrap up in the next 10 minutes.
Yeah.
Anyway, I’d be interested in talking more about how to reconcile. It’s currently more interesting to me, so I’d be up for that if you wanted to do more of that. If you wanted to do some of that.
Yeah, I think also talking about the technical side, and the relevance for civilization having functional discourse. Functional discourse is also something I would like to flesh out more with you.
Yeah, okay. Sounds good.
Conversation 3 — April 20th, 2024
We continued our dialogue over Zoom on April 20th 2024. The audio was also transcribed and lightly edited.
On the nature of evil
Religious prophets vs the Comet King on the problem of evil
How are you doing?
I’m good.
Before we pick any particular direction to go, let’s enumerate possible directions.
Before we do that, can you tell me what how you would compare the religious prophets and the Comet King? I want to know.
(Spoilers for UNSONG below.)
Okay. I think the Comet King is baffled by the problem of evil and saw evil as something to destroy. I think he resists evil.
And I think part of what I found interesting was at the end of Unsong, there’s this thing about how Thamiel actually has all along been doing God’s will. And everything that looked like evil actually was done with noble intentions or something, that I found… it got me thinking that Scott Alexander might be wiser than the Comet King character he wrote.
But my general sense was that I felt like Scott and the Comet King were sort of grasping around. Scott was pointing in the right direction for how to grapple with the problem of evil, but there were some things he was missing that I think Jesus and Muhammad and the Buddha understood.
Wait. So first, why did you think he was wiser than the Comet King character?
Well, I think he was wiser than the Comet King, at least when the Comet King tried to destroy hell directly, in that at the end of Unsong, by the end… If I recall correctly, part of what happened was when all the minds were emerging, Thamiel revealed himself to be an instrument of God all along.
And something like, all the evil actions that were done under Thamiel’s name in some sense had noble intentions behind them—or, in other words, was in some sense innocent. And that feels like a key part of the problem of evil… I think the prophets get that, and I think the Comet King didn’t get that, at least when he was trying to destroy hell.
Yeah. I don’t know. It seems a bit naive to me. I can see some noble intentions behind it, but I also can see a perspective where Scott is implicitly not getting that some people just want to be evil or something.
Evil is like cancer, maybe
In my current model, I think evil is a lot like cancer.
So, Mike Levin has done some research where he figured out how to take cells that were slightly cancerous, and basically going off and doing their own thing separate from the rest of what the other cells in the organism were doing. And he found a way to reconnect the cancerous cell with the rest of the cells in the organism, and then the cell stopped being cancerous. (2:26:00 − 2:26:35)
Sorry. You said that they could disconnect the cells from the other cancerous cells, and then reconnect and the cells are healed?
Mike has done research on bioelectricity and organisms, which he basically recognizes as a software layer above the hardware level of DNA that modulates a lot of the activity within organisms. And what cancerous cells do is they sort of disconnect from the main bioelectric network of the organism. And he’s managed to get pre-cancerous cells and slightly cancerous cells reconnected to the main bioelectric network, and in so doing, gotten them to stop becoming cancerous.
That’s interesting.
I agree. I think this also has parallels to when people have traumas that they don’t recognize as traumas and just think of as being who they are, which is my current model of where evil comes from. And that trauma that they’re endorsing as who they are, rather than as a trauma, sort of takes on a life of its own, which seems structurally similar to a cancer. And the healing of that trauma, by reconsolidating the emotional memory via forging a connection through the mental mountain separating it from the rest of your psyche, seems structurally analogous to reconnecting the cancer to the rest of the organism’s bioelectric network.
I recently came up with a new frightening thought, or, one of the ways I can empathize with people who seem to be wicked and cruel.
Suppose someone hurts you. It does not get righted and no one really notices it, but you notice it and you’re tracking it. From your perspective you think “I have learned, that if you’re a person like me in situation A, the counterparty’s response was to do X, and X hurts me, but the harm doesn’t get repaid”.
And then situation A’ comes up where you’re in the opposite position, a situation that seems to you structurally similar and your counterparty thinks “Well, I assume you’ll be nice and friendly to me.” But you’re like, “Well, you don’t get it. The thing that happens to you in situation A is that X happens to you. And no one cared about it when it happened to me. And so I’m going to behave the same, and I’ll show you that this is what happens, so that you’re not naive. If you would like, we could fight together to make sure the harm is accounted for across the world whenever situation A comes up, but that seems very ambitious, and in the meantime I’m going to continue to show you how this situation currently works, and so I’ll take an action that hurts you and I will not apologize for it. This isn’t me hurting you, it’s me teaching you how the world works.” But then from the other guy’s perspective, you were just was randomly cruel and nasty to them.” They’re like, “Wow, that guy is just randomly cruel and nasty.”
The reason I find this frightening is that, insofar as this is a common behavioral pattern, whenever a good norm is broken and not punished, the person hurt may start to propagate the norm breaking themselves, and the next people will too, and quickly norms may dissolve.
Yeah, yeah. This is basically what I… the thing you described feels like a pretty central example of the kind of thing I’m pointing at with trauma that you endorse as just how things are or something, as opposed to viewing of the trauma.
I guess for me, the thing that’s weird about it is it comes from this noble kind of feeling. It comes from a desire to set things straight, where you’re trying to track what’s good and what’s right and wrong, and every person and institution around you seems to not be tracking it. So around you, things seems to have gone wrong, and you’re trying to set things right and teach people the rules of the game, but you’re doing it in a fairly...
Misguided way.
...well, what’s misguided about it? Somehow it’s a bit zero-sum or something, but I can’t figure out why. I don’t understand it. I’d have to spend some more time to properly articulate it.
You might also find people who are like you, and it may feel like you click with them, because you both share an internalized norm and can connect on that level and not get mad at each other about it.
How does that relate to doing evil things or cruel things? Are you saying we all might agree to a certain cruel norm?
I mean that there’s something contextual about the nature of evaluating what’s misguided. For example, I think a bunch of Chinese norms seem pretty antisocial by western standards, and I think some many western norms seem pretty antisocial by Chinese standards.
But I think that’s just a difference of protocols. If you interpret these actions in the other protocols, then you’ll get the wrong results, but we pick these protocols and we think they’re a good idea. And then there’s–
Yeah. I think also, part of what I’m saying is that judgments of evil are protocol-relative.
You mean, I should not judge a Chinese person’s actions the same way as I judge an American’s person actions, even if they’re the same actions because they have different background protocol assumptions. That was what I was hearing you say?
More like the judgments of some person’s actions as evil are relative to a particular protocol they’re operating from. And there are some things that are pretty universally considered evil like murder, but I would just say that most human protocols count that as evil.
Sure. Well, wrong and evil are different.
But yeah, another frame I have is… Jordan Peterson had this good point about bringing up kids where he was like, “You can bring up your kids poorly, and prepare them for the world poorly, by giving them an overly fine and easy environment.” The point is to prepare them for the world that they’re going to arrive in rather than to be like, “Everything is fine. Don’t worry about a thing until you become 18.” Insofar as there is something that will normally be punished at in the world, it is actively helpful to this new person to give them negative feedback for the thing, so that they form accurate expectations. Maybe not punish them as hard as the world would, but make sure that they anticipate that they will get a negative reward.
And similarly it reminds me a bit of the book series ‘A Series of Unfortunate Events’ that also has a similar philosophy about kids, which is not that you can protect them from the world, but you have to prepare them for it.
I feel like there’s a similar impulse when someone is going around doing a lot of evil things, which is something like, “Well, this is how the world works and I’m teaching you. You keep going around being trusting and kind, and let me tell you, bad things are going to happen to you and I’m going to show you that bad things are going to happen to you as a result, so that you learn.” And then the other person is like, “You just made the world worse. You are the cause of the world being worse.” And they’re like, “Well no, I think the world is worse and I’m just teaching you,” as I said.
Yeah, that resonates a lot.
Yeah. And I think that connects with like, there’s often a naivety that more evil people have towards good people or something. ‘You’re all counting on the world being nice and fairies and kittens, but definitely people are going to steal your stuff and rape you. So, you idiots. Don’t judge me.’ Anyway, I just care about this a lot because I’ve been trying to figure out what evil is.
Mistake vs conflict theory on cancer / evil
So, I was drawing an analogy between evil and cancer.
You were also saying you felt that somewhat like the Comet King didn’t understand evil, as well as the prophets did.
Yeah. There’s sort of taking the conflict theory view of cancer, where the cancer is just in conflict with the rest of the organism and there’s sort of a mistake theory view of it, where actually, the thing the cancer wants is to reintegrate with the rest of the organism.
Utter idiot.
But yeah, it doesn’t know how to.
Well, it ‘doesn’t know how to’ as hard as you can fail at ‘not knowing how to’ — in that it kills the rest of the organism. It’s such an idiot.
Yeah. And I think the mistake theory view of evil, I think, is pretty central to all the deal about forgiveness. And yeah, I think the Comet King sort of had a conflict theory view of evil when he was trying to destroy hell.
Can you give an example of how one of the prophets understood evil better?
Yeah. I think Jesus on the cross being like, “Forgive them, for they know not what they do”… it’s like, yeah, even the evil that results in me getting crucified, I’m still going to take the mistake theory view of it.
I mean, the reason you would want to not do that is because it makes it so meaningless. It’s just so annoying, you’re being… It’s like, “No, this person’s evil and this is a big fight of good against evil and we need to kill them,” versus “They’re just screwed up and now I’m going to get killed and I’m going to get tortured and every good thing that could have happened isn’t going to happen.” It just feels more meaningless when it’s a mistake rather than a fight.
Okay. Yeah, I think there are more datapoints in steel-Islam, for how to navigate the fact that even though cancers might be mistakes, you sometimes maybe still need to kill them anyway.
Yeah.
The general vibe Muhammad took toward people acting toward him in evil ways was sort of trying to see through it, and connect with the person “behind the evil” who knew not what they did.
It seems like an ambitious high variance strategy, but I hope it works out. I’m sure it works out sometimes.
I mean, in the charitable accounts of his success, he did not always employ this for the people who were trying to kill him and his army.
Yeah, yeah, yeah. But if he was talking to them, he would probably try to talk to the person behind them.
Yeah. So one particular anecdote about Muhammad that for me, captures the heart of Islam for me actually, is a story where he was resting under a tree and one of the polytheists who hated him was like, “Oh, this is my chance to kill him.” So he goes up to Muhammad and raises a sword over Muhammad’s head and says, “Who will protect you now?” And then Muhammad with complete calm and confidence responds, “God will protect me.” And this confidence shook the polytheist so much that he trembled in fear and dropped his sword.
Basically, what Muhammad is saying is, “My God is more powerful than your gods. And if you fuck with me, you’re going to regret it, because my God is more powerful than yours.” And he basically transmitted this vibe in his interaction with his would-be would be assailant. So the assailant dropped the sword, and then Muhammad picks up the sword, and then holds it over his head and says, “Who will protect you now?” And the polytheist says, “Nobody. This is it. I’m done for.” And then Muhammad drops the sword and says, “Wrong, the same God will protect you too.”
Okay.
And then he converts.
...do you want to go out and try and convert some people together with a sword?
Well, it would require total confidence that I would be protected from someone holding a sword above my head, which I don’t have. There are stories of people being outnumbered three to one, but then scaring of all of their would-be assailants with their body language. I could not do that to someone threatening to murder me.
No. I mean, maybe you could. I don’t really know what would happen. Those are weird situations. But I didn’t like that he said, “My God will.” I feel like I wish he’d said… I don’t know, this is a bad answer, but the concept of goodness will, which you and I share. I don’t know if a person would do that.
Well, they didn’t share it. I think back then, God was the best language they had for talking about a lot of these things. The ontologies were just super, super different.
What with?
The ontologies of the people in Arabia around the year 600 are very different from ours.
I didn’t like that you said, “My God will.” What were you connecting that to that we were talking about?
The prophets seeing through evil.
Oh. So how do you talk to someone through evil? I understand that’s supposed to be an example, but it was a very low detail example. It was only one phrase. You have to find some part of them that’s not… I was not convinced. I don’t know.
It sort of views them in a mistake theory way, where they’re actually just mistaken about the actions they endorse doing and they just haven’t recognized that yet.
There’s still like a, which part of them hasn’t recognized it? There’s some belief about what they’re made up of in their mind or in their soul.
The part of them that endorses acting in a way, in an evil way.
Look, you can take me to some random evil person, I don’t know, like a terrorist or something. And we can connect over parts of the world that are not about that part of the world. We could probably cook together and eat a meal and hang out in nature, and there would be some connection there. But how do you connect… I don’t know how to… It’s not quite the right phrase, but tempt them away from evil.
They have to decide for themselves, but you can help facilitate that process, is sort of how I think about it.
I don’t quite get how to do it without knowing them very in detail.
Forgiving evil in ourselves
Okay. I think the easiest sub-case to focus on is the evil in ourselves.
All right.
I think generally, anytime we’re judging ourselves and thinking to ourselves, we deserve not to be forgiven for something, we’re taking a conflict theory view of some evil part of ourselves, rather than a mistake theory view. And I think of the forgiveness and self-acceptance as a shift toward the mistake theory view of why we did the bad things that we’ve done.
I wish this story had more detail to it. Do you have any other stories of talking to people, talking past the evil of them? I feel like I just want some more detail. Or of you talking past it to yourself I guess, is probably also good.
Yeah. I’ve done things in romantic relationships that I considered unforgivable. They’re pretty bad. I’m not going to go into them right now. And for a while, I just thought I was a bad person. And I basically came to realize that those parts of me don’t just exist in a vacuum. They largely came from my mom having treated me in ways that were like that while I was a teenager.
And then I had a phase of blaming my mom for treating me that way for a while like, “Yeah, I’m a bad person, but that’s because you’re a bad person too, mom.” But then I realized that she was just doing the best she could and she really had no intention to hurt me. And part of her doing the best she could was her mom treating her basically that same… She would tell me stories of her life as a kid in China, and her mom would be treating her in a similar kind of way.
And then there was a way in which I saw the patterns that we were enacting as impersonal, and just the way things were in some sense, and not deep reflections of worthiness of love or of punishment. And once I took that stance, I was able to shift the frame on myself towards “oh yeah, I’m a person who’s capable of doing some pretty bad things because of his trauma, and I should be careful about that, and heal the trauma” and away from “I am a bad person who does bad things to people.” I was also able to extend that same attitude toward my mom and her mom.
What’s the difference between… So sometimes you’ve wronged someone and you owe them some debt, and you don’t really notice or care. Then you realize at some point you go, “Oh, dear. I wronged them. I owe them some debt. I can’t fix this. I can’t right this wrong.” Not all things can just be undone. It’s not how lots of things work. The world is just worse now. And you could make some effort to fix it or not to fix it, or to send a credible signal that you would fix it if you could.
I think that’s part of what an apology is is to say, “I recognize that I’m in your debt and I would fix it if I could. I don’t know how to, but I want to register that this is the state of things.” And then once it’s firmly in the accounting, the counterparty can choose to forgive the debt (or not). What’s the difference between the thing I said, realizing that you would make it right if you could and the thing you said, which seems different, where it seemed like you already would’ve fixed it if you could have and then something changed where you forgave yourself?
Sorry. What’s the thing I would do if I could, like repay the debt?
Or undo the hurt you had caused.
For me, that’s part of the process of forgiving myself. If I want to correct the mistake, that means living in alignment with being the kind of person who doesn’t make that kind of mistake in the first place, and also tries to make amends with the people who he has made those mistakes with.
But how much were you not that before, and then how much were you that after you changed your perspective on yourself?
In this particular case, I think I was pretty willing to make restitutions, if that’s the right way of saying it, pretty soon. But that didn’t feel sufficient. The harder bit for me was like, “Oh, I have a part that’s capable of doing this kind of evil and I don’t trust myself to not do this kind of evil in the future.”
I think it was only when I took an orientation of self-forgiveness that I was able to stop judging myself for the evil while still being realistic that I’m going to make these kinds of mistakes in the future, but letting that be okay while also actually taking the steps to be better at not making that mistake in the future. When I’m making myself wrong, it’s a lot harder to find the root cause of why I made that mistake in the first place.
Oh sorry, say that last bit again, what you did–
When I’m making myself wrong for the evil, it’s much harder to find out what the root cause of the evil is in the first place.
When you’re marking yourself as wrong or you’re castigating yourself?
Yeah, when I’m castigating myself for my wrongdoing.
Insofar as you believe you can’t change and then you just try and punish yourself every time.
Right. When I allowed myself to take the stance that my evil knew not what it was doing, then there was room for getting curious for what… the evil knows not and what it does. Then there was room in me to get curious about what kinds of mistakes I was making that led me to take the evil actions in the first place.
It sounds like the relevant thing was something like a belief that you could change?
That feels like a core piece, but it feels downstream of allowing myself to even look at the evil in the first place rather than just thinking of it as something to exterminate
I do hold, as a pretty high virtue, a virtue of being able to look at any part of reality. I think, as far as you are doing a bunch of all things, I do hold it as pretty high that you need to be able to look at that with a certain level of lightness and just be like, “Hey, I wonder what’s happening over there. That’s interesting. What happens if I do that? Oh, it does this.” It seems important to me.
Yeah, I think basically humans have trapped priors that make it very hard for them to look at evil for what it actually is, without trying to castigate it. I think that’s a core thing.
Definitely people have a difficult time with self-reflection and self-awareness.
Did you want to get something you wanted? I was getting things I wanted, but I think you wanted something slightly different.
Does mistake theory toward sociopaths really make sense?
Cool. I mean, I’d wanted to enumerate possible topics to talk about. Now that we’re on this thread, I think it’s good to keep going down it. Yeah, so there are some things about the nature of evil that I think my steelmen of the religions get better than the Comet King.
I think I agree. When you were like “forgive them, for they know not what they do”, I didn’t see that in the Comet King. I felt like he had something more of a fairly childish notion of ethics or morality.
I would have called it standard rather than childish.
Well, I feel like he couldn’t empathize. I don’t know if I was right. I don’t know if this is right, but I feel like he couldn’t empathize with evil people. And I always think being unable to empathize is not a virtue.
Got it. I do want to make the caveat that I’ve only read the excerpts on Paul Christiano’s blog post plus a couple of pages in the actual book and so I’m going off of-
It’s been a bunch of years since I read it, so I was also kind of going off of those, I’d have to re-read it to have a more confident stance.
Okay, cool.
I think the thing that’s happening right now is I’m just annoyed by evil and would like to understand it and so I’m just flailing about in this conversation hoping that at the end of it I will.
Yeah. What about it feels most annoying to you right now?
Well, it’s just that it’s out there and I don’t know how to engage with it or think about it and, I don’t know, I thought I might be able to as a result of some of the things you said. I thought I had some threads but I don’t know about what’s annoying me about it? I know.
Feynman says you should always have a dozen big open problems in the back of your mind and whenever you hear a good technique you try and apply it to all of them.
Yeah.
Anyway, you said something that made me feel like “Oh, maybe I can finally understand it,” I can’t remember what you said. Because I just keep wandering around trying to because it’s always metaphysically weird when you talk about good and evil. Everyone’s like, “That’s not a thing, is it?”
Yes.
It’s not like an element, not like a carbon or something.
I think part of the crux of it for me is taking a mistake theory rather than conflict theory view of evil, which doesn’t imply that I’m not going to get into conflicts with evil. Sometimes I don’t have any means at my disposal to communicate that there’s a mistake. But also that leads me to take a pretty different orientation toward evil. It’s easier for me to empathize. Rather than thinking of them as just fundamentally bad people, if I instead think of them as good people who made a mistake somewhere and are now in this hellhole relative to how I operate… as you were saying earlier, there’s this way they’re just being unkind to others that they just think is normal.
It seems like a helpful fiction to imagine everyone’s a good person just trying and failing. But I don’t know if it’s true.
I mean I think it’s true on the level that cancers are mistaken. It’s like cancers are mistaken. Also, sometimes you should zap them with chemo because we don’t know how to correct those mistakes yet. I feel like one concrete difference between the conflict versus mistake view of evil is that on the mistake view of evil, when you feel safe from the evil, there’s room for you to turn the other cheek and then that helps plant the seed for the evil to realize its mistake.
So, I was trying to emphathize with choosing to hurt people… and playing the role of evil in their lives… and I’m trying to figure out what mistake you might make to get there. But I think you can just be that the whole time. Can you not just terminally value hurting people?
I think that’s anti-natural.
Why? There’s people who like torturing animals and there’s people around who we won’t ever be able to fix and just go around hurting people all their days. There are sociopaths and–
I don’t know about whether we’ll ever be able to fix them. We certainly don’t have remotely the technology now to be able to help them.
I also think they tend to be pretty unkind to themselves and miserable in a particular way,
But they also don’t care about it as much. “Be kind to yourself,” and they go, “No.” I don’t really know what they say. Perhaps they say, “That’s not what I’m interested in. You’d like it if I was kind to myself, wouldn’t you? Watch how unkind I can be to myself,” he says cutting off his own finger. I don’t know.
There’s another way things can be. You can come to just pretty much learn the heuristic that hurting people is often a pretty good thing according to you or a thing to move towards. There’s a bunch of reasons to think that, a bunch of ways that people can get to that heuristic, a bunch of conflict situations where they practice that heuristic and it goes well for them.
Okay. I think a fairly concrete example for me is I’ve met a bunch of pathological liars who were raised in Eastern Europe, where my sense is that during the Soviet era, pathological lying was rampant, and it was an extremely common survival strategy.
Yeah, that’s messed up.
One part of me is disappointed that these kinds of people are able to exist at all. I’m like, “Wouldn’t it be nicer if that strategy caused your civilization to crumble immediately?” And they’re like, “Nah, the people lasted for 200 years” or something. I just like truth a lot, I wish it was obvious and easily the best strategy.
Yeah, when I meet these people, my first thought is like, “Holy shit, I hate that you’re lying to me and I can’t trust you. Fuck you.”
And the second thought is like, “Oh wait, you were raised in a culture where this was normalized and in some sense there’s nothing personal about this.”
And the third thought is like, “Holy shit, you’re fucking lying all the time. That sounds fucking terrible. You can’t trust your own beliefs, other people can’t trust you and you can’t trust other people because you think they’re also all kind of lying to you.”
Compared to how I live, that sounds like a hellhole. But they don’t recognize that, and there’s nothing you can do to force them to recognize that. They have to make the decision for themselves to stop lying and tell the truth. There are things you can do to help facilitate that decision, but ultimately it rests on them.
The mistake theory aspect comes from thinking of them as not fundamentally being liars at their core, but instead something they learned in the course of growing up.
I feel like it’s possible that the first person they were was a liar and they never were not that and you can be like, “Well, I can hope that you might change into a different person.” But I don’t know that they ever were.
They weren’t liars when they were babies. At some point-
Maybe it’s not that they got put into a rough culture and then they became a liar. Maybe they were just a liar from the start.
I think liar from the start can be a thing as well. But even then, my story wouldn’t be like, “Oh, it’s because they were acculturated into that.” It would be like, “Oh, you developed in a very unfortunate way”. It’s like if you have a birth defect that causes you to not have a limb, but instead it’s a “birth defect” that gives you the false belief that it’s better to live your life as a liar.
But just as a question of are they a good person with bad parts clumped on top or are they just a bunch of good and bad parts and they could probably become a different person. You seem to be a bit essentialist about, “No, there’s a good person there and they just have some issues,” whereas I’m like, “Here’s a person and here’s some of their properties. We would like to change which person they are because this person utterly sucks.”
Mistake versus conflict theory are both valid self-fulfilling prophecies one could use to model someone, and I think I’m less being an essentialist about people being intrinsically good, and more insisting on mistake theory being a self-fulfilling prophecy that wins more, in that I think it gets you strictly more degrees of freedom in how you can interact with them.
Conflict theory kind of defaults to an “eye for an eye” view. Mistake theory leaves room for this when you’re not in a position of strength, but also leaves room for you to turn the other cheek when you’re in a position of strength, which can help them recognize their mistake.
I want to rescue conflict theory from your criticisms.
Okay.
I’m not sure, but I think forgiveness is more relevant foe a conflict theorist than a mistake theorist. I think the goal should be to be a conflict theorist who can forgive your enemies rather than a mistake theorist who can forgive your enemies and it’s much–
Okay. I think of forgiveness as going from conflict theory to mistake theory.
No, there’s conflicts and you should be the sort of person who can forgive people you’re in conflicts with, even if it doesn’t fix anything and even if you’re still in conflicts with them afterwards.
That makes sense, but seems consistent with my views. I don’t think mistake theory necessarily implies fixing anything (although it leaves more room for things to be fixed), nor do I think it implies not being in conflicts with them afterwards (although it does leave more room for conflicts to get resolved).
I can grant that some pathological liars didn’t pick it up culturally, and kind of just have fucked up genetics. But I still prefer to view them as a person who would ultimately actually prefer truth but took a wrong turn at some point, than as someone who is just fundamentally in their core a pathological liar. I think, with the technologies available today, me viewing them as a mistake theorist does not lead to substantially different actions from me viewing them as a conflict theorist, except perhaps that I might be more attuned to the suffering they’re going through for having such a deeply entrenched false belief about the best way to live their life.
Was this coming down from a discussion of who understands evil better, Unsong or the prophets?
Yeah. Then you were like, “I’m still annoyed about evil somewhere,” and then I just started going to my mistake versus conflict theory models.
I think you said that you think that the prophets had some better understanding of evil than the Comet King and that was why you found their stories better. Is that what you said?
Those were one of the ways, yeah. I like them more than the Comet King.
Or you find their stories more useful to learn from or whatever.
Yeah.
I also wanted to bring up HPMOR!Voldemort as an example of someone who’s pretty far down the evil category, an evil sort-of-rationalist who is relatively quite self-aware about their evil nature. Harry has a reflective conversation with him toward the end of HPMOR and was scared by how confident Voldemort was that he would end up evil. Voldemort was like, “You’re right. I will now start to consider strategies where I’m kind to people. That will be a better way of getting my goals sometimes.” But Voldemort was also like, “At the end of it, I’ll still be evil and kill people because it’s just the way I want to be.” I think that’s something people can choose.
And I feel like that’s just a thing in the world that you can get to… like, orthogonality and so forth. I think you can get to an entity that’s just like, “Yeah, I decided I kept self modifying towards being evil and it kept being better every time and now this is where I’m at and I’m in an attractor state and any move away from this is locally not good and kind of anti-integrity.”
Yeah, I think it’s anti-natural for such an entity to exist, and that’s a crux for religion-according-to-Alex. In particular, I think such an entity would be mistaken so long as they hold such beliefs, although I don’t think it’s impossible for such entities to mistakenly hold on to such beliefs for as long as they exist.
Yeah. So I kind of want to defend it. You can argue against it. I mean, what’s your argument against that being-
Well, would you classify the genetically defective liar as fundamentally evil?
Look, I’m all for hope and believing in people. I just don’t want to assume, in principle, everyone definitely can get better and everyone is definitely a good person. I think in practice hopefully, but I don’t want to like… You’d be like, “What about this evil person?” You go, “Actually they’re definitely fine, don’t worry about it. They’re definitely just a good person covered with some problems.” I mean maybe it’s true, but maybe humans just all are all good. But I don’t know. I don’t know most people.
I feel you’re like, “No matter how far gone they are, they can always come back from it.” And I want to be like, “You can get far gone enough that you can’t come back from it,” and this seems like a meaningful difference in our models of the world. And I’m not saying that any particular–
Okay. I think people can be so far gone that, with the technologies available today, they’re not going to–
I’m saying, in principle, they can’t come back from it. Like you could rearrange the atoms to get a good person, but the same person would no longer be there.
So, if you rearrange the atoms of the cancer cell so that it stopped being cancerous and just want it to become a normal cell, would you say the cancer is still there?
I don’t view cancers as having a concept of personal identity.
Well, the Buddhists make a point about how most of our concepts of personal identity are pretty confused, and I’m with them on that.
Sure. I’m sure my concepts of personal identity are confused. That seems surely the case, given that I haven’t thought about them… I’ve thought about them a fair bit. I don’t know. I think I mostly identify with my virtues rather than my ego. Everyone keeps telling me this is wrong.
It seems more correct to me to identify with your virtues than your ego.
Well, then, I’m like, “And I don’t really like myself.” And then, they’re like, “Well, you should.” And I’m like, “But I’m not that good. I’m kind of weak in a bunch of ways except for some virtues, and those are the main things I care about.”
Just for the record, I do think there’s more to who you are than your virtues. I think who you are includes both your virtues and your ego. But I’m not going to go into that right now. In the meantime, I propose we make a list of possible other topics to talk about.
Rescuing old religions vs competing with old religions
Cool. All right. Let’s talk about how you’re sick and tired of me trying to rescue the old religions.
I knew you’d pick that.
It was one of the two things on my list.
I want to get closer to where you think we approach or think about religion differently.
Right.
Well, they’re just like the oldest, most epistemically, corrupt institution on the world, and they have so much power. And if you’re like, “Wouldn’t it be nice to rescue them?” I’d be like, “Wouldn’t it be nice to burn them to the ground, and then, build something better?” And every time you’re like, “What about rescuing them?” I’m like, “Look at all the horrors. They can’t think or change their minds. They rape kids and have so much power over individual families’ lives. People are like, ‘Yes, let’s just have them decide how my life works.’” I’m like, “This institution wouldn’t stand up to Aella running five surveys on their members. We can’t rescue this institution. We’ve got to start new institutions.” I really think that they just can’t live up to the level of individualism where you’re just like, “What does everyone believe?” Because if everyone just says what they believe, you’d go, “Well, we don’t have an institution anymore because we assumed everyone agreed on everything.” But it turns out they all have their own opinions. Oh no.
Alex doesn’t personally find institutionalized religion compelling
For some reason, the thing I’m wanting to say in response to that is how I feel about the Baha’i Faith. Have you heard of the Baha’i Faith?
Who is it? I’ve heard of it, but I don’t know who it is.
It’s a new religion that started within the last 200 years that’s basically… in some sense their doctrine is basically just “Here’s how to interpret all the religions. Here’s the core thing they have in common that we’re going to express that to the best of our ability. All the religions are fundamentally trying to say the same thing, and we’re just trying to be the religion that unites all the other religions.” And their doctrine seemed super reasonable and legible. It wasn’t like you have to believe these weird supernatural stories. Instead, it was like, “Yeah, here are reasonable ways you can interpret all the aspects of all these other religions.” And their prophet’s first principle, the highest principle for this religion, is the independent investigation of the truth. It seemed so reasonable that I’d even considered joining.
And then, I just did more research on this, and they were like, “Yeah, homosexuality is bad.” And I’m like, “Well, where did they say that?” And then, they cited one of the prophet’s original sayings, and one way you could interpret that was as homosexuality being bad. But in the original context, he was saying something like, “Yeah, you know the thing that our society does where men have sex with young boys? That thing is bad.” And somehow that got generalized to all homosexuality is bad. And I’m like, “Okay, so my independent pursuit of truth leads me to not think homosexuality is bad, but apparently, if I join your religion, whose core tenants resonate with me and seem reasonable and non-dogmatic, I’m expected to come to eventually hold the belief that homosexuality is bad because of some great-grandson of the original prophet interpreting something the original prophet said in a particular way… ghat seems lame. I’m not going to join this religion.”
Hey, let me just read the Wikipedia page on Baha’i Faith for a moment. I just pulled it up.
Sure. I’m going to get some water.
That’s funny. I want to meet some of these people that claim to be prophets. I like that they had a guy who was like, “There’s another prophet coming soon.” And then, another guy who was like, “And I am that prophet.” And then, he was exiled and imprisoned. It seems like a pretty classic trope.
I’ve read books about a lot of these people. They’re interesting people.
Rejects notions of racism, sex. Well, they do seem to have these false beliefs about God, but that’s to be expected.
I think there are ways you can interpret them that are not false.
The Baha’i writing state that human beings have a rational soul. Sure. Isn’t trying to sell itself to me in particular. Anyway. So, I don’t like that, but that’s fine. I can’t quite tell what sort of religion it’s like. Did they write all the books themselves, the Bab? Or is that an old book? No, it’s from the 1820s or so on. Oh interesting, it was fastest growing religion between 1910 and 2010.
I think if I were forced to choose a religion to join, where I would just like… I’m in a city and someone was like, “Yep, there’s a community of every single world religion here, and you’re forced to pick one to join,” I think I would pick the Baha’i.
I would see if I could make my own one, and if not, I’d probably pick one that didn’t have any metaphysical beliefs in God.
I don’t know. I hear this about the Unitarians, or the Buddhists. Or maybe some Jewish group who just clearly didn’t believe in God.
Let’s read the prohibitions. Backbiting and gossiping are prohibited and denounced. That’s an issue. I like to gossip. I think it’s good. Drinking and selling alcohol are forbidden. That’s also an issue. I like drinking and selling alcohol. Okay. Participation in partisan politics is forbidden. That’s kind of cool, but also kind of obviously dysfunctional if the entire population converts. Begging is forbidden as a profession. What? They must have had some problems with that when they lost all their money when they became religious. Okay, it’s a new religion.
So, what were we talking about? I was like, “You need to respect individualism,” and you’re like, “This one is the best.” And I’m like, “It doesn’t seem different enough from the other religions for me to be excited about it.”
Sorry. I think that I brought this up as the most reasonable-sounding religion I could find, in some sense.
Yeah. It’s interesting.
And it not passing my bar for wanting to join it.
Yeah. Okay.
I think I’m saying that as a level on which I feel like I’m agreeing with you about the inadequacies of religions, especially when institutionalized.
Mistake vs conflict theory toward existing religions
You keep talking about uniting and interpreting the religions. And I’m like… I don’t know. It is missing something like “We’re in conflict with them”. I’m like, “If the Christian Church comes by—”
Oh. I think we’re in conflict with the exclusivists, but not the pluralists.
No! I think we’re in conflict with the pluralists.
Okay, that’s a crux for me. I don’t feel like I’m in conflict with the pluralists.
I’m like, a bunch of Christianity and Islam does not have a good interpretation. It’s just mistaken and dumb. And as far as people are trying to live it out, we should stop them. And as far as they’re trapped in it, we should disempower this institution.
Okay. I think a lot of pluralists do feel that way, although they have more of a mistake theory rather than conflict theory view of the kind of mistakes you’re naming among the exclusivists.
I think the correct relationship to the religions, even if you’re getting the good stuff out of them, is to be an atheist, or to be like, “Cool, I’m starting my new religion. You guys can come and hang out sometimes, and we’ll talk about how to live well in the world, but you guys are messing up really badly on loads of things”. Sorry. You agree with that?
The good pluralists I’ve talked to would agree with that. They would be like, “Yeah, it’s a real mess out there. Things are pretty fucked. Lots of reform is needed. Atheist critiques of religion are largely correct.”
But I feel like you’re arguing for a reformed Christianity, and I’m arguing for a destroyed Christianity.
This feels like an instance of a mistake versus conflict theory thing actually.
No, it doesn’t. It’s similar to you being like, “Let’s fix the FDA.” And I’m like, “Let’s just end the FDA, and then, start any new regulations separately.” Some institutions should die. Surely you agree with this. I don’t think you think one should always reform the institutions. Sometimes, it’s like, “Please end the institution. Everyone will be strictly better off as soon as it’s over, and then, we can start working on maybe making some new institutions.”
I think Joel Spolsky wrote a blog post about how programmers who think that way about old legacy code bases, and they’re just like, “Fuck that. We’re just going to start anew,” tend to get bitten in the ass because, for all the shortcomings of the old code base, there’s a lot of adaptation to weird edge cases that you can only learn through contact with reality that you’re going to lose if you just throw the whole thing out.
No, I understand that sometimes, people are wrong in the other way.
I’m also curious what you think are the most wrong metaphysical beliefs.
Okay. We’ll get to that in a minute.
There’s a good point in the Moral Mazes sequence by Zvi. He talks about how institutions become corrupted and over time, where as everyone goodharts on gaining power within the institution, the very culture of it changes to reward power seeking rather than to reward actually object-level achieving the goals of the institution. And I think one of the things that he says is very good is that especially these big companies have competition such that a very mazy, culturally dysfunctional company can be outcompeted by a very healthy startup, or something like that. And that’s something I believe, which is that competition is healthy and good and incentivizes against corruption.
Yeah. Yes.
And you just wake up one day and you’d be like, “Oh, my company is dead now because someone else made a better company than me.” I’ve got no right to own the next company.
Religious pluralism coalition memetically outcompeting religious exclusivism
I think Christian pluralism, Muslim pluralism, Buddhist pluralism, and “atheist pluralism” are all compatible with each other. And I want the mutually compatible thing to memetically outcompete the exclusivists’ memes. I think this is just another articulation of the thing that I’m wanting, which is maybe consistent with destroying the exclusivist memes, and maybe also consistent with salvaging them.
Yeah. I want this thing to have memetic dominance. I think some of that dominance is going to come from converting exclusivists to pluralists, and some of that is just going to be from pushing them out and winning, or something. And maybe we have different views about what the relative fractions of those are, but that doesn’t feel cruxy for the thing I care about here.
With the companies, you’re in some pretty competitive industry, and everything’s a bit corrupt and mazy. And then, some new technology’s invented that’s much better than all the existing stuff, and someone makes a new startup and out competes all the other stuff. So, on one side, I agree, the person doesn’t have to be like, “Grr, I hate these other companies. They’re evil and need to be destroyed.” They’re just like, “I’m just going to do a good thing for the world.” And the result is that the other companies just fade away. So, I can agree that I don’t need to be going out to war with them or something. At the same time, the method isn’t like, “How can I bring those companies together and all agree to use a new technology?” I’m just like, “I’m just going to go and make a better technology and out compete them.”
I’m not trying to convert the companies, or reform the companies, or bring the companies together in spiritual healing. I’m just going to go and outcompete the companies, and this is better for the world. And maybe I’ll read some books. I’ll read Working Backwards by some of the high-level people at Amazon, about the core institutional tech that they have, like writing memos and starting with writing the public press release for the product before building anything. And there’s stuff I can learn from them, and that seems good as well when I’m building my own company. But I’m not reforming Amazon when I’m outcompeting them. I’m stealing from Amazon, I’m studying them, and then I’m doing something better.
I would also take this stance with regards to governments. My favorite way to improve governments is to let me please, please let me start a new country. I would much prefer this to trying to reform the FDA (and every other part of government). I think it would be faster and better. Unfortunately, the world is messed up and I’m not allowed to, but I do think it would probably be a better way of improving the state of government.
Okay, I want to make an analogy with Muhammad. Muhammad first tried reforming Mecca by preaching to the Meccans, and it didn’t work. They tried to kill him. Muhammad then managed to succeed in reforming Mecca by capturing it.
With an army or what?
With an army that the Meccans tried to kill many times. And then Mecca was reformed.
I think something analogous to that might have to happen, with Muhammad’s army being analogous to the pluralist coalition, and Mecca analogous to the religious exclusivists.
You’d like to become Pope?
Would I what?
Like to become Pope? I was teasing a bit.
Probably not. There’s a lot I respect about Catholicism, but I don’t consider myself a Catholic.
Sure.
Right. So the Muhammad picture, I think, is actually pretty compatible with your picture. If Amazon is Mecca, then Muhammad’s thing is a new startup.
Well, it more sounded like he bought… No, no, no, no. Muhammad’s thing is like Elon Musk and Twitter. He didn’t outcompete Twitter. He showed up with all the money in the world and bought Twitter and then fired more than 50% of the staff. And now he runs Twitter, and it’s a bit better (e.g. Community Notes).
I think my main point is I think there’s less difference in our visions than it might have initially seemed. In terms of what I’m trying to do, it’s to figure out intellectual foundations for religions that let you translate between their core claims, and interpret their metaphysical claims in sensible ways. And getting a bunch of political clout around that, in favor of religious pluralism, seems pretty compatible with your picture of things?
I don’t think we are agreeing as much as you’re saying we’re agreeing. I think you’re trying to-
Okay. Where do you think are the concrete disagreements?
I think you are saying your plan is to… I mean, again, your plan is more of a collaborative reform, whereas my proposal would be if you would like to fix religion would be to study the religions and learn what you can from them, and then start a better one.
Sorry. I’m collaborating with the pluralists, not the exclusivists.
Part of my conception of the pluralists that I’m starting with here is that the pluralist’s views are already pretty much mutually compatible. And there’s also a thing where pluralists are already kind of shunned by the mainstream of the religions to a large extent. There’s one pluralist Baptist that I know who was faculty at a Baptist university, and the students there led a prayer group for the salvation of his soul.
He was a what? He was what?
He is a Baptist. He is a pluralist Baptist. And he was a professor at a Baptist university, I think. The students there led a prayer group for the salvation of his soul because they thought he had gone totally off the deep end. And I’m like, “Yeah, that guy is the kind of pluralist I want to collaborate with.” Not so much the students at the university praying for his soul.
Reminds me about how some of us here feel about you!
So you want to take the pluralists, who are all the religious people who only care about the things the religions have got in common?
More like who think that the things they have in common are more important than the things that are different between them.
They’re unique to their specific local brand?
Yeah, that’s right.
There’s one way of interpreting the rites and rituals and dogmas of religion as the One True Way To Live, and if you don’t live according to that, then fuck you. And there’s another way where you can think of them as pointers for the masses of people to live in alignment with the real thing, but the real thing is what it’s really about. And when you know what the real thing is, the rites and rituals and dogmas don’t matter so much, and therefore it doesn’t really matter if other people from other religions aren’t doing the rites and rituals or adhering to your dogmas. That’s a common pluralist perspective, I think.
So you’re doing this pluralist group, and with them, you are trying to figure out what the important things are that are common between all the religions. And then what’s the next step?
Have this coalition gain memetic dominance over the exclusivists’ memes.
Why Alex doesn’t want to start a new religion
Okay. Why do you not want to start a new religion? I feel like in your shoes, that would be my next thought, and I don’t know why it isn’t yours.
One thing is I don’t want to be a prophet. Another thing is, what do you think are the substantive differences between starting a new religion and the kind of thing I’m describing?
It’s got a name, and it has some recommendations for how to live one’s life that are very concrete and involved with things about having a rest day in the week and so on.
I don’t find that kind of thing very interesting.
Religions also have a set of stories that become common knowledge amongst tens of millions of people.
I think an anthology of the best of the existing stories would work well. I think the existing stories are extremely powerful. They’re very Lindy.
Look, stories are powerful and some institutions are there to tell stories about how to live life well. These are religions.
Yeah. Sorry, I am trying to tell a story about how a positive singularity might happen, which is kind of a new story, where I’m invoking aspects of existing religions’ conceptions of something like their version of a positive singularity.
Does the “Allah will protect me” story suck?
Now, I do think your Muhammad story under the tree kind of sucks, and I could write a better version if I tried for a year.
Strongly disagree there. But what do you think sucks about it?
Just most of the time, that doesn’t work, and it didn’t tell you why it worked in this instance. Most of the time someone comes with your sword and you say, “Who’s going to save you?” You say, “God.” They go, “No,” and then you die.
Yeah. I think my crux for whether it’s good is whether it’s plausible to me that someone could actually be so charismatic that that could work.
Charismatic can’t be the right word.
Yeah. I more mean “embody the source code for which that works”. I’ve read stories of cult leaders who make eye contact and the people feel like their lives are changed from that eye contact experience.
But also, I agree that if there’s no way for you to picture someone saying “God will protect me”, and then the other person being persuaded by that, the story seems kind of lame. But I think if you have prior context, including from other anecdotal reports of modern cult leaders who have done analogous things, I think it seems less crazy as a story.
Jesus’s empirical claim about evil
All right. There was something else on my mind. I was thinking about the story of Jesus being forgiving as he is being killed.
Yep.
I’m trying to understand what was good about it. I mean, it’s partly he’s in a situation where it’s very hard to forgive someone. But I think that’s maybe not just it. It’s also… It’s arguing… He’s implicitly saying it’s correct to forgive that person or he is explicitly saying it, I guess. Forgive the people who are hurting him.
I would say better rather than correct, but yeah.
I guess the they know not what they do is an actual claim about not just I’m choosing to forgive them, but I believe… This is a fact about the world. I genuinely think that’s an empirical claim that they don’t understand.
Yes.
...they lack self-awareness of what they’re doing.
Yes. In contrast to the view you were putting forth earlier of some people just are evil or something.
Or self-aware of their evil and hold onto it.
Yeah. I think there’s a way in which their self-awareness has a blind spot. Their self-awareness knows not what it does.
Yeah, that’s possible.
It’s like a pathological liar who is self-aware that they’re a liar and they’re just like, “Yep, this is the best thing for me to do and I endorse that.” And I look at that, I’m like, “I see that. I think you’re wrong anyway.”
Yeah. Although a part of me would want to be like, “He does know that he’s lying right now. That’s true.” There’s still a sense in which they don’t understand. Anyway, I guess I was trying to think about it with other virtues or something. Like what would be a similar story to the Jesus story with the virtue of curiosity? And at first, I was just like, “What’s the least adaptive situation for curiosity or the hardest time to be curious?” There’s probably a situation in which curiosity is being beaten out of you by some government or something, or some situation where the world is very uninteresting and everyone is justifiably kind of bored by it
But I was trying to think about it for a while, and it wasn’t just that it was disincentivized very harshly, but it was also that somehow it was still the correct virtue in that situation. It wasn’t just that Jesus was doing a costly signaling of forgiving. It was also that he was saying, “Surprisingly, they actually don’t know what they’re doing.” That’s relevant. And similarly, if you were trying to be curious in a situation that was very unrewarding of being curious, a good story for curiosity would be one where it actually was the right thing to do, even though… I don’t know. Even though it didn’t save you personally, it still was the right sort of thing to do spiritually. But I don’t know what that is. It’s hard to write the story. Anyway. I’ll think about it more.
Alex doesn’t want to tell people how to live
I don’t want to start a religion because I don’t want to tell people how to live. I don’t like doing that.
On the other hand, I think a lot of people don’t know how to live and and would probably appreciate it.
I am happy to make a blog about things that have helped me and a bunch of other people live their lives, but I wouldn’t want to generalize it to the whole world. It’s related to how for every piece of advice, there’s equal and opposite counter-advice that some people need to hear more.
I’m pretty for writing out things I’ve learned on my spiritual journey, and for other people to do that, and for all that kind of stuff to come together on Steel-Arbital or something. But the thing that I’m wanting feels fundamentally decentralized rather than fundamentally centralized, while still remaining coherent, and telling people “you should live your life in XYZ way” feels too centralized to me.
But… Sorry. There’s a distinction between starting a new religion and being a prophet, although I understand that typically they’re the same guy. But I think there are answers to questions like, what society-wide protocols would help people a lot? And I think a Sabbath is a pretty reasonable answer to that question. I don’t think it’s in-principle impossible for you to come up with good answers to these questions.
I agree about that, but that also doesn’t feel like the kind of question I’m interested in answering. For example, I think there’s something good that comes from the ritual of praying five times a day at a set time as a culture. I don’t think the world would be better off if everyone who’s doing it just stops doing that, nor do I think it would be better if everyone in the world started doing that. And I’m saying this as a datapoint to illustrate why I don’t want to be giving recommendations for how people should live their lives.
I feel like a religion is aspiring to be a way of life for masses of people. I agree that one marginal person quitting their local religious traditions may hurt them, but I think if a whole city switches religious traditions… I mean, actually the first generation might also run into some issues or the first decade, but… I don’t know. It depends on what time scale you’re trying to invest in civilization at. But I think the five times a day one, it wouldn’t surprise me if that one, on net, it would be better if everyone gave it up and picked up some other traditions instead.
I mean, I’m not certain. I haven’t really thought about this one. I’ve not really chatted with people who do it. My guess is it’s good to be able to focus on a single thing for more than five hours at a time. If you sleep for eight hours, then you’ve only got 16 hours, which means every three hours you’ve got to pray. I think you need to be able to focus on something for more than three hours is important. So my guess is that this one is not one that would survive contact with reality or that should not.
But I think once a week, most people taking a rest day makes more sense to me, though I’m not certain about it.
Yeah. I want the pluralist representatives of each religion/atheism to be able to translate to the inclusivists who can then translate to the exclusivists. Where the exclusivists are like, “My way is the only way.” And the inclusivists are like, “My way is the best way, but the other ways have good things, too.” And the pluralists are like, “My way is one way out of many.”
I think I want to summarize my position around the thing I’m more interested in doing… Steel-Arbital is really at the heart of what I want, along with a vision for the future that pluralists from each world religion and “pluralist atheists” can all get behind, and then sort of have that vision percolate outward through the rest of the world through Steel-Arbital.
In closing
All right. Nonetheless, I think it was important to get on the same page about, or to at least talk through, my issues with trying to rescue or reform religions. I think that was a part of the model that I was pretty suspicious of and would’ve felt bad about not talking through.
Yeah. I’m glad we talked that through.
I think relevantly, I brought up my criticisms and it didn’t seem to obviously disagree with your plan. I think there’s still other things we’ll touch on, about what it looks like if you succeed at the thing you’re trying to do.
Yeah.
But it was still good to chat through this.
Yeah. Sweet.
I’ll see you next time. Thanks, Alex.
Yep. Thank you, Ben. Goodbye.
Conversation 4 — April 28th 2024
We continued our dialogue over Zoom on April 20th 2024. The audio was also transcribed and lightly edited.
Intro
I think I had a sense that there’s a bunch of things you want to get from unifying the religious pluralists and the conceptions of good and so forth. And there’s a bunch of challenges for how to enact this in the world such that everyone engages with them well. And there’s a bunch of collective epistemology issues.
I think that you had some like leads for, “Here’s how I want to go about doing it or here...” I don’t quite know how far it was like, “Here is my plan for doing it versus here are some bits that would need to go well versus...” I don’t know, I’m not exactly sure what your relationship. But it seemed like-
Yeah, it’s more like the latter, like here’s the big picture for how things can go well. Or, if things go well, here’s what that could look like. Here’s a direction humanity could take that seems like it has some hope according to me.
Did you want to add anything else to where your attention was going?
I didn’t really come up with anything else. I would just mention that what you’d said around steel-Arbital and steel-UpTrust point to the same thing in my head. I’ll just call it steel-Arbital in this context.
Yeah. All right. Do you want to start telling me a bit about either that or any other parts of the puzzle that you think you see? Or how to get those things to go well, or what would have to happen if things to go well?
Apparently: Agent Foundations = Religious Metaphysics = SteelArbital
One thing is that I think:
the technical challenges involved in solving Agent Foundations
the technical challenges involved in finding mathematical foundations for metaphysics, in a way that can give accounts for the metaphysics shared by the mystical traditions of the major world religions
the technical challenges that we need to solve in order to build steel-Arbital
have very substantial overlap. Why I think this might be very non-obvious, and I think this is maybe an important thing to unpack before continuing too much further down.
Yeah, I’m skeptical. Can you pick two of them and point how they’re connected?
Yeah. Let’s pick the first two first, which is Agent Foundations and the religious metaphysics.
So, Agent Foundations is asking questions like where are the agents? What is an agent? How can you tell if something’s an agent? What makes something more or less an agent? How should an agent make decisions?
How should one generally make decisions? How should one form beliefs about the world? There’s a bunch of bounded theoretical questions here that no one has answers to.
Yes, that seems like a fair overview.
The religious metaphysics is asking… I don’t know what is that. “Here is a story for how the whole universe works (that’s wrong).”
What’s wrong about it?
You don’t actually rebirth a lot of times. There aren’t a bunch of devils who will burn you in hell for eternity. The soul is not a fundamental unit of physics.
I can see them as connecting to same questions, but I don’t see them as like… I don’t think the religious people are going to come up with any good math. I expect the Agent Foundations to answer some helpful questions about a place in the universe, not unlike, I don’t know, Darwin’s theory of evolution and helped answer some relevant questions in the universe. I wouldn’t say that Darwin’s theory of evolution was roughly the same thing as a religious metaphysics. I think they had very different methodologies for understanding the world and very different details as to how to apply them to understanding the world. So, it seems confusing to lump them. It’s the same thing.
That’s a very helpful starting point.
Is Steel-Arbital harder than coordinating around not building AGI?
For now, I’ll continue by going into more detail around my vision for steel-Arbital.
You’re a person. You’ve got conflicts with other people in terms of beliefs and actions. Fortunately, AIs give you the best versions of where they’re coming from and show you ways you can coordinate with them, in addition to the ways that you’re in conflict with them,. This results in gradual increases of mutual understanding and coordination, with the end result being mutual understanding and coordination on a global level. That’s one piece.
The second piece is that I don’t imagine this looking like everybody agreeing with each other. I more think that there will emerge / congeal a Schelling coalition of people who actually care about truth and doing good things, and understanding how to coordinate with the other people who are doing that.
And I imagine this coalition winning in the world, where part of winning involves Moloch not being an existential threat to humanity anymore, whether from AI or other things. That’s the high level of what I’m picturing. And importantly, I think this Schelling coalition is going to involve a bunch of pluralist religious leaders and “pluralist atheists”.
This is all quite vague. What’s Arbital got to do with this, or what UI design causes this that has not already happened? And why is it different from the current state of the world? Sorry. You might not have answers, but these are the sorts of questions I would like answers for.
The tech that I specified was extremely vague. It’s...
I think I’m going to give up on trying to not be rude in this conversation because I think we’re friends, and it’s fine. And...
Yeah, go for it. I said at the beginning that at a high level, this is what I want the tech to do. I think asking about what the UI should be is missing the point of what I’m trying to say.
I’ll also give up not being rude. I think the default vision that AI people have of a positive singularity is repulsive to most of the world, and for good reason. What I’m trying to do is rescue the good parts and find a thing that’s more compatible with what the rest of the world cares about.
Also, the positive vision that most religious people have for the end of the world is repulsive to the other people who are not part of their religion, and for good reason. And the thing I’m trying to do is rescue the good parts and put them together, paint a picture of what it looks like and a rough picture of what it might look like to get there.
There’s a thing we want the tech to be doing. And I want to emphasize where the tech can help with peacemaking, instead of just scientific development and material abundance. I think scientific progress and material abundance are very cool and may end up being very helpful, but I think peacemaking is a more central component of my picture.
I’m not especially arguing for specific vision of the future. I don’t have a particular governance proposal in mind for post-utopia. If I wanted to get one, I’d probably go and read Bostrom’s new book. And he’ll probably have some interesting suggestions. I’m mostly against the “definitely lose control of the future” aspect where you just build a different species and then it kills you and moves on. I think that’s the minimum future that I’m thinking we can all get behind. That’s the response to one of these things.
We are aligned on that. I think the next piece of the picture I’m trying to gesture at is getting clearer on what it actually looks like for tech to help get the world to not be vulnerable anymore.
Yeah, I know. And you were saying you have dreams where everyone can come together and coordinate really well. But that seems vague and stronger than what I need. I just need everyone to agree to not build AGI until we solve the alignment problem. This is a far weaker problem than I would like to solve than all coordination between all humans.
I think that’s politically impossible. I think...
But your thing is harder than that.
I disagree. How are you going to beat Moloch? How are you going to stamp down Moloch? Bostrom once proposed mass surveillance, but I think that’s not going to be politically feasible. I think any such approach would collapse because of internal politics, and it also won’t have moral legitimacy with the rest of the world. That’s why a “benevolent dictatorship” is not going to work.
I guess I was going to make everyone feel a bit ashamed for doing it and low status and scared, similar to how lots of people aren’t doing human genetic enhancement—
That’s definitely not going to work.
—lots of people don’t do human genetic enhancement just because they’re like, “Everyone will think I’m shitty for doing it and be mean to me.” That’s a pretty strong force in human civilization.
That is nothing near the strength of the forces of wanting military and economic advantage. So, it’s not going to work.
We can argue about whether my thing works. And I’m happy to defend a little bit more. I was making it up on the spot, but again, I was like, I guess I don’t understand why your thing isn’t a superset of my thing. And I’m like, “We will have to solve the specific problem.”
And you were like, “Not only will we solve this problem, we will solve all problems about coordination.” I don’t understand how your thing isn’t just like… mine isn’t just… I’m just like we should solve this coordination problem. You’re like, oh, well, the software will solve all coordination problems, and then we’ll manage to easily solve this one.
I don’t think I’m trying to say “solve all coordination problems”. I think I am trying to say that whatever approach for the coordination problem you have in mind runs into fundamental difficulties, especially around overcoming Moloch. If you think the problem is to beat Moloch, I’d be on board with you, but I’d ask you how you would beat Moloch without a benevolent dictator, because I don’t think benevolent dictatorship is really a thing.
What are you saying? I guess, still, I was like, “Be specific about what this Arbital looks like.” And you were like, “That’s not the point. The point is to have a decentralized process for coming up with true arguments that everyone trusts.”
That’s not the full picture. That, I think, is going to be critical infrastructure for the Schelling coalition to have the moral authority to win in the world—in other words, to have a “decentralized benevolent dictatorship”, so to speak.
By winning in the world, you mean build a global governance or dissolve national borders or something. Not like that’s the definition of it, but that’s a part of it.
Something like global governance is closer to the mark of what I’m picturing, but in a way that’s decentralized and continuously earning the trust of everyone it’s governing. It would be through a style of governance that’s very different from anything that currently exists.
But I guess I may be hearing you maybe mistakenly. You’re saying, “I’m not going to go out and build a global governance. I’m going to and go out and build technology for causing people to have trust in these arguments in general. And then they will figure out that they should have some global governance. And they will trust it. But the thing I’m doing is not launching a flag for global governance. It’s for people being able to congeal arguments.”
And including for what should be done. I think it should also congeal values.
Well, what? Values...
And by congeal, I don’t mean everyone should have the same values. I mean figure out where they can be reconciled and where it’s too expensive to reconcile them.
Steel-Arbital as a preference synthesizer
Have I given you my analogy of the elephant with a blind man?
It’s a standard analogy. Five guys...
Yeah. So, the picture I have in mind is there’s something like moral realism that’s built into the steel-Arbital. I think you can’t have a thing that does good discernment for reasoning without a solution to moral realism. I can go into that as a separate bit, but the TLDR is that the discernment of whether an argument is correct is value-loaded. And I don’t think that means arguments are irreconcilable, because I believe something-like-moral-realism is true.
I think all our values come from somewhere and can be ultimately reconciled. We can go into that, but I’m not going to go into that right now. My main point is if you imagine an upgrade of democracy where people have conflicting views about how society should be run in service of the common good, but you think of these conflicting views as different parts of the elephant of how society actually should be run, the thing that I want from the technology is to be able to figure out that synthesis and communicate that synthesis to each person.
I am just now realizing that “convergence toward a synthesis of preferences” is just as important as “convergence toward a synthesis of beliefs” in my vision of steel-Arbital, and I want to apologize for not making that clearer earlier.
Can you say slightly more detail about how you think the preference synthesizer thing is suposed to work?
Well, yeah. An idealized version would be like a magic box that’s able to take in a bunch of people with conflicting preferences about how they ought to coordinate (for example, how they should govern their society), figure out a synthesis of their preferences, and communicate this synthesis to each person in a way that’s agreeable to them.
And this is the project that my colleague, Jacob Lagerros, is currently working on. He’s building an app called Octopus, where we all sit down and we all… The dumb thing it currently is for is what restaurant do you all want to go for dinner? And you all chat with Octopus. And you talk to her about preferences and so on. And at some point, it just tells you, you are all going to this place. And everyone was indeed down for it. He’s trying to—
Sweet.
—make that work for other things. Anyway, maybe I shouldn’t have interrupted you.
I liked that.
Okay. So, you want a preference synthesizer, or like a policy-outputter that everyone’s down for?
Yes, with a few caveats, one being that I think preference synthesis is going to be a process that unfolds over time, just like truth-seeking dialogue that bridges different worldviews. I think the idealization of a magic box that instantaneously spits out synthesized preferences might not be possible.
Another caveat is that people will only be happy with the synthesized preferences if they’re willing to admit where they’re wrong. If a young earth creationist tried to have their preferences synthesized with a bunch of reasonable scientists around what schools should teach in their local community, they might not be happy with the preference synthesizer telling them that the earth isn’t 6000 years old.
Killing Moloch with a decentralized singleton running on Steel-Arbital
(What would you do with a trusted preference synthesizer?)
The core hope I have is that the set of people who can actually coordinate with this preference synthesizing process can out-compete the kinds of people who wouldn’t, in terms of getting to run the world.
That’s one of the words that commonly hides the most complexity. “Out-compete”, a phrase which here means “starts new countries and wins wars and builds new religion”. It’s just such a deceptive little word...
It’s not necessarily starting a new country. There’s one version of this where enough people inside existing governments are able to coordinate with this and then do reforms from the inside out. And nations still exist, but they’re just all way friendlier with each other.
Sure. I want new countries so bad. I think if we could have new countries, I might be like, “Finally, there’s hope.” Anyway. It’s good that your plan is in fact quite ambitious.
Again, I feel like it’s less of plan and more like… there’s one vision for defeating Moloch, which is benevolent dictatorship, and I’m proposing a different vision of preference synthesis.
Well, I don’t know what you want exactly, but I feel like you need to say something that leaves me with a reason to believe that there’s clear gains to be had in preference synthesis. I think using machine-learning assistants to do it is of a different type signature than most other proposals. And that’s somewhat tempting.
Sorry. The clear gain is that it’s a possible attractor state of the world in which the thing that’s in power isn’t a centralized dictatorship and isn’t eaten up by Moloch.
Why isn’t it eaten up by Moloch?
Part of my vision is that it outcompetes the people who aren’t participating in this process. And the Molochian incentives that misalign with the common good would get noticed by this preference synthesis process.
All right, but I’m not sure whether you’re saying, “By the way, Ben, unfortunately, I think in order to save the world, we’d have to kill Moloch. We probably can’t do that. So, we’re all screwed,” or whether you’re like, “Ben, I think I have some details on how we can kill Moloch. And I think it can work out.”
It’s more like the latter. Is your current position like, “We probably can’t kill Moloch. So, we’re probably all screwed”?
[Long pause]
I think my current position is I haven’t heard of a plan that would kill Moloch. I don’t think I would say the words “We definitely can’t kill Moloch,” but I have no plan or I have no concrete bets. I couldn’t tell whether you were saying, “Well, we will have to, even though I don’t know how to,” or whether you were saying “I have a good way of solving it”.
I think my picture is basically that the Schelling coalition / the preference synthesis coalition is going to have minimal Moloch internally within it, and that they become powerful enough to determine how the world runs. This is kind of like killing Moloch, and the process by which this all happens is pretty much how I interpret religious prophecies about the end of the world.
So, is this a singleton but it’s internally decentralized?
Yes.
Okay. Seems like a nice idea. No, we could try and talk more about, concretely, about preference synthesis and what would happen in the world if people were better at it. Maybe I’ll just keep—
That seems worthwhile to do. I think, insofar as there is skepticism behind your “would be nice,” or something, I think I want to hear more about that. Or is it more just like, “I don’t really know how to picture it so it’s hard for me to comment on it.”?
I wouldn’t say I had a very specific skepticism to share, which kind of sounds like I don’t have a concrete enough vision of it to comment on. Yeah, I think that’s probably where I’m at on it.
Steel-Arbital should reveal and forgive blind spots
Yeah. I think the thing I’m wanting to say right now is a potentially very relevant detail in my conception of the preference synthesis process, which is that to the extent that individual people in there have deep blind spots that lead them to pursue things that are at odds with the common good, this process would reveal those blind spots while also offering the chance to forgive them if you’re willing to accept it and change.
Okay. I still can’t tell when you say these things whether you’re saying them as, “These are just the 17 aspirational things we’ve got to hit,” versus you’re like, “I think I can see how you can get these.”
Closer to the former, but it’s more like “if the process is working at all, this is going to be a crucial component of it.”
Yeah.
The reason it felt relevant for me to name is because I think in my implicit model of people’s implicit models of conflicts, a lot of where conflicts ground out is other people just not getting how the thing they’re doing is bad, and furthermore seems to be more interested in doubling down than caring about getting it… but also, everyone’s kind of like this.
Well, yes. I think this is most of the way that things go badly is that people don’t understand the bad… We could go into detail on that and I think we would end up agreeing with it. I currently don’t bet that it’s literally all of it. I think sometimes people like to-
I wouldn’t either.
Yeah. Just to follow up on that, I generally have noticed that when I’m in conversations with folks where I’m like, “I’m pretty sure the other person here has wronged someone,” there’s a pattern I’ve noticed where they seem unable… they specifically, they use the word bad. They’re like, “Oh, that’s bad,” and they weren’t able to concretely say what was bad about it. They just said that it was bad. They couldn’t describe the detail of the cost that was imposed or whatever. So, I have a personal strong heuristic to taboo the word bad pretty heavily, to go, “Oh, yes, it’d be bad if that happens. Sorry, what I mean to say is, ‘This specific cost would be imposed on these specific people,’” or whatever. I generally also get the sense that people have blind spots around the places where they inflict harms, but anyway, we could ground that, I think.
Cool.
Why was it relevant to you?
A critical mass of people sort of letting go of core blind spots is just a concrete thing that I think would be part of this process, that I think is the sort of thing that most people don’t think about very much as a possible way the future could go.
Religion as a tool for coordinating with the masses
It seems hopeful. Seems kind of unrealistic, but maybe that’s a bit on me for not really chatting with half the world whose IQ is below 100.
I think the way you coordinate with those people is with celebrities or something.
I really would prefer stories. Or, that seems wrong-headed to me, but I feel like you just said the way you help people in their marriages and the way you help people become better parents is via celebrities, or the way you help improve schools is via celebrities and I’m like, I think–
The way you help people improve is by showing them people they trust who are doing the thing that would be an improvement over where they’re at and the people that most normal people trust are celebrities. The people they look up to and think to themselves, “I want to be more like this person.”
There were two things. One, just because someone is successful in front of me, it doesn’t mean you understand how to be successful. They’re like, “My kid hurt me in this way and I forgave them for it.”
“Ah, I guess I should always forgive my kids.” It’s not the rule. The rule is a much more complicated negotiation. Just because you can see someone nailing something doesn’t mean you’ve now learned how to nail that thing. Secondly–
Sorry. I’m saying this is the general avenue with which you can reach people like this. If there’s celebrity-endorsed content that goes into the specifics of how you can actually do things better… the only thing I’m trying to say is that it is very tractable to communicate things to normal people.
Hey, but you’re into religions. Why didn’t you say the great way to do it is to have a priest in every town who read some good stories about how to be virtuous?
Because people find celebrities more compelling than religions.
Well, I think in their free time, but I think fewer people would be willing to-
For the devoutly religious, then yeah, framing in terms of the Bible would also work. The point is just to, for any given person, take whoever their moral authorities are, who are actually aligned with the coalition, and then filter it through that. Filter the message through whatever language the moral authorities would use.
Look, if we had 50 years to build a new religion, I think we could do better than celebrities. I think we could have better books and stories and reading groups and local preachers and better websites and communication technology, like Google Docs but better, for discussing ethics and getting problems solved and having anonymous conversations with people. I think people should have more anonymous conversations about the problems in their lives with other people.
For the kinds of normal people who generally seek religious advice from priests, the thing I would want the AI to do is to quote Bible references in a compelling way, and in particular in a way that is compelling relative to what they’re experienced with.
I’m saying if I had 50 years, we could write more stories and the AIs could talking through those stories in sort of personalized one-on-one dialogues when they needed it.
That’s part of my picture as well. It’s just that if you’re starting with the Bible as a moral code, I think there are pluralist interpretations of the Bible that are compatible with how to actually be a good person. I don’t think the fundamentalists would have much of a role in the preference synthesis process in the beginning. They would more be like people in the outskirts who would get integrated later on, if ever. Zooming out a bit, I do feel that part of what’s going on here is optimism around coordinating with normal people versus pessimism and hopelessness around that, or something like that.
Right. It’s not just quote-unquote “typical” people, but it’s also the existing structures of the world, like governments and countries and regulations and trades. I think it’s not hard to get a bunch of people in a room to agree to a superior option. I think it is much harder to in fact change legal structures or change the way companies work. That takes longer time and has a lot of annoying shear forces.
Steel-Arbital vs religious visions for a fixed world
I agree with you, and for what it’s worth, in my visions for how this would work, there would be a coherent decentralized process that’s pretty small at first. But as existing power structures fall apart and dissolve under their own weight, that’s when the Schelling Coalition actually expands in power. It’s when the existing structures dissolve and people are needing alternatives that people will turn to the Schelling Coalition, as the Schelling alternative that’s more nimble and stable and trustworthy than whatever they were leaning on before.
I don’t know how to do that, especially quickly or even very slowly, with nation states, or whatever you’re supposed to call them, state actors. If I could be like, “I have a new country. You can just come to my country, and it’s got a better legal system that I can start afresh with,” then I could compete with it relatively quickly. I don’t buy a story of, like, “Ah, yes, within 20 years, France and Canada’s legal structures will just fall apart and they’ll be looking for some other better one.” I’m like, “Nah. They’ve lasted hundreds of years. They’ll stick around. Probably they will just stagnate and persist. They will persist, rather.”
By the way, this conversation isn’t not fun.
It’s kind of fun to just make up how to fix the whole world.
But to be clear, it’s fun in the way that it’s probably fun to write fanfic.
It feels relevant that I kind of am picturing the Schelling Coalition as a meta… It’s not like a new religion or a new country, but it is kind of like a meta-religion and a meta-country that has a coherent worldview instead of beliefs and norms about how to operate. Maybe there will be a president from the Schelling Coalition and that’s one way a lot of change could happen.
Sorry, there’ll be a president? Okay.
Yeah.
Yeah, anyway, I’m getting a sense you’re like, “My plan is to fix the world and here is a bunch of details for how to do it.” Sorry, plan is a strong phrase. “I think we need to fix the world and here is a bunch of details about how to do it.”
More like a vision for a fixed world.
Okay. “I have a vision for fixed world. Here’s a bunch of details to it,” and I’m not sure I buy any of the details.
Yeah, like which details?
“There will be a political coalition amongst all the players in the world and it’ll be the best one and it’ll beat everyone else. They will be able to solve preference problems. They will be able to synthesize all their preferences into a unique thing via some unspecified computational method.”
Hold on. The goodness of the coalition comes from the goodness of truth. There’s this trope where good people are better at coordinating with each other than evil people, and I’m basically picturing this as like, yeah, the coalition of the good people, and in the epistemic domain, the coalition of people who are actually trying to figure out truth in a real way.
Oh, all right. I feel like there’s this thing I thought when I was reading through the previous dialogues where I was like, “Insofar as you think religions can save the world, I want to point out that we already had a period of history of religious dominance where they got really good shots at saving the world.”
Or they got, like-
“Every smart person was religious and they had all the power in the world, and then they didn’t save the world and they were kind of stupid, and it was good that they’re in less power today because they’re epistemically corrupt, and they hurt a lot of people.” I think that trying that again doesn’t sound to me like a winning strategy. I mean, you can be like, “We’ll do it better,” and I’m like, so-
Trying what exactly again?
I don’t know, having a big power structure over the people’s lives that is built around stories of virtue and goodness and that has the storytelling leaders in every town and is-
Okay, all right. I object to the notion that they’ve already tried to save the world many times and failed. They all have prophecies that say, “We’re just starting. The world will get saved in the end days when the truth of religion actually ends up dominating.” And under exclusivist interpretations, this is kind of dumb, but under pluralist interpretations, they’re all basically saying the same thing, according to me at least.
Well, that’s interesting. That seems like it’s a standard attractor, then.
Yes.
Doesn’t mean it’s truth-tracking.
That’s right. I think whether the attractor gets realized is an empirical question that’s up to the actions we take as a civilization. I don’t think it’s preordained to happen.
What religions say about starting new religions
I want to hear more skepticism.
Why do you want to hear more skepticism, Alex?
As inroads to cruxes.
Why do you want cruxes, Alex?
Because I want to double crux with you.
Why do you want to double crux with me, Alex?
Because I appreciate this attempt at bridging worldviews. I appreciate this attempt to find where my informational gaps are, and also getting clarity on where there are weaknesses in the pictures that I have.
Yeah, I still don’t really get why you don’t want to start a new religion.
Because I respect the existing ones.
You know there’s, like, dozens?
Yeah, I respect the ones that I’ve interfaced with.
No, I know, but each one of them thinks the thing you’re supposed to do is start a new religion.
Losing their edge, I tell you! Nevermind. Okay, I didn’t know that.
And the Jews, as well. I don’t know what you mean about “losing their edge”. This was in the Quran. I think Muhammad saw himself as-
So, you make your new religion and it’s better than the other ones, and then you’d be like, “Had I existed at the time of the Quran, the Quran would’ve said that the true believers of my religion would also go to heaven, or be saved.” It’s not an argument against starting a new religion.
Revisiting agent foundations vs religious metaphysics vs steel-Arbital
Okay, I think the place I want to go for a sec… I want to go way back to the beginning, around… steel-Arbital (/ preference synthesis) versus agent foundation versus religious metaphysics. I think the ways in which those feel very similar feel very relevant to my picture.
I think physicalism is wrong. I think you can interpret souls as the source code that is sort of the object of FDT. That’s not a material thing that ends when you die.
Yes. Math also exists. Multiple truths are true, irrespective-
There also isn’t as clean a distinction between math and matter as one might commonly assume.
I agree with these things.
Yeah. I think the heart of religion and the heart of what decision theory says about how we should act are in alignment. I think for the right notion of identity, which is related more to your FDT source code, you can reasonably think in terms of reincarnation.
But it’s also very easy to make dumb mistakes around reincarnation if you don’t have the right metaphysical sophistication, which Buddhists heavily emphasize around how they talk about rebirth, for example.
I’m not saying it’s never a nice idea to think about it as though you were being reincarnated, but I think the parts of me that are in the other people are my virtues, but not a lot of my personal and individual characteristics, and it’s certainly not my memories.
I’m with you there.
Of course you’re with me!
I think you’re stereotyping religions as having wrong metaphysics, and I think that’s true of naive interpretations of religion, but-
Also, all the things they would’ve said 500 years ago, they would’ve just kept saying false things. “God directly made the people,” as opposed to, “Evolution evolved the people.” That’s not a great phrasing, but you know what I mean.
Yes. The people who say that still are pretty embarrassing.
I’m not trying to be rude, but they would be like, “No, no. They were right the whole time, if you think about it.”
I’m like, “No, they were wrong. The things they said were false.” There was some useful stuff in what they said, but anyway, I don’t know why-
Yeah. They wrote about a lot of things and they made a bunch of major mistakes. You’d previously characterized religions as having obviously wrong metaphysics and I feel like there’s something meaty there in our disagreement. When I was talking to Catholic monks two days ago, I thought to myself that their metaphysics seemed wrong. And so on that level I agree, but that’s not the thing that I’m trying to talk about with religion here.
All right. If you want disagreements and stuff, I can try and say more things like, “I think it would be reasonable to try and make a new religion, and there are interesting and useful things to learn from the previous ones about how to relate to death and child-rearing and good and evil, and you should go and take those from it, but I’m not on the page with you of reforming them. They seem obviously awful and horrendous and dysfunctional. Your attempts to reform these institutions seems obviously dumb.” Sorry, I’m not sure I… whatever. But it seems like there is stuff that they have been able to talk about with regards to good and evil and child-rearing and death and so forth that is useful to get from it.
I’m not trying to reform, I’m trying to work with what exists rather than against. I think in the ways that matter, the sort of Schelling Coalition I’m describing is, for the relevant intents and purposes, a new religion. In many ways it’s going to be way more powerful than any of the past religions because it’s got the power of technology. Rather than designing a standard set of new stories, the AIs in this can craft their own stories that are good, in terms of being persuasive and fruitful and ethical. In some ways, it is a new kind of nation or a new kind of religion.
Sorry, did you follow up on the promise of why agent foundations are the same as preference synthesis is the same as the religious metaphysics?
I was giving overlaps between agent foundations and religious metaphysics. I haven’t spoken about preference synthesis yet, but basically I think you need a moral-realism-like-thing—universal laws for coordination—that the preference synthesizer would have to be based on. Otherwise, you can just have irreconcilable values that can’t be synthesized.
I think these universal laws of coordination are at the heart of what religions are actually about, and require something resembling the metaphysics that I claim are shared in the mystical traditions of all the major world religions in order to properly formulate.
If we figure out agent foundations, I think we’re also going to figure out this metaphysics, and these universal laws of coordination. And I think having a technical understanding of these universal laws of coordination are a necessary prerequisite for building trustworthy preference synthesis technology.
Cruxes around coordinating with the masses
Why don’t you ask me a question about my worldview?
[Long pause]
Why are you so down on coordinating with the rest of the world?
I mean, I believe in arguments and evidence and conceptual understanding and research and science and breakthroughs. I don’t believe in social media and I guess I don’t believe in most governments as a space to reliably have true things happen to you and not moral mazes.
Generally, I don’t know, communication seems hard.
What seems hard about it?
Different question, actually.
Well, briefly, the answer is something like reality has a surprising amount of detail and when you try to get two people to look at the same bit of reality and to have a shared model of it. It’s not in fact a simple task. I’m not saying it’s hard. I’m saying it’s not even simple to get on the same page about that part of reality because the reality itself is not simple.
What was your other question?
How do you feel about how HPMOR Harry treated Ron Weasley?
I think the real Harry Potter, the original Harry Potter would’ve been upset about it. I think it seems like a very natural way for things to go in-universe.
I think he was an asshole.
Which character?
HPMOR Harry to Ron Weasley.
I think you got to let that sort of thing go. I mean, you don’t want to get into fights with everyone who’s dismissive or doesn’t get along with you. You’re going to meet thousands of people in your life. Not that important.
If this is your attitude, it makes sense why we might have different views about coordinating with the rest of the world.
Why? I might be forgetting how he treated him, but I mostly remember him being fairly dismissive and uninterested and maybe once or twice a little bit rude. I don’t remember anything especially egregious. Was there anything especially different than my recollection? In your recollection？
Your recollection sounds fairly accurate. For what it’s worth, I’m pretty up for people not wanting to hang out with someone or spend time with them or learn that much from them.
They’re not a married couple. They’re not even… they’re essentially strangers to each other.
Yeah. I think he is rude to strangers. He was rude to that stranger. I’m not holding this… I’m not like “fuck Eliezer for that”. I’m more like, well, obviously if you’re someone who has a superiority complex around people like Ron Weasley, you’re obviously not going to be able to think you can coordinate with the rest of the world. Because whatever dimensions along which you could coordinate, you’re kind of just preemptively dismissing. You’re like, I’ve got my ways of operating and my ways of coordination and if you can’t meet that, then you’re not worth my time.
I’m tempted to argue in defense of the opposing position out of an instinctual disagreeableness that seems healthy, but insofar as it seems like strongly morally flavored to you, I think my main response is I haven’t really thought about it very much and I don’t have a strong personal take.
I don’t endorse any moralistic elements of what I’m saying here, and to the extent I’m being moralistic I don’t endorse it. I’m just trying to say, from a matter-of-fact standpoint, that if this is your attitude toward normal people, you probably won’t consider it tractable to coordinate with them.
I mean, is there a part of the world that I should be coordinating with that I didn’t mention? I think about companies and industries and I think about governments and politicians and I think about activism and protest movements and I think about sort of online communities and social medias. I’m not thinking specifically about certain dudes I know who clearly have IQ 75, who I knew when I was a teenager, but I’m not thinking about those specific people.
In particular, all of the parts of human psychology that are irrational and unreasonable, I think you are not very willing to coordinate with. I think you are willing to, when you see them in individual people, put them aside and coordinate with the rest of them, but I don’t think of you as being very willing to coordinate with the irrationalities in and of themselves.
Do you mean I’m not aware of them or do you mean I’m not willing to...
Work with them. I can imagine a dialogue with you where I’m like, Ben, what if we coordinate with the Ron Weasleys of the world? And you’re like, but they’re so lame. And I’m like, okay, they are kind of lame in the ways you’re saying, but they’re also people, they want to help and they can be very helpful, and actually most of the world is made up of them.
I mean I’m open to the feedback slash critique that I’m not very in touch with median people or below median people, and being like, I think you would actually have a better model of the world if you did that more often is I think I would take that as a pointer in that direction and a reason to move it up to actually prioritize it.
Yeah, I think you’re saying something slightly different, which is something like...
Well, I also, I don’t know how much they respect each other.
You don’t what?
I don’t know. This whole conversation is kind of, it sounds kind of super classist or something in a way that I don’t quite endorse. But anyway, I don’t know how much such folk respect each other.
You don’t know how much what?
I don’t know how much the population of people whose IQ is between 70 and 100… I don’t know how much they respect each other and whether they respect each other more than I’m respecting them. I think many such folk just getting through the day and everyone around them is occasionally source of interests and sources of annoyance.
I don’t know. Probably that’s too cynical of me.
Anyway, we can go hang out with some low IQ people sometime, next time you’re in town if you like. Or I guess maybe I’ll just find a place to do it.
I can do that. I have a friend in mind.
You have a friend in town that maybe sometime we could hang out?
Yes. I am serious. I’ve hung out with him and I really like hanging out with him.
Yeah, I’d be up for that sometime next time you’re in town.
I should wrap up soon-ish.
Yeah, I think we’re 10 minutes over. I also was hoping to leave on time.
Ben finally gets why Zhu doesn’t want to start a new religion
Cool. Yeah. I have one closing thought. Something feels important about the way you kept asking “why aren’t we just trying to start a new religion rather than… isn’t that clearly better than trying to reform the existing religions that are kind of obviously fucked, Alex?” Whereas my frame is more like, the thing I’m doing feels like it effectively is starting a new religion, in the same sort of way that it’s also starting a new government. But, part of the deal is that this sort of “new religion” or “new government” is so different from what people typically mean by “religion” or “government” that using the same term feels misleading.
But even still, on a vibe level, when I imagine you saying “new religion”, it’s got more of a vibe of sort of just brushing aside all of the existing religions, whereas in my conception of things, it’s very important to work with the existing stuff rather than brushing it aside.
Yeah. I feel more like a Jeff Bezos kind of like our relationship to our competitors. Amazon is that when anyone says, what do you think about competitors? I say, what do you think about the users? Focus on the users. And again when you’re like how do you build good stories and institutions that help people live good lives? I’m not like, well what does Christianity, what do the other people say? I’m like, what is the answer to that question? And how can we build an institution that embodies it?
And you might be like, by the way, do you know how this other religion, I was like, oh, that’s a good hint. That’s a good tip. I like some things there. I’ll take some of that, but I’m not trying to reform it. It’s not my first go-to. My first go-to is just focus on the answers to the question and to figure out how to build an institution around it that’s fresh. It just, anyway, they’re just so old. They were made thousand years ago. Most questions, they’re kind of like, you should make Google Docs or you should have Glowfick. There’s so many of their bottle mix we don’t have anymore. So I don’t expect their answers to help that much. And so many of the environments that there are wisdom was set up for very different to these current ones.
I do want to say something else in wrapping up in a second, but maybe you can respond to that first. More the logistical side effects.
Yeah, in some sense, a core thesis of where I’m coming from is that I feel like working with people’s irrationalities is like focusing on the users rather than the competition. For people in Christian Western culture, I think using the language of Christianity in good ways can be a very effective way to reach the users. That feels to me more like the kind of thing I’m doing than reforming existing religions. I’m agnostic as to what happens to the existing religions. They might just crumble. They might successfully get reformed from the inside, or they might turn out to be too bureaucratic and decrepit for that to happen. I don’t know, and I don’t really care.
Okay, so you’re saying you’re trying to talk with people about how to live life well and so forth. And this is the most direct way you have to do it at the minute. And that’s why you’re using religion. Okay, That makes sense to me. That helps understanding your perspective.
Going forward
So it sounded like maybe, so this is the fourth one of these we’ve done, I think? The first one was written and then this is the third call.
Yeah, that’s right.
Okay. I want to say something that’s puzzling to me, so it sounded like maybe we were going to have to wrap these up anyway for a while because we’re busy. And I was going to say, I feel like we’ve done a fair bit of exploration, and it seems plausible that maybe we should pause here and come back if we have a different idea, and maybe try something else in the future, if we want to. But also we could just stop.
I’m down to keep going. I definitely feel like there’s still more to talk about, but I think I’m down either way.
Okay. Nonetheless, I think you would say you were like, I’m busy the next two, three weeks.
Yeah.
Okay. Yeah, that also helps me, it works for me because I have a lot of work that I need to do and I’m probably going to be working weekends starting tomorrow.
Okay.
It’s also plausible to me that… I can imagine that conversation being very different if we have a third person involved or something.
I would be open to that.
But I guess I meant someone, I didn’t, to be clear, I didn’t mean a bridging person. I meant a person with their own perspective or something that was a bit distinct from either of us.
What about my Imam friend?
That sounds so annoying, but it does sound like it would probably be somewhat interesting.
I think you’d be surprised.
I also think I’d be surprised. I also think I would be annoyed.
Sorry. I think you’d be surprised by how not annoyed you might be.
I mean, it depends what we talked about.
Yeah.
Anyway, that sounds incredibly annoying, but I’m probably open to it anyway.
So let’s pick a date to check in. We should try something slightly different than. My vote is we don’t just have exactly the same calls, just some more of them. I think we should do something slightly different. I don’t know, watch a talk together, or invite a third person, or go and hang out with your friend.
That sounds good. Let’s just have something on the calendar. I should really get going now. Thanks, Ben.
Thank you too. Okay, bye-bye!
Goodbye.
It’s very relevant to me in discussing the value of religions that this is NOT Catholic doctrine. I’m pretty sure that this is a heresy, in that it contradicts a core tenet of Christian doctrine: the exclusivity of Christ.
A key component of Christian theology, since the at least shortly after the crucifixion, is that salvation is found uniquely through Jesus Christ.
Now, I’ve read books about the Christian / Jewish mystical traditions that state otherwise: putting forward that Jesus was not the Messiah but a Messiah. I agree that some religious people have ever expressed that perspective. (I think, for some meanings of Messiah, it’s even true. )
The most charitable interpretation is that state of affairs is this is the secret esoteric meaning, accessible only to elite elite initiates (and/or repeatedly rediscovered by mystics), as distinct from the simple stories taught to the masses. But even that interpretation is a stretch: beliefs like this one are explicitly heretical to the explicit, enforced (though less so than in previous eras, see my other comment), doctrine of the organized religions. Proponents of those views were often excommunicated for expressing them.
That’s pretty cruxy for me with regards to my attitudes about religion. I believe there are a few religious pluralists, and I probably like, and maybe agree, with a lot of them. But the thing that they’re doing, which we both think is cool, is generally expressly forbidden/denied by institutionalized religion (at least in the West—I don’t know enough about non-Abrahamic institutionalized religions to say one way or the other).
My read of your statements is that they’re they’re giving too much credit to the Catholic Priesthood, and the Mormon priesthood, and the community of Protestant ministers, because there are a tiny number of religious pluralists who are expressing views and attitudes that those much larger organizational structures explicitly deny.
It seems like you’re doing the opposite of throwing out the baby with the bathwater—refusing to throw out old, dirty, bathwater because...there’s a small chunk of soap, in it or something. (I’m aware I’m staining the analogy). If your message was “hey: there’s some useful soap in this bathwater—we should take it out and use it”, I would be sympathetic. But your rhetoric reads to me as much more conciliatory than that, “yeah this bathwater isn’t perfect, but it’s actually pretty good!”
This is admittedly in the connotation, not the detonation. I expect we agree about most of the facts on the ground. But my impression of your overall attitude is that it’s not accurately representing organized religions as a class in their actual expressed views and their actual behavior and impacts on the world.
I don’t know much about religion, but my impression is the Pope disagrees with your interpretation of Catholic doctrine, which seems like strong counterevidence. For example, seethis quote:
And this one:
Semi-related ACX post that came out today: Against The Cultural Christianity Argument.
“Thou shalt have no other Schelling points before me” is a pretty strong attractor for (at least naive) coordination tech.
An organisation such as the Catholic Church primarily wants to perpetuate its own existence, so of course the official doctrine is that they are The One True Church.
An individual Catholic, on the other hand, might genuinely believe that the benefits of religion are also available from other suppliers.
But by believing that they automatically become not Catholic any more, according to the definition of Catholic given by the Catholic Boss who is also the only one with the right to make the rules. If they state that openly they are liable to be excommunicated, though of course most of the times no one will care (even in much darker times the Inquisition probably wouldn’t come after every nobody who said something blasphemous once).
Possibly similar dilemma with e.g. UK political parties, who generally have a rule that publicly supporting another party’s candidate will get you expelled.
An individual party member, on the other hand, may well support the party’s platform in general, but think that that one particular candidate is an idiot who is unfit to hold political office—but is not permitted to say so,
(There is a joke about the Whitby Goth Weekend that everyone thinks half the bands are rubbish, but there is no consensus on which half that is. Something similar seems to hold for Labour Party supporters.)
I think there is rather a lot of soap to be found… but it’s very much not something you can find by taking official doctrine as an actual authority.
Well, perhaps, but due to global commerce I can just go to the store and buy a bar of soap much more easily.
And perhaps you are fond of that particular type of soap and it’s a bit harder to find the specific type that you’re looking for but it’s still not really worth saving the old bathwater for it, instead of just looking for that specific type of soap?
I suspect this issue can be side-stepped if you point to some “pluralist (neo)-Hinduism” that makes Jesus an avatar of Vishnu.
Some quick reactions:
I believe that religions contain lot of psychological and cultural insight, and it’s plausible to me that many of them contain many of the same insights.
Religions can be seen as solutions to the coordination problem of how to get many very different people to trust each other. However, most of them solve it in a way which conflicts with other valuable cultural technologies (like science, free speech, liberal democracy, etc). I’m also sympathetic to the Nietzschean critique that they solve it in a way which conflicts with individual human agency and flourishing.
Religions are, historically, a type of entity that consolidate power. E.g. Islam right now has a lot of power over a big chunk of the world. We should expect that the psychological insights within religions (and even the ones shared across religions) have been culturally selected in part for allowing those religions to gain power.
So my overall position here is something like: we should use religions as a source of possible deep insights about human psychology and culture, to a greater extent than LessWrong historically has (and I’m grateful to Alex for highlighting this, especially given the social cost of doing so).
But we shouldn’t place much trust in the heuristics recommended by religions, because those heuristics will often have been selected for some combination of:
Enabling the religion as a whole (or its leaders) to gain power and adherents.
Operating via mechanisms that break in the presence of science, liberalism, individualism, etc (e.g. the mechanism of being able to suppress criticism).
Operating via mechanisms that break in the presence of abrupt change (which I expect over the coming decades).
Relying on institutions that have become much more corrupt over time.
Where the difference between a heuristic and an insight is something like the difference between “be all-forgiving” and “if you are all-forgiving it’ll often defuse a certain type of internal conflict”. Insights are about what to believe, heuristics are about what to do. Insights can be cross-checked against the rest of our knowledge, heuristics are much less legible because in general they don’t explain why a given thing is a good idea.
IMO this all remains true even if we focus on the heuristics recommended by many religions, i.e. the pluralistic focus Alex mentions. And it’s remains true even given the point Alex made near the end: that “for people in Christian Western culture, I think using the language of Christianity in good ways can be a very effective way to reach the users.” Because if you understand the insights that Christianity is built upon, you can use those to reach people without the language of Christianity itself. And if you don’t understand those insights, then you don’t know how to avoid incorporating the toxic parts of Christianity.
I think this:
is the actual reason religions matter so much, and to a larger extent why they were created so much.
In slogan form, religion turns prisoner’s dilemmas into stag hunts.
Thanks a lot for the kind words!
I think we’re interpreting “pluralism” differently. Here are some central illustrations of what I consider to be the pluralist perspective:
the Catholic priest I met at the Parliament of World Religions who encouraged someone who had really bad experiences with Christianity to find spiritual truth in Hinduism
the passage in the Quran that says the true believers of Judaism and Christianity will also be saved
the Vatican calling the Buddha and Jesus great healers
I don’t think “lots of religions recommend X” means the pluralist perspective thinks X is good. If anything, the pluralist perspective is actually pretty uncommon / unusual among religions, especially these days.
I think this doesn’t work for people with IQ ⇐ 100, which is about half the world. I agree that an understanding of these insights is necessary to avoid incorporating the toxic parts of Christianity, but I think this can be done even using the language of Christianity. (There’s a lot of latitude in how one can interpret the Bible!)
If I change “i.e. the pluralist focus Alex mentions” to “e.g. the pluralist focus Alex mentions” does that work? I shouldn’t have implied that all people who believe in heuristics recommended by many religions are pluralists (in your sense). But it does seem reasonable to say that pluralists (in your sense) believe in heuristics recommended by many religions, unless I’m misunderstanding you. (In the examples you listed these would be heuristics like “seek spiritual truth”, “believe in (some version of) God”, “learn from great healers”, etc.)
I personally don’t have a great way of distinguishing between “trying to reach these people” and “trying to manipulate these people”. In general I don’t even think most people trying to do such outreach genuinely know whether their actual motivations are more about outreach or about manipulation. (E.g. I expect that most people who advocate for luxury beliefs sincerely believe that they’re trying to help worse-off people understand the truth.) Because of this I’m skeptical of elite projects that have outreach as a major motivation, except when it comes to very clearly scientifically-grounded stuff.
If your main point is “don’t follow religious heuristics blindly, only follow them if you actually understand why they’re good” I’m totally with you. I think I got thrown off a bit because, AFAIU, the way people tend to come to adopt pluralist views is by doing exactly that, and thereby coming to conclusions that go against mainstream religious interpretations. (I am super impressed that the Pope himself seems to have been going in this direction. The Catholic monks at the monastery I visited generally wished the Pope were a lot more conservative.)
I use heuristics similar to those for communicating to young children.
This is why I mostly want religious pluralist leaders who already have an established track record of trustworthiness in their religious communities to be in charge of getting the message across to the people of their religion.
Reminded me of this:
In physics, the objects of study are mass, velocity, energy, etc. It’s natural to quantify them, and as soon as you’ve done that you’ve taken the first step in applying math to physics. There are a couple reasons that this is a productive thing to do:
You already derive benefit from a very simple starting point.
There are strong feedback loops. You can make experimental predictions, test them, and refine your theories.
Together this means that you benefit from even very simple math and can scale up smoothly to more sophisticated. From simply adding masses to F=ma to Lagrangian mechanics and beyond.
It’s not clear to me that those virtues apply here:
I don’t see the easy starting point, the equivalent of adding two masses.
It’s not obvious that the objects of study are quantifiable. It’s not even clear what the objects of study are.
Formal statements about religion must be unfathomably complex?
I don’t see feedback loops. It must be hard to run experiments, make predictions, etc.
Perhaps these concerns would be addressed by examples of the kind of statement you have in mind.
It could also help for zhukeepa to give any single instance of such a ‘Rosetta Stone’ between different ideologies or narratives or (informal) worldviews. I do not currently know what to imagine, other than a series of loose analogies, which can be helpful, but are a bit of a difficult target to point at and I don’t expect to find with a mathematical framework.
It’s relevant that I think of the type signature of religious metaphysical claims as being more like “informal descriptions of the principles of consciousnes / the inner world” (analogously to informal descriptions of the principles of the natural world) than like “ideology or narrative”. Lots of cultures independently made observations about the natural world, and Newton’s Laws in some sense could be thought of as a “Rosetta Stone” for these informal observations about the natural world.
I’m not sure exactly what you’re asking—I wonder how much my reply to Adam Shai addresses your concerns?
I will also mention this quote from the category theorist Lawvere, whose line of thinking I feel pretty aligned with:
I think getting technical precision on philosophical concepts like these will play a crucial role in the kind of math I’m envisioning.
Very helpful, thank you.
Thank you for recording and posting these, I feel like I learned a lot, both about how to have conversations and lots of little details like the restaurant thing as proto preference synthesizer and the trauma cancer analogy and the Muhammad story and the disendorsing all judgements/resentments thing.
Thanks, this really warmed my heart to read :) I’m glad you appreciated all those details!
My read, from my very limited perspective mostly engaging with new-age folks, and my serious-Christian highschool peers, as a teenager, is that this is mostly a response to increased secularism and waning traditional religious influence.
Enlightenment, post-Westphalian norms are very heavy on freedom of religion. As the influence of religions has waned, and societal mores have come to rest more and more on the secular morality. As an example, compare how in the 1950s, the question of whether someone “a good person” was loaded on whether they were an upstanding christian, and in 2010, the question of whether someone is “a good person” is much more highly loaded on whether they’re racist.
In that process, insisting on religious exclusivism has become kind of goshe—it makes you seem like a backwards, bigoted, not-very cosmopolitan, person.
So how can religions respond? They can double down on calling out how mainstream society is sinful and try to fight the culture war. Or they can get with the program, and become hip and liberal, which means being accepting of other religions, to the point of saying that they’re all paths up the same mountain, in defiance of centuries of doctrine and literal bloody violence.
And in practice, different religious communities have taken either one or some combination of these paths.
But this doesn’t represent an awakening to the underlying metaphysical truth underlying all religions (which again, I’m much more sympathetic to than many). It mostly represents, religious institutions adapting to the pressures of modernity and secularism.
It looks to me like mostly a watering down of religion, rather than an expression of some deep spiritual impulse.
It’s basically the same situation as regards doctrine on evolution by natural selection. The church fought that one, tooth and nail, for centuries, declaring that it was an outright. But secular culture eventually (mostly) beat the church in that culture war, and only now, to stay relevant, does the church backslide putting forward much weaker versions of their earlier position, whereby evolution is real, but compatible with divine creation. It’s confused to look at that situation and attribute it to the Church’s orientation to religion, all along, being more correct than we gave it credit for.
And the sociology matters here because makes different predictions about what will accelerate and increase this trend. If I’m correct, and this shift is mostly downstream of religions losing their influence, compared broader secular mores and values, then we won’t get more of it by increasing the importance, salience, of religion.
Granted, I’ll give you that much of the world is still religious, and maybe the best way to bridge to them to produce the transformations the world needs, is through their religion. But I think you’re overstating how much religious pluralism is a force on the rise, as opposed to a kind of weak-sauce reaction to external forces.
And FWIW, there’s something cruxy for me here. I would be more interested in religions if I thought that there was a positive rising force of religious pluralism (especially one that was grounded in the mystical traditions of the various religions), instead of reactive one.
You’re what I call “agreeing to agree”.
(shit, I guess I never published that blog post, or the solution to it. Uh. this comment is a placeholder for me.)
My recommendation is to sidestep taking about propositions that you agree with or disagree with, and generate hypothetical or real situations, looking for the ones where you’re inclined to take different actions. In my experience, that surfaces real disagreement better than talking about what you believe (for a number of reasons).
eg
If you had the option to press a button to cut the number of observant christians in the world by half (they magically become secular atheists) would you press it? What if you have to choose between doubling the number of observant christians or halving the number, with no status quo option?
If could magically cause 10% of the people in the world to have read a single book, what book? (With the idea being that maybe Alex would recommend a religious book, and Ben probably wouldn’t.)
What attitude do each of you have about SBF or FTX? Do you have some joy about his getting jail time? What would each of you do if you were the judge of his case?
Same question but for other FTX employees.
If Ben catches a person outright lying on LessWrong, what should he do?
[possibly unhelpfully personal] Should Ben have done anything different in his interactions with Nonlinear?
(Those are just sugestions. Probably you would need to generate a dozen or so, and tweak and iterate until you find a place where you have different impulses for what to do in that situation.)
If you find an example where you take different actions or have a different attitude, now you’re in a position to start trying to find cruxes.
Personally, I’m interested: does this forgiveness business suggest any importantly different external actions? Or is it mostly about internal stance?
My answer to a few of Eli’s prompts below.
Half, naturally. Best to rip the band-aid off and then seek new things that aren’t fundamentally based in falsehoods and broken epistemologies.
10% of the world is way too many people. I’m not sure a lot of them would be capable of comprehending a lot of the material I’ve read, given the distribution of cognitive and reading ability in the world. It would probably have to be something written for most people to read, not like ~anything on LessWrong. Like, if HPMOR could work I’d pick that. If not, perhaps some Dostoyevsky? But, does 10% of the world speak English? I am confused about the hypothetical.
But not the Bible or the Quran or what-have-you, if that’s what you’re thinking.
I don’t think ‘joy’ is what I’d feel, but I am motivated by the prospect of him getting jail time. If I could work to counterfactually increase his jail time or cause it to happen at all, I’d be very motivated to spend a chunk of my life doing so. If someone were to devote much of their adult life to making sure SBF went to jail, I’d think that a pretty good way to spend one’s life. The feeling is one of ‘setting things right’. Justice is important and motivating. There’s something healing about horrendous behavior on a mass scale getting punished, rather than people behaving in sickening ways and then getting to simply move forward with their lives with no repercussions or accountability. Something is very broken when people can betray your trust and intentionally hurt you that much for no good reason, and for them to get away with it can really make the hurt person quite twisted and unable to trust other people to be good. And so it is good to set things right.
If someone verifiably lies, by default I’ll just call it out and have it be a part of their reputation (and bring it up in the future when their reputation is relevant). That’s how most things are dealt with, and in this scene things will likely go pretty badly for them. If they want to apologize and make amends then they can perhaps settle up the costs they imposed on both the people they deceived and the damage they did to social norms, but we have rules about what we moderate (e.g. sockpuppeting), and it isn’t “all unethical behavior”. Most people have lied in their lives (including me), I am not going to personally hold accountable (or rate limit) everyone who has ever behaved badly. In general, someone having behaved badly in their lives does not mean they cannot make good contributions to public discourse. I wouldn’t stop most people involved in major crimes like murder and fraud from commenting on LessWrong about areas of interest / importance, e.g. Caroline Ellison would be totally able to comment on LW about most topics we discuss.
The idea of this conversational technique is that you can shape the hypothetical to find one where the two of you have strong, clear, differing intuitions.
If you’re like “IDK man, most people won’t even understand most of the books that I think are important, and so most of the problem is figuring out something that ‘works’ at all, not picking the best thing”, you could adjust the hypothetical, accordingly. What about 10% of the global population sampled randomly from people who have above 110 IQ, and if they’re not english speakers they get a translation? Does that version of the hypothetical give you a clearer answer?
Or like (maybe this is a backwards way to frame things but) I would guess[1] that there’s a version of some question like this to which you would answer the sequences, or something similar, since it seems like your take is “[one of] the major bottleneck[s] in the world is making words mean things.” Is there a version that does return the sequences or similar?
FYI, I feel interested in these answers and wonder if Alex disagrees with either the specific actions or something about the spirit of the actions.
For instance, my stereotype of religious prophets, is that they don’t dedicate their life to taking down, or prosecuting a particular criminal. My personal “what would Jesus / Buddha do?” doesn’t return “commit my life to making sure that guy gets jail time.” Is that an “in” towards your actual policy differences (the situations in the world where you would make different tradeoffs)?
Though obviously, don’t let my guesses dictate your attitudes. Maybe you don’t actually think anything like that!
I like the technique.
Yeah, I should probably write these up. I called this “action-oriented operationalization” (in contrast to prediction-oriented operationalization) and at least part of the credit goes to John Salvatier for developing it.
Really appreciated this exchange, Ben & Alex have rare conversational chemistry and ability to sense-make productively at the edge of their world models.
I mostly agree with Alex on the importance of interfacing with extant institutional religion, though less sure that one should side with pluralists over exclusivists. For example, exclusivist religious groups seem to be the only human groups currently able to reproduce themselves, probably because exclusivism confers protection against harmful memes and cultural practices.
I’m also pursuing the vision of a decentralized singleton as alternative to Moloch or turnkey totalitarianism, although it’s not obvious to me how the psychological insights of religious contemplatives are crucial here, rather than skilled deployment of social technology like the common law, nation states, mechanism design, cryptography, recommender systems, LLM-powered coordination tools, etc. Is there evidence that “enlightened” people, for some sense of “enlightened” are in fact better at cooperating with each other at scale?
If we do achieve existential security through building a stable decentralized singleton, it seems much more likely that it would be the result of powerful new social tech, rather than the result of intervention on individual psychology. I suppose it could be the result of both with one enabling the other, like the printing press enabling the Reformation.
This is sus.
In an effort to assert that everyone is basically good, we’ve found ourselves asserting that people who are very psychologically different than us, who were born that way, are fundamentally mistaken, at a genetic level?
I at least want to be open to the possibility of the sociopath who straightforwardly likes being a sociopath—it works for them, they sincerely don’t want to be different, they think it would be worse to be neurotypical. Maybe, by their own, coherent lights, they’re just right about that.
(In much the same way that many asexuals look at the insanity of sexual desire and think “why would I want to be like that?!”, while the normally-sexed people would absolutely not choose delete their sex-drive.)
Fantastic. I would love to figure out of this is true.
It’s also a crux for my worldview too. A Double Crux!
From my personal experience with you, Alex, I agree that you seem unusually good at earning the (at least a certain kind of) trust of people I encounter, from a broad array of background. I don’t particularly attribute that to the forgiveness thing? More-so to the fact that you’re friendly, and have a unusual-for-humans ability and inclination to pass ITTs.
Is there a minimal operationalized version of this? Something that is the smallest formal or empirical result one could have that would count to you as small progress towards this goal?
I’m not sure how much this answers your question, but:
I actually think Buddhism’s metaphysics is quite well-fleshed-out, and AFAIK has the most fleshed-out metaphysical system out of all the religious traditions. I think it would be sufficient for my goals to find a formalization of Buddhist metaphysics, which I think would be detailed and granular enough to transcend and include the metaphysics of other religious traditions.
I think a lot of Buddhist claims can be described in the predictive processing framework—see e.g. this paper giving a predictive processing account of no-self, and this paper giving a predictive processing account of non-dual awareness in terms of temporal depth. I would consider these papers small progress towards the goal, insofar as they give (relatively) precise computational accounts of some of the principles at play in Buddhism.
I don’t think the mathematical foundations of predictive processing or active inference are very satisfactory yet, and I think there are aspects of Buddhist metaphysics that are not possible to represent in terms of these frameworks yet. Chris Fields (a colleague of Michael Levin and Karl Friston) has some research that I think extends active inference in directions that seem promising for putting active inference on sounder mathematical footing. I haven’t looked too carefully into his work, but I’ve been impressed by him the one time I talked with him, and I think it’s plausible that his research would qualify as small progress toward the goal.
I’ve considered any technical model that illustrates the metaphysical concepts of emptiness and dependent origination (which are essentially the central concepts of Buddhism) to be small progress towards the goal. Some examples of this:
In his popular book Helgoland, Carlo Rovelli directly compares the core ideas of relational quantum mechanics to dependent origination (see also the Stanford Encyclopedia of Philosophy page on RQM).
It can be very hard to wrap one’s mind around how the Buddhist concept of emptiness could apply for the natural numbers. I found John Baez’s online dialogue about how “‘the’ standard model [of arithmetic] is a much more nebulous notion than many seem to believe” to be helpful for understanding this.
Non-classical logics that don’t take the law of excluded middle for granted helped me to make sense of the concept of emptiness in the context of truth vaues; the Kochen-Specker theorem helped me make sense of the concept of emptiness in the context of physical properties; and this paper giving a topos perspective on the Kochen-Specker theorem helped me put the two together.
Thanks this was clarifying. I am wondering if you agree with the following (focusing on the predictive processing parts since that’s my background):
There are important insights and claims from religious sources that seem to capture psychological and social truths that aren’t yet fully captured by science. At least some of these phenomenon might be formalizable via a better understanding of how the brain and the mind work, and to that end predictive processing (and other theories of that sort) could be useful to explain the phenomenon in question.
You spoke of wanting formalization but I wonder if the main thing is really the creation of a science, though of course math is a very useful tool to do science with and to create a more complete understanding. At the end of the day we want our formalizations to comport to reality—whatever aspects of reality we are interested in understanding.
Yes, I agree with this claim.
That feels resonant. I think the kind of science I’m hoping for is currently bottlenecked by us not yet having the right formalisms, kind of like how Newtonian physics was bottlenecked by not having the formalism of calculus. (I would certainly want to build things using these formalisms, like an ungameable steel-Arbital.)
(I didn’t read most of the dialogue so this may be addressed elsewhere)
I think this is subtly but importantly wrong. I think what you’re actually supposed to be trying to get at is more like creating preferences than reconciling preferences.
I’m not sure how you’re interpreting the distinction between creating a preference vs reconciling a preference.
Suppose Alice wants X and Bob wants Y, and X and Y appear to conflict, but Carol shows up and proposes Z, which Alice and Bob both feel like addresses what they’d initially wanted from X and Y. Insofar as Alice and Bob both prefer Z over X and Y and hadn’t even considered Z beforehand, in some sense Carol created this preference for them; but I also think of this preference for Z as reconciling their conflicting preferences X and Y.
I’m saying that a religious way of being is one where the minimal [thing that can want, in the fullest sense] is a collective.
I don’t really get how what you just said relates to creating vs reconciling preferences. Can you elaborate on that a bit more?
I’ll try a bit but it would take like 5000 words to fully elaborate, so I’d need more info on which part is unclear or not trueseeming.
One piece is thinking of individual humans vs collectives. If an individual can want in the fullest sense, then a collective is some sort of combination of wants from constituents—a reconciliation. If an individual can’t want in the fullest sense, but a collective can, then: If you take several individuals with their ur-wants and create a collective with proper wants, then a proper want has been created de novo.
The theogenic/theopoetic faculty points at creating collectives-with-wants, but it isn’t a want itself. A flowerbud isn’t a flower.
The picture is complicated of course. For example, individual humans can do this process on their own somewhat, with themselves. And sometimes you do have a want, and you don’t understand the want clearly, and then later come to understand the want more clearly. But part of what I’m saying is that many episodes that you could retrospectively describe that way are not really like that; instead, you had a flowerbud, and then by asking for a flower you called the flowerbud to bloom.
Thanks for the elaboration. Your distinction about creating vs reconciling preferences seems to hinge on the distinction between “ur-want” and “proper want”. I’m not really drawing a type-level distinction between “ur-want” and “proper want”, and think of each flower as itself being a flowerbud that could further bloom. In my example of Alice wanting X, Bob wanting Y, and Carol proposing Z, I’d thought of X and Y as both “proper wants” and “ur-wants that bloomed into Z”
I strongly agree with the bolded part, and have often been in the position of advocating for this “spiritual stance”.
But I feel like this answer is kind of dodging the question of “which frame is actually a more accurate model of reality?” I get that they’re frames, and you can use multiple frames to describe the same phenomenon.
But the “people are basically good underneath their trauma” and the stronger form “all people are basically good underneath their trauma” do make specific predictions about how those people would behave under various circumstances.
Specifically, I’m tracking at least three models:
Almost everyone is traumatized, to various degrees. that trauma inhibits their ability to reason freely about the world, and causes them to respond to various stimuli with long-ago-learned not-very-calibrated flailing responses. Their psychology is mostly made of a stack of more-or-less reflective behavioral patterns and habits which developed under the constraints of the existing psychological ecosystem of behaviors and patterns. but underneath all of that is an forgotten, but ever present Buddha-nature of universal love for all beings.
vs.
Almost everyone is traumatized, to various degrees. that trauma inhibits their ability to reason freely about the world, and causes them to respond to various stimuli with long-ago-learned not-very-calibrated flailing responses. Their psychology is mostly made of a stack of more-or-less reflective behavioral patterns and habits which developed under the constraints of the existing psychological ecosystem of behaviors and patterns. but underneath all of that is an forgotten, but if that trauma is cleared that mind is a basically-rational optimizer for one’s personal and genetic interests.
vs.
Almost everyone is traumatized, to various degrees. that trauma inhibits their ability to reason freely about the world, and causes them to respond to various stimuli with long-ago-learned not-very-calibrated flailing responses. Their psychology is mostly made of a stack of more-or-less reflective behavioral patterns and habits which developed under the constraints of the existing psychological ecosystem of behaviors and patterns. but underneath all of that is an forgotten, but underneath that is a pretty random cluge of evolutionary adaptions.
Obviously, reality will be more complicated than any of these three simplifications, and is probably a mix of all of them to various degrees.
But I think it is strategically important to have a model of 1) what the underlying motivational stack for most humans is like and 2) how much variation is there in underlying motivations, between humans.
This is not my understanding of the psychology of most literal clinical sociopaths. It is my read of, say Stalin.
In reference to some of my other comments here, I would be much more sympathetic to this take if the historical situation was that Mohamed led wars of self defense against aggressors. But he lead wars of conquest in the name of Islam, which continued for another century after his death.
As Jesus says, “by your fruits you will know them.”
Okay. I think the Comet King is baffled by the problem of evil and saw evil as something to destroy. I think he resists evil. And I think part of what I found interesting was at the end of Unsong, there’s this thing about how Thamiel actually has all along been doing God’s will. And everything that looked like evil actually was done with noble intentions or something, that I found… it got me thinking that Scott Alexander might be wiser than the Comet King character he wrote
@Ben Pace, this should have a note that says “spoilers for unsong”, and spoiler tags over the offending sentences?
Yes! I have added them to the post, and also to your comment. Thanks.
Thank you!
So like, there’s already a whole cottage industry of interpretations of religious texts. Jordan Peterson, for instance, gives a bunch of “psychological” readings of the meaning of the bible, which got attention recently. (And notably, he has a broadly pluralistic take on religion).
But there are lots and lots of rabbis and ministers and so on promoting their own interpretations or promoting interpretations, many of which are hundreds of years old. There’s a vast body of scholarship regarding what these texts mean and how to interpret them.
Alex, it sounds like you hope to add one more interpretation to the mix, with firmer mathematical rigor.
Do you think that that one will be taken up by some large fraction of Christians, Muslims, Jews, or Buddhists? Is there some group that will see the formalizations and recognize them as self-evidently the truth that their personal religious tradition was pointing at?
My sense is that Alex feels the relevant thing is that speaking to people from his interpretation of religious texts is a more effective way of communicating with many/most people in the world than not-doing that, which isn’t quite the same as saying “and it will definitely become the most widespread interpretation of these texts”, though is a part of a path toward having the most widespread interpretation.
FYI @Ben Pace, as a reader, I resonate with your annoyance in this conversation.
For me, I agree with many of Alex’s denotational claims, but feel like he’s also slipping in a bunch of connotation that I think is mostly wrong, but because he’s agreeing with the denotation for any particular point you bring up, it feels slippery, or like I’m having one pulled over on me.
It has a motte-and-bailey feel to it. Like Alex will tell me that “oh of course Heaven and Hell are not literal places under the earth and in the sky, that’s obviously dumb.” But, when he goes to talk with religious people, who might think that, or something not quite as wrong as that, but still something I think is pretty wrong, he’ll talk about heaven and hell, without clarifying what hem means, and it will seem like they’re agreeing, but the communitive property of agreement between you and Alex and Alex and the religious person doesn’t actually hold.
Like, it reads to me like Alex keeps trying to say “you and I Ben, we basically agree”, and as an onlooker, I don’t think you actually agree, on a bunch of important but so-far-inexplicit points.
[Ironically, if Alex does succeed in his quest of getting formal descriptions of all this religious stuff, that might solve this problem. Or at least solve it with religious people who also happen to be mathematicians.]
I don’t know if that description matches your experience, this is just my take as an onlooker.
Why?
Yes, but it sounds like you’re not mostly interested in reading about their stories. You’re interested in reading about the stories of prophets, who, overall, have had a mixed impact on the world.
How literally true is this? I would be very interested any examples of secular / non-religious heroes of this type, to help me triangulate the concept you’re pointing at.
Is there a further claim that those insights can really be gotten from other venues? Or that, even though they’re not necessarily a good gateway for any particular person), they’re still better than the alternatives?
I may be totally off, but whenever I read you (zhukeepa) elaborating on the preference synthesizer idea I kept thinking of democratic fine-tuning (paper: What are human values, and how do we align AI to them?), which felt like it had the same vibe. It’s late night here so I’ll butcher their idea if I try to explain them, so instead I’ll just dump a long quote and a bunch of pics and hope you find it at least tangentially relevant:
Example moral graph, which “charts out how much agreement there is that any one value is wiser than another”:
Also, “people endorse the generated cards as representing their values—in fact, as representing what they care about even more than their prior responses. We paid for a representative sample of the US (age, sex, political affiliation) to go through the process, using Prolific. In this sample, we see a lot of convergence. As we report further down, people overwhelmingly felt well-represented with the cards, and say the process helped them clarify their thinking”, which is why I paid attention to DFT at all:
Yeah, I also see broad similarities between my vision and that of the Meaning Alignment people. I’m not super familiar with the work they’re doing, but I’m pretty positive on the the little bits of it I’ve encountered. I’d say that our main difference is that I’m focusing on ungameable preference synthesis, which I think will be needed to robustly beat Moloch. I’m glad they’re doing what they’re doing, though, and I wouldn’t be shocked if we ended up collaborating at some point.
Seems like Monica Anderson was trying to do something like that with BubbleCity. (pdf, podcast)