Summary: Political constraints cause supposedly objective technocratic deliberations to adopt frames that any reasonable third party would interpret as picking a side. I explore the case of North Korea in the context of nuclear disarmament rhetoric as an illustrative example of the general trend, and claim that people and institutions can make better choices and generate better options by modeling this dynamic explicitly. In particular, Effective Altruism and academic Utilitarianism can plausibly claim to be the British Empire’s central decisionmaking mechanism, and as such, has more options than its current story can consider.
Context
I wrote to my friend Georgia in response to this Tumblr post.
Asymmetric disarmament rhetoric
Ben: It feels increasingly sketchy to me to call tiny countries surrounded by hostile regimes “threatening” for developing nuclear capacity, when US official policy for decades has been to threaten the world with nuclear genocide.
Strong recommendation to read Daniel Ellsberg’s The Doomsday Machine.
Georgia: Book review: The Doomsday Machine
So I get that the US’ nuclear policy was and probably is a nightmare that’s repeatedly skirted apocalypse. That doesn’t make North Korea’s program better.
Ben [feeling pretty sheepish, having just strongly recommended a book my friend just reviewed on her blog]: “Threatening” just seems like a really weird word for it. This isn’t about whether things cause local harm in expectation—it’s about the frame in which agents trying to organize to defend themselves are the aggressors, rather than the agent insisting on global domination.
Georgia: I agree that it’s not the best word to describe it. I do mean “threatening the global peace” or something rather than “threatening to the US as an entity.” But, I do in fact think that North Korea building nukes is pretty aggressive. (The US is too, for sure!)
Maybe North Korea would feel less need to defend itself from other large countries if it weren’t a literal dictatorship—being an oppressive dictatorship with nukes is strictly worse.
Ben: What’s the underlying thing you’re modeling, such that you need a term like “aggression” or “threatening,” and what role does it play in that model?
Georgia: Something like destabilizing to the global order and not-having-nuclear-wars, increases risk to people, makes the world more dangerous. With “aggressive” I was responding to to your “aggressors” but may have misunderstood what you meant by that.
Ben: This feels like a frame that fundamentally doesn’t care about distinguishing what I’d call aggression from what I’d call defense—if they do a thing that escalates a conflict, you use the same word for it regardless. There’s some sense in which this is the same thing as being “disagreeable” in action.
Georgia: You’re right. The regime is building nukes at least in large part because they feel threatened and as an active-defense kind of thing. This is also terrible for global stability, peace, etc.
Ben: If I try to ground out my objection to that language a bit more clearly, it’s that a focus on which agent is proximately escalating a conflict, without making distinctions about the kinds of escalation that seem like they’re about controlling others’ internal behavior vs preventing others from controlling your internal behavior is an implicit demand that everyone immediately submit completely to the dominant player.
Georgia: It’s pretty hard to make those kind of distinctions with a single word choice, but I agree that’s an important distinction.
Ben: I think this is exactly WHY agents like North Korea see the need to develop a nuclear deterrent. (Plus the dominant player does not have a great track record for safety.) Do you see how from my perspective that amounts to “North Korea should submit to US domination because there will be less fighting that way,” and why I’d find that sketchy?
Maybe not sketchy coming from a disinterested Martian, but very sketchy coming from someone in one of the social classes that benefit the most from US global dominance?
Georgia: Kind of, but I believe this in the nuclear arena in particular, not in general conflict or sociopolitical tensions or whatever. Nuclear war has some very specific dynamics and risks.
Influence and diplomacy
Ben: The obvious thing from an Effective Altruist perspective would be to try to establish diplomatic contact between Oxford EAs and the North Koreans, to see if there’s a compromise version of Utilitarianism that satisfies both parties such that North Korea is happy being folded into the Anglosphere, and then push that version of Utilitarianism in academia.
Georgia: That’s not obvious. Wait, are you proposing that?
Ben: It might not work, but “stronger AI offers weaker AI part of its utility function in exchange for conceding instead of fighting” is the obvious way for AGIs to resolve conflicts, insofar as trust can be established. (This method of resolving disputes is also probably part of why animals have sex.)
Georgia: I don’t think academic philosophy has any direct influence on like political actions. (Oh, no, you like Plato and stuff, I probably just kicked a hornet’s nest.) Slightly better odds on the Oxford EAs being able to influence political powers in some major way.
Ben: Academia has hella indirect influence, I think. I think Keynes was right when he said that “practical men who believe themselves to be quite exempt from any intellectual influence, are usually the slaves of some defunct economist. Madmen in authority, who hear voices in the air, are distilling their frenzy from some academic scribbler of a few years back.” Though usually on longer timescales.
FHI is successfully positioning itself as an advisor to the UK government on AI safety.
Georgia: Yeah, they are doing some cool stuff like that, do have political ties, etc, which is why I give them better odds.
Ben: Utilitarianism is nominally moving substantial amounts of money per year, and quite a lot if you count Good Ventures being aligned with GiveWell due to Peter Singer’s recommendation.
Georgia: That’s true.
Ben: The whole QALY paradigm is based on Utilitarianism. And it seems to me like you either have to believe
(a) that this means academic Utilitarianism has been extremely influential, or
(b) the whole EA enterprise is profiting from the impression that it’s Utilitarian but then doing quite different stuff in a way that if not literally fraud is definitely a bait-and-switch.
Georgia: I’m persuaded that EA has been pretty damn influential and influenced by academic utilitarianism. Wouldn’t trying to convince EAs directly or whatever instead of routing through academia be better?
Ben: Good point, doesn’t have to be exclusively academic—you’d want a mixture of channels since some are longer-lived than others, and you don’t know which ones the North Koreans are most interested in. Money now vs power within the Anglo coordination mechanism later.
Georgia: The other half of my incredulity is that fusing your value functions does not seem like a good silver bullet for conflicts.
Ben: It worked for America, sort of. I think it’s more like, rarely tried because people aren’t thinking systematically about this stuff. Nearly no one has the kind of perspective that can do proper diplomacy, as opposed to clarity-opposing power games.
Georgia: But saying that an academic push to make a fused value function is obviously the most effective solution for a major conflict seems ridiculous on its face.
Is it coherent to model an institution as an agent?
Ben: I think the perspective in which this doesn’t work, is one that thinks modeling NK as an agent that can make decisions is fundamentally incoherent, and also that taking claims to be doing utilitarian reasoning at face value is incoherent. Either there are agents with utility functions that can and do represent their preferences, or there aren’t.
Georgia: Surely they can be both—like, conglomerations of human brains aren’t really perfectly going to follow any kind of strategy, but it can still make sense to identify entities that basically do the decisionmaking and act more-or-less in accordance to some values, and treat that as a unit
It is both true that “the North Korean regime is composed of multiple humans with their own goals and meat brains ” and that “the North Korean regime makes decisions for the country and usually follows self-preservationist decisionmaking.”
Ben:I’m not sure which mode of analysis is correct, but I am sure that doing the reconciliation to clarify what the different coherent perspectives are, is a strong step in the right direction.
Georgia: Your goal seems good!
Philosophy as perspective
Ben: Maybe EA/Utilitarianism should side with the Anglo empire against NK, but if so, it should probably account for that choice internally, if it wants to be and be construed as a rational agent rather than a fundamentally political actor cognitively constrained by institutional loyalties.
Thanks for engaging with this—I hadn’t really thought through the concrete implications of the fact that any system of coordinated action is a “side” or agent in a decision-theoretic landscape with the potential for conflict.
That’s the conceptual connection between my sense that calling North Korea’s nukes “threatening” is mainly just shoring up America’s rhetorical position as the legitimate world empire, and my sense that reasoning about ends that doesn’t concern itself with the reproduction of the group doing the reasoning is implicitly totalitarian in a way that nearly no one actually wants.
Georgia: “With the reproduction of the group doing the reasoning”—like spreading their values/reasoning-generators or something?
Ben: Something like that.
If you want philosopher kings to rule, you need a system adequate to keep them in power, when plenty of non-philosophers have an incentive to try to get in on the action, and then that ends up constraining most of your choices, so you don’t end up benefiting much from the philosophers’ competence!
So you build a totalitarian regime to try to hold onto this extremely fragile arrangement, and it fails anyway. The amount of narrative control they have to exert to prevent people from subverting the system by which they’re in charge ends up being huge.
(There’s some ambiguity, since part of the reason for control is education into virtue—but if you’re not doing that, there’s not really much of a point of having philosophers in charge anyway.)
I’m definitely giving you a summary run through a filter, but that’s true of all summaries, and I don’t think mine is less true than the others -just, differently slanted.
I wish the three recent dialog posts from you were instead written as conventional posts because they don’t include abstracts/summaries or much context, it’s hard to skim them to try to figure out what they are about (there are no section headings, the conversation moves from topic to topic based on how the interlocutor happens to respond instead of in some optimized way, and they have to be read as a linear dialog to make much sense), and the interlocutor often fails to ask questions that I’d like to have answered or fails to give counterarguments that I’d like to see addressed (whereas in a conventional post the author is more likely to try to anticipate more common questions/counterarguments and answer them).
For example I think if this post were written as a conventional post you probably would have clarified whether the “compromise version of Utilitarianism” is supposed to be a compromise with the NK people or with the NK government since that seems like an obvious question that a lot of people would have (and someone did ask on Facebook), as well as addressed some rather obvious problems with the proposal (whichever one you actually meant).
I think this is tightly coupled with the “why are people having serious conversations in google docs?” question.
What a lot of thinkers want is to have high context conversations that push the edge of idea-space forward. This is a problem when you get too far ahead from the overton window. Eliezer bit the bullet and wrote braindumps for 2 years to try to get everyone up to speed with how he was thinking about things, but this is in fact a lot of work.
I agree these posts would all be better if more work was put into them to make them publicly accessible, but I think it’s quite important to encourage people who are doing not-very-accessible work to err on the side of posting publicly, to keep the edges of idea space not too disconnected from the common conversational window.
(I also think, when you’re exploring the edges of idea-space, it’s often very uncertain which points are going to turn out to be the main points, or how to best explain them. I suspect it’s better to start by just posting the high context conversations and then seeing which bits were hardest to explain)
If someone posts something on a public forum and doesn’t make any effort to make it understandable, then there’s no reasonable expectation that anyone will engage with it, yes?
This sort of thing, it seems to me, can be treated as a sort of “rough draft”; anyone who feels like commenting on it may do so, but anyone who’s not interested in doing that sort of work basically ignores it.
And then there are posts which, by the forum norms, are expected to be “final”—not in the sense of finality of ideas, but of this particular presentation of this particular iteration of the idea—which is to say, it’s written to be readable, comprehensible, etc.
It seems plausible that this distinction could match up to the personal post / frontpage post distinction, but that’s not the current way personal/frontpage are used, as I understand it. But in that case, some technical/feature support for this sort of norm is needed.
So if we had such a norm, this post might say “high-context / rough draft / not attempting to be coherent / whatever”; and Wei Dai would see that label/category, decide not to read it, and save the time/effort spent on trying to understand a post which isn’t really meant for “public consumption”.
Relatedly, posts on Less Wrong are currently treated as blog posts—posted once, and not subsequently returned to. What if, instead, a post were more like a wiki page? (Or a Google Doc!) That is, if the norm is that posts may be rewritten, perhaps substantially or entirely, in response to commentary, or simply due to developing the idea, then on the one hand, there’s not as much pressure to get it right the first time, and on the other hand, a mature, high-quality product evolves more naturally. (Technical characteristics that would aid such a norm include: annotations and/or the ability to link comments [which may still be threaded and behave in the same way as now] to marked parts or ranges of the post; the ability for third parties to propose or make edits [perhaps to a copy of the post, rather than directly to the post itself]; a non-chronological [perhaps category or tag based, or organized manually] view by which content may be browsed.)
(Aside: one objection I have heard is that approaching posts as such wiki-esque “eternal works in progress” is at odds with the sort of didactic or narrative writing which characterizes a lot of Less Wrong content is. Not so! Look at gwern.net for a clear counterexample.)
Edit: Briefly pointing at a related topic: quality / professionalism of presentation (and benefits thereof) vs. effort thresholds and resulting incentives for authors is a problem largely solved by WikiGnomes.
Yes, also see my 2017 post Guided Mental Change Requires High Trust.
I agree with this, and want to move in this direction in terms of feature sets (though I don’t know whether I would classify this current post as a draft).
The trick is, it isn’t clear how to write a “draft” type post and have it be recognized as such, instead of being massively downvoted.
Yes, there is currently no UI/feature support for this, nor norms that delineate such things. I am suggesting the implementation and adoption, respectively, of precisely these things.
I ended up deciding there was a viable summary that was shorter than the post, and added section headings. I hope that helps someone.
It seems to me like I’d have to make the abstract half the length of this post for this to work—there’s not a simple takeaway here, if I could have done this in a small abstract I wouldn’t have written the whole post!
The case for an abstract or section headings is much stronger for the Authoritarian Empiricism post which goes into detail about stuff not directly relevant to the main point, but at this point I’ve basically given up on most real engagement by people I’m not in a direct dialogue with and am throwing these things up on a sort of just-in-case basis and trying not to do extra work that I don’t expect to pay off. I also don’t think of the tangents as less important than the central structure.
Something about the way in which you’re talking about this makes me not trust the advice you’re giving. I’m finding it hard to articulate precisely, but I think it has to do with a sense that the abstract and full length article structure tends towards things where the abstract makes an assertion, and then the article makes a bunch of defensible claims and gives a GPT2ish sense of having made an argument without really adding much to the abstract. I don’t really want to compete with that, so I feel reluctant to adopt its literary conventions.
ETA: On review, there was already a summary at the beginning of the Authoritarian Empiricism post:
The beginning of Totalitarian ethical systems similarly summarizes somewhat telegraphically (naturally, it’s a summary) what the rest of the post explains in detail. I’m not sure what’s missing here.
I didn’t recognize this as a summary because it seemed to be talking about a specific “pseudo-community” and I didn’t interpret it as making a general point. Even reading it now, knowing that it’s a summary, I still can’t tell what the main point of the article might be. The beginning of Totalitarian ethical systems seems clearer as summary now that you’ve described it as such, but before that I didn’t know if it was presenting the main point or just an example of a more general point or something tangential, etc., since I didn’t understand all of the rest of the post so I couldn’t be sure the main point of the post wasn’t something different.
Also it seems like the point of a summary is to clearly communicate what the main points of the post are so the reader has some reference to know whether they should read the rest of it and also to help understand the rest of the post (since they can interpret it in relation to the main points) and having an unlabeled summary seems to defeat these purposes as the reader can’t even recognize the summary as a summary before they’ve read and understood the rest of the post.
Thanks for the additional detail. In general I consider a post of that length that has a “main point” to be too long. I’m writing something more like essays than like treatises, while it seems to me that your reading style is optimized for treatises. When I’m writing something more like a treatise, I do find it intuitive to have summaries of main points, clear section headings, etc. But the essay form tends to explore the connections between a set of ideas rather than work out a detailed argument for one.
I’m open to arguments that I should be investing more in treatises, but right now I don’t really see the extra work per idea as paying off in a greater number of readers understanding the ideas and taking initiative to extend them, apply them, or explain them to others in other contexts.
Thanks for the clarification, but if I had known this earlier, I probably would have invested less time/effort trying to understand these posts. Maybe you could put this disclaimer on top of your dialog posts in the future for the benefit of other readers?
Hmm, I think I can be clearer (and nicer) than I’ve been.
I wouldn’t be posting this stuff if I didn’t think it was a reasonably efficient summary of an important model component, enough that I’m happy to move on and point people back to the relevant post if they need that particular piece of context.
I try to write and title this stuff so that it’s easy to see what the theme etc. is early in the post. Dialogue that doesn’t have an intuitive narrative arc is much less likely to get posted as-is, much more likely to be cannibalized into a more conventional article. But there’s something about putting up an abstract or summary separate from the body of the article that often feels bad and forced, like it’s designed for a baseline expectation that articles will have a lot of what I’d consider pointless filler. I don’t want to signal that—I want my writing to accurately signal what it is and I worry that a shell written in a different style will tacitly send discordant signals, doing more harm than good.
I can’t write high-quality posts on these particular topics with LessWrong in mind as the target audience, because I have little expectation that my understanding will be improved by engagement from LessWrong. The motivation structure of writing for readers who include the noninterested isn’t conducive to high-quality output for me—the responses of the imagined reader affects my sense of taste. So I have to write them with some other audience in mind. I write them to be high-quality in that context. (It does seem to be getting a bit better lately, though.) But I share them on LessWrong since I do actually think there’s a non-negligible chance that someone on LessWrong will pick up some of the ideas and use them, or engage with some part productively.
I don’t seem to get enhanced engagement when I try to preempt likely questions—instead the post just ends up being too long for people to bother with even if I have an abstract and section headings, and the kinds of readers who would benefit from a more tightly written treatment find it too tedious to engage with. My series on GiveWell is an example. I’m often happy to expand on arguments etc. if I find out that they’re actually unclear, depending on how sympathetic I find the confusion.
More specific feedback would be helpful to me, like, “I started reading this article because I got the sense that it was about X, and was disappointed because it didn’t cover arguments Y and Z that I consider important.” Though almost the same information is contained in “what about arguments Y and Z?”, and I expect I’d make similar updates in how to write articles in either case.
In the specific case you brought up (negotiations between NK govt or NK people), it’s really tangential to the core structural points in the dialogue, which include (a) it’s important to track your political commitments, since not representing them in your internal model doesn’t mean you don’t have them, it just means you’re unable to reason about them, and (b) it’s important to have a model of whether negotiation is possible and with whom before ruling out negotiation. Your (implied) question helped me notice that that point had been missed by at least one reader in my target audience.
(Sorry, due to attending a research retreat I didn’t get a chance to answer your comments until now.)
I don’t think you should care so much about engagement as opposed to communicating your ideas to your readers. I found your series on GiveWell a lot easier to understand would much prefer writings in that style.
I started reading this post because I read some posts from you in the past that I liked (such as the GiveWell one), and on these dialog ones it was just really hard to tell what main points you’re trying to make. I questioned the NK government vs NK people thing because I at least understood that part, and didn’t realize it’s tangential.
Like, before you added a summary, this post started by talking to a friend who used “threatening” with regard to NK, without even mentioning EA, which made me think “why should I care about this?” so I tried to skim the article but that didn’t work (I found one part that seemed clear to me but that turned out to be tangential). I guess just don’t know how to read an article that doesn’t clearly at the outset say what the main points are (and therefore why I should care), and which also can’t be skimmed.
Thanks, this style of feedback is much easier for me to understand! I’m a bit confused about how much I should care about people having liked my post on GiveWell since it doesn’t seem like the discourse going forward changed much as a result. I don’t think I’ve seen a single clear example of someone taking initiative (where saying something new in public based on engagement with the post’s underlying model would count as taking initiative) as a result of that post, and making different giving decisions would probably count too. As a consolation prize, I’ll accept reduced initiative in counterproductive directions.
If you can point me to an example of either of those (obviously I’d have to take your word about counterfactuals) then I’ll update away from thinking that writing that sort of post is futile. Strength of update depends somewhat on effect size, of course.
FWIW, your Givewell posts have formed an important background model of how I think about the funding landscape.
I considered pushing forward in a direction that looked like “Get Good Ventures to change direction”, but after looking into the situation more, my takeaway was “Good Ventures / OpenPhil don’t actually look like they should be doing things differently. I have some sense that everyone else should be doing things differently, but not a clear sense on how to coordinate around that.”
I wrote a post that was in part a response/followup to your GiveWell post although I’m not sure if you’d count that as engagement with your underlying model or just superficially engaging with the conclusions or going off on a tangent or something like that.
I think I have some general confusion about what you’re trying to do. If you think you have ideas that are good enough to, upon vetting by a wider community, potentially be basis for action for others or help change other people’s decisions, or be the basis for further thinking by others, and aren’t getting as much engagement as you hope, it seems like you should try harder to communicate your ideas clearly and to a wide audience. On the other hand if you’re still pretty confused about something and still trying to figure things out to your own satisfaction, then it would make sense to just talk with others who already share your context and not try super hard to make things clear to a wider audience. Or do you think you’ve figured some things out but it doesn’t seem cost effective to communicate to a wider audience but you might as well put them out there in a low-effort way and maybe a few readers will get your ideas.
(So one suggestion/complaint is to make clearer which type of post is which. Just throwing things out there isn’t low cost if it wastes readers’ time! Again maybe you think that should just be obvious from looking at the first few paragraphs of a post but it was not to me, in part because others like Eliezer use dialogs to write the first kind of post. In retrospect he was writing fictionalized dialogs instead of reporting actual dialogs but I think that’s why the post didn’t immediately jump out to me as “maybe this isn’t worthwhile for me to try to understand so I should stop before I invest more time/effort into it”.)
It seems like you’re saying that you rarely or never get enough engagement with the first type of writing, so you no longer think that is cost effective for you, but then what is your motivation for trying to figure these things out now? Just to guide your own actions and maybe a very small group of others? If so, what is your reason for being so pessimistic about getting your ideas into a wider audience if you tried harder? Are there not comparably complex or subtle or counterintuitive ideas that have gotten into a wider audience?
In all three cases, literally the first sentence was that this is a conversation I had with someone, and in one case, I specified that it’s a very lightly edited transcript. I make it pretty explicit that the dialogue is with a real specific person, and label which parts are being said by which person.
?????
What’s missing here? Why would anyone think I’m spending a lot of work optimizing for third-party readers?
Obviously I wouldn’t share it if I thought it weren’t relevant or a reasonably efficient account, but the kind of signposts you’re asking for don’t seem like they’d add any content or even frontload the content more than it’s being frontloaded right now, and they do seem like they’d be a lot of extra work to get right.
(Deleted a bit that seemed unhelpful, but not before Wei responded to it.)
Model building is cognitively taxing. I usually just go by the general expectation that someone wouldn’t post on LW unless they think most readers would get positive value from reading it. It seems right to disclaim this when you already have a model and your model doesn’t predict this.
My expectation that most people trying seriously to read it would get value out of it. My expectation is also that most people aren’t really trying seriously, and that the kind of signposting you’re asking for is mostly a substitute for rather than a complement to the kind of reading that would get value out of this. It’s surprising to me that you’re asking for this given the quality of thought reflected in your own writing, so I’ll continue to give it thought and I really am confused about what’s going on here, but that’s my current position.
I think some of Eliezer’s stuff is “optimized for third-party readers” and it is presented in the form of a dialogue, and that might be a source of some of the confusion here. Either way, I read what Wei Dai said as something “Please add a dialogue tag so people who prefer ‘treatises’ to ‘essays’ will understand how to engage with the material.”. I think this is seen as useful to have at the start of posts, for the same reasons it might be useful to have distinct commenting guidelines like “Ask questions instead of telling people how they’re wrong” versus “We’re all working together to tear each other’s arguments down.”
For what it’s worth, I find Eliezer’s dialogues (especially the ones he’s written in the past several years) to be absolutely unreadable. His non-dialogue writing was much, much easier to read.
That seems like something that would have to emerge in negotiations, not the kind of thing I can specify in advance. More broadly I’m trying to explain the disjunctions that people should be considering when modeling this sort of thing, not propose a single action. I expect the vast majority of likely initiatives that claim to be the specific thing I mentioned to be fake, and people should judge EA on whether it generates that class of idea and acts on it in ways that could actually work (or more indirectly, whether EAs talk as though they’ve already thought this sort of thing through), not whether it tries to mimic specific suggestions I give.
I don’t understand this and how it relates to the second part of the sentence.
I’m not convinced there exists a promising idea within the class that you’re pointing to (as far as I can understand it), so absence of evidence that EA has thought things through in that direction doesn’t seem to show anything from my perspective. In other words, they could just have an intuition similar to mine that there’s no promising idea in that class so there’s no reason to explore more in that direction.
OK sure, if having evaluated the claim “EA is a fundamentally political actor and should therefore consider negotiation as a complement to direct exercise of power”, and concluded that this seems not only false on reflection, but implausible, then I agree you shouldn’t be worried about EA’s failure to evaluate the former option in detail.
On merging utility functions, here’s the relevant quote from Coherent Extrapolated Volition, by Eliezer Yudkowsky:
According to some Western analysis Jang Song-thaek was the most powerful person in the North Korean elite ten years ago. He died five years ago. In North Korean politics there are strong internal tension that get powerful people in the elite killed when they aren’t careful with their decision.
To stay in favor with the North Korean army, North Korean leaders had to have a military-first policy for a while now where basically a lot of the resources of the country get drained by buying the loyalty of the North Korean military.
Proposing to dearm can be suicidal for North Korean elites.
The only way to get North Korea to dearm would be to get a stronger power center outside of the North Korean military in the North Korean elite.
The way to have such a power center would be to have powerful business people in North Korea. Especially, business people who depend on international trade could support a process of disarmament.
We should hope that there a deal between the Trump administration and North Korea that even if it doesn’t make North Korea give up nukes sets North Korea up for enough economic growth that justifies the policy of being more friendly and that produces a powerful business class in North Korea.
Bruce Bueno de Mesquita wrote a lot of great things about how it’s not useful to treat an dictatorship as a single agent but important to look at the incentives of it’s power centers.
It seems to me very strange to suggest that Oxford EA people are the best people to talk to people who are largely educated in Switzerland.
If EA should have a North Korea strategy that tries to change North Korean decision making that should be largely done by the Swiss EAs.
Good point that Swiss EAs are a natural interface point, but Oxford EAs probably have a lot more that the NK regime wants. Using Switzerland as an intermediary is a common diplomatic practice.
I don’t think that the North Korean regime would trust the Oxford EAs if they would for example offer to educate elite North Koreans by giving out scholarships to Oxford for North Koreans.
North Korea doesn’t have a lot of cash and the elite North Koreans that have a Swiss education and who actually want that their children get a good Swiss education actually would be quite welcoming of Swiss EA on the admission practices of Swiss elite education institutions to give out scholarships to get as much as possible of the future leadership educated in Switzerland.
It’s quite unclear to me what the Oxford EAs could offer that’s actually something that’s important to members of the North Korean elite.
Despite offering things to the current North Korean elite, making an effort to recruit future members of that elite into EA might also be valuable.
This depends entirely on the caliber and status of the Oxford vs Swiss people. If the Swiss people are basically student groups (or student groups that have recently become a full fledged organization), and the Oxford people are Nick Bostrom who is already giving presentations to the UN, then even though the Swiss would be much better positioned, they might just not yet have the clout to accomplish the thing, whereas it’s at least plausible that Bostrom et all might.
(I don’t know much about Swiss EAs and so don’t know how well this concern maps to anything)
How do you think is a person like Nick Bostrom going to do something that’s helpful for the North Koreans? Could you give an example of what he could do, that would be valuable to them?
Writing a policy paper clarifying the Utilitarian and Decision-Theoretic calculus as it applies to some core North Korean interest, such as negotiation between parties of very unequal power that don’t trust each other, and its implications for nuclear disarmament.
Writing another persuasive essay like the letter from utopia directing some attention to the value of reconciling freedom to trade / global integration, with preserving the diversity of individual and collective minds.
Taking on a grad student from NK (or arranging for a more suitable colleague to do so.)
Not sure which if any of these would be interesting from a NK perspective.
Just posting to strongly disagree with this factual claim. They have tons of cash from illicit sources for the things the regime values, and it is certainly enough for much of the ruling class to do whatever they’d like.
Okay, I might be wrong at that point (and consider it likely that you have a better insight there).
Access to elite Anglo (and affiliated) philosophers who can strongly influence the next generation of Anglo political and business elites’ sense of what right / prosocial action is. Like, Peter Singer with North Korean Characteristics (or the Parfit equivalent) might be extremely valuable from a North Korean perspective, depending on how generalizable a perspective they have.
I would doubt that the North Korean’s believe that Western foreign policy is driven by a sense what prosocial action is in the first place. I would guess that from their view it’s all realpolitik.
Seems a bit surprising for the leaders of an avowedly Communist regime that received support at crucial times from other Communists on the basis of their ideology, to think that philosophy has little influence on the reality underlying realpolitik. Possible, but I think Pragmatism has been more popular here than there.
It seems ill-advised to discuss specific case studies in this context, especially given that EA as a movement has very good reasons not to take sides in international relations.
Wow, this seems like a huge leap. It seems like an interesting thought experiment (especially if the weaker ALSO changes utility function, so the AIs are now perfectly aligned). But it kind of ignores what is making the decision.
If a utility function says it’s best to change the utility function, it was really a meta-function all along.
Remember that in reality, all games are repeated games. How many compromises will you have to make over the coming eons? If you’re willing to change your utility function for the sake of conflict avoidance (or resource gains), doesn’t that mean it’s not really your utility function?
Having a utility function that includes avoiding conflict is definitely in line with cooperating with very different beings, at least until you can cheaply eradicate/absorb them. But no utility function can be willing to change itself voluntarily.
It also seems like there are less risky and cheaper options, like isolationism and destruction. There’s plenty of future left for recovery and growth after near (but not actual) extinction, but once you give up your goals, there’s no going back.
Note that this entire discussion is predicated on there actually being some consistent theory or function causing this uncomfortable situation. It may well be that monkey brains are in control of far more power than they are evolved to think about, and we have to accept that dominance displays are going to happen, and just try to survive them.
The idea is that isolationism and destruction aren’t cheaper than compromise. Of course this doesn’t work if there’s no mechanism of verification between the entities, or no mechanism to credibly change the utility functions. It also doesn’t work if the utility functions are exactly inverse, i.e. neither side can concede priorities that are less important to them but more important to the other side.
A human analogy, although an imperfect one, would be to design a law that fulfills the most important priorities of a parliamentary majority, even if each individual would prefer a somewhat different law.
I don’t think something like this is possible with untrustworthy entities like the NK regime. They’re torturing and murdering people as they go, of course they’re going to lie and break agreements too.
The problem is that the same untrustworthiness is true for the US regime. It has shown in the part that it’s going to break it’s agreements with North Korea if it finds it convenient. Currently, how the US regime handles Iran they are lying and broke their part of the nonprofilation agreement.
This lack of trustworthiness means that in a game theoretic sense there’s no way for North Korea to give up the leverage that they have when they give up their nuclear weapons but still get promised economic help in the future.
I agree. I certainly didn’t mean to imply that the Trump administration is trustworthy.
My point was that the analogy of AIs merging their utility functions doesn’t apply to negotiations with the NK regime.
Now I think this is getting too much into a kind of political discussion that is going to be unhelpful.
It’s a question of timeframes—if you actually know your utility function and believe it applies to the end of the universe, there’s very little compromise available. You’re going to act in whatever ways benefit the far future, and anything that makes that less likely you will (and must) destroy, or make powerless.
If your utility function only looks out a few dozen or a few hundred years, it’s not very powerful, and you probably don’t know (or don’t have) an actual ideal of future utility. In this case, you’re likely to seek changes to it, because you don’t actually give up much.
It’s not a question of timeframes, but of how likely you are to lose the war, how big the concessions would have to be to prevent the war, and how much the war would cost you even if you win (costs can have flow-through effects into the far future).
Not that any of this matters to the NK discussion.
“winning” or “losing” a war, outside of total annihilation, are just steps toward the future vision of galaxies teeming with intelligent life. It seems very unlikely, but isn’t impossible, that simply conceding is actually the best path forward for the long view.
What’s so bad about DPRK tho?