The âpublic debateâ about AI is confusing for the general public and for policymakers because it is a three-sided debate
Summary of Argument: The public debate among AI experts is confusing because there are, to a first approximation, three sides, not two sides to the debate. I refer to this as a đșthree-sided framework, and I argue that using this three-sided framework will help clarify the debate (more precisely, debates) for the general public and for policy-makers.
Broadly speaking, under my proposed đșthree-sided framework, the positions fall into three broad clusters:
AI âpragmatistsâ or realists are most worried about AI and power. Examples of experts who are (roughly) in this cluster would be Melanie Mitchell, Timnit Gebru, Kate Crawford, Gary Marcus, Klon Kitchen, and Michael Lind. For experts in this group, the biggest concern is how the use of AI by powerful humans will harm the rest of us. In the case of Gebru and Crawford, the âpowerful humansâ that they are most concerned about are large tech companies. In the case of Kitchen and Lind, the âpowerful humansâ that they are most concerned about are foreign enemies of the U.S., notably China.
AI âdoomersâ or extreme pessimists are most worried about AI causing the end of the world. @Eliezer Yudkowsky is, of course, the most well-known to readers of LessWrong but other well-known examples include Nick Bostrom, Max Tegmark, and Stuart Russell. I believe these arguments are already well-known to readers of LessWrong, so I wonât repeat them here.
AI âboostersâ or extreme optimists are most worried that we are going to miss out on AI saving the world. Examples of experts in this cluster would be Marc Andreessen, Yann LeCun, Reid Hoffman, Palmer Luckey, Emad Mostaque. They believe that AI can, to use Andreessenâs recent phrase, âsave the world,â and their biggest worry is that moral panic and overregulation will create huge obstacles to innovation.
These three positions are such that, on almost every important issue, one of the positions is opposed to a coalition of the other two of the positions
AI Doomers + AI Realists agree that AI poses serious risks and that the AI Boosters are harming society by downplaying these risks
AI Realists + AI Boosters agree that existential risk should not be a big worry right now, and that AI Doomers are harming society by focusing the discussion on existential risk
AI Boosters and AI Doomers agree that AI is progressing extremely quickly, that something like AGI is a real possibility in the next few years, and that AI Realists are harming society by refusing to acknowledge this possibility
Why This Matters. The âAI Debateâ is now very much in the public consciousness (in large part, IMHO, due to the release of ChatGPT), but also very confusing to the general public in a way that other controversial issues, e.g. abortion or gun control or immigration, are not. I argue that the difference between the AI Debate and those other issues is that those issues are, essentially two-sided debates. Thatâs not completely true, there are nuances, but, in the publicâs mind at their essence, they come down to two sides.
To a naive observer, the present AI debate is confusing, I argue, because various experts seem to be talking past each other, and the âexpert positionsâ do not coalesce into the familiar structure of a two-sided debate with most experts on one side or the other. When there are three sides to a debate, then one fairly frequently sees what look like âtemporary alliancesâ where A and C are arguing against B. They are not temporary alliances. They are based on principles and deeply held beliefs. Itâs just that, depending on how you frame the question, you wind up with âstrange bedfellowsâ as two groups find common ground on one issue even though they are sharply divided on others.
Example: the recent Munk Debate showed a team of âdoomersâ arguing with a mixed team of one booster and one realist
The recent Munk Debate on AI and existential risk illustrates the three-sided aspect of the debates and how, IMHO the đșthree-sided framework can help make sense of the conflict.
To summarize, Yoshua Bengio and Max Tegmark argued that âAI research and development poses an existential threat.â In other words, what Iâm calling the âAI Doomerâ position.
Arguing for the other side was a team made up of Yann LeCun and Melanie Mitchell. When I first listened to the debate, I thought both Mitchell and LeCun made strong arguments, but I found it hard to make their arguments fit together.
But after applying the đșthree-sided framework, it appeared to me that LeCun and Mitchell were both strongly opposed to the âdoomerâ position but for very different reasons.
LeCun, as an đ AI Booster, argued for the vast potential and positive impact of AI. He acknowledged challenges but saw these as technical issues to be resolved rather than insurmountable obstacles or existential threats. He argued for AI as a powerful tool that can improve society and solve complex problems.
On the other hand, Mitchell, representing the âïž AI Realist perspective, questioned whether, in the near term, AI could ever reach a stage where it could pose an existential threat. While she agreed that AI presents risks, she argued that the most important risks are related to immediate, tangible concerns like job losses or the spread of disinformation.
Analyzed under the đșthree-sided framework, then, the Munk Debate was between:
An âAll Doomerâ team of Bengio and Tegmark, each making Doomer arguments; versus
A âMixed Realist/âBoosterâ team of âRealistâ Mitchell making Realist anti-Doomer arguments and âBoosterâ LeCun, making Booster anti-Doomer arguments.
Thought experiment: other debates.
Letâs imagine another debate on the question âat this stage over-regulation of AI is a much bigger threat to humanity than under-regulation.â My take is that the people taking what Iâm calling the AI Booster position, such as Marc Andreessen, Yan LeCun, Reid Hoffman, would agree with this proposition. While both the AI Doomers, such as Yudkowsky, Bostrom, Soares, and the AI Realists, such as Melanie Mitchell and Timnit Gebru would agree that under regulation is a greater danger
Yet another debate could be on the question: âWe are likely to see something like AGI in the next 5 yearsâ Here, the Realistsâmany of whom believe that claims for AI are vastly overhypeâwould argue the ânoâ side, while both the Booster and the Doomers would team up against the Realists to argue that yes, AGI is likely in the relatively near term.
- A Model-based ApÂproach to AI ExÂisÂtenÂtial Risk by (25 Aug 2023 10:32 UTC; 45 points)
- Helpful exÂamÂples to get a sense of modÂern auÂtoÂmated manipulation by (12 Nov 2023 20:49 UTC; 33 points)
- 's comment on An apÂpeal to peoÂple who are smarter than me: please help me clarÂify my thinkÂing about AI by (EA Forum; 6 Aug 2023 14:09 UTC; 1 point)
I think thereâs some truth to this framing, but Iâm not sure that peopleâs views cluster as neatly as this. In particular, I think there is a âhow dangerous is existential riskâ axis and a âhow much should we worry about AI and Powerâ axis. I think you rightly identify the âboosterâ cluster (x-risk fake, AI +power nothing to worry about) and ârealistâ (x-risk sci-fi, AI + power very concerning) but I think you are missing quite a lot of diversity in peopleâs positions along other axes that make this arguably even more confusing for people. For example, I would characterise Bengio as being fairly concerned about both x-risk and AI+power, wheras Yudkowsky is extremely concerned about x-risk and fairly relaxed about AI+power.
I also think itâs misleading to group even âdoomersâ as one cluster because thereâs a lot of diversity in the policy asks of people who think x-risk is a real concern, from âmore research neededâ to âshut it all downâ. One very important group you are missing are people who are simultaneously quite (publicly) concerned about x-risk, but also quite enthusiastic about pursuing AI development and deployment. This group is important because it includes Sam Altman, Dario Amodei and Demis Hassabis (leadership of the big AI labs), as well as quite a lot people who work developing AI or work on AI safety. You might summarise this position as âAI is risky, but if we get it right it will save us allâ. As they are often working at big tech, I think these people are mostly fairly un-worried or neutral about AI + power. This group is obviously important because they work directly on the technology, but also because this gives them a loud voice in policy and the public sphere. You might think of this as a âhow hard is mitigating x-riskâ axis. This is another key source of disagreement : going from public statements alone, I think (say) Sam Altman and Eliezer Yudkowsky agree on the âhow dangerousâ axis and are both fairly relaxed Silicon Valley libertarians on the âAI+powerâ axis, and mainly disagree on how difficult is it to solve x-risk. Obviously peopleâs disagreements on this question have a big impact on their desired policy!
I am quite sure that in a world where friendly tool AIs were provably easy to build and everyone was gonna build them instead of something else and the idea even made sense, basically a world where we know we donât need to be concerned about x-risk, Yudkowsky would be far less ârelaxedâ about AI+power. In absolute terms maybe heâs just as concerned as everyone else about AI+power, but that concern is swamped by an even larger concern.
Maybe I shouldnât have used EY as an example, I donât have any special insight into how he thinks about AI and power imbalances. Generally I get the vibe from his public statements that heâs pretty libertarian and thinks pros outweigh cons on most technology which he thinks isnât x-risky. I think Iâm moderately confident that hes more relaxed about, say, misinformation or big tech platforms dominance than (say) Melanie Mitchell but maybe iâm wrong about that.
Thanks for that feedback. Perhaps this is another example of the tradeoffs in the âhow many clusters are there in this group?â decision. Iâm kind of thinking of this as a way to explain, e.g., to smart friends and family members, a basic idea of what is going on. For that purpose I tend, I guess, to lean in favor of fewer rather than more groups, but of course there is always a danger there of oversimplifying.
I think I may also need to do a better job distinguishing between describing positions vs describing people. Most of the people thinking and writing about this have complicated, evolving views on lots of topics, and perhaps many donât fit neatly, as you say. Since the Munk Debate, Iâve been trying to learn more about, e.g. Melanie Mitchellâs views, and in at least one interview I heard, she acknowledged that existential risk was a possibility, she just thought it was a lower priority than other issues.
I need to think more about the âexistential risk is a real problem but we are very confident that we can solve it on our current pathâ typified by Sam Altman and (maybe?) the folks at Anthropic. Thanks for raising that.
As you note, this view contrasts importantly with both the (1) boosters and (2) the doomers.
My read is that the booster arguments put forth by, Marc Andreessen or Yann LeCun, argue that âexistential riskâ concerns are like worrying about âwhat happens if Aliens invade our future colony on Mars?ââview that âthis is going to be airplane developmentâyes there are risks but we are going to handle it!â
I think youâve already explained very well the difference between the Sam Altman view and the Doomer view. Maybe this needs to be a 2 by 2 matrix? OTOH, perhaps there, in the oversimplified framework, there are two âboosterâ positions on why we shouldnât be inordinatetly worried about existential risk: (1) itâs just not a likely possibility (Andreessen, LeCun) (2) âyes itâs a problem but we are going to solve it and so we donât need to, e.g. shut down AI developmentâ (Altman)
Thinking about another debate question, I wonder about the question
âWe should pour vastly more money and resources into
fixing[eta: solving] the alignment problemâI think(??) that Altman and Yudkowsky would both argue YES, and that Andreessen and LeCun would (I think?) argue NO.
Any post along the lines of yours needs a âpolitical compassâ diagram lol.
I mean itâs hard to say what Altman would think in your hypothetical debate: assuming he has reasonable freedom of action at OpenAI his revealed preference seems to be to devote â 20% of the resources available to his org to âthe alignment problemâ. If he wanted to assign more resources into âsolving alignmentâ he could probably do so. I think Altman thinks heâs basically doing the right thing in terms of risk levels. Maybe thatâs a naive analysis, but I think itâs probably reasonable to take him more or less at face value.
I also think that itâs worth saying that easily the most confusing argument for the general public is exactly the Anthropic/âOpenAI argument that âAI is really risky but also we should build it really fastâ. I think you can steelman this argument more than Iâve done here, and many smart people do, but thereâs no denying it sounds pretty weird, and I think itâs why many people struggle to take it at face value when people like Altman talk about x-riskâit just sounds really insane!
In constrast, while people often think itâs really difficult and technical, I think yudkowskyâs basic argument (building stuff smarter than you seems dangerous) is pretty easy for normal people to get, and many people agree with general âbig tech badâ takes that the ârealistsâ like to make.
I think a lot of boosters who are skeptical of AI risk basically think âAI risk is a load of horseshitâ for various not always very consistent reasons. Itâs hard to overstate how much âdonât anthropomorphiseâ and âthinking about AGI is distracting sillyness by people who just want to sit around and talk all dayâ are frequently baked deep into the souls of ML veterans like LeCun. But I think people who would argue no to your proposed alignment debate would, for example, probably strongly disagree that âthe alignment problemâ is like a coherent thing to be solved.
Itâs worth remembering that the explicit goal of a lot of AI alignment research is to make AI controllable, which makes AI âdoomersâ in a sense allied with (or at least, building a superweapon for) the people in power that the AI ârealistsâ are worried about.
AI alignment is more often than not assumed to be with some abstract âhuman valuesâ which are ought to be collected from the general public and aggregated with some clever, yet undeveloped algorithms. I donât think anyone in AI alignment, once âsolvedâ this problem, would hand the technology over to POTUS or any other powerful leader or a clique.
Maybe sometimes? Mostly I disagree.
My impression is that MIRI would prefer some sort of TESCREAList future, with emulated minds colonizing all of space, or something like that. This is probably something most people would be afraid of. MIRI isnât the only AI alignment research org, but I donât really see other rationalist alignment researchers opposing this desire, I think because theyâre all TESCREALists?
Of course MIRI doesnât seem to be on track towards grabbing control over the world themselves to implement TESCREALism, so more realistically MIRI might produce something that the leading tech companies like OpenAI can incorporate in their work. But âtech companiesâ were one of the original sides mentioned in the OP that ârealistsâ were worried about.
And furthermore, the tech companies have recently decided that they donât know how to control the AI and begged the US government to take control. Rationalists donât really expect the government to do well with this on its own, but I think there is hope that AI safety researchers could produce some tools that the US government could make mandates about using? But that would also mean that âdoomersâ are giving the US government a lot of power?
I think you mischaracterise whatâs happening a lot. Tech companies donât âbeg US government to control AIâ, they beg it to regulate the industry. Thatâs a very big difference. They didnât yet say anything about the control of AI, once itâs build, and (presumably) âalignedâ, apart from well-sounding for PR phrases like âwe will increasingly involve input from more people about where do we take AIâ. In reality, they just donât know yet who and how should âcontrolâ AI (if anyone), and align with whose âvaluesâ. They hope that the âalignment MVPâ, a.k.a. AGI which is an aligned AI safety and alignment scientist, will actually tell them the ârightâ answers to these questions (as per OpenAIâs âsuperalignmentâ agenda).
Your three sided debate maps on to the Collective Intelligence Projectâs âTransformative Technology Trilemmaâ, which has corners on âparticipationâ, âsafetyâ, and âprogressâ. https://ââcip.org/ââwhitepaper#trilemma
Was not aware of this Collective Intelligence Project and glad to learn about them. Iâll take a look. Thanks.
Iâm very eager to find other âthree-sidedâ frameworks that this one might map onto.
Maybe OT, but I also have been reading Arnold Klingâs âThree Languages of Politicsâ but so far have been having trouble mapping Klingâs framework onto this one
Realist and pragmatist donât seem like the best choices of terms, since they pre-judge the issue a bit in the direction of that view.
Maybe something like âmundane-istâ would be better. The ârealistsâ are people who think that AI is fundamentally âmundaneâ and that the safety concerns with AI are basically the same as safety concerns with any new technology (increases inequality by making the powerful more powerful, etc.) But of course âmundane-istâ isnât a real word, which is a bit of a problem.
Thanks. To be honest, I am still wrestling with the right term to use for this group. I came up with ârealistâ and âpragmatistâ as the âleast badâ options after searching for a term that meets the following criteria:
short, ideally one word
conveys the idea of prioritizing (a) current or near-term harms over (b) far-term consequences
minimizes the risk that someone would be offended if the label were applied to them
I also tried playing around with an acronym like SAFEr for âSkeptical, Accountable, Fair, Ethicalâ but couldnât figure out an acronym that I liked.
Would very much appreciate feedback or suggestions on a better term. FWIW, I am trying to steelman the position but not pre-judge the overall debate.
Not exactly what youâre asking for, but maybe a 2x2 could be food for thought.
Thanks. I think this is useful and Iâm trying to think through who is in the upper left hand corner. Are there âAI researchersâ or, more broadly, people who are part of the public conversation who believe (1) AI isnât moving all that fast towards AGI and (2) that itâs not that risky?
I guess my initial reaction is that people in the upper left hand corner just generally think âAI is kind of not that big a dealâ and that there are other societal problems to worry about. does that sound right? Any thoughts on who should be placed in the upper left?
Yeah, I think so. But since those people generally find AI less important (thereâs both less of an upside and less of a downside) they generally participate less in the debate. Hence thereâs a bit of a selection effect hiding those people.
There are some people who arguably are in that corner who do participate in the debate, thoughâe.g. Robin Hanson. (He thinks some sort of AI will eventually be enormously important, but that the near-term effects, while significant, will not be at the level people on the right side think).
Looking at the 2x2 I posted I wonder if you could call the lower left corner something relating to ânon-existential risksâ. That seems to capture their views. It might be hard to come up with a catch term, though.
The upper left corner could maybe be called âscepticsâ.
A useful model for why itâs both appealing and difficult to say âDoomers and Realists are both against dangerous AI and for safetyâletâs work together!â.
AI realism also risks a Security theater that obscures existential risks of AI.
Yes agreed. Indeed one of the things that motivated me to propose this three-sided framework is watching discussions of the following form:
1. A & B both state that they believe that AI poses real risks that the public doesnât understand.
2. A takes (what I now call) the âdoomerâ position that existential risk is serious and all other risks pale in comparison: âwe are heading toward an iceberg and so it is pointless to talk about injustices on the ship re: third class vs first class passengersâ
3. B takes (what I now call) the ârealistâ or âpragmatist positionâ that existential risk is, if not impossible, very remote and a distraction from more immediate concerns, e.g. use of AI to spread propaganda or to deny worthy people of loans or jobs: âall this talk of existential risk is science fiction and obscuring the REAL problemsâ
4. A and B then begin vigorously arguing with each other, each accusing the other of wasting time on unimportant issues.
My hypothesis/âtheory/âargument is that at this point the general public throws up its hands because both the critics/âexperts canât seem to agree on the basics.
By the way, I hope itâs clear that Iâm not accusing A or B of doing anything wrong. I think they are both arguing in good faith from deeply held beliefs.
yes, this has been very much on my mind: if this three-sided framework is useful/âvalid, what does it mean for the possibility of the different groups cooperating?
I suspect that the depressing answer is that cooperation will be a big challenge and may not happen at all. Especially as to questions such as âis the European AI Act in its present form a good start or a dangerous waste of time?â It strikes me that each of the three groups in the framework will have very strong feelings on this question
realists: yes, because, even if it is not perfect, it is at least a start on addressing important issues like invasion of privacy.
boosters: no, because it will stifle innovation
doomers: no, because you are looking under the lamp post where the light is better, rather than addressing the main risk, which is existential risk.
The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year.
Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?
I suspect most of us occupy more than one position in this taxonomy. Iâm a little bit doomer and a little bit accelerationist. I theres significant, possibly world ending, danger in AI, but I also think as someone who works on climate change in my day job, that climate change is a looming significant civilization ending risk or worse (20%-ish) for humanity and worry humans alone might not be able to solve this thing. Lord help us if the siberian permafrost melts,we might be boned as a species.
So as a result, I just donât know how to balance these two potential x risk dangers. No answers from me, alas, but I think we need to understand that for many, maybe most of us, we havenât really planted our flag in any of these camps exclusively, weâre still information gathering.
Thanks for this comment. I totally agree and I think I need to make this clearer. I mentioned elsewhere ITT that I realized that I need to do a better job distinguishing between âpositionsâ vs. âpeopleâ especially when debate gets heated.
These are tough issues with a lot of uncertainty and I think smart people are wrestling with these.
If I can risk the âpersonal blogâ tag: I showed an earlier draft of the essay that became this post to a friend who is very smart but doesnât follow these issues very closely. He put me on the spot by asking me âok, so which are you? You are putting everyone else in a box, so which box are you in?â And it made me realize, âI donât knowâit depends on which day you ask me!â
Autonomous lethal weapons (ALWs; we need a more eerie, memetic name) could make the difference. Against the ârealistsâ, whereas bias is not a new problem, ALWs emphatically are. Likewise no reflexive optimism from the boosters lessens the need for sober regulation to lessen the self-evident risk of ALWs.
And this provides a ânarrative through-lineâ for regulationâwe must regulate ALWs, and so, AI systems that could design ALWs. It follows, we must regulate AI systems that design other AI systems in general, and so too, we must therefore regulate AI artificial intelligence researchers, or recursive self-improving systems. The regulations can logically follow, and lead to the (capability, at least) of regulating projects conducive to AGI.
All this suggests a scenario plan: on assumption ALWs will be used in combat, and there are confirmed fatalities therefrom, we publicise like hell: names, faces, biographiesâwhich the ALW didnât, and couldnât have, appreciated, but it killed them anyway. We observe that it chose to kill themâwhy? On what criteria? What chain of reasoning? No one alive knows. With the anxiety from ALWs in general, and such a case in particular, we are apt to have more public pressure for regulation in general.
If that regulation focuses on ALWs, and what ensures more safety, to the âstair-stepsâ of regulation against risk to humans in general, we have a model that appeals to ârealistsâ: ALWs given a photograph, of epicanthic folds, dark skin, blue eyes, whateverâenables the ultimate âdiscriminationâ, of genocide. Whereas the boosters have nothing to say against regulations, since such lethal uses by AI canât be âmade safeâ, particularly if multiple antagonists have them.
We leverage ALW regulation to get implicitly existence-risk averse leadership into regulatory bodies (since in a bureaucracy, who wins the decision makers wins the decisions). Progress.
OPâs analysis seems soundâbut observe that the media are also biased toward booster-friendly, simpler, hyperbolic narratives; whereas theyâve no mental model of, not robots with human minds, but the minds themselves supplanted. Not knowing whatâs happening, they default to their IT shibboleths, ârealistâ-friendly bias concerns. As for âdoomersâ, they donât know what to do.
If somebody knows how to make a press release for such a use described: go for it.
Thereâs already a more eerie, memetic name. Slaughterbots.
I think ALWs are already more of a ârealistâ cause than a doomer cause. To doomers, theyâre a distractionâa superintelligence can kill you with or without them.
ALWs also seem to be held to an unrealistic standard compared to existing weapons. With present-day technology, theyâll probably hit the wrong target more often than human-piloted drones. But will they hit the wrong target more often than landmines, cluster munitions, and over-the-horizon unguided artillery barrages, all of which are being used in Ukraine right now?
Although I donât see anything wrong with your doomer/âbooster delineation here, your doomer/ârealist delineation is catastrophic. Although nearterm AI power realities are ultimately a distraction from AI risk, they are also essential for understanding AGI macrostrategy e.g. race dynamics.
By lumping things like US-China affairs in the same camp as spin doctors like Gebru and Mitchell, youâre basically labeling those factors as diametrically opposed to the AI safety community, which will only give bad actors more control/âmonopoly over the overton window (both the overton window for the general public and the separate one within the AI safety community).
The nearterm uses of AI are critical to understand the gameboard and to weigh different strategies. Itâs not intrinsically valuable, but itâs instrumentally convergent to understand the worldâs current âAI situationâ. Itâs a bad idea to lump it in the same category with opportunistic woke celebrities.
I direct skepticism at boosters supporting fast enough timelines to reach AGI within the near future, that sounds like a doomer only position.
I donât think all boosters would agree with âWe are likely to see something like AGI in the next 5 yearsâ. Yan LeCun Has explicitly said that LLMs are an off-ramp on the path to AI, and that we havenât made an AI that as smart as a cat at this point so AGI is still a long way off.