I googled for mentions of [confidence building measures] on Lesswrong and felt mildly saddened and surprised that there were so few, but happy that there were at least some. A confidence building measure (or “CBM”) is a sort of “soft power” technique where people find ways to stay chill and friendly somewhat far away from possible battle lines, in order to make everyone more confident that grownups are in charge on both sides. They seem like they could help rule out large classes of failure modes, if they work as they claim to work “on the tin”.
In the search, I found 5 hits. One from me in 2011 and another from me in 2017. Two mentions are “incidental” (in biblios) but there was a pretty solid and recent and direct hit with [AN #146]: Plausible stories of how we might fail to avert an existential catastrophe. which is pointing to a non-less-wrong text on CBMs. Quoting the core of AN #146 here:
This paper explores confidence-building measures (CBMs) as a way to reduce the negative effects of military AI use on international stability. CBMs were an important tool during the Cold War. However, as CBMs rely on a shared interest to succeed, their adoption has proven challenging in the context of cybersecurity, where the stakes of conflict are less clear than in the Cold War. The authors present a set of CBMs that could diminish risks from military use of AI and discuss their advantages and downsides. On the broad side, these include building norms around the military use of AI, dialogues between civil actors with expertise in the military use of AI from different countries, military to military dialogues, and code of conducts with multilateral support. On the more specific side, states could engage in public signalling of the importance of Test and Evaluation (T&E), transparency about T&E processes and push for international standards for military AI T&E. In addition, they could cooperate on civilian AI safety research, agree on specific rules to prevent accidental escalation (similar to the Incidents at Sea Agreement from the Cold War), clearly mark autonomous systems as such, and declare certain areas as off-limits for autonomous systems.
The paper itself appears to be authored by… maybe this Michael Horowitz and then this Paul Scharre seems like an excellent bet for the other author. A key feature here in my mind is that Horowitz appears to already have a functional enduring bipartisan stance that seems to have survived various US presidential administration changes? This is… actually kinda hopeful! :-)
(Edit: Oh wait. Hopefulness points half retracted? The “AI IR Michael Horowitz” is just a smart guy I think, and not specifically a person with direct formal relationships with the elected US govt officials. Hmm.)
States have long used established “rules of the road” to govern the interaction of military forces operating with a high degree of autonomy, such as at naval vessels at sea, and there may be similar value in such a CBM for interactions with AI-enabled autonomous systems. The 1972 Incidents at Sea Agreement and older “rules of the road” such as maritime prize law provide useful historical examples for how nations have managed analogous challenges in the past. Building on these historical examples, states could adopt a modern-day “international autonomous incidents agreement” that focuses on military applications of autonomous systems, especially in the air and maritime environments. Such an agreement could help reduce risks from accidental escalation by autonomous systems, as well as reduce ambiguity about the extent of human intention behind the behavior of autonomous systems.
Another interesting aspect is that the letters “T&E” occur 31 times in the Horowitz & Scharre paper which is short for the “Testing & Evaluation” of “new AI capabilities”. They seem to want to put “T&E” processes into treaties and talks as a first class object, basically?
States could take a variety of options to mitigate the risks of creating unnecessary incentives to shortcut test and evaluation, including publicly signaling the importance of T&E, increasing transparency about T&E processes, promoting international T&E standards, and sharing civilian research on AI safety.
Neither italics nor bold in original.
But still… like… wisdom tournament designs? In treaties? That doesn’t seem super obviously terrible if we can come up with methods that would be good in principle and in practice.
I googled for mentions of [confidence building measures] on Lesswrong and felt mildly saddened and surprised that there were so few, but happy that there were at least some. A confidence building measure (or “CBM”) is a sort of “soft power” technique where people find ways to stay chill and friendly somewhat far away from possible battle lines, in order to make everyone more confident that grownups are in charge on both sides. They seem like they could help rule out large classes of failure modes, if they work as they claim to work “on the tin”.
In the search, I found 5 hits. One from me in 2011 and another from me in 2017. Two mentions are “incidental” (in biblios) but there was a pretty solid and recent and direct hit with [AN #146]: Plausible stories of how we might fail to avert an existential catastrophe. which is pointing to a non-less-wrong text on CBMs. Quoting the core of AN #146 here:
The paper itself appears to be authored by… maybe this Michael Horowitz and then this Paul Scharre seems like an excellent bet for the other author. A key feature here in my mind is that Horowitz appears to already have a functional enduring bipartisan stance that seems to have survived various US presidential administration changes? This is… actually kinda hopeful! :-)
(Edit: Oh wait. Hopefulness points half retracted? The “AI IR Michael Horowitz” is just a smart guy I think, and not specifically a person with direct formal relationships with the elected US govt officials. Hmm.)
There are specific ideas, backstory, phrases, and policy proposals in the article itself: AI and International Stability: Risks and Confidence-Building Measures. Here’s a not-entirely-random but not especially critical part of the larger article:
I guess the thing I’d want to communicate is that it doesn’t seem to be the case that there are literally ZERO competent grownups anywhere in the world, even though I sometimes feel like that might be the case when I look at things like the world’s handling of covid. From what I can tell, the grownups that get good things done get in, apply competence to do helpful things, then bounce out pretty fast afterwards?
Another interesting aspect is that the letters “T&E” occur 31 times in the Horowitz & Scharre paper which is short for the “Testing & Evaluation” of “new AI capabilities”. They seem to want to put “T&E” processes into treaties and talks as a first class object, basically?
Neither italics nor bold in original.
But still… like… wisdom tournament designs? In treaties? That doesn’t seem super obviously terrible if we can come up with methods that would be good in principle and in practice.