On the Chatham House Rule

[Scott Garrabrant]

I have gone to several events operating under the Chatham House Rule, and have overall found it more annoying than useful. In this post, I share why I dislike the rule, how I think it can be improved, and hopefully spark others to give ideas on how to improve it. In particular, I think the part about not revealing who was at the event should be opt in. Partially, my goal is to eventually develop a modified version that might be used at future events.

The Chatham House Rule states that “When a meeting, or part thereof, is held under the Chatham House Rule, participants are free to use the information received, but neither the identity nor the affiliation of the speaker(s), nor that of any other participant, may be revealed.”

Note that the rule looks slightly ambiguous. If I am in a small conversation, clearly I cannot share who was in that conversation, but what about the list of participants as a whole? I think if you read the rule carefully, you will see that you cannot share who was at the event. Indeed, the official Chatham House website has more explanation, and explicitly states that “the list of attendees should not be circulated beyond those participating in the meeting.” However a significant number of people I have asked at Chatham House Rule events were unaware that this was part of the rule.

Keeping participant lists a secret is hard

This is by far the most annoying part of the rule, and the fact that many participants often are not aware that it is part of the rule means that the attempts by people trying to follow the rule are mostly useless. It almost feels like an information hazard to me to make more people aware of what the Chatham House Rule says, since following this part of the rule is so annoying.

I have personally violated this part of the rule many times. Once, I got an email with information on logistics for the event. Inside the email were a bunch of links to google docs with details. One of the docs said that the event was under the Chatham House Rule, without explaining what it was. I at the time did not know that this applied to the list of participants. I forwarded the email to my wife, so she would know where I was and what to pack for me. The email was sent directly to me and all the rest of the participants, so the attendees were visible.

Another time, I went to a Chatham House rule event, and when I arrived, I was given a schedule of talks with an entire list of participants in the back. When I first received it thought to myself “bleh, do I have to shred this?” I didn’t dispose of it, I left it in my luggage. My wife later found it and asked me if I wanted it, while starting to flip through it quickly. I told her to just throw it away. She didn’t see names, but she could have, which means I messed up.

After that same event, when asked how the event was, I mentioned that I saw “someone” give a talk on X. I mentioned this because I thought X was interesting. The person I was talking to said they knew who the person was, since they had seen that talk.

I went to another event, and afterwords, someone emailed me asking for advice on research projects. In the email they mentioned that they enjoyed talking to me at the event. I wanted to add a third person to the email thread who I knew would have a project that would be a good fit for him. Instead I responded by telling him about the project, and saying that he could add the person to the thread himself, explaining that I didn’t want to do it because of the Chatham House rule. He misunderstood, and instead of adding him to the thread, asked me if he could add him to the thread. I got frustrated and decided to interpreted his question as permission and just added him myself. (I am clearly being pedantic here and the question clearly was permission, but I want to illustrate how annoying a literal interpretation of the rule is.)

Worse, the information about who was at the event is differentially hard. It is much harder to keep the information about who was there a secret than it is to keep information about who said what a secret. If you only have to keep information about what is said a secret, it is usually a valid response to say that you can’t answer a question because of the Chatham House Rule. This is much less the case for information about who was there, because it can come up when you are already talking about a specific person. It is not uncommon to be asked if I know a specific person. How do I respond if I met them at a Chatham House Rule event?

I think this part of the rule is doing harm by making people take the other part less seriously. All while failing to provide benefit because not everyone even knows about it.

How to fix it

It seems we can do much better just by having people opt in to the part where their participation in the event should be a secret.

Note that we probably could not practically have people opt in the the whole of Chatham House Rule. This is because I expect something like half of people would opt in, and you cant keep track of that many people. Also, it is convenient to have everything said at the event under the Chatham House Rule, since otherwise it can be hard to remember what things that were said under the rule.

I expect that only a couple people (perhaps no people) at any given event will opt in to not being revealed to have been there. If this is wrong, this plan will not work. Also, if people want to not be singled out as opting in, this could cause some harm.

One thing to be concerned about is the cost of having two rules. Often there is a cost for having two standards, and I tend to avoid having to pay that cost, even if it means not introducing a better standard. However, in this case, I think that there is little benefit of having only one standard. If people spend 3 minutes at the beginning of each event thinking about what they are agreeing to, this would be better for achieving common knowledge. The main benefit of having one standard is having to keep track of which event used which rule, and if an individual does not want to pay that cost, they could just pretend that it is always the stricter rule.

Other possible minor changes

Some other changes that are probably not all good, but might be worth considering are:

Having formal talks be not under the Chatham House Rule, unless stated otherwise.

Introducing a mechanism for people to report themselves when they make mistakes.

Having a time at the beginning in which everyone agrees to the rule.

Having a time at the end where people can waive their right to Chatham House Rule if they feel that they didn’t say anything they dont mind being public.

[habryka]

A significant fraction of events I go to are operated under Chatham House rules. A significant fraction of the organizers of those events don’t seem to understand the full consequences of those rules, and I’ve referenced this post multiple times when talking to people about those rules.

[Raemon]

One of my major focuses this year has been on how to improve norms around confidentiality, and meta-trust. This post is a clear articulation of why many default secrecy norms aren’t very robust, and why we need something better.

[Ben Pace]

I reviewed this post here.

[Scott Garrabrant]

I think the comments here point out just how much we do not have common knowledge about this thing that we are pretending we have common knowledge about.

[Paul Crowley]

“If you’re running an event that has rules, be explicit about what those rules are, don’t just refer to an often-misunderstood idea” seems unarguably a big improvement, no matter what you think of the other changes proposed here.

[Davis_Kingsley]

I generally agree with this post. In my experience with several events operating under this rule:′

1. Many people disregard the rule or don’t take it very seriously.

2. Others may not hear that the rule is in effect at all, especially if they arrive late or otherwise miss orientation.

3. This creates a negative selection effect where the only ones openly discussing specifics of an event that is covered by the rule are those who don’t take the rule very seriously—generally speaking, these are not the people who would be most optimal as the public face of the event.

I do think the principle behind the rule is useful, but in practice I have noticed that it often seems more of a hindrance than a boon. I somewhat worry that having multiple rules will increase noncompliance or misunderstandings, however, which seem frequent even as it stands.

[Dacyn]

On the Chatham House website I see

Q. Can participants in a meeting be named as long as what is said is not attributed?
A. It is important to think about the spirit of the Rule. For example, sometimes speakers need to be named when publicizing the meeting. The Rule is more about the dissemination of the information after the event—nothing should be done to identify, either explicitly or implicitly, who said what.

which seems reasonable. The comment about not circulating the attendee list beyond the participants is a response to the question “Can a list of attendees at the meeting be published?”, and my impression is that it is only meant as an answer to this question: i.e. such a list should not be published outside of the meeting, but it is OK if some people happen to come across it randomly. So I think you are just taking the Chatham Rule much more literally than it is intended.

[ryan_b]

The core problem is that people are unfamiliar with the rule, and the organizers are being very lazy in their implementation of it. In what way could we change the rule such that it solves the problem of people being ignorant of it or indifferent to it?

I blame the organizers for this entirely. I recommend a solution of two parts:

1. When announcing the Chatham House Rule applies, tell people what it is.

2. Organizers shouldn’t break the rule immediately be sending out lists in email blasts.

On the other hand it seems like who-said-what is much more of a concern for you, so you could propose a much weaker rule that gets around the logistical question:

Masked Ball Rule: Do not attribute any information from this event.

[ozziegooen]

I’m curious, what kinds of events follow Chatham House Rules? I’ve never heard of them until now. Is it just official ones from the Chatham House itself, or have other organizations been using them?

[Zvi]

I’ve attended one event under Chatham House rules. Not only was keeping who was there a secret costly, but people reliably considered it unreasonable that I actually kept that secret. “Oh, come on” and variants were used often, because actually keeping to the rule was annoying and they didn’t see the point.

People treating it as unreasonable does make keeping the rule even more expensive, and raises the probability it will be ignored—I believe others took the information part seriously but not the who was there part. But that also makes it really important we find a way to do the full no-one-knows-you-are-there thing when you need to do it, without it giving away that there was true need for it. If you say who attended until the moment you really can’t say, you’re doing Glomarization /​ Meta-Honesty wrong...

[Scott Garrabrant]

The FLI Beneficial AI workshop and the CHAI annual workshops have both been under the Chatham House Rule, for example. I don’t know about outside of AI safety.

[habryka]

Most workshops organized by CEA follow Chathams House rules, at least the EA Leaders forum and the individual outreach forum both did so. Anna Salamon organized a big-picture AI workshop a year ago that also followed Chathams House rules.

[Vaniver]

I think you mean, “someone organized” :P

[ChristianKl]

Having a time at the end where people can waive their right to Chatham House Rule if they feel that they didn’t say anything they dont mind being public.

It’s my feeling that it isn’t a good idea for this to be part of the main rule set because different events organizers have different ideas about how to end an event and the ending of the event is one of the parts that’s likely the best remembered by the participants for cognitive science reasons. I would prefer not to end events with bureaucracy like that.

As an aside, even when the main rules simply allow people to explicitly opt-out individual events can have such a section at the end.

[ChristianKl]

I did run the last LessWrong meetups in Berlin under the Chatham House rule and I share the feeling that the part about keeping participant lists secret is costly, hard to follow and while most people intuitively understand that sharing personal content can be problematic many people don’t have the same intuitive sense for the attendee list. That means that while it’s easy for them to remember the part of the rule about content it’s not easy to remember and follow the other part of the rule.

[sapphire]

My intuitive reaction is that you are following this rule more strictly than intended. If I held an event with Chatham house rules and someone’s wife accidentally saw the attendee list briefly this would not even register as a problem. I would expect the attendee to tell their wife not to talk about who was at the event. I also think its expected people will occasionally break the rules for the greater good (ex adding a trustworthy person to an email thread so they can work on a research problem). If someone asks whether you know someone just say ‘yes but the details are private’ (this is not breaking the rules imo).

When people chose to use Chatham house rules they are trying to prevent information becoming public and let people stay off the record. They usually do not expect the rules to be treated as sacred.

[Scott Garrabrant]

I agree that most people do not expect the rules to be treated as sacred. I still want the rules to be such that someone could (without great cost) treat them as sacred if they wanted to.

That or it should be explicitly stated that you are only expected to loosely follow the spirit of the rule.

[sapphire]

That runs into the problem that if you say the rules are absolute many people will ‘follow their spirit’ and if you say ‘follow the spirit of the rules’ then people will be way too lax about the rules. Eliezer mentions this issue in his meta-honesty.

[musicmage4114]

I feel like you might be appealing to consequences here a bit. Whether or not it is tractable in theory to follow the rule as an absolute (that is, to the letter) is a different problem from whether or not people will actually choose to do so.

It seems that Scott is more concerned about finding a formulation that can be followed to the letter in theory, while at the same time he has already conceded that most people will not choose to do so regardless of the formulation (and will thus follow the spirit instead).

[Vaniver]

My intuitive reaction is that you are following this rule more strictly than intended.

This is not my understanding, since the original context involves foreign policy issues where it might be highly important for someone to not have officially been at an event.

I also think it’s pretty horrible to have rules where people take them seriously to different degrees, in part because the rule is costly to follow and the benefit of following it rapidly falls off as other people don’t follow it. (Remember that the core goal of the rules is to allow people to say things that they don’t want attributed to them, even in a loose sense, but if you are half-following the rule that means people still pay the costs of being careful about what they say outside the event, and can’t trust that they can speak freely inside the event.) At one such event, someone made a claim of the form “okay, now we’ve established common knowledge about Chatham House Rules” which was obviously not true (and also made me worried about what that person thought common knowledge meant!).

[sapphire]

A later comment suggests the Chatham house website is sympathetic to my interpretation:

Q. Can participants in a meeting be named as long as what is said is not attributed?
A. It is important to think about the spirit of the Rule. For example, sometimes speakers need to be named when publicizing the meeting. The Rule is more about the dissemination of the information after the event—nothing should be done to identify, either explicitly or implicitly, who said what.

[ChristianKl]

Even when it comes to high level foreign policy issues I wouldn’t expect that no content is shared with spouses. Sharing content in an attributed way with spouses doesn’t prevent attendees from speaking freely.

[DanielFilan]

For what it’s worth, if I asked somebody to keep something secret and not tell anybody, and they said yes, I would consider it a breach of trust if they told their spouse.

[leggi]

I see a simple fix.

Rather than trying to go full “Chatham House”, make your own rules that work for the group (that’s what Chatham House did), call it what you like.

Define those rules and make sure everyone understands. (a ‘traffic light’ system for levels of security at different meetings if necessary?)

Rules should be sensible, simple and clear - and for everyone. (Opt-in opt-out—who can remember who’s agreed to what?).

How do I respond if I met them at a Chatham House Rule event?

Vaguely. Move the conversation swiftly on. The art of distraction. “Ah! that’s a long story” or “top secret” with a smile will work—if you’ve nothing else - but never lie!

Patient/​client confidentiality means ‘never say who’ comes easily to me. It’s a skill to be learned. Spotting the ‘trap’ before you get to it.

(The examples your violations might have serious consequences if you’re a spy/​diplomat/​gangster/​involved with something deadly, but otherwise… they’re probably forgivable :)

[Robert Miles]

This makes me think of the Creative Commons Licences. They’re neatly named to show what is and isn’t allowed. If I see that a piece of work is marked “CC BY-NC”, I can easily see that it’s published under a creative commons licence (CC), I’m free to share it as long as I give attribution so people know who the work is by (BY), and it’s used only for non-commercial purposes (NC). Perhaps we could design a set of rules like that, with the various optional parts separated out and clearly labelled. “This event is CH NA-AL”, or whatever.