Thank you for this update—I appreciate the clear reasoning. I also personally feel that the AI policy community is overinvested in the “say things that will get you points” strategy and underinvested in the “say true things that help people actually understand the problem” strategy. Specifically, I feel like many US policymakers have heard “be scared of AI because of bioweapons” but have not heard clear arguments about risks from autonomous systems, misalignment, AI takeover, etc.
A few questions:
To what extent is MIRI’s comms team (or technical governance team) going to interact directly with policymakers and national security officials? (I personally suspect you will be more successful if you’re having regular conversations with your target audience and taking note of what points they find confusing or unconvincing rather than “thinking from first principles” about what points make a sound argument.)
To what extent is MIRI going to contribute to concrete policy proposals (e.g., helping offices craft legislation or helping agencies craft specific requests)?
To what extent is MIRI going to help flesh out how its policy proposals could be implemented? (e.g., helping iron out the details of what a potential international AI compute governance regime would look like, how it would be implemented, how verification would work, what society would do with the time it buys)
Suppose MIRI has an amazing resource about AI risks. How does MIRI expect to get national security folks and important policymakers to engage with it?
(Tagging @lisathiergart in case some of these questions overlap with the work of the technical governance team.)
Got it– thank you! Am I right in thinking that your team intends to influence policymakers and national security officials, though? If so, I’d be curious to learn more about how you plan to get your materials in front of them or ensure that your materials address their core points of concern/doubt.
Put a bit differently– I feel like it would be important for your team to address these questions insofar as your team has the following goals:
The main audience we want to reach is policymakers – the people in a position to enact the sweeping regulation and policy we want – and their staff.
We are hopeful about reaching a subset of policy advisors who have the skill of thinking clearly and carefully about risk, particularly those with experience in national security.
In this reply I am speaking just about the comms team and not about other parts of MIRI or other organizations.
We want to produce materials that are suitable and persuasive for the audiences I named. (And by persuasive, I don’t mean anything manipulative or dirty; I just mean using valid arguments that address the points that are most interesting / concerning to our audience in a compelling fashion.)
So there are two parts here: creating high quality materials, and delivering them to that audience.
First, creating high quality materials. Some of this is down to just doing a good job in general: making the right arguments in the right order using good writing and pedagogical technique; none of this is very audience specific. There is also an audience-specific component, and to do well on that, we do need to understand our audience better. We are working to recruit beta readers from appropriate audience pools.
Second, delivering them to those audiences. There are several approaches here, most of which will not be executed by the comms team directly, we hand off to others. Within comms, we do want to see good reach and engagement with intelligent general audiences.
Thank you! I still find myself most curious about the “how will MIRI make sure it understands its audience” and “how will MIRI make sure its materials are read by policymakers + natsec people” parts of the puzzle. Feel free to ignore this if we’re getting too in the weeds, but I wonder if you can share more details about either of these parts.
There is also an audience-specific component, and to do well on that, we do need to understand our audience better. We are working to recruit beta readers from appropriate audience pools.
There are several approaches here, most of which will not be executed by the comms team directly, we hand off to others
Thank you for this update—I appreciate the clear reasoning. I also personally feel that the AI policy community is overinvested in the “say things that will get you points” strategy and underinvested in the “say true things that help people actually understand the problem” strategy. Specifically, I feel like many US policymakers have heard “be scared of AI because of bioweapons” but have not heard clear arguments about risks from autonomous systems, misalignment, AI takeover, etc.
A few questions:
To what extent is MIRI’s comms team (or technical governance team) going to interact directly with policymakers and national security officials? (I personally suspect you will be more successful if you’re having regular conversations with your target audience and taking note of what points they find confusing or unconvincing rather than “thinking from first principles” about what points make a sound argument.)
To what extent is MIRI going to contribute to concrete policy proposals (e.g., helping offices craft legislation or helping agencies craft specific requests)?
To what extent is MIRI going to help flesh out how its policy proposals could be implemented? (e.g., helping iron out the details of what a potential international AI compute governance regime would look like, how it would be implemented, how verification would work, what society would do with the time it buys)
Suppose MIRI has an amazing resource about AI risks. How does MIRI expect to get national security folks and important policymakers to engage with it?
(Tagging @lisathiergart in case some of these questions overlap with the work of the technical governance team.)
All of your questions fall under Lisa’s team and I will defer to her.
Got it– thank you! Am I right in thinking that your team intends to influence policymakers and national security officials, though? If so, I’d be curious to learn more about how you plan to get your materials in front of them or ensure that your materials address their core points of concern/doubt.
Put a bit differently– I feel like it would be important for your team to address these questions insofar as your team has the following goals:
In this reply I am speaking just about the comms team and not about other parts of MIRI or other organizations.
We want to produce materials that are suitable and persuasive for the audiences I named. (And by persuasive, I don’t mean anything manipulative or dirty; I just mean using valid arguments that address the points that are most interesting / concerning to our audience in a compelling fashion.)
So there are two parts here: creating high quality materials, and delivering them to that audience.
First, creating high quality materials. Some of this is down to just doing a good job in general: making the right arguments in the right order using good writing and pedagogical technique; none of this is very audience specific. There is also an audience-specific component, and to do well on that, we do need to understand our audience better. We are working to recruit beta readers from appropriate audience pools.
Second, delivering them to those audiences. There are several approaches here, most of which will not be executed by the comms team directly, we hand off to others. Within comms, we do want to see good reach and engagement with intelligent general audiences.
Thank you! I still find myself most curious about the “how will MIRI make sure it understands its audience” and “how will MIRI make sure its materials are read by policymakers + natsec people” parts of the puzzle. Feel free to ignore this if we’re getting too in the weeds, but I wonder if you can share more details about either of these parts.
Your curiosity and questions are valid but I’d prefer not to give you more than I already have, sorry.
Valid!