It is notoriously difficult to figure out what research to focus on in AI alignment. I’ll argue that a focus on threat models can help significantly. Roughly speaking, a threat model is a story detailing how we get from today’s world to an AI-caused existential catastrophe. Once you have a good threat model, you can evaluate a piece of research by seeing to what extent it makes the story less likely to happen.
Rohin has recently become a Research Scientist on the technical AGI safety team at DeepMind after finishing his PhD at the Center for Human-Compatible AI (CHAI). He is particularly interested in clarifying big-picture questions around AI alignment. He is the editor and principal content creator of the Alignment Newsletter, and has had several talks and podcast appearances, particularly around distilling and synthesizing AI alignment research.
Rohin Shah at the SSC Online Meetup
Dr. Rohin Shah will speak on ” The importance of threat models for AI alignment,” and then will take questions.
After that, we will socialize virtually at the LessWrong Walled Garden in GatherTown.
Please register here and we will send you an invitation by Feb. 13.
The meetup is Feb. 14, 2021 at 18:30 UTC, 20:30 Israel Standard Time, 10:30 Pacific Standard Time.
Subscribe for email notifications of further such events here.
Abstract:
It is notoriously difficult to figure out what research to focus on in AI alignment. I’ll argue that a focus on threat models can help significantly. Roughly speaking, a threat model is a story detailing how we get from today’s world to an AI-caused existential catastrophe. Once you have a good threat model, you can evaluate a piece of research by seeing to what extent it makes the story less likely to happen.
For background, please see this talk by Rohin, “What’s been happening in AI alignment?”,
Bio:
Rohin has recently become a Research Scientist on the technical AGI safety team at DeepMind after finishing his PhD at the Center for Human-Compatible AI (CHAI). He is particularly interested in clarifying big-picture questions around AI alignment. He is the editor and principal content creator of the Alignment Newsletter, and has had several talks and podcast appearances, particularly around distilling and synthesizing AI alignment research.