TLDR; Participate online or in-person on the weekend 3rd to 5th May in an exciting and fun AI safety research hackathon focused on demonstrating and extrapolating risks to democracy from real-life threat models. We invite researchers, cybersecurity professionals, and governance experts to join but it is open for everyone, and we will introduce starter code templates to help you kickstart your team’s projects. Join here.
Why demonstrate risks to democracy?
Despite some of the largest potential risks from AI being related to our democratic institutions and the fragility of society, there is surprisingly little work demonstrating and extrapolating concrete risks from AI to democracy.
By putting together actual demonstrations of potential dangers and mindfully extrapolating these risks into the late 2020s, we can raise awareness among key decision-makers and stakeholders, thus driving the development of mitigation strategies.
This type of research will also be informative for dangerous capability evaluations and our understanding of catastrophic risk in the context of societal stability. As a result, your participation in this hackathon will contribute to a growing body of knowledge that will help shape the future of AI governance.
We are excited to see you there and hack away with you to develop impactful research!
What is a research hackathon?
The AI x Democracy Hackathon is a weekend-long event where you participate in teams (1-5) to create interesting, fun, and impactful research. You submit a PDF report that summarizes and discusses your findings in the context of AI safety. These reports will be judged by our jury and you can win up to $1,000!
It runs from 3rd May to 5th May (in two weeks) and we’re excited to welcome you for a full weekend of engaging research, though we also welcome partial participation and keep it open to all participants. You will hear fascinating talks about real-world projects tackling these types of questions, get the opportunity to discuss your ideas with experienced mentors, and you will get reviews from top-tier researchers in the field of AI safety to further your research abilities.
Everyone can participate and we encourage you to join especially if you’re considering AI safety from another career. We give you code templates and ideas to kickstart your projects and you’ll be surprised what you can accomplish in just a weekend – especially with your new-found community!
Read more about how to join, what you can expect, the schedule, and what previous participants have said about being part of the hackathon below.
What are some examples of Democracy x AI projects I could do in a weekend?
You can check out a bunch of interesting ideas on AI Safety Ideas to red-team democracy by creating a demonstration of an actual risk model.
For example, you could develop an LLM to contain a sleeper agent that activates on election day, train agents to skew poll results to inflate support for particular policies, or use an LLM to draft uncontroversial legislative proposals that, if implemented, indirectly impacts more contentious or harmful policies.
These are ideas to get you started and you can read previous hackathon projects to see examples of the types of projects you can develop during just one weekend, e.g. EscalAItion found that LLMs had a propensity to escalate in military scenarios and was accepted at the multi-agent security workshop at NeurIPS 2023 following more research.
Inspiration
Here is some interesting material to get inspired for the hackathon:
There’s loads of reasons to join! Here are just a few:
See how fun and interesting AI safety can be
Get to know others who are into the overlap of empirical ML safety and AI governance
Win up to $1,000, helping you towards your first H100 GPU
Get practical experience with LLM evaluations and AI governance research
Show the AI safety labs what you are able to do to increase your chances at some amazing jobs
Get a certificate at the end
Get proof that your work is awesome so you can get that grant to pursue the AI safety research that you always wanted to pursue
The best teams are offered to participate in the Apart Lab program, which supports teams in their journey towards publishing groundbreaking AI safety and security research
Please join! This can be your first foray into AI and ML safety and maybe you’ll realize that there are exciting low-hanging fruit that your specific skillset is adapted to. Even if you normally don’t find it particularly interesting, this time you might see it in a new light!
There’s a lot of pressure from AI safety to perform at a top level and this seems to drive some people out of the field. We’d love it if you consider joining with a mindset of fun exploration and get a positive experience out of the weekend.
What are previous experiences from the research hackathon?
Yoann Poupart, BlockLoads CTO: “This Hackathon was a perfect blend of learning, testing, and collaboration on cutting-edge AI Safety research. I really feel that I gained practical knowledge that cannot be learned only by reading articles.”
Lucie Philippon, France Pacific Territories Economic Committee: “It was great meeting such cool people to work with over the weekend! I did not know any of the other people in my group at first, and now I’m looking forward to working with them again on research projects! The organizers were also super helpful and contributed a lot to the success of our project.”
Akash Kundu, now an Apart Lab fellow: “It was an amazing experience working with people I didn’t even know before the hackathon. All three of my teammates were extremely spread out, while I am from India, my teammates were from New York and Taiwan. It was amazing how we pulled this off in 48 hours in spite of the time difference. Moreover, the mentors were extremely encouraging and supportive which helped us gain clarity whenever we got stuck and helped us create an interesting project in the end.”
Nora Petrova, ML Engineer at Prolific: “The hackathon really helped me to be embedded in a community where everyone was working on the same topic. There was a lot of curiosity and interest in the community. Getting feedback from others was interesting as well and I could see how other researchers perceived my project. It was also really interesting to see all the other projects and it was positive to see other’s work on it.”
Chris Mathwin, MATS Scholar: “The Interpretability Hackathon exceeded my expectations, it was incredibly well organized with an intelligently curated list of very helpful resources. I had a lot of fun participating and genuinely feel I was able to learn significantly more than I would have, had I spent my time elsewhere. I highly recommend these events to anyone who is interested in this sort of work!”
What is the agenda for the hackathon?
The schedule runs from 7PM CET / 10AM PST Friday to 4AM CET Monday / 7PM PST Sunday. We start with an introductory talk and end the event during the following week with an awards ceremony. Join the public ICal here where you will also see events preparing you for the hackathon.
CET / PST
Fri 7 PM / 10 AM
Introduction to the hackathon, what to expect, and a talk from an expert. Splitting into teams.
Fri 8:30 PM / 11:30 AM
Hacking begins!
Mon 4 AM / Sun 7 PM
Deadline for submissions. Reviewers begin judging the projects.
Wed 5 PM / 8 AM
Award ceremony and lightning talks from the teams.
Who will give my project reviews?
We are delighted to have experts in AI technical safety research and AI governance join us!
Nina Rimsky from Anthropic will provide you feedback on your project as part of the jury
Simon Lermen will share his work on Bad Agents and mass spear-phishing
Konrad Seifert from the Simon Institute for Longterm Governance will be part of our jury and give insightful comments on your project
Besides emphasizing the introduction of concrete mitigation ideas for the risks presented, we are aware that projects emerging from this hackathon might pose a risk if disseminated irresponsibly.
Join by clicking the “Sign Up” button on the hackathon page and read more about the hackathons on our home page.
If you have any feedback, suggestions, or resources we should be aware of, feel free to reach out to sprints@apartresearch.com, submit pull requests, add ideas, or write any comments here. We are happy to take constructive suggestions.
Demonstrate and evaluate risks from AI to society at the AI x Democracy research hackathon
Link post
TLDR; Participate online or in-person on the weekend 3rd to 5th May in an exciting and fun AI safety research hackathon focused on demonstrating and extrapolating risks to democracy from real-life threat models. We invite researchers, cybersecurity professionals, and governance experts to join but it is open for everyone, and we will introduce starter code templates to help you kickstart your team’s projects. Join here.
Why demonstrate risks to democracy?
Despite some of the largest potential risks from AI being related to our democratic institutions and the fragility of society, there is surprisingly little work demonstrating and extrapolating concrete risks from AI to democracy.
By putting together actual demonstrations of potential dangers and mindfully extrapolating these risks into the late 2020s, we can raise awareness among key decision-makers and stakeholders, thus driving the development of mitigation strategies.
This type of research will also be informative for dangerous capability evaluations and our understanding of catastrophic risk in the context of societal stability. As a result, your participation in this hackathon will contribute to a growing body of knowledge that will help shape the future of AI governance.
We are excited to see you there and hack away with you to develop impactful research!
What is a research hackathon?
The AI x Democracy Hackathon is a weekend-long event where you participate in teams (1-5) to create interesting, fun, and impactful research. You submit a PDF report that summarizes and discusses your findings in the context of AI safety. These reports will be judged by our jury and you can win up to $1,000!
It runs from 3rd May to 5th May (in two weeks) and we’re excited to welcome you for a full weekend of engaging research, though we also welcome partial participation and keep it open to all participants. You will hear fascinating talks about real-world projects tackling these types of questions, get the opportunity to discuss your ideas with experienced mentors, and you will get reviews from top-tier researchers in the field of AI safety to further your research abilities.
Everyone can participate and we encourage you to join especially if you’re considering AI safety from another career. We give you code templates and ideas to kickstart your projects and you’ll be surprised what you can accomplish in just a weekend – especially with your new-found community!
Read more about how to join, what you can expect, the schedule, and what previous participants have said about being part of the hackathon below.
What are some examples of Democracy x AI projects I could do in a weekend?
You can check out a bunch of interesting ideas on AI Safety Ideas to red-team democracy by creating a demonstration of an actual risk model.
For example, you could develop an LLM to contain a sleeper agent that activates on election day, train agents to skew poll results to inflate support for particular policies, or use an LLM to draft uncontroversial legislative proposals that, if implemented, indirectly impacts more contentious or harmful policies.
These are ideas to get you started and you can read previous hackathon projects to see examples of the types of projects you can develop during just one weekend, e.g. EscalAItion found that LLMs had a propensity to escalate in military scenarios and was accepted at the multi-agent security workshop at NeurIPS 2023 following more research.
Inspiration
Here is some interesting material to get inspired for the hackathon:
A paper presenting a framework for AI and Democracy
Article by Yoshua Bengio in the Journal of Democracy
Why it Matters podcast on The Year of AI and Elections
Guardian article about the influence of AI on the US elections
80000 hours podcast with Nina Schick on disinformation and the rise of synthetic media
You can also see more on the evaluations starter resources page.
Why should I join?
There’s loads of reasons to join! Here are just a few:
See how fun and interesting AI safety can be
Get to know others who are into the overlap of empirical ML safety and AI governance
Win up to $1,000, helping you towards your first H100 GPU
Get practical experience with LLM evaluations and AI governance research
Show the AI safety labs what you are able to do to increase your chances at some amazing jobs
Get a certificate at the end
Get proof that your work is awesome so you can get that grant to pursue the AI safety research that you always wanted to pursue
The best teams are offered to participate in the Apart Lab program, which supports teams in their journey towards publishing groundbreaking AI safety and security research
And many many more… Come along!
Do I need experience in AI safety to join?
Please join! This can be your first foray into AI and ML safety and maybe you’ll realize that there are exciting low-hanging fruit that your specific skillset is adapted to. Even if you normally don’t find it particularly interesting, this time you might see it in a new light!
There’s a lot of pressure from AI safety to perform at a top level and this seems to drive some people out of the field. We’d love it if you consider joining with a mindset of fun exploration and get a positive experience out of the weekend.
What are previous experiences from the research hackathon?
Yoann Poupart, BlockLoads CTO: “This Hackathon was a perfect blend of learning, testing, and collaboration on cutting-edge AI Safety research. I really feel that I gained practical knowledge that cannot be learned only by reading articles.”
Lucie Philippon, France Pacific Territories Economic Committee: “It was great meeting such cool people to work with over the weekend! I did not know any of the other people in my group at first, and now I’m looking forward to working with them again on research projects! The organizers were also super helpful and contributed a lot to the success of our project.”
Akash Kundu, now an Apart Lab fellow: “It was an amazing experience working with people I didn’t even know before the hackathon. All three of my teammates were extremely spread out, while I am from India, my teammates were from New York and Taiwan. It was amazing how we pulled this off in 48 hours in spite of the time difference. Moreover, the mentors were extremely encouraging and supportive which helped us gain clarity whenever we got stuck and helped us create an interesting project in the end.”
Nora Petrova, ML Engineer at Prolific: “The hackathon really helped me to be embedded in a community where everyone was working on the same topic. There was a lot of curiosity and interest in the community. Getting feedback from others was interesting as well and I could see how other researchers perceived my project. It was also really interesting to see all the other projects and it was positive to see other’s work on it.”
Chris Mathwin, MATS Scholar: “The Interpretability Hackathon exceeded my expectations, it was incredibly well organized with an intelligently curated list of very helpful resources. I had a lot of fun participating and genuinely feel I was able to learn significantly more than I would have, had I spent my time elsewhere. I highly recommend these events to anyone who is interested in this sort of work!”
What is the agenda for the hackathon?
The schedule runs from 7PM CET / 10AM PST Friday to 4AM CET Monday / 7PM PST Sunday. We start with an introductory talk and end the event during the following week with an awards ceremony. Join the public ICal here where you will also see events preparing you for the hackathon.
Who will give my project reviews?
We are delighted to have experts in AI technical safety research and AI governance join us!
Nina Rimsky from Anthropic will provide you feedback on your project as part of the jury
Simon Lermen will share his work on Bad Agents and mass spear-phishing
Konrad Seifert from the Simon Institute for Longterm Governance will be part of our jury and give insightful comments on your project
See other collaborators in the speakers and collaborators section of the hackathon page.
What if my research seems too risky to share?
Besides emphasizing the introduction of concrete mitigation ideas for the risks presented, we are aware that projects emerging from this hackathon might pose a risk if disseminated irresponsibly.
For all of Apart’s research dissemination and events, we follow our Responsible Disclosure Policy.
Where can I read more about this?
Join our community Discord server
The AI x Democracy hackathon page
The Apart Sprints website
Read our blog
See in-person and online events
An overview of hackathon locations this research sprint
Our Africa location hosted by Jonas Kgomo
Our Vietnam location hosted by Jord Nguyen
Ideas and inspiration for the AI x democracy hackathon (you are welcome to add more)
Starter code templates and inspiration for this hackathon
Previous hackathons
Our collaboration with METR to develop the task standard
Our collaboration with CAIF to demonstrate multi-agent risk from advanced AI
Join by clicking the “Sign Up” button on the hackathon page and read more about the hackathons on our home page.
If you have any feedback, suggestions, or resources we should be aware of, feel free to reach out to sprints@apartresearch.com, submit pull requests, add ideas, or write any comments here. We are happy to take constructive suggestions.