Regarding (2), I think the best AI gov research training org is substantially different from MATS’ idealized form because:
To the extent possible, I think MATS should be trying to solve a compartmentalization of the alignment problem (i.e., the technical part, to the extent that’s separable from governance) because I think people probably grow more as deep researchers from academic cohorts with a dominant focus;
The Bay Area, where MATS is based, is not the governance hub of the US;
One should market differently to pol. sci., international relations, and complex systems researchers compared to ML, maths, physics, and neuroscience researchers;
The MATS seminar program is geared towards technical alignment research and an ideal AI governance seminar program would look substantially different;
I don’t understand the governance literature as well as I understand the technical safety literature and there are probably unknown unknowns;
MATS is quite large (55 in-person scholars for winter);
There are good alignment mentors MATS could support that might get bumped out if we add governance mentors;
At some point, more MATS team members (to support a larger cohort) might make it hard for us to work effectively (as per Lightcone’s model);
A small, cohesive MAGS team could work in parallel with the MATS team for greater impact than a unified team;
GovAI are already doing something like the hypothetical MAGS, and the OpenAI governance team might want to do the same, which would mean a lot of potential mentors (maybe enough for a MATS-sized program, funders + mentors willing).
The Bay Area, where MATS is based, is not the governance hub of the US;
The Bay is an AI hub, home to OpenAI, Google, Meta, etc., and therefore an AI governance hub. Governance is not governments. Important decisions are being made there—maybe more important decisions than in DC. To quote Allan Dafoe:
AI governance concerns how humanity can best navigate the transition to a world with advanced AI systems[1]. It relates to how decisions are made about AI[2], and what institutions and arrangements would help those decisions to be made well.
Also, many, many AI governance projects go hand-in-hand with technical expertise.
Regarding (2), I think the best AI gov research training org is substantially different from MATS’ idealized form because:
To the extent possible, I think MATS should be trying to solve a compartmentalization of the alignment problem (i.e., the technical part, to the extent that’s separable from governance) because I think people probably grow more as deep researchers from academic cohorts with a dominant focus;
The Bay Area, where MATS is based, is not the governance hub of the US;
One should market differently to pol. sci., international relations, and complex systems researchers compared to ML, maths, physics, and neuroscience researchers;
The MATS seminar program is geared towards technical alignment research and an ideal AI governance seminar program would look substantially different;
I don’t understand the governance literature as well as I understand the technical safety literature and there are probably unknown unknowns;
Currently MATS expects applicants to have background knowledge at the level of the AGI Safety Fundamentals AI Alignment Curriculum and not the AI Governance Curriculum;
I’d rather do one thing well than two things poorly.
Thoughts, Mauricio?
Additional reasons why MATS + MAGS > MATS + governance patch:
MATS is quite large (55 in-person scholars for winter);
There are good alignment mentors MATS could support that might get bumped out if we add governance mentors;
At some point, more MATS team members (to support a larger cohort) might make it hard for us to work effectively (as per Lightcone’s model);
A small, cohesive MAGS team could work in parallel with the MATS team for greater impact than a unified team;
GovAI are already doing something like the hypothetical MAGS, and the OpenAI governance team might want to do the same, which would mean a lot of potential mentors (maybe enough for a MATS-sized program, funders + mentors willing).
The Bay is an AI hub, home to OpenAI, Google, Meta, etc., and therefore an AI governance hub. Governance is not governments. Important decisions are being made there—maybe more important decisions than in DC. To quote Allan Dafoe:
Also, many, many AI governance projects go hand-in-hand with technical expertise.
Maybe more broadly, AI strategy is part of AI governance.