I initially thought MIRI dropping the AF team was a really bad move, and wrote (but didn’t publish) an open letter aiming to discourage this (tl;dr thesis: This research might be critical, we want this kind of research to be ready to take advantage of a possible AI assisted research window).
After talking with the team more, I concluded that actually having an institutional home for this kind of work which is focused on AF would be healthier, as they’d be able to fundraise independently, self-manage, set their own agendas entirely freely, have budget sovereignty, etc, rather than being crammed into an org which was not hopeful about their work.
I’ve been talking in the background and trying to set them up with fiscal sponsorship and advising on forming an org for a few weeks now, it looks like this will probably work for most of the individuals, but the team has not cohered around a leadership structure or agenda yet. I’m hopeful that this will come together, as I think that this kind of theoretical research is one of the most likely classes of progress we need to navigate the transition to superintelligence. Most likely an umbrella org which hosts individual researchers is the short term solution, hopefully coalescing into a more organized team at some point.
but the team has not cohered around a leadership structure or agenda yet. I’m hopeful that this will come together
I don’t expect the most effective strategy at present to be [(try hard to) cohere around an agenda]. An umbrella org hosting individual researchers seems the right starting point. Beyond that, I’d expect [structures and support to facilitate collaboration and self-organization]to be ideal. If things naturally coalesce that’s probably a good sign—but I’d prefer that to be a downstream consequence of exploration, not something to aim for in itself.
To be clear, this is all on the research side—on the operations side organization is clearly good.
Yeah, I mostly agree with the claim that individuals pursuing their own agendas is likely better than trying to push for people to work more closely. Finding directions which people feel like converging on could be great, but not at the cost of being able to pursue what seems most promising in a self-directed way.
I think I meant I was hopeful about the whole thing coming together, rather than specifically the coherent agenda part.
I initially thought MIRI dropping the AF team was a really bad move, and wrote (but didn’t publish) an open letter aiming to discourage this (tl;dr thesis: This research might be critical, we want this kind of research to be ready to take advantage of a possible AI assisted research window).
After talking with the team more, I concluded that actually having an institutional home for this kind of work which is focused on AF would be healthier, as they’d be able to fundraise independently, self-manage, set their own agendas entirely freely, have budget sovereignty, etc, rather than being crammed into an org which was not hopeful about their work.
I’ve been talking in the background and trying to set them up with fiscal sponsorship and advising on forming an org for a few weeks now, it looks like this will probably work for most of the individuals, but the team has not cohered around a leadership structure or agenda yet. I’m hopeful that this will come together, as I think that this kind of theoretical research is one of the most likely classes of progress we need to navigate the transition to superintelligence. Most likely an umbrella org which hosts individual researchers is the short term solution, hopefully coalescing into a more organized team at some point.
Broadly I agree.
I’m not sure about:
I don’t expect the most effective strategy at present to be [(try hard to) cohere around an agenda]. An umbrella org hosting individual researchers seems the right starting point. Beyond that, I’d expect [structures and support to facilitate collaboration and self-organization] to be ideal.
If things naturally coalesce that’s probably a good sign—but I’d prefer that to be a downstream consequence of exploration, not something to aim for in itself.
To be clear, this is all on the research side—on the operations side organization is clearly good.
Yeah, I mostly agree with the claim that individuals pursuing their own agendas is likely better than trying to push for people to work more closely. Finding directions which people feel like converging on could be great, but not at the cost of being able to pursue what seems most promising in a self-directed way.
I think I meant I was hopeful about the whole thing coming together, rather than specifically the coherent agenda part.