I think writing this post was helpful to me in thinking through my career options. I’ve also been told by others that the post was quite valuable to them as an example of someone thinking through their career options.
Interestingly, I left METR (then ARC Evals) about a month and a half after this post was published. (I continued to be involved with the LTFF.) I then rejoined METR in August 2024. In between, I worked on ambitious mech interp and did some late stage project management and paper writing (including some for METR). I also organized a mech interp workshop at ICML 2024, which, if you squint, counts as “onboarding senior academics”.
I think leaving METR was a mistake ex post, even if it made sense ex ante. I think my ideas around mech interp when I wrote this post weren’t that great, even if I thought the projects I ended up working on were interesting (see e.g. Compact Proofs and Computation in Superposition). While the mech interp workshop was very well attended (e.g. the room was so crowded that people couldn’t get in due to fire code) and pretty well received, I’m not sure how much value it ended up producing for AIS. Also, I think I was undervaluing the resources available to METR as well as how much I could do at METR.
If I were to make a list for myself in 2023 using what I know now, I’d probably have replaced “onboarding senior academics” with “get involved in AI policy via the AISIs”, and instead of “writing blog posts or takes in general”, I’d have the option of “build common knowledge in AIS via pedagogical posts”. Though realistically, knowing what I know now, I’d have told my past self to try to better leverage my position at METR (and provided him with a list of projects to do at METR) instead of leaving.
Also, I regret both that I called it “ambitious mech interp”, and that this post became the primary reference for what this term meant. I should’ve used a more value-neutral name such as “rigorous model internals” and wrote up a separate post describing it.
I think writing this post was helpful to me in thinking through my career options. I’ve also been told by others that the post was quite valuable to them as an example of someone thinking through their career options.
Interestingly, I left METR (then ARC Evals) about a month and a half after this post was published. (I continued to be involved with the LTFF.) I then rejoined METR in August 2024. In between, I worked on ambitious mech interp and did some late stage project management and paper writing (including some for METR). I also organized a mech interp workshop at ICML 2024, which, if you squint, counts as “onboarding senior academics”.
I think leaving METR was a mistake ex post, even if it made sense ex ante. I think my ideas around mech interp when I wrote this post weren’t that great, even if I thought the projects I ended up working on were interesting (see e.g. Compact Proofs and Computation in Superposition). While the mech interp workshop was very well attended (e.g. the room was so crowded that people couldn’t get in due to fire code) and pretty well received, I’m not sure how much value it ended up producing for AIS. Also, I think I was undervaluing the resources available to METR as well as how much I could do at METR.
If I were to make a list for myself in 2023 using what I know now, I’d probably have replaced “onboarding senior academics” with “get involved in AI policy via the AISIs”, and instead of “writing blog posts or takes in general”, I’d have the option of “build common knowledge in AIS via pedagogical posts”. Though realistically, knowing what I know now, I’d have told my past self to try to better leverage my position at METR (and provided him with a list of projects to do at METR) instead of leaving.
Also, I regret both that I called it “ambitious mech interp”, and that this post became the primary reference for what this term meant. I should’ve used a more value-neutral name such as “rigorous model internals” and wrote up a separate post describing it.