Overall, my very tentative and subjective impression is that the program shaved ~3 years off the median participant’s Path of Alignment Maturity; they seem-to-me to be coming up with project ideas about on par with a typical person 3 years further in. The shoulder John/Eliezer exercises were relatively costly and I don’t think most groups should try to duplicate them, but other than those I expect most of the MATS content can scale quite well, so in principle it should be possible to do this with a lot more people.
This seems like super amazing news! If this is true, your potential work improving and scaling this stuff seems clearly much higher EV than your research (in the next year, say) (on average time-invested, and also on the margin unless you’re working on this way more than I thought). Do you agree; what are you planning?
Edit: relatedly, I think this is the highlight of the post and the title misses the point.
I still put higher EV on my technical research, because this isn’t the only barrier to scaling research. Indeed, I expect technical work itself will be a necessary precondition to scaling research in the not-very-distant future; scaling research ultimately requires a paradigm, and paradigm discovery is a technical problem.
But I do think this is high-value, plans are under way to scale up for the next round of MATS, and I’m also hoping to figure out how to offload most of the work to other people.
Edit: relatedly, I think this is the highlight of the post and the title misses the point.
Yeah, a lot of my posts over the past month or two have been of frankly mediocre quality by my usual standards. I previously had a policy of mostly not talking directly about alignment strategy, critiques of other peoples’ research, and other alignment meta stuff. For various reasons I’ve lately been writing down a bunch of it very quickly, which does trade off to some extent with quality. Hopefully I’ll run out of such material soon and go back to writing better posts about more interesting things.
This seems like super amazing news! If this is true, your potential work improving and scaling this stuff seems clearly much higher EV than your research (in the next year, say) (on average time-invested, and also on the margin unless you’re working on this way more than I thought). Do you agree; what are you planning?
Edit: relatedly, I think this is the highlight of the post and the title misses the point.
I still put higher EV on my technical research, because this isn’t the only barrier to scaling research. Indeed, I expect technical work itself will be a necessary precondition to scaling research in the not-very-distant future; scaling research ultimately requires a paradigm, and paradigm discovery is a technical problem.
But I do think this is high-value, plans are under way to scale up for the next round of MATS, and I’m also hoping to figure out how to offload most of the work to other people.
Yeah, a lot of my posts over the past month or two have been of frankly mediocre quality by my usual standards. I previously had a policy of mostly not talking directly about alignment strategy, critiques of other peoples’ research, and other alignment meta stuff. For various reasons I’ve lately been writing down a bunch of it very quickly, which does trade off to some extent with quality. Hopefully I’ll run out of such material soon and go back to writing better posts about more interesting things.