I commented on that thread, and also wrote this post a few months ago, partially to express some thoughts on the announcement of Lightcone (& their compensation philosophy). Recently I’ve even switched to looking for direct work opportunities in the AI alignment sphere.
As is usual with ACX posts on the subject of AI alignment, the comments section has a lot of people who are totally unfamiliar with the space and are therefore extremely confident that various inadequacies (such as the difficulties EA orgs have in hiring qualified people) can easily be solved by doing the obvious thing. This is not to say that everything they suggest is wrong. Some of their suggestions are not obviously wrong. But that’s mostly by accident.
I agree that “pay people market rate” is a viable strategy and should be attempted. At the very least I haven’t yet seen many strong arguments for what benefit is accrued by paying below-market rates (in this specific set of circumstances), and thus default to “don’t shoot yourself in the foot”. I also think that most people are deeply confused about the kinds of candidates that MIRI (& similarly focused orgs) are looking for, in terms of researchers. There are some problems with operating in a largely pre-paradigmatic field which impose non-trivial constraints on the kinds of people it even makes sense to hire:
To the extent that you have “research directions”, they’re usually much less defined than “demonstrate a highly specific, narrow result at the edge of a well-established field”
As a result, you mostly need incoming researchers to be self-sufficient or bootstrap to that state very quickly, because every hour spent on mentorship and guidance by an existing researcher is an hour not spent doing research (it’s probably not a total loss, but it’s not great)
If you say “ok, bite the bullet, and spend 6 months − 2 years training interested math/cs/etc students to be competent researchers”, they’ve definitely tried this on a smaller scale and my understanding is that it doesn’t tend to work very well to produce the kinds of researchers who can make sustained progress on attacking relevant problems without so much help from existing researchers that it stops being worth it
So the general attitude present of “just hire a proper recruiter!” is missing all the ways in which that kind of suggestion is a total non-starter. The kinds of candidates MIRI is looking for aren’t legibly distinguished by their LinkedIn profiles. Recruiters in general have niches, and (to the best of my knowledge) this is not a niche that is well-served by recruiters. Superficially qualified candidates who don’t actually believe in the problem that they’re being paid to attack are probably not going to make much progress on it, but they will take up the time and attention of existing researchers.
Could MIRI do a better job of advertising their own job postings? Maybe, unless what they’re doing is effective at preferentially filtering out bad candidates without imposing an overwhelming barrier to good candidates. MIRI is hardly the only player in the game at this point, though—Redwood, ARC, Anthropic, and Alignment AI are all hiring researchers and none of them (officially) require candidates to work their way up to being Alignment Forum regulars.
“ok, bite the bullet, and spend 6 months − 2 years training interested math/cs/etc students to be competent researchers”—but have they tried this with Terry Tao?
People have tried to approach top people in their field and offer them >$500k/yr salaries, though not at any scale and not to a ton of people. Mostly people in ML and CS are already used to seeing these offers and frequently reject them, if they are in academia.
There might be some mathematicians for whom this is less true. I don’t know specifically about Terence Tao. Sometimes people also react quite badly to such an offer and it seems pretty plausible they would perceive it as a kind of bribe and making such an offer could somewhat permanently sour relations, though I think it’s likely one could find a framing for making such an offer that wouldn’t run into these problems.
My takeaway from this datapoint is that people are motivated by status more than money. This suggests ramping up advocacy via a media campaign, and making alignment high-status among the scientifically-minded (or general!) populace.
That is indeed a potential path that I am also more excited about. There are potentially ways to achieve that with money, though most ways I’ve thought about are also very prone to distorting what exactly is viewed as high-status, and creating strong currents in the field to be more prestigious, legible, etc. However, my guess is there are still good investments in the space, and I am pursuing some of them.
I’d love to hear more about this (unless secrecy is beneficial for some reason). I’d also like to know if there’s any way for people to donate to such an effort.
Yes, I think reaching out to differentiated candidates (like TT) is at least worth trying, since if it doesn’t work the downside is bounded by the relatively small number of attempts you can make.
I commented on that thread, and also wrote this post a few months ago, partially to express some thoughts on the announcement of Lightcone (& their compensation philosophy). Recently I’ve even switched to looking for direct work opportunities in the AI alignment sphere.
As is usual with ACX posts on the subject of AI alignment, the comments section has a lot of people who are totally unfamiliar with the space and are therefore extremely confident that various inadequacies (such as the difficulties EA orgs have in hiring qualified people) can easily be solved by doing the obvious thing. This is not to say that everything they suggest is wrong. Some of their suggestions are not obviously wrong. But that’s mostly by accident.
I agree that “pay people market rate” is a viable strategy and should be attempted. At the very least I haven’t yet seen many strong arguments for what benefit is accrued by paying below-market rates (in this specific set of circumstances), and thus default to “don’t shoot yourself in the foot”. I also think that most people are deeply confused about the kinds of candidates that MIRI (& similarly focused orgs) are looking for, in terms of researchers. There are some problems with operating in a largely pre-paradigmatic field which impose non-trivial constraints on the kinds of people it even makes sense to hire:
To the extent that you have “research directions”, they’re usually much less defined than “demonstrate a highly specific, narrow result at the edge of a well-established field”
As a result, you mostly need incoming researchers to be self-sufficient or bootstrap to that state very quickly, because every hour spent on mentorship and guidance by an existing researcher is an hour not spent doing research (it’s probably not a total loss, but it’s not great)
If you say “ok, bite the bullet, and spend 6 months − 2 years training interested math/cs/etc students to be competent researchers”, they’ve definitely tried this on a smaller scale and my understanding is that it doesn’t tend to work very well to produce the kinds of researchers who can make sustained progress on attacking relevant problems without so much help from existing researchers that it stops being worth it
So the general attitude present of “just hire a proper recruiter!” is missing all the ways in which that kind of suggestion is a total non-starter. The kinds of candidates MIRI is looking for aren’t legibly distinguished by their LinkedIn profiles. Recruiters in general have niches, and (to the best of my knowledge) this is not a niche that is well-served by recruiters. Superficially qualified candidates who don’t actually believe in the problem that they’re being paid to attack are probably not going to make much progress on it, but they will take up the time and attention of existing researchers.
Could MIRI do a better job of advertising their own job postings? Maybe, unless what they’re doing is effective at preferentially filtering out bad candidates without imposing an overwhelming barrier to good candidates. MIRI is hardly the only player in the game at this point, though—Redwood, ARC, Anthropic, and Alignment AI are all hiring researchers and none of them (officially) require candidates to work their way up to being Alignment Forum regulars.
“ok, bite the bullet, and spend 6 months − 2 years training interested math/cs/etc students to be competent researchers”—but have they tried this with Terry Tao?
People have tried to approach top people in their field and offer them >$500k/yr salaries, though not at any scale and not to a ton of people. Mostly people in ML and CS are already used to seeing these offers and frequently reject them, if they are in academia.
There might be some mathematicians for whom this is less true. I don’t know specifically about Terence Tao. Sometimes people also react quite badly to such an offer and it seems pretty plausible they would perceive it as a kind of bribe and making such an offer could somewhat permanently sour relations, though I think it’s likely one could find a framing for making such an offer that wouldn’t run into these problems.
My takeaway from this datapoint is that people are motivated by status more than money. This suggests ramping up advocacy via a media campaign, and making alignment high-status among the scientifically-minded (or general!) populace.
That is indeed a potential path that I am also more excited about. There are potentially ways to achieve that with money, though most ways I’ve thought about are also very prone to distorting what exactly is viewed as high-status, and creating strong currents in the field to be more prestigious, legible, etc. However, my guess is there are still good investments in the space, and I am pursuing some of them.
I’d love to hear more about this (unless secrecy is beneficial for some reason). I’d also like to know if there’s any way for people to donate to such an effort.
Yes, I think reaching out to differentiated candidates (like TT) is at least worth trying, since if it doesn’t work the downside is bounded by the relatively small number of attempts you can make.