It’s a lot easier than university departments, which require you to get multiple degrees (taking up something like a decade of your life depending on how you count it) before your application will even be considered.
By this stage of their careers, they already have those bits of paper. MIRI are asking people who don’t a priori highly value alignment research to jump through extra hoops they haven’t already cleared, for what they probably perceive as a slim chance of a job outside their wheelhouse. I know a reasonable number of hard science academics, and I don’t know any who would put in that amount of effort in the application for a job they thought would be highly applied for by more qualified applicants. The very phrasing makes it sound like they expect hundreds of applicants and are trying to be exclusive. If nothing else is changed, that should be.
Maybe they do in fact receive hundreds of applicants and must exclude most of them?
It’s not MIRI’s fault that there isn’t a pre-existing academic discipline of AI alignment research.
Imagine SpaceX had a branch office in some very poor country that literally didn’t have any engineering education whatsoever. Should they then lower their standards and invite applicants who never studied engineering? No, they should just deal with the fact that they won’t have very many qualified people, and/or they should do things like host workshops and stuff to help people learn engineering.
It wasn’t Los Alamos’ fault that there wasn’t a pre-existing academic discipline of nuclear engineering, but they got by anyway, because they had Von Neumann and other very smart people. If MIRI is to get by, they need to recruit Von Neumann-level people. Like maybe Terry Tao.
Just to be clear, there was a thriving field of nuclear engineering, and Los Almost was run mostly by leading figures in that field. Also, money was never a constraint on the Manhattan Program and it’s success had practically nothing to do with the availability of funding, but instead all to do with the war, the involvement of a number of top scientists, and the existence of a pretty concrete engineering problem that one could throw tons of manpower at.
The Manhattan project itself did not develop any substantial nuclear theory, and was almost purely an an engineering project. I do not know what we would get by emulating it at this point in time. The scientists involved in the Manhattan project did not continue running things like the Manhattan project, they went into other institutions optimized for intellectual process that were not capable of absorbing large amounts of money or manpower productively, despite some of them likely being able to get funding for similar things (some of them went on and built giant particle colliders, though this did not generally completely revolutionize or drastically accelerate the development of new scientific theories, though it sure was helpful).
Universities aren’t known for getting things done. Corporations are. Are you trying to signal exclusivity and prestige, or are you trying to save the lightcone?
Universities are pretty well known for getting things done. Most nobel prize winning work happens in them, for instance. It’s just that the things they do are not optimized for being the things corporations do.
I’m not saying we shouldn’t be actually trying to hire people. In fact I have a post from last year saying that exact thing. But if you think corporations are the model for what we should be going for, I think we have very different mechanistic models of how research gets done.
Getting an advanced degree in, say, CS, qualifies me to work for many different companies. Racking up karma posting mathematically rigorous research on the Alignment Forum qualifies me to work at one (1) place: MIRI. If I take the “PhD in CS” route, I have power to negotiate my salary, to be selective about who I work for. Every step I take along the Alignment Forum path is wasted[1] unless MIRI deigns to let me in.
I’m under the impression that that’s partially a sign of civilizational failure from metastasizing bureaucracies. I’ve always heard that the ultra-successful Silicon Valley companies never required a degree (and also that that meritocratic culture has eroded and been partially replaced by credentialism, causing stasis).
EDIT: to be clear, this means I disagree with the ridiculous hyperbole upthread of it being “cultish”, and in a lot of ways I’m sure the barriers to employment in traditional fields are higher. Still, as an outsider who’s missing lots of relevant info, it does seem like it should be possible to do a lot better.
I think you’re buying the hype of how much Alignment Forum posts help you even get the attention of MIRI way too much. I have a much easier time asking university departments for feedback, and there is a much smoother process for applying there.
Those multiple degrees are high cost but very low risk, because even if you don’t get into the university department, these degrees will give you lots of option value, while a 6 month gap in your CV trying to learn AI Safety on your own does not. More likely you will not survive the hit on your mental health.
I personally decided not to even try AI Safety research for this reason.
Grad school is infamously bad for your mental health. Something like one-third of my classmates dropped out. My AI safety researcher friends are overall less depressed and more happy, as far as I can tell, though it’s hard to tell because it’s so subjective.
These less depressed people you talk about, are they already getting paid as AI safety researchers, or are they self-studying to (hopefully) become AI safety researchers?
In any case, I’m clearly generalising from my own situation, so it may not extend very far. To flesh out this data point: I had 2 years of runway, so money wasn’t a problem, but I already felt beaten down by LW to the extent that I couldn’t really take any more hits to my self-esteem, so I couldn’t risk putting myself up for rejection again. That’s basically why I mostly left LW.
It’s a lot easier than university departments, which require you to get multiple degrees (taking up something like a decade of your life depending on how you count it) before your application will even be considered.
By this stage of their careers, they already have those bits of paper. MIRI are asking people who don’t a priori highly value alignment research to jump through extra hoops they haven’t already cleared, for what they probably perceive as a slim chance of a job outside their wheelhouse. I know a reasonable number of hard science academics, and I don’t know any who would put in that amount of effort in the application for a job they thought would be highly applied for by more qualified applicants. The very phrasing makes it sound like they expect hundreds of applicants and are trying to be exclusive. If nothing else is changed, that should be.
Maybe they do in fact receive hundreds of applicants and must exclude most of them?
It’s not MIRI’s fault that there isn’t a pre-existing academic discipline of AI alignment research.
Imagine SpaceX had a branch office in some very poor country that literally didn’t have any engineering education whatsoever. Should they then lower their standards and invite applicants who never studied engineering? No, they should just deal with the fact that they won’t have very many qualified people, and/or they should do things like host workshops and stuff to help people learn engineering.
It wasn’t Los Alamos’ fault that there wasn’t a pre-existing academic discipline of nuclear engineering, but they got by anyway, because they had Von Neumann and other very smart people. If MIRI is to get by, they need to recruit Von Neumann-level people. Like maybe Terry Tao.
Just to be clear, there was a thriving field of nuclear engineering, and Los Almost was run mostly by leading figures in that field. Also, money was never a constraint on the Manhattan Program and it’s success had practically nothing to do with the availability of funding, but instead all to do with the war, the involvement of a number of top scientists, and the existence of a pretty concrete engineering problem that one could throw tons of manpower at.
The Manhattan project itself did not develop any substantial nuclear theory, and was almost purely an an engineering project. I do not know what we would get by emulating it at this point in time. The scientists involved in the Manhattan project did not continue running things like the Manhattan project, they went into other institutions optimized for intellectual process that were not capable of absorbing large amounts of money or manpower productively, despite some of them likely being able to get funding for similar things (some of them went on and built giant particle colliders, though this did not generally completely revolutionize or drastically accelerate the development of new scientific theories, though it sure was helpful).
I don’t think we disagree?
I think you do in practice, because you seem to believe that the best way to recruit those people is via a strategy like the above.
Universities aren’t known for getting things done. Corporations are. Are you trying to signal exclusivity and prestige, or are you trying to save the lightcone?
Universities are pretty well known for getting things done. Most nobel prize winning work happens in them, for instance. It’s just that the things they do are not optimized for being the things corporations do.
I’m not saying we shouldn’t be actually trying to hire people. In fact I have a post from last year saying that exact thing. But if you think corporations are the model for what we should be going for, I think we have very different mechanistic models of how research gets done.
You took the words right out of my draft comment.
Corporations also often require advanced degrees in specific fields. Or multiple years of work experience.
Getting an advanced degree in, say, CS, qualifies me to work for many different companies. Racking up karma posting mathematically rigorous research on the Alignment Forum qualifies me to work at one (1) place: MIRI. If I take the “PhD in CS” route, I have power to negotiate my salary, to be selective about who I work for. Every step I take along the Alignment Forum path is wasted[1] unless MIRI deigns to let me in.
Not counting positive externalities
See my reply to benjamin upthread.
I’m under the impression that that’s partially a sign of civilizational failure from metastasizing bureaucracies. I’ve always heard that the ultra-successful Silicon Valley companies never required a degree (and also that that meritocratic culture has eroded and been partially replaced by credentialism, causing stasis).
EDIT: to be clear, this means I disagree with the ridiculous hyperbole upthread of it being “cultish”, and in a lot of ways I’m sure the barriers to employment in traditional fields are higher. Still, as an outsider who’s missing lots of relevant info, it does seem like it should be possible to do a lot better.
I think you’re buying the hype of how much Alignment Forum posts help you even get the attention of MIRI way too much. I have a much easier time asking university departments for feedback, and there is a much smoother process for applying there.
Those multiple degrees are high cost but very low risk, because even if you don’t get into the university department, these degrees will give you lots of option value, while a 6 month gap in your CV trying to learn AI Safety on your own does not. More likely you will not survive the hit on your mental health.
I personally decided not to even try AI Safety research for this reason.
Grad school is infamously bad for your mental health. Something like one-third of my classmates dropped out. My AI safety researcher friends are overall less depressed and more happy, as far as I can tell, though it’s hard to tell because it’s so subjective.
These less depressed people you talk about, are they already getting paid as AI safety researchers, or are they self-studying to (hopefully) become AI safety researchers?
In any case, I’m clearly generalising from my own situation, so it may not extend very far. To flesh out this data point: I had 2 years of runway, so money wasn’t a problem, but I already felt beaten down by LW to the extent that I couldn’t really take any more hits to my self-esteem, so I couldn’t risk putting myself up for rejection again. That’s basically why I mostly left LW.
Ah, good point, mostly the former category. I only know a few people in the latter category.