I’m not particularly sold on how Ought’s current focus (Elicit) translates to AI alignment. I’m particularly pessimistic about the governance angle, but I also don’t see how an automated research assistant is moving the needle on AI alignment research (as opposed to research in other domains, where I can much more easily imagine it being helpful).
This is possibly a failure of my understanding of their goals, or just of my ability to imagine helpful ways to use an automated research assistant (which won’t be as usable for research that advances capabilities?). I’m certainly open to changing my mind here. The other problem is that they seem to mostly be looking for front-end/full-stack engineers, and while I’m ok with working on the front-end of an existing codebase, I would not describe that as my particular area of expertise.
I also forgot to include Anthropic on my list.
Fathom seems interesting, but fundamentally the idea seems to be “apply economic incentives to try to tilt the field in a more safety-conscious direction”, which has most of the same problems as “throw money at the problem of AI alignment” but one step removed. They’re also grappling with the fact that their R&D efforts are fundamentally capabilities-focused and their only levers are to say “we’ll charge you more” or “we won’t sell this to you”, which I can’t say seems like a huge improvement over “this doesn’t actually exist for you to buy”.
I may very well solicit a consultation with 80000 Hours soon :)
I’m not particularly sold on how Ought’s current focus (Elicit) translates to AI alignment. I’m particularly pessimistic about the governance angle, but I also don’t see how an automated research assistant is moving the needle on AI alignment research (as opposed to research in other domains, where I can much more easily imagine it being helpful).
This is possibly a failure of my understanding of their goals, or just of my ability to imagine helpful ways to use an automated research assistant (which won’t be as usable for research that advances capabilities?). I’m certainly open to changing my mind here. The other problem is that they seem to mostly be looking for front-end/full-stack engineers, and while I’m ok with working on the front-end of an existing codebase, I would not describe that as my particular area of expertise.
I also forgot to include Anthropic on my list.
Fathom seems interesting, but fundamentally the idea seems to be “apply economic incentives to try to tilt the field in a more safety-conscious direction”, which has most of the same problems as “throw money at the problem of AI alignment” but one step removed. They’re also grappling with the fact that their R&D efforts are fundamentally capabilities-focused and their only levers are to say “we’ll charge you more” or “we won’t sell this to you”, which I can’t say seems like a huge improvement over “this doesn’t actually exist for you to buy”.
I may very well solicit a consultation with 80000 Hours soon :)