I strongly believe that, given the state of things, we really should spend way more on higher-quality people and see what happens. Up to and including paying Terry Tao 10 million dollars. I would like to emphasize I am not joking about this.
I’ve heard lots of objections to this idea, and don’t think any of them convince me it is not worth actually trying this.
One objection I can conceive of why they’re not doing this is that most fields medalists probably already have standing offers from places like Jane Street, and aren’t taking them because they prefer academia. But if that were the case, and they’d already tried, that just tells me that MIRI might be really bad at recruiting. There has to be a dozen different reasons someone who wouldn’t work at Jane Street would still be willing to do legitimate maths research, for the same amount of money as they’d get in finance, to save the damn world.
I’m pretty sure MIRI has on the order of ten billion dollars at it’s disposal. If this is the case, then they can outbid Jane Street. In fact, trying to snipe Jane Street engineers isn’t a bad recruitment strategy, as a lot of the filtering has already been done for you.
I always imagine that as we got closer to the sort of gray zone that you were talking about earlier, the best thing to do might be to pause the pushing of the performance of these systems so that you can analyze down to minute detail exactly and maybe even prove things mathematically about the system so that you know the limits and otherwise of the systems that you’re building. At that point I think all the world’s greatest minds should probably be thinking about this problem. So that was what I would be advocating to you know the Terence Tao’s of this world, the best mathematicians. Actually I’ve even talked to him about this—I know you’re working on the Riemann hypothesis or something which is the best thing in mathematics but actually this is more pressing. I have this sort of idea of like almost uh ‘Avengers assembled’ of the scientific world because that’s a bit of like my dream.
I wonder how fleshed out the full plan is? The fact that there is a plan does give me some hope. But as Tomás B. says below, this needs to be put into place now, rather than waiting for a fire alarm that may never come.
I strongly believe that, given the state of things, we really should spend way more on higher-quality people and see what happens. Up to and including paying Terry Tao 10 million dollars. I would like to emphasize I am not joking about this.
I’ve heard lots of objections to this idea, and don’t think any of them convince me it is not worth actually trying this.
One objection I can conceive of why they’re not doing this is that most fields medalists probably already have standing offers from places like Jane Street, and aren’t taking them because they prefer academia. But if that were the case, and they’d already tried, that just tells me that MIRI might be really bad at recruiting. There has to be a dozen different reasons someone who wouldn’t work at Jane Street would still be willing to do legitimate maths research, for the same amount of money as they’d get in finance, to save the damn world.
I’m pretty sure MIRI has on the order of ten billion dollars at it’s disposal.If this is the case, then they can outbid Jane Street. In fact, trying to snipe Jane Street engineers isn’t a bad recruitment strategy, as a lot of the filtering has already been done for you.We don’t have 10 billion dollars.
So you are funding-constrained.
SBF/FTX does though.
In a March 15th episode of the DeepMind podcast, Demis Hassabis said he has talked to Terence Tao about working on AI safety:
His dream team is contingent on a fire alarm. This really needs to happen right now.
I’m glad Demis is somewhat reasonable. How tf can we pull the fire alarm? A petition?
Did you change your mind about Demis being reasonable or a petition being a good idea? And why? I’m considering sending him an email.
Interesting. I note that they don’t actually touch on x-risk in the podcast, but the above quote implies that Demis cares a lot about Alignment.
I wonder how fleshed out the full plan is? The fact that there is a plan does give me some hope. But as Tomás B. says below, this needs to be put into place now, rather than waiting for a fire alarm that may never come.
I support this.
Here’s a version of this submitted as a project idea for the FTX Foundation.
Here’s a more fleshed out version, FAQ style. Comments welcome.