Funding someone to set up more contact with whoever is working on AI risk in China. I recall Jeffrey Ding mentioned that Tencent has some people articulating concerns over AI in his paper Deciphering China’s AI Dream. I asked Dr. Ding about this, and let’s see if he replies.
I’m also working on surveying the top 0.1% of mathematicians to find how much $ would be required to get them to work on AI safety happily for a while. Or to join a workshop for a week. I think these questions are more likely to get a serious answer in person. This is obviously parallelizable, requires some consistency in the survey questions and is worth co-ordinating on. I’d like to organise EAs on this topic, and maybe even pay people in local communities who might otherwise not be bothered to go in person. Better that then travelling there myself, and trying to leverage the communities connections to get some time with these guys.
Also, the more we interview, the more people we can namedrop and hence the more respectable our endeavour will appear.
Do you have a survey or are you just doing them personally?
One concern is not having well-specified problems in their specific expertise (eg we don’t have mesa-optimizers specified as a problem in number theory, and it may not be useful actually), so there’s an onboarding process. Or a level of vetting/trust that some of the ones picked can understand the core difficulty and go back-and-forth from formalization to actual-problem-in-reality.
Having both more ELK-like questions and set of lit reviews for each subfield would help. It’d be even better if someone specifically formalized these problems in different math fields (if it made sense to do so), but that already requires someone knowledgeable in the field to do the formalizing. So a bit of iterative-bootstrapping would be useful.
I’m devising the survey and thinking about how to approach these people. My questions would probably be of the form
How much would it take for you to attend a technical workshop?
How much to take a sabbatical to work in a technical field?
How much for you to spend X amount of time on problem Y?
Yes, we do need something they can work on. That’s part of what makes the suvey tricky, because I expect if you said “work on problem X which is relevant to your field” vs “work on problem Y that you know nothing about, and attend a workshop to get you up to speed” would result in very different answers. And knowing what questions to ask them requires a fair bit of background knowledge in AI safety and the mathematicians subfield, so this limits the pool of people that can sensibly work on this.
Which is why trying to parralelise things and perhaps set up a group where we can discuss targets and how to best approach them would be useful. I’d also like to be able to contact AI safety folks on who they think we should go after, and which problems we should present as well as perhaps organising some background reading for these guys as we want to get them up to speed as quickly as possible.
Funding someone to set up more contact with whoever is working on AI risk in China. I recall Jeffrey Ding mentioned that Tencent has some people articulating concerns over AI in his paper Deciphering China’s AI Dream. I asked Dr. Ding about this, and let’s see if he replies.
I’m also working on surveying the top 0.1% of mathematicians to find how much $ would be required to get them to work on AI safety happily for a while. Or to join a workshop for a week. I think these questions are more likely to get a serious answer in person. This is obviously parallelizable, requires some consistency in the survey questions and is worth co-ordinating on. I’d like to organise EAs on this topic, and maybe even pay people in local communities who might otherwise not be bothered to go in person. Better that then travelling there myself, and trying to leverage the communities connections to get some time with these guys.
Also, the more we interview, the more people we can namedrop and hence the more respectable our endeavour will appear.
Do you have a survey or are you just doing them personally?
One concern is not having well-specified problems in their specific expertise (eg we don’t have mesa-optimizers specified as a problem in number theory, and it may not be useful actually), so there’s an onboarding process. Or a level of vetting/trust that some of the ones picked can understand the core difficulty and go back-and-forth from formalization to actual-problem-in-reality.
Having both more ELK-like questions and set of lit reviews for each subfield would help. It’d be even better if someone specifically formalized these problems in different math fields (if it made sense to do so), but that already requires someone knowledgeable in the field to do the formalizing. So a bit of iterative-bootstrapping would be useful.
I’m devising the survey and thinking about how to approach these people. My questions would probably be of the form
How much would it take for you to attend a technical workshop?
How much to take a sabbatical to work in a technical field?
How much for you to spend X amount of time on problem Y?
Yes, we do need something they can work on. That’s part of what makes the suvey tricky, because I expect if you said “work on problem X which is relevant to your field” vs “work on problem Y that you know nothing about, and attend a workshop to get you up to speed” would result in very different answers. And knowing what questions to ask them requires a fair bit of background knowledge in AI safety and the mathematicians subfield, so this limits the pool of people that can sensibly work on this.
Which is why trying to parralelise things and perhaps set up a group where we can discuss targets and how to best approach them would be useful. I’d also like to be able to contact AI safety folks on who they think we should go after, and which problems we should present as well as perhaps organising some background reading for these guys as we want to get them up to speed as quickly as possible.