Alternatively, has anyone considered… just asking him to?
That sounds naive. Maybe it is. But maybe it isn’t. Maybe smart people like Terry can be convinced of something like “Oh shit! This is actually crazy important and working on it would be the best way to achieve my terminal values.”
(Personally I’m working on the “get 10 million dollars” part. I’m not sure what the best thing would be to do after that, but paying Terry Tao doesn’t sound like a bad idea.)
Edit: Information about contacting him can be found here. If MIRI hasn’t already, it seems to me like it’d be a good idea to try reaching out. It also seems worth being at least a little bit strategic about it as opposed to, say, a cold email. More generally, I think this probably applies to, say, the top 100 mathematicians in the world, not just to Terry. (I hesitate to say this because of some EMH-like reasoning: if it made sense MIRI would have done it already, so I shouldn’t waste time saying this. But noticing and plucking all of the low hanging fruit is actually really hard, so despite my very high opinion of MIRI, I think it is at least plausible if not likely that there is a decent amount of low hanging fruit left to be plucked.)
A reply to comments showing skepticism about how mathematical skills of someone like Tao could be relevant:
Last time I thought I would understood anything of Tao’s blog was around ~2019. Then he was working on curious stuff, like whether he could prove there can be finite-time blow-up singularities in Navier-Stokes fluid equations (coincidentally, solving the famous Millenium prize problem showing non-smooth solution) by constructing a fluid state that both obeys Navier-Stokes and also is Turing complete and … ugh, maybe I quote the man himself:
[...] one would somehow have to make the incompressible fluid obeying the Navier–Stokes equations exhibit enough of an ability to perform computation that one could programme a self-replicating state of the fluid that behaves in a manner similar to that described above, namely a long period of near equilibrium, followed by an abrupt reorganization of the state into a rescaled version of itself. However, I do not know of any feasible way to implement (even in principle) the necessary computational building blocks, such as logic gates, in the Navier–Stokes equations.
However, it appears possible to implement such computational ability in partial differential equations other than the Navier–Stokes equations. I have shown5 that the dynamics of a particle in a potential well can exhibit the behaviour of a universal Turing machine if the potential function is chosen appropriately. Moving closer to the Navier–Stokes equations, the dynamics of the Euler equations for inviscid incompressible fluids on a Riemannian manifold have also recently been shown6,7 to exhibit some signs of universality, although so far this has not been sufficient to actually create solutions that blow up in finite time.
The relation (if any, to proving stuff about computational agents alignment people are interested in) is probably spurious (I myself don’t follow either Tao’s work or alignment literature), but I am curious if he’d be interested in working on a formal system of self-replicating / self-improving / aligning computational agents, and (then) capable of finding something genuinely interesting.
There’s a finite resource that gets used up when someone contacts Person in High Demand, which is roughly, that person’s openness to thinking about whether your problem is interesting.
The following is probably moot because I think it’s best for AI research organizations (hopefully ones with some prestige) to be the ones who pursue this, but in skimming through the paper, I don’t get the sense that it is applicable here.
From the abstract (emphasis mine):
In some situations a number of agents each have the ability to undertake an initiative that would have significant effects on the others. Suppose that each of these agents is purely motivated by an altruistic concern for the common good. We show that if each agent acts on her own personal judgment as to whether the initiative should be undertaken, then the initiative will be undertaken more often than is optimal.
Toy example from the introduction:
A sports team is planning a surprise birthday party for its coach. One of the players decides that it would be more fun to tell the coach in advance about the planned event. Although the other players think it would be better to keep it a surprise, the unilateralist lets word slip about the preparations
underway.
With Terry, I sense that it isn’t a situation where the action of one would have a significant affect on others (well, on Terry). For example, suppose Alice, a reader of LessWrong, saw my comment and emailed Terry. The most likely outcome here, I think, is that it just gets filtered out by some secretary and it never reaches Terry. But even if it did reach Terry, my model of him/people like him is that, if he in fact is unconvinced by the importance of AI safety, it would only be a mild annoyance and he’d probably forget it ever happened.
On the other hand, my model is also that if dozens and dozens of these emails reach him to the point where it starts to be an inconvenience to deal with them, at that point I think it would make him more notably annoyed, and I expect that this would make him less willing to join the cause. However, I expect that it would move him from thinking like a scout/weak sports fan to thinking like a weak/moderate sports fan. In other words, I expect the annoyance to make him a little bit biased, but still open to the idea and still maintaining solid epistemics. That’s just my model though.
I think the model clearly applies, though almost certainly the effect is less strictly binary than in the surprise party example.
I expect the annoyance to make him a little bit biased, but still open to the idea and still maintaining solid epistemics.
This is roughly a crux for me, yeah. I think dozens of people emailing him would cause him to (fairly reasonably, actually!) infer that something weird is going on (e.g., people are in a crazy echo chamber) and that he’s being targeted for unwanted attention (which he would be!). And it seems important, in a unilateralist’s curse way, that this effect is probably unrelated to the overall size of the group of people who have these beliefs about AI. Like, if you multiply the number of AI-riskers by 10, you also multiply by 10 the number of people who, by some context-unaware individual judgement, think they should cold-email Tao. Some of these people will be correct that they should do something like that, but it seems likely that many of such people will be incorrect.
Yeah, random internet forum users emailing eminent mathematician en masse would be strange enough to be non-productive. I for one wasn’t thinking anyone would to, I don’t think it was what OP suggested. To anyone contemplating sending one, the task is best delegated to someone who not only can write coherent research proposals that sound relevant to the person approached, but can write the best one.
Mathematicians receive occasional crank emails about solutions to P ?= NP, so anyone doing the reaching needs to be reputable to get past their crank filters.
fwiw, I don’t think someone’s openness to thinking about an idea necessarily goes down as more people contact them about it. I’d expect it to go up. Although this might not necessarily be true for our target group
If MIRI hasn’t already, it seems to me like it’d be a good idea to try reaching out. It also seems worth being at least a little bit strategic about it as opposed to, say, a cold email.
+1 especially to this—surely MIRI or a similar x-risk org could attain a warm introduction with potential top researchers through their network from someone who is willing to vouch for them.
Alternatively, has anyone considered… just asking him to?
That sounds naive. Maybe it is. But maybe it isn’t. Maybe smart people like Terry can be convinced of something like “Oh shit! This is actually crazy important and working on it would be the best way to achieve my terminal values.”
(Personally I’m working on the “get 10 million dollars” part. I’m not sure what the best thing would be to do after that, but paying Terry Tao doesn’t sound like a bad idea.)
Edit: Information about contacting him can be found here. If MIRI hasn’t already, it seems to me like it’d be a good idea to try reaching out. It also seems worth being at least a little bit strategic about it as opposed to, say, a cold email. More generally, I think this probably applies to, say, the top 100 mathematicians in the world, not just to Terry. (I hesitate to say this because of some EMH-like reasoning: if it made sense MIRI would have done it already, so I shouldn’t waste time saying this. But noticing and plucking all of the low hanging fruit is actually really hard, so despite my very high opinion of MIRI, I think it is at least plausible if not likely that there is a decent amount of low hanging fruit left to be plucked.)
A reply to comments showing skepticism about how mathematical skills of someone like Tao could be relevant:
Last time I thought I would understood anything of Tao’s blog was around ~2019. Then he was working on curious stuff, like whether he could prove there can be finite-time blow-up singularities in Navier-Stokes fluid equations (coincidentally, solving the famous Millenium prize problem showing non-smooth solution) by constructing a fluid state that both obeys Navier-Stokes and also is Turing complete and … ugh, maybe I quote the man himself:
The relation (if any, to proving stuff about computational agents alignment people are interested in) is probably spurious (I myself don’t follow either Tao’s work or alignment literature), but I am curious if he’d be interested in working on a formal system of self-replicating / self-improving / aligning computational agents, and (then) capable of finding something genuinely interesting.
minor clarifying edits.
Please keep the unilateralist’s curse in mind when considering plans like this. https://nickbostrom.com/papers/unilateralist.pdf
There’s a finite resource that gets used up when someone contacts Person in High Demand, which is roughly, that person’s openness to thinking about whether your problem is interesting.
The following is probably moot because I think it’s best for AI research organizations (hopefully ones with some prestige) to be the ones who pursue this, but in skimming through the paper, I don’t get the sense that it is applicable here.
From the abstract (emphasis mine):
Toy example from the introduction:
With Terry, I sense that it isn’t a situation where the action of one would have a significant affect on others (well, on Terry). For example, suppose Alice, a reader of LessWrong, saw my comment and emailed Terry. The most likely outcome here, I think, is that it just gets filtered out by some secretary and it never reaches Terry. But even if it did reach Terry, my model of him/people like him is that, if he in fact is unconvinced by the importance of AI safety, it would only be a mild annoyance and he’d probably forget it ever happened.
On the other hand, my model is also that if dozens and dozens of these emails reach him to the point where it starts to be an inconvenience to deal with them, at that point I think it would make him more notably annoyed, and I expect that this would make him less willing to join the cause. However, I expect that it would move him from thinking like a scout/weak sports fan to thinking like a weak/moderate sports fan. In other words, I expect the annoyance to make him a little bit biased, but still open to the idea and still maintaining solid epistemics. That’s just my model though.
I think the model clearly applies, though almost certainly the effect is less strictly binary than in the surprise party example.
This is roughly a crux for me, yeah. I think dozens of people emailing him would cause him to (fairly reasonably, actually!) infer that something weird is going on (e.g., people are in a crazy echo chamber) and that he’s being targeted for unwanted attention (which he would be!). And it seems important, in a unilateralist’s curse way, that this effect is probably unrelated to the overall size of the group of people who have these beliefs about AI. Like, if you multiply the number of AI-riskers by 10, you also multiply by 10 the number of people who, by some context-unaware individual judgement, think they should cold-email Tao. Some of these people will be correct that they should do something like that, but it seems likely that many of such people will be incorrect.
Yeah, random internet forum users emailing eminent mathematician en masse would be strange enough to be non-productive. I for one wasn’t thinking anyone would to, I don’t think it was what OP suggested. To anyone contemplating sending one, the task is best delegated to someone who not only can write coherent research proposals that sound relevant to the person approached, but can write the best one.
Mathematicians receive occasional crank emails about solutions to P ?= NP, so anyone doing the reaching needs to be reputable to get past their crank filters.
I think the people cold emailing Terry in this scenario should at least make sure they have the $10M ready!
fwiw, I don’t think someone’s openness to thinking about an idea necessarily goes down as more people contact them about it. I’d expect it to go up.
Although this might not necessarily be true for our target group
+1 especially to this—surely MIRI or a similar x-risk org could attain a warm introduction with potential top researchers through their network from someone who is willing to vouch for them.