Thank you for this thoughtful comment Linda—writing this replying has helped me to clarify my own thinking on growth and depth. My basic sense is this:
If I meet someone who really wants to help out with AI safety, I want to help them to do that, basically without reservation, regardless of their skill, experience, etc. My sense is that we have a huge and growing challenge in navigating the development of advanced AI, and there is just no shortage of work to do, though it can at first be quite difficult to find. So when I meet individuals, I will try to help them find out how to really help out. There is no need for me to judge whether a particular person really wants to help out or not; I simply help them see how they can help out, and those who want to help out will proceed. Those who do not want to help out will not proceed, and that’s fine too—there are plenty of good reasons for a person to not want to dive head-first into AI safety.
But it’s different when I consider setting up incentives, which is what @Larks was writing about:
My basic model for AI safety success is this:
Identify interesting problems. As a byproduct this draws new people into the field through altruism, nerd-sniping, apparent tractability.
Solve interesting problems. As a byproduct this draws new people into the field through credibility and prestige.
I’m quite concerned about “drawing people into the field through credibility and prestige” and even about “drawing people into the field through altruism, nerd-sniping, and apparent tractability”. The issue is not the people who genuinely want to help out, whom I consider to be a boon to the field regardless of their skill or experience. The issue is twofold:
Drawing people who are not particularly interested in helping out into the field via incentives (credibility, prestige, etc).
Tempting those who do really want to help out and are already actually helping out to instead pursue incentives (credibility, prestige, etc).
So I’m not skeptical of growth via helping individuals, I’m skeptical of growth via incentives.
Ok, that makes sense. Seems like we are mostly on the same page then.
I don’t have strong opinions weather drawing in people via prestige is good or bad. I expect it is probably complicated. For example, there might be people who want to work on AI Safety for the right reason, but are too agreeable to do it unless it reach some level of acceptability. So I don’t know what the effects will be on net. But I think it is an effect we will have to handle, since prestige will be important for other reasons.
On the other hand, there are lots of people who really do want to help, for the right reason. So if growth is the goal, helping these people out seems like just an obvious thing to do. I expect there are ways funders can help out here too.
I would not update much on the fact that currently most research is produced by existing institutions. It is hard to do good research, and even harder with out collogues, sallary and other support that comes with being part of an org. So I think there is a lot of room for growth, by just helping the people who are already involved and trying.
On the other hand, there are lots of people who really do want to help, for the right reason. So if growth is the goal, helping these people out seems like just an obvious thing to do
So I think there is a lot of room for growth, by just helping the people who are already involved and trying.
Thank you for this thoughtful comment Linda—writing this replying has helped me to clarify my own thinking on growth and depth. My basic sense is this:
If I meet someone who really wants to help out with AI safety, I want to help them to do that, basically without reservation, regardless of their skill, experience, etc. My sense is that we have a huge and growing challenge in navigating the development of advanced AI, and there is just no shortage of work to do, though it can at first be quite difficult to find. So when I meet individuals, I will try to help them find out how to really help out. There is no need for me to judge whether a particular person really wants to help out or not; I simply help them see how they can help out, and those who want to help out will proceed. Those who do not want to help out will not proceed, and that’s fine too—there are plenty of good reasons for a person to not want to dive head-first into AI safety.
But it’s different when I consider setting up incentives, which is what @Larks was writing about:
I’m quite concerned about “drawing people into the field through credibility and prestige” and even about “drawing people into the field through altruism, nerd-sniping, and apparent tractability”. The issue is not the people who genuinely want to help out, whom I consider to be a boon to the field regardless of their skill or experience. The issue is twofold:
Drawing people who are not particularly interested in helping out into the field via incentives (credibility, prestige, etc).
Tempting those who do really want to help out and are already actually helping out to instead pursue incentives (credibility, prestige, etc).
So I’m not skeptical of growth via helping individuals, I’m skeptical of growth via incentives.
Ok, that makes sense. Seems like we are mostly on the same page then.
I don’t have strong opinions weather drawing in people via prestige is good or bad. I expect it is probably complicated. For example, there might be people who want to work on AI Safety for the right reason, but are too agreeable to do it unless it reach some level of acceptability. So I don’t know what the effects will be on net. But I think it is an effect we will have to handle, since prestige will be important for other reasons.
On the other hand, there are lots of people who really do want to help, for the right reason. So if growth is the goal, helping these people out seems like just an obvious thing to do. I expect there are ways funders can help out here too.
I would not update much on the fact that currently most research is produced by existing institutions. It is hard to do good research, and even harder with out collogues, sallary and other support that comes with being part of an org. So I think there is a lot of room for growth, by just helping the people who are already involved and trying.
I very much agree with these two: