Claim 3: The Open Philanthropy Project’s Open AI grant represents a shift from evaluating other programs’ effectiveness, to assuming its own effectiveness.
If you want to discuss this claim, I encourage you to do it as a reply to this comment.
Wanting a board seat does not mean assuming that you know better than the current managers—only that you have distinct and worthwhile views that will add to the discussion that takes place in board meetings. This may be true even if you know worse than the current managers.
Is the idea that someone might think that current managers are wrongly failing to listen to them, but if forced to listen, would accept good ideas and reject bad ones? That seems plausible, though the more irrational you think the current managers are in the relevant ways, the more you should expect your influence to be through control rather than contributing to the discourse. Overall this seems like a decent alternative hypothesis.
I don’t think this grant alone represents a specific turning point, but OpenAI definitely is not old enough for its effectiveness and contribution to AI safety to actually be measured. I think the actual shifting point probably happened a long time ago, but it was readily apparent that it occurred when Open Phil decided that it should shift from transparency to “openness” and “information sharing”. In the beginning transparency was one of its core values and something it promised to its donors, whereas now it haschanged so that information sharing is basically only a tool it uses when necessary to ensure that it maintains its reputation.
I would actually be fine with Open Phil being very careful and circumspect about how it explains its thinking and processes behind what it does, as long as its success can still be measured. Consider how one might invest in a money management firm. Obviously hedge funds do not explain the details of their thinking and methods and have an incentive to keep them a secret, although they might explain a bit about their overall philosophy and give a general overview of their approach. But this is ok, because it is easy to evaluate the success of investments if the goal is financial gain. With philanthropy, this is much much harder, but EA was an improvement on what already existed because it specifically tries to quantify the impact of different philanthropic efforts. But when that changed to giving towards abstract causes, such as AI risk, we lost the ability to measure impacts. Is it even possible to measure success in mitigating the risks of AI? AI risk has an enormous amount of uncertainty among abstract causes, and the size of this grant suggests that Open Phil was extremely certain about the effect of this grant compared to its other grants. That’s very difficult to feel comfortable with unless I feel extremely confident in Open Phil’s thinking, which I can’t be because they have elected to keep most of the details of their thinking confidential.
I think Open Phil not even trying to bridge the gap between:
> (a) the reasons we believe what we believe and (b) the reasons we’re able to share publicly and relatively efficiently.
is deeply problematic.
The reasons given in the post you link to are, to my mind, not convincing at all. We are talking about directing large sums of money to AI research that could have done a lot of good if directed in a different way. The objection is that giving the justification for would just take too long, and also any objections to it from non-AI-specialists would not be worth listening to.
But given that the sums are large, spending time explaining the decision is crucial, because if the reasoning does not support the conclusion, it’s imperrative that this be discovered. And limiting input to AI-experts introduces what I would have thought is a totally unacceptable selection effect: these people are bound to be much more likely than average to belive that directing money to AI-research is very valuable.
Claim 3: The Open Philanthropy Project’s Open AI grant represents a shift from evaluating other programs’ effectiveness, to assuming its own effectiveness.
If you want to discuss this claim, I encourage you to do it as a reply to this comment.
Wanting a board seat does not mean assuming that you know better than the current managers—only that you have distinct and worthwhile views that will add to the discussion that takes place in board meetings. This may be true even if you know worse than the current managers.
Is the idea that someone might think that current managers are wrongly failing to listen to them, but if forced to listen, would accept good ideas and reject bad ones? That seems plausible, though the more irrational you think the current managers are in the relevant ways, the more you should expect your influence to be through control rather than contributing to the discourse. Overall this seems like a decent alternative hypothesis.
I don’t think this grant alone represents a specific turning point, but OpenAI definitely is not old enough for its effectiveness and contribution to AI safety to actually be measured. I think the actual shifting point probably happened a long time ago, but it was readily apparent that it occurred when Open Phil decided that it should shift from transparency to “openness” and “information sharing”. In the beginning transparency was one of its core values and something it promised to its donors, whereas now it has changed so that information sharing is basically only a tool it uses when necessary to ensure that it maintains its reputation.
I would actually be fine with Open Phil being very careful and circumspect about how it explains its thinking and processes behind what it does, as long as its success can still be measured. Consider how one might invest in a money management firm. Obviously hedge funds do not explain the details of their thinking and methods and have an incentive to keep them a secret, although they might explain a bit about their overall philosophy and give a general overview of their approach. But this is ok, because it is easy to evaluate the success of investments if the goal is financial gain. With philanthropy, this is much much harder, but EA was an improvement on what already existed because it specifically tries to quantify the impact of different philanthropic efforts. But when that changed to giving towards abstract causes, such as AI risk, we lost the ability to measure impacts. Is it even possible to measure success in mitigating the risks of AI? AI risk has an enormous amount of uncertainty among abstract causes, and the size of this grant suggests that Open Phil was extremely certain about the effect of this grant compared to its other grants. That’s very difficult to feel comfortable with unless I feel extremely confident in Open Phil’s thinking, which I can’t be because they have elected to keep most of the details of their thinking confidential.
I think Open Phil not even trying to bridge the gap between:
> (a) the reasons we believe what we believe and (b) the reasons we’re able to share publicly and relatively efficiently.
is deeply problematic.
The reasons given in the post you link to are, to my mind, not convincing at all. We are talking about directing large sums of money to AI research that could have done a lot of good if directed in a different way. The objection is that giving the justification for would just take too long, and also any objections to it from non-AI-specialists would not be worth listening to.
But given that the sums are large, spending time explaining the decision is crucial, because if the reasoning does not support the conclusion, it’s imperrative that this be discovered. And limiting input to AI-experts introduces what I would have thought is a totally unacceptable selection effect: these people are bound to be much more likely than average to belive that directing money to AI-research is very valuable.