I’m not trying to say “it’s bad to give large sums of money to any group because humans have a tendency to to seek power.”
I’m saying “you should be exceptionally cautious about giving large sums of money to a group of humans with the stated goal of constructing an AGI.”
You need to weight any reassurances they give you against two observations:
The commonly observed pattern of individual humans or organisations seeking power (and/or wealth) at the expense of the wider community.
The strong likelihood that there will be an opportunity for organisations pushing ahead with AI research to obtain incredible wealth or power.
So, it isn’t “humans seek power therefore giving any group of humans money is bad”. It’s “humans seek power” and, in the specific case of AI companies, there may be incredibly strong rewards for groups that behave in a self-interested way.
The general idea I’m working off is that you need to be skeptical of seemingly altruistic statements and commitments made by humans when there are exceptionally lucrative incentives to break these commitments at a later point in time (and limited ways to enforce the original commitment).
That seems like a valuable argument. It might be worth updating the wording under premise 2 to clarifying this? To me it reads as saying that the configuration, rather than the aim, of OpenAI was the major red flag.
This does not feel super cruxy as the the power incentive still remains.
In that case OP’s argument would be saying that donors shouldn’t give large sums of money to any sort of group of people, which is a much bolder claim
(I’m the OP)
I’m not trying to say “it’s bad to give large sums of money to any group because humans have a tendency to to seek power.”
I’m saying “you should be exceptionally cautious about giving large sums of money to a group of humans with the stated goal of constructing an AGI.”
You need to weight any reassurances they give you against two observations:
The commonly observed pattern of individual humans or organisations seeking power (and/or wealth) at the expense of the wider community.
The strong likelihood that there will be an opportunity for organisations pushing ahead with AI research to obtain incredible wealth or power.
So, it isn’t “humans seek power therefore giving any group of humans money is bad”. It’s “humans seek power” and, in the specific case of AI companies, there may be incredibly strong rewards for groups that behave in a self-interested way.
The general idea I’m working off is that you need to be skeptical of seemingly altruistic statements and commitments made by humans when there are exceptionally lucrative incentives to break these commitments at a later point in time (and limited ways to enforce the original commitment).
That seems like a valuable argument. It might be worth updating the wording under premise 2 to clarifying this? To me it reads as saying that the configuration, rather than the aim, of OpenAI was the major red flag.