A suggestion to blacklist anyone who decided to give $30 million (a paltry sum of money for a startup) to OpenAI.
I agree with many of the points you have made in this post, but I strongly disagree with the characterisation of $30 million as a “paltry sum”.
1. My limited research indicates that $30 million was likely a significant amount of money for OpenAI at the time
I haven’t been able to find internal financial reports from 2017 OpenAI* but the following quote from wikipedia describes OpenAI’s operating expenses in that year.
”In 2017, OpenAI spent $7.9 million, or a quarter of its functional expenses, on cloud computing alone”
So, while OpenAI is currently worth tens of billions, $30 million appears to have been a significant sum for them in 2017.
Again, I haven’t been able to find internal financial reports (not claiming they aren’t available).
My understanding is Open Phil would have access to reports which would show that $30 million was or wasn’t a significant amount of money at the time, although they’re probably bound by confidentiality agreements which would forbid them from sharing.
2. $30 million was (and still is) a substantial amount of money for AI Safety Research.
This can be seen by simply looking at the financial reports of various safety orgs. In my original shortform post I believe I compared that amount to a few years of MIRI’s operating expenses.
But you can take your pick of safety orgs and you’ll see that $30 million buys you a lot. AI Safety researchers are (relatively) cheap.
It’s a lot for AI safety but for OpenAI at the time, with the backing of Elon Musk and the most respected AI researchers in the country, they could have raised a similar amount for series A funding at the time. (I’m unsure if they were a capped profit yet). Likewise, 1B was pledged to them at their founding, but it’s hard to tell how much was actually distributed out by 2017.
Agree with 2, but Safety research also seems hard to fund.
with the backing of Elon Musk and the most respected AI researchers in the country, they could have raised a similar amount for series A funding at the time
It’s not clear that Elon Musk was willing to give them even $30m in March 2017 (early 2017) when OpenPhil did, nor would anyone there believe that selling a large fraction of the company for just $30m in a seed round was a good idea. The rupture was not far away, that led to Musk completely leaving OA after trying & failing to eject Sam Altman. See https://openai.com/index/openai-elon-musk/
We spent a lot of time trying to envision a plausible path to AGI. In early 2017, we came to the realization that building AGI will require vast quantities of compute. We began calculating how much compute an AGI might plausibly require. We all understood we were going to need a lot more capital to succeed at our mission—billions of dollars per year, which was far more than any of us, especially Elon, thought we’d be able to raise as the non-profit.
And note Musk’s quoted emails later in 2018 are taking the all-or-nothing attitude of OA should either find a path to raise billions per year (not total), or give up and shut down. (Hence Andrej Karpathy’s proposal there to merge OA into Tesla and try to become a trillion-dollar company on self-driving AI, to make AGI pay its way.) If you believe that a non-profit needs >$2000m/year or it’s a total waste of money, how eager are you going to be to donate $30m, once? Seems like a good way to waste $30m by throwing good money after bad...
I agree with many of the points you have made in this post, but I strongly disagree with the characterisation of $30 million as a “paltry sum”.
1. My limited research indicates that $30 million was likely a significant amount of money for OpenAI at the time
I haven’t been able to find internal financial reports from 2017 OpenAI* but the following quote from wikipedia describes OpenAI’s operating expenses in that year.
”In 2017, OpenAI spent $7.9 million, or a quarter of its functional expenses, on cloud computing alone”
So, while OpenAI is currently worth tens of billions, $30 million appears to have been a significant sum for them in 2017.
Again, I haven’t been able to find internal financial reports (not claiming they aren’t available).
My understanding is Open Phil would have access to reports which would show that $30 million was or wasn’t a significant amount of money at the time, although they’re probably bound by confidentiality agreements which would forbid them from sharing.
2. $30 million was (and still is) a substantial amount of money for AI Safety Research.
This can be seen by simply looking at the financial reports of various safety orgs. In my original shortform post I believe I compared that amount to a few years of MIRI’s operating expenses.
But you can take your pick of safety orgs and you’ll see that $30 million buys you a lot. AI Safety researchers are (relatively) cheap.
It’s a lot for AI safety but for OpenAI at the time, with the backing of Elon Musk and the most respected AI researchers in the country, they could have raised a similar amount for series A funding at the time. (I’m unsure if they were a capped profit yet). Likewise, 1B was pledged to them at their founding, but it’s hard to tell how much was actually distributed out by 2017.
Agree with 2, but Safety research also seems hard to fund.
It’s not clear that Elon Musk was willing to give them even $30m in March 2017 (early 2017) when OpenPhil did, nor would anyone there believe that selling a large fraction of the company for just $30m in a seed round was a good idea. The rupture was not far away, that led to Musk completely leaving OA after trying & failing to eject Sam Altman. See https://openai.com/index/openai-elon-musk/
And note Musk’s quoted emails later in 2018 are taking the all-or-nothing attitude of OA should either find a path to raise billions per year (not total), or give up and shut down. (Hence Andrej Karpathy’s proposal there to merge OA into Tesla and try to become a trillion-dollar company on self-driving AI, to make AGI pay its way.) If you believe that a non-profit needs >$2000m/year or it’s a total waste of money, how eager are you going to be to donate $30m, once? Seems like a good way to waste $30m by throwing good money after bad...
If OpenAI didn’t get the 30m from any other donor, they’d probably just turn into a capped profit earlier and raise money that way.
Also I never said Elon would have been the one to donate. They had 1B pledged, so they could have conceivably gotten that money from any other donors.
By the backing of Elon Musk, I mean the startup is associated with his brand. I’d imagine this would make raising funding easier.