It’s not clear to me that there was actually an option to build a $100B company with competent people around the world who would’ve been united in conditionally shutting down and unconditionally pushing for regulation. I don’t know that the culture and concepts of people who do a lot of this work in the business world would allow for such a plan to be actively worked on.
You maybe right. Maybe the top talent wouldn’t have gotten on board with that mission, and so it wouldn’t have gotten top talent.
I bet Illya would have been in for that mission, and I think a surprisingly large number of other top researchers might have been in for it as well. Obviously we’ll never know.
And I think if the founders are committed to a mission, and they reaffirm their commitment in every meeting, they can go surprisingly far in making in the culture of an org.
Maybe there’s a hope there, but I’ll point out that many of the people needed to run a business (finance, legal, product, etc) are not idealistic scientists who would be willing to have their equity become worthless.
Those people don’t get substantial equity in most business in the world. They generally get paid a salary and benefits in exchange for their work, and that’s about it.
I know little enough that I don’t know whether this statement is true. I would’ve guessed that in most $10B companies anyone with a title like “CFO” and “CTO” and “COO” is paid primarily in equity, but perhaps this is mostly true of a few companies I’ve looked into more (like Amazon).
Ilya is demonstrably not in on that mission, since his step immediately after leaving OpenAI was to found an additional AGI company and thus increase x-risk.
It’s not clear to me that there was actually an option to build a $100B company with competent people around the world who would’ve been united in conditionally shutting down and unconditionally pushing for regulation. I don’t know that the culture and concepts of people who do a lot of this work in the business world would allow for such a plan to be actively worked on.
You maybe right. Maybe the top talent wouldn’t have gotten on board with that mission, and so it wouldn’t have gotten top talent.
I bet Illya would have been in for that mission, and I think a surprisingly large number of other top researchers might have been in for it as well. Obviously we’ll never know.
And I think if the founders are committed to a mission, and they reaffirm their commitment in every meeting, they can go surprisingly far in making in the culture of an org.
Maybe there’s a hope there, but I’ll point out that many of the people needed to run a business (finance, legal, product, etc) are not idealistic scientists who would be willing to have their equity become worthless.
Those people don’t get substantial equity in most business in the world. They generally get paid a salary and benefits in exchange for their work, and that’s about it.
I know little enough that I don’t know whether this statement is true. I would’ve guessed that in most $10B companies anyone with a title like “CFO” and “CTO” and “COO” is paid primarily in equity, but perhaps this is mostly true of a few companies I’ve looked into more (like Amazon).
Ilya is demonstrably not in on that mission, since his step immediately after leaving OpenAI was to found an additional AGI company and thus increase x-risk.
I don’t think that’s a valid inference.