They did the opposite, incentivizing themselves to reach the profit cap. I’m talking about making sure that any net worth beyond a billion goes to someone else.
The value of the startup is only loosely correlated with being positive for AI safety (capabilities are valuable, but they’re not the only valuable thing). Ideally the startup would be worth billions if and only if AI safety was solved.
A startup could disincentivize itself from becoming worth more than a billion dollars by selling an option to buy it for a billion dollars.
Not to say it’s impossible, but OpenAI tried to do this with the profit cap and it didn’t work.
They did the opposite, incentivizing themselves to reach the profit cap. I’m talking about making sure that any net worth beyond a billion goes to someone else.
I didn’t track this previously: how did they incentive themselves to reach the cap?
If your investors only get paid up to 100x their investment, you want to go for strategies that return much more than 100x if they work.
The value of the startup is only loosely correlated with being positive for AI safety (capabilities are valuable, but they’re not the only valuable thing). Ideally the startup would be worth billions if and only if AI safety was solved.