I personally think it’s most important to have at least some technical employees who have the knowledge/expertise to evaluate the actual situation, and who also have the power to do something about it. I’d want this to include some people who’s primary job is more like “board member” and some people who’s primary job is more like “alignment and/or capabilities researcher.”
But there is a sense in which I’d feel way more comfortable if all technical AI employees (alignment and capabilities), and policy / strategy people, didn’t have profit equity, so they didn’t have inceptive to optimize against what was safe. So, there’s just a lot of eyes on the problem, and the overall egregore steering the company has one fewer cognitive distortion to manage.
This might be too expensive (OpenAI and Anthropic have a lot of money, but, like, that doesn’t mean they can just double everyone’s salary)
An idea that occurs to me is construct “windfall equity”, which only pays out if the company or world generates AI that is safely, massively improving the world.
I personally think it’s most important to have at least some technical employees who have the knowledge/expertise to evaluate the actual situation, and who also have the power to do something about it. I’d want this to include some people who’s primary job is more like “board member” and some people who’s primary job is more like “alignment and/or capabilities researcher.”
But there is a sense in which I’d feel way more comfortable if all technical AI employees (alignment and capabilities), and policy / strategy people, didn’t have profit equity, so they didn’t have inceptive to optimize against what was safe. So, there’s just a lot of eyes on the problem, and the overall egregore steering the company has one fewer cognitive distortion to manage.
This might be too expensive (OpenAI and Anthropic have a lot of money, but, like, that doesn’t mean they can just double everyone’s salary)
An idea that occurs to me is construct “windfall equity”, which only pays out if the company or world generates AI that is safely, massively improving the world.