The majority of the leadership, engineers, etc. at OpenAI/DeepMind/Anthropic don’t agree that we’d be collectively better off if they all shut down.
The majority do agree, they just aren’t solving the collective action problem.
If (2) is the case, has anyone thought about using a dominant assurance contract?
The dominant assurance contract adds a simple twist to the crowdfunding contract. An entrepreneur commits to produce a valuable public good if and only if enough people donate, but if not enough donate, the entrepreneur commits not just to return the donor’s funds but to give each donor a refund bonus. To see how this solves the public good problem consider the simplest case. Suppose that there is a public good worth $100 to each of 10 people. The cost of the public good is $800. If each person paid $80, they all would be better off. Each person, however, may choose not to donate, perhaps because they think others will not donate, or perhaps because they think that they can free ride.
Now consider a dominant assurance contract. An entrepreneur agrees to produce the public good if and only if each of 10 people pay $80. If fewer than 10 people donate, the contract is said to fail and the entrepreneur agrees to give a refund bonus of $5 to each of the donors. Now imagine that potential donor A thinks that potential donor B will not donate. In that case, it makes sense for A to donate, because by doing so he will earn $5 at no cost. Thus any donor who thinks that the contract will fail has an incentive to donate. Doing so earns free money. As a result, it cannot be an equilibrium for more than one person to fail to donate. We have only one more point to consider. What if donor A thinks that every other donor will donate? In this case, A knows that if he donates he won’t get the refund bonus, since the contract will succeed. But he also knows that if he doesn’t donate he won’t get anything, but if does donate he will pay $80 and get a public good which is worth $100 to him, for a net gain of $20. Thus, A always has an incentive to donate. If others do not donate, he earns free money. If others do donate, he gets the value of the public good. Thus donating is a win-win, and the public good problem is solved.[2]
Maybe this would look something like: We offer a contract to engineers at specific major AI labs. If at least 90% of the engineers at each of those specific labs sign the contract by end of 2024, they agree to all mass quit their jobs. If not, everyone who signed the contract gets $500 at the end of 2024.
I’m guessing that coordination among the leadership has already been tried and failed. But if not, another idea is to structure the dominance assurance contract as an investment round, so it ends up being a financial boost for safety-conscious organizations that are willing to sign the contract, if not enough organizations sign.
One story for why coordination does not materialize:
Meta engineers self-select for being unconcerned with safety. They aren’t going to quit any time soon. If offered a dominance assurance contract, they won’t sign either early or late.
DeepMind engineers feel that DeepMind is more responsible than Meta. They think a DeepMind AGI is more likely to be aligned than a Meta AGI, and they feel it would be irresponsible to quit and let Meta build AGI.
OpenAI engineers feel that OpenAI is more responsible than Meta or DeepMind, by similar logic it’s irresponsible for them to quit.
Anthropic engineers feel that Anthropic is more responsible than OpenAI/DeepMind/Meta, by similar logic it’s irresponsible for them to quit.
Overall, I think people are overrating the importance of a few major AI labs due to their visibility. There are lots of researchers at NeurIPS, mostly not from the big AI labs in the OP. Feels like people are over-focused on OpenAI/DeepMind/Anthropic due to their visibility and social adjacency.
Seems like there are 2 possibilities here:
The majority of the leadership, engineers, etc. at OpenAI/DeepMind/Anthropic don’t agree that we’d be collectively better off if they all shut down.
The majority do agree, they just aren’t solving the collective action problem.
If (2) is the case, has anyone thought about using a dominant assurance contract?
https://www.cato-unbound.org/2017/06/07/alex-tabarrok/making-markets-work-better-dominant-assurance-contracts-some-other-helpful/
Maybe this would look something like: We offer a contract to engineers at specific major AI labs. If at least 90% of the engineers at each of those specific labs sign the contract by end of 2024, they agree to all mass quit their jobs. If not, everyone who signed the contract gets $500 at the end of 2024.
I’m guessing that coordination among the leadership has already been tried and failed. But if not, another idea is to structure the dominance assurance contract as an investment round, so it ends up being a financial boost for safety-conscious organizations that are willing to sign the contract, if not enough organizations sign.
One story for why coordination does not materialize:
Meta engineers self-select for being unconcerned with safety. They aren’t going to quit any time soon. If offered a dominance assurance contract, they won’t sign either early or late.
DeepMind engineers feel that DeepMind is more responsible than Meta. They think a DeepMind AGI is more likely to be aligned than a Meta AGI, and they feel it would be irresponsible to quit and let Meta build AGI.
OpenAI engineers feel that OpenAI is more responsible than Meta or DeepMind, by similar logic it’s irresponsible for them to quit.
Anthropic engineers feel that Anthropic is more responsible than OpenAI/DeepMind/Meta, by similar logic it’s irresponsible for them to quit.
Overall, I think people are overrating the importance of a few major AI labs due to their visibility. There are lots of researchers at NeurIPS, mostly not from the big AI labs in the OP. Feels like people are over-focused on OpenAI/DeepMind/Anthropic due to their visibility and social adjacency.