Anthropic has an unconventional governance mechanism: an independent “Long-Term Benefit Trust” elects some of its board. Anthropic sometimes emphasizes that the Trust is an experiment, but mostly points to it to argue that Anthropic will be able to promote safety and benefit-sharing over profit.[1]
But the Trust’s details have not been published and some information Anthropic has shared is concerning. In particular, Anthropic’s stockholders can apparently overrule, modify, or abrogate the Trust, and the details are unclear.
Anthropic has not publicly demonstrated that the Trust would be able to actually do anything that stockholders don’t like.
The facts
There are three sources of public information on the Trust:
They say there’s a new class of stock, held by the Trust/Trustees. This stock allows the Trust to elect some board members and will allow them to elect a majority of the board by 2027.
But:
Morley et al.: “the Trust Agreement also authorizes the Trust to be enforced by the company and by groups of the company’s stockholders who have held a sufficient percentage of the company’s equity for a sufficient period of time,” rather than the Trustees.
I don’t know what this means.
Morley et al.: the Trust and its powers can be amended “by a supermajority of stockholders. . . . [This] operates as a kind of failsafe against the actions of the Voting Trustees and safeguards the interests of stockholders.” Anthropic: “the Trust and its powers [can be changed] without the consent of the Trustees if sufficiently large supermajorities of the stockholders agree.”
It’s impossible to assess this “failsafe” without knowing the thresholds for these “supermajorities.” Also, a small number of investors—currently, perhaps Amazon and Google—may control a large fraction of shares. It may be easy for profit-motivated investors to reach a supermajority.
Maybe there are other issues with the Trust Agreement — we can’t see it and so can’t know.
Vox: the Trust “will elect a fifth member of the board this fall,” viz. Fall 2023.
Anthropic has not said whether that happened nor who is on the board these days (nor who is on the Trust these days).
Conclusion
Public information is consistent with the Trust being quite subordinate to stockholders, likely to lose their powers if they do anything stockholders dislike. (Even if stockholders’ formal powers over the Trust are never used, that threat could prevent the Trust from acting contrary to the stockholders’ interests.)
Anthropic knows this and has decided not to share the information that the public needs to evaluate the Trust. This suggests that Anthropic benefits from ambiguity because the details would be seen as bad. I basically fail to imagine a scenario where publishing the Trust Agreement is very costly to Anthropic—especially just sharing certain details (like sharing percentages rather than saying “a supermajority”)—except that the details are weak and would make Anthropic look bad.[2]
Maybe it would suffice to let an auditor see the Trust Agreement and publish their impression of it. But I don’t see why Anthropic won’t publish it.
Maybe the Trust gives Anthropic strong independent accountability — or rather, maybe it will by default after (unspecified) time- and funding-based milestones. But only if Anthropic’s board and stockholders have substantially less power over it than they might—or if they will exercise great restraint in using their power—and the Trust knows this.
Unless I’m missing something, Anthropic should publish the Trust Agreement (and other documents if relevant) and say whether and when the Trust has elected board members. Especially vital is (1) publishing information about how the Trust or its powers can change, (2) committing to publicly announce changes, and (3) clarifying what’s going on with the Trust now.
Note: I don’t claim that maximizing the Trust’s power is correct. Maybe one or more other groups should have power over the Trust, whether to intervene if the Trust collapses or does something illegitimate or just to appease investors. I just object to the secrecy.
Thanks to Buck Shlegeris for suggestions. He doesn’t necessarily endorse this post.
Maybe Anthropic’s Long-Term Benefit Trust is powerless
Crossposted from AI Lab Watch. Subscribe on Substack.
Introduction
Anthropic has an unconventional governance mechanism: an independent “Long-Term Benefit Trust” elects some of its board. Anthropic sometimes emphasizes that the Trust is an experiment, but mostly points to it to argue that Anthropic will be able to promote safety and benefit-sharing over profit.[1]
But the Trust’s details have not been published and some information Anthropic has shared is concerning. In particular, Anthropic’s stockholders can apparently overrule, modify, or abrogate the Trust, and the details are unclear.
Anthropic has not publicly demonstrated that the Trust would be able to actually do anything that stockholders don’t like.
The facts
There are three sources of public information on the Trust:
The Long-Term Benefit Trust (Anthropic 2023)
Anthropic Long-Term Benefit Trust (Morley et al. 2023)
The $1 billion gamble to ensure AI doesn’t destroy humanity (Vox: Matthews 2023)
They say there’s a new class of stock, held by the Trust/Trustees. This stock allows the Trust to elect some board members and will allow them to elect a majority of the board by 2027.
But:
Morley et al.: “the Trust Agreement also authorizes the Trust to be enforced by the company and by groups of the company’s stockholders who have held a sufficient percentage of the company’s equity for a sufficient period of time,” rather than the Trustees.
I don’t know what this means.
Morley et al.: the Trust and its powers can be amended “by a supermajority of stockholders. . . . [This] operates as a kind of failsafe against the actions of the Voting Trustees and safeguards the interests of stockholders.” Anthropic: “the Trust and its powers [can be changed] without the consent of the Trustees if sufficiently large supermajorities of the stockholders agree.”
It’s impossible to assess this “failsafe” without knowing the thresholds for these “supermajorities.” Also, a small number of investors—currently, perhaps Amazon and Google—may control a large fraction of shares. It may be easy for profit-motivated investors to reach a supermajority.
Maybe there are other issues with the Trust Agreement — we can’t see it and so can’t know.
Vox: the Trust “will elect a fifth member of the board this fall,” viz. Fall 2023.
Anthropic has not said whether that happened nor who is on the board these days (nor who is on the Trust these days).
Conclusion
Public information is consistent with the Trust being quite subordinate to stockholders, likely to lose their powers if they do anything stockholders dislike. (Even if stockholders’ formal powers over the Trust are never used, that threat could prevent the Trust from acting contrary to the stockholders’ interests.)
Anthropic knows this and has decided not to share the information that the public needs to evaluate the Trust. This suggests that Anthropic benefits from ambiguity because the details would be seen as bad. I basically fail to imagine a scenario where publishing the Trust Agreement is very costly to Anthropic—especially just sharing certain details (like sharing percentages rather than saying “a supermajority”)—except that the details are weak and would make Anthropic look bad.[2]
Maybe it would suffice to let an auditor see the Trust Agreement and publish their impression of it. But I don’t see why Anthropic won’t publish it.
Maybe the Trust gives Anthropic strong independent accountability — or rather, maybe it will by default after (unspecified) time- and funding-based milestones. But only if Anthropic’s board and stockholders have substantially less power over it than they might—or if they will exercise great restraint in using their power—and the Trust knows this.
Unless I’m missing something, Anthropic should publish the Trust Agreement (and other documents if relevant) and say whether and when the Trust has elected board members. Especially vital is (1) publishing information about how the Trust or its powers can change, (2) committing to publicly announce changes, and (3) clarifying what’s going on with the Trust now.
Note: I don’t claim that maximizing the Trust’s power is correct. Maybe one or more other groups should have power over the Trust, whether to intervene if the Trust collapses or does something illegitimate or just to appease investors. I just object to the secrecy.
Thanks to Buck Shlegeris for suggestions. He doesn’t necessarily endorse this post.
E.g. 1, 2, and 3, and almost every time Anthropic people talk about Anthropic’s governance.
Unlike with some other policies, the text of the Trust Agreement is crucial; it is a legal document that dictates actors’ powers over each other.