Do we know if @paulfchristiano or other ex-lab people working on AI policy have non-disparagement agreements with OpenAI or other AI companies? I know Cullen doesn’t, but I don’t know about anybody else.
I know NIST isn’t a regulatory body, but it still seems like standards-setting should be done by people who have no unusual legal obligations. And of course, some other people are or will be working at regulatory bodies, which may have more teeth in the future.
To be clear, I want to differentiate between Non-Disclosure Agreements, which are perfectly sane and reasonable in at least a limited form as a way to prevent leaking trade secrets, and non-disparagement agreements, which prevents you from saying bad things about past employers. The latter seems clearly bad to have for anybody in a position to affect policy. Doubly so if the existence of the non-disparagement agreement itself is secretive.
Sam Altman appears to have been using non-disparagements at least as far back as 2017-2018, even for things that really don’t seem to have needed such things at all, like a research nonprofit arm of YC.* It’s unclear if that example is also a lifetime non-disparagement (I’ve asked), but nevertheless, given that track record, you should assume the OA research non-profit also tried to impose it, followed by the OA LLC (obviously), and so Paul F. Christiano (and all of the Anthropic founders) would presumably be bound.
This would explain why Anthropic executives never say anything bad about OA, and refuse to explain exactly what broken promises by Altman triggered their failed attempt to remove Altman and subsequent exodus.
(I have also asked Sam Altman on Twitter, since he follows me, apropos of his vested equity statement, how far back these NDAs go, if the Anthropic founders are still bound, and if they are, whether they will be unbound.)
* Note that Elon Musk’s SpaceX does the same thing and is even worse because they will cancel your shares after you leave for half a year, and if they get mad at you after that expires, they may simply lock you out of tender offers indefinitely—which is not much different inasmuch as SpaceX may never IPO. Given the close connections there, Musk may be where Altman picked this bag of tricks up.
It seems really quite bad for Paul to work in the U.S. government on AI legislation without having disclosed that he is under a non-disparagement clause for the biggest entity in the space the regulator he is working at is regulating. And if he signed an NDA that prevents him from disclosing it, then it was IMO his job to not accept a position in which such a disclosure would obviously be required.
I am currently like 50% Paul has indeed signed such a lifetime non-disparagement agreement, so I do think I don’t buy that a “presumably” is appropriate here (though I am not that far away from it).
It would be bad, I agree. (An NDA about what he worked on at OA, sure, but then being required to never say anything bad about OA forever, as a regulator who will be running evaluations etc...?) Fortunately, this is one of those rare situations where it is probably enough for Paul to simply say his OA NDA does not cover that—then either it doesn’t and can’t be a problem, or he has violated the NDA’s gag order by talking about it and when OA then fails to sue him to enforce it, the NDA becomes moot.
At the very least I hope he disclosed it to the gov’t (and then it was voided at least in internal government communications, I don’t know how the law works here), though I’d personally want it to be voided completely or at least widely communicated to the public as well.
lol Paul is a very non-disparaging person. He always makes his criticism constructive, i don’t know if there’s any public evidence of him disparaging anyone regardless of NDAs
+1. Also curious about Jade Leung (formerly Open AI)– she’s currently the CTO for the UK AI Safety Institute. Also Geoffrey Irving (formerly DeepMind), who is a research director at the UKAISI.
Geoffrey Irving was one of the first people to publicly say some very aggressive + hard-to-verify things about Sam Altman during the November board fiasco, so hopefully this means he’s not bound (or doesn’t feel bound) by a very restrictive non-disparagement agreement.
Great point! (Also oops– I forgot that Irving was formerly OpenAI as well. He worked for DeepMind in recent years, but before that he worked at OpenAI and Google Brain.)
Do we have any evidence that DeepMind or Anthropic definitely do not do non-disparagement agreements? (If so then we can just focus on former OpenAI employees.)
Do we know if @paulfchristiano or other ex-lab people working on AI policy have non-disparagement agreements with OpenAI or other AI companies? I know Cullen doesn’t, but I don’t know about anybody else.
I know NIST isn’t a regulatory body, but it still seems like standards-setting should be done by people who have no unusual legal obligations. And of course, some other people are or will be working at regulatory bodies, which may have more teeth in the future.
To be clear, I want to differentiate between Non-Disclosure Agreements, which are perfectly sane and reasonable in at least a limited form as a way to prevent leaking trade secrets, and non-disparagement agreements, which prevents you from saying bad things about past employers. The latter seems clearly bad to have for anybody in a position to affect policy. Doubly so if the existence of the non-disparagement agreement itself is secretive.
Sam Altman appears to have been using non-disparagements at least as far back as 2017-2018, even for things that really don’t seem to have needed such things at all, like a research nonprofit arm of YC.* It’s unclear if that example is also a lifetime non-disparagement (I’ve asked), but nevertheless, given that track record, you should assume the OA research non-profit also tried to impose it, followed by the OA LLC (obviously), and so Paul F. Christiano (and all of the Anthropic founders) would presumably be bound.
This would explain why Anthropic executives never say anything bad about OA, and refuse to explain exactly what broken promises by Altman triggered their failed attempt to remove Altman and subsequent exodus.
(I have also asked Sam Altman on Twitter, since he follows me, apropos of his vested equity statement, how far back these NDAs go, if the Anthropic founders are still bound, and if they are, whether they will be unbound.)
* Note that Elon Musk’s SpaceX does the same thing and is even worse because they will cancel your shares after you leave for half a year, and if they get mad at you after that expires, they may simply lock you out of tender offers indefinitely—which is not much different inasmuch as SpaceX may never IPO. Given the close connections there, Musk may be where Altman picked this bag of tricks up.
It seems really quite bad for Paul to work in the U.S. government on AI legislation without having disclosed that he is under a non-disparagement clause for the biggest entity in the space the regulator he is working at is regulating. And if he signed an NDA that prevents him from disclosing it, then it was IMO his job to not accept a position in which such a disclosure would obviously be required.
I am currently like 50% Paul has indeed signed such a lifetime non-disparagement agreement, so I do think I don’t buy that a “presumably” is appropriate here (though I am not that far away from it).
It would be bad, I agree. (An NDA about what he worked on at OA, sure, but then being required to never say anything bad about OA forever, as a regulator who will be running evaluations etc...?) Fortunately, this is one of those rare situations where it is probably enough for Paul to simply say his OA NDA does not cover that—then either it doesn’t and can’t be a problem, or he has violated the NDA’s gag order by talking about it and when OA then fails to sue him to enforce it, the NDA becomes moot.
At the very least I hope he disclosed it to the gov’t (and then it was voided at least in internal government communications, I don’t know how the law works here), though I’d personally want it to be voided completely or at least widely communicated to the public as well.
lol Paul is a very non-disparaging person. He always makes his criticism constructive, i don’t know if there’s any public evidence of him disparaging anyone regardless of NDAs
Examples of maybe disparagement:
Arguing with Soares about whether AIs will kill everyone
Saying OpenAI is probably wrong and has motivated cognition in this comment
Wow, good point. I’ve never considered that aspect.
+1. Also curious about Jade Leung (formerly Open AI)– she’s currently the CTO for the UK AI Safety Institute. Also Geoffrey Irving (formerly DeepMind), who is a research director at the UKAISI.
Geoffrey Irving was one of the first people to publicly say some very aggressive + hard-to-verify things about Sam Altman during the November board fiasco, so hopefully this means he’s not bound (or doesn’t feel bound) by a very restrictive non-disparagement agreement.
Great point! (Also oops– I forgot that Irving was formerly OpenAI as well. He worked for DeepMind in recent years, but before that he worked at OpenAI and Google Brain.)
Do we have any evidence that DeepMind or Anthropic definitely do not do non-disparagement agreements? (If so then we can just focus on former OpenAI employees.)
Relevant market