I’m coming to this late, but this seems weird. Do I understand correctly that many people were saying that Anthropic, the AI research company, had committed never to advance the state of the art of AI research, and they believed Anthropic would follow this commitment? That is just… really implausible.
This is the sort of commitment which very few individuals are psychologically capable of keeping, and which ~zero commercial organizations of more than three or four people are institutionally capable of keeping, assuming they actually do have the ability to advance the state of the art. I don’t know whether Anthropic leadership ever said they would do this, and if they said it then I don’t know whether they meant it earnestly. But even imagining they said it and meant it earnestly there is just no plausible world in which a company with hundreds of staff and billions of dollars of commercial investment would keep this commitment for very long. That is not the sort of thing you see from commercial research companies in hot fields.
If anyone here did believe that Anthropic would voluntarily refrain from advancing the state of the art in all cases, you might want to check if there are other things that people have told you about themselves, which you would really like to be true of them, but you have no evidence for other than their assertions, and would be very unusual if they were true.
I also strongly expected them to violate this commitment, though my understanding is that various investors and early collaborators did believe they would keep this commitment.
I think it’s important to understand that Anthropic was founded before the recent post-Chat-GPT hype/AI-interest-explosion. Similarly to how OpenAIs charter seemed plausible as something that OpenAI could adhere to for people early on, so did it seem possible that commercial pressures would not cause a fully-throated arms-race between all the top companies, with billions to trillions of dollars for the taking for whoever got to AGI first, which I do agree made violating this commitment a relatively likely conclusion.
To be blunt, this is why I believe there was a game of telephone that happened, because I agree that this commitment was unlikely to be held, so I don’t think they promised this would happen (though the comms from Anthropic are surprisingly unclear on such an important point.)
This comment is inspired by Raemon’s comment below this paragraph, and I’ll elaborate on a problematic/toxic dynamic in criticizing orgs that might be right but also have a reasonable probability of being shady is that the people who do try to criticize such orgs are often selected for both more conflict than is optimal, and more paranoia than a correctly calibrated person, which means it’s too easy for even shady organizations to validly argue away any criticism, no matter how serious, and the honest organizations will almost certainly respond the same way, which means you get much less evidence and data for your calibration, and this can easily spiral into being far too paranoid/insane about an organization to the point of elaborating false conspiracy theories about it:
My own take on the entire affair is that Anthropic comms definitely needs to be more consistent and clear, but also we should try to be much more careful around the qualifiers, and importantly to treat 2 similar sounding sentences as potentially extremely different, because every word does matter for these sorts of high-stakes situations.
More generally, it’s important to realize early when a telephone game is happening, so that you can stop the spread of misconceptions.
I’m coming to this late, but this seems weird. Do I understand correctly that many people were saying that Anthropic, the AI research company, had committed never to advance the state of the art of AI research, and they believed Anthropic would follow this commitment? That is just… really implausible.
This is the sort of commitment which very few individuals are psychologically capable of keeping, and which ~zero commercial organizations of more than three or four people are institutionally capable of keeping, assuming they actually do have the ability to advance the state of the art. I don’t know whether Anthropic leadership ever said they would do this, and if they said it then I don’t know whether they meant it earnestly. But even imagining they said it and meant it earnestly there is just no plausible world in which a company with hundreds of staff and billions of dollars of commercial investment would keep this commitment for very long. That is not the sort of thing you see from commercial research companies in hot fields.
If anyone here did believe that Anthropic would voluntarily refrain from advancing the state of the art in all cases, you might want to check if there are other things that people have told you about themselves, which you would really like to be true of them, but you have no evidence for other than their assertions, and would be very unusual if they were true.
I also strongly expected them to violate this commitment, though my understanding is that various investors and early collaborators did believe they would keep this commitment.
I think it’s important to understand that Anthropic was founded before the recent post-Chat-GPT hype/AI-interest-explosion. Similarly to how OpenAIs charter seemed plausible as something that OpenAI could adhere to for people early on, so did it seem possible that commercial pressures would not cause a fully-throated arms-race between all the top companies, with billions to trillions of dollars for the taking for whoever got to AGI first, which I do agree made violating this commitment a relatively likely conclusion.
To be blunt, this is why I believe there was a game of telephone that happened, because I agree that this commitment was unlikely to be held, so I don’t think they promised this would happen (though the comms from Anthropic are surprisingly unclear on such an important point.)
This comment is inspired by Raemon’s comment below this paragraph, and I’ll elaborate on a problematic/toxic dynamic in criticizing orgs that might be right but also have a reasonable probability of being shady is that the people who do try to criticize such orgs are often selected for both more conflict than is optimal, and more paranoia than a correctly calibrated person, which means it’s too easy for even shady organizations to validly argue away any criticism, no matter how serious, and the honest organizations will almost certainly respond the same way, which means you get much less evidence and data for your calibration, and this can easily spiral into being far too paranoid/insane about an organization to the point of elaborating false conspiracy theories about it:
https://www.lesswrong.com/posts/wn5jTrtKkhspshA4c/?commentId=9qfPihnpHoESSCAGP
My own take on the entire affair is that Anthropic comms definitely needs to be more consistent and clear, but also we should try to be much more careful around the qualifiers, and importantly to treat 2 similar sounding sentences as potentially extremely different, because every word does matter for these sorts of high-stakes situations.
More generally, it’s important to realize early when a telephone game is happening, so that you can stop the spread of misconceptions.