I wouldn’t want to give an “official organizational probability distribution”, but I think collectively we average out to something closer to “a uniform prior over possibilities” without that much evidence thus far updating us from there. Basically, there are plausible stories and intuitions pointing in lots of directions, and no real empirical evidence which bears on it thus far.
(Obviously, within the company, there’s a wide range of views. Some people are very pessimistic. Others are optimistic. We debate this quite a bit internally, and I think that’s really positive! But I think there’s a broad consensus to take the entire range seriously, including the very pessimistic ones.)
People who work in alignment, as opposed to capabilities, aren’t examples of people who are writing their own death warrants.
If someone switched from capability to alignment, that would be evidence that technical insight into capability leads to belief in AI Doom . But the people you have named have always been on the alignment side of the fence. That leads to the “theologians mostly believe in God” problem.
Please refer back to your original claim:
You haven’t named anyone who is.
evhub was at 80% about a year ago (currently at Anthropic, interned at OpenAI).
Daniel Kokotajlo was at 65% ~2 years ago; I think that number’s gone up since then.
Quite a few other people at Anthropic also have pessimistic views, according to Chris Olah:
The Deepmind alignment team probably has at least a couple people who think the odds are bad, (p(doom) > 50%) given the way Vika buckets the team, combined with the distribution of views reflected by DeepMind alignment team opinions on AGI ruin arguments.
People who work in alignment, as opposed to capabilities, aren’t examples of people who are writing their own death warrants.
If someone switched from capability to alignment, that would be evidence that technical insight into capability leads to belief in AI Doom . But the people you have named have always been on the alignment side of the fence. That leads to the “theologians mostly believe in God” problem.