I hope for us to enter the field as a neutral group, looking to collaborate widely and shift the dialog towards being about humanity winning rather than any particular group or company.
Greg and Ilya (to Elon):
The goal of OpenAI is to make the future good and to avoid an AGI dictatorship. You are concerned that Demis could create an AGI dictatorship. So do we. So it is a bad idea to create a structure where you could become a dictator if you chose to, especially given that we can create some other structure that avoids this possibility.
Greg and Ilya (to Altman):
But we haven’t been able to fully trust your judgements throughout this process, because we don’t understand your cost function.
We don’t understand why the CEO title is so important to you. Your stated reasons have changed, and it’s hard to really understand what’s driving it.
Is AGI truly your primary motivation? How does it connect to your political goals? How has your thought process changed over time?
From Altman: [...] Admitted that he lost a lot of trust with Greg and Ilya through this process. Felt their messaging was inconsistent and felt childish at times. [...] Sam was bothered by how much Greg and Ilya keep the whole team in the loop with happenings as the process unfolded. Felt like it distracted the team.
Apparently airing such concerns is “childish” and should only be done behind closed doors, otherwise it “distracts the team”, hm.
I thought the part you quoted was quite concerning, also in the context of what comes afterwards:
Hiatus: Sam told Greg and Ilya he needs to step away for 10 days to think. Needs to figure out how much he can trust them and how much he wants to work with them. Said he will come back after that and figure out how much time he wants to spend.
Sure, the email by Sutskever and Brockman gave some nonviolent communication vibes and maybe it isn’t “the professional thing” to air one’s feelings and perceived mistakes like that, but they seemed genuine in what they wrote and they raised incredibly important concerns that are difficult in nature to bring up. Also, with hindsight especially, it seems like they had valid reasons to be concerned about Altman’s power-seeking tendencies!
When someone expresses legitimate-given-the-situation concerns about your alignment and your reaction is to basically gaslight them into thinking they did something wrong for finding it hard to trust you, and then you make it seem like you are the poor victim who needs 10 days off of work to figure out whether you can still trust them, that feels messed up! (It’s also a bit hypocritical because the whole “I need 10 days to figure out if I can still trust you for thinking I like being CEO a bit too much,” seems childish too.)
(Of course, these emails are just snapshots and we might be missing things that happened in between via other channels of communication, including in-person talks.)
Also, I find it interesting that they (Sutskever and Brockman) criticized Musk just as much as Altman (if I understood their email correctly), so this should make it easier for Altman to react with grace. I guess given Musk’s own annoyed reaction, maybe Altman was calling the others’ email childish to side with Musks’s dismissive reaction to that same email.
Lastly, this email thread made me wonder what happened between Brockman and Sutskever in the meantime, since it now seems like Brockman no longer holds the same concerns about Altman even though recent events seem to have given a lot of new fire to them.
I get their concerns about Google, but I don’t get why they emphasize Demis. Makes it seem like there’s more to it than “he happens to be DeepMind’s CEO atm”
The fact that Demis is a champion Diplomacy player suggests that there is more to him than meets the eye. Diplomacy is a game won by pretending to be allies with as many people as possible for as long as possible before betraying them at the most optimal time. Infamous for harming friendships when played with friends.
Not that I think this suggests Demis is a bad person, just that there is reason to be extra unsure about his internal stance from examining his public statements.
Edit:
@lc gave a ‘skeptical’ react to this comment. I’m not sure which bit is causing the skepticism. Maybe lc is skeptical that being a champion level player in games of strategic misdirection is reason to believe someone is skilled at strategic misdirection? Or maybe the skepticism is about this being relevant to the case at hand? Perhaps the people discussing Demis and ambitions of Singleton-creation and world domination aren’t particularly concerned about specifically Demis, but rather generally about an individual competent and ambitious enough to pull off such a feat?
I dunno. I feel more inclined to put my life in Demis’ hands than Sam’s or Elon’s if forced to make a choice, but I would prefer not to have to. I also would take any of the above having Singleton-based Decisive Strategic Advantage over a nuclear-and-bioweapon-fought-WWIII.
So hard to forsee consequences, and we have only limited power as bystanders. Not no power though.
Hassabis is a five-times winner of the all-round world board games championship (the Pentamind), and an expert player of many games including:[34]
Chess: achieved Master standard at age 13 with ELO rating of 2300 (at the time the second-highest in the world for his age after Judit Polgár)[144]
Diplomacy: World Team Champion in 2004, 4th in 2006 World Championship[145]
Poker: cashed at the World Series of Poker six times including in the Main Event[146]
Multi-games events at the London Mind Sports Olympiad: World Pentamind Champion (five times: 1998, 1999, 2000, 2001, 2003)[147] and World Decamentathlon Champion (twice: 2003, 2004)
A few quotes that stood out to me:
Greg:
Greg and Ilya (to Elon):
Greg and Ilya (to Altman):
Also this:
Apparently airing such concerns is “childish” and should only be done behind closed doors, otherwise it “distracts the team”, hm.
I thought the part you quoted was quite concerning, also in the context of what comes afterwards:
Sure, the email by Sutskever and Brockman gave some nonviolent communication vibes and maybe it isn’t “the professional thing” to air one’s feelings and perceived mistakes like that, but they seemed genuine in what they wrote and they raised incredibly important concerns that are difficult in nature to bring up. Also, with hindsight especially, it seems like they had valid reasons to be concerned about Altman’s power-seeking tendencies!
When someone expresses legitimate-given-the-situation concerns about your alignment and your reaction is to basically gaslight them into thinking they did something wrong for finding it hard to trust you, and then you make it seem like you are the poor victim who needs 10 days off of work to figure out whether you can still trust them, that feels messed up! (It’s also a bit hypocritical because the whole “I need 10 days to figure out if I can still trust you for thinking I like being CEO a bit too much,” seems childish too.)
(Of course, these emails are just snapshots and we might be missing things that happened in between via other channels of communication, including in-person talks.)
Also, I find it interesting that they (Sutskever and Brockman) criticized Musk just as much as Altman (if I understood their email correctly), so this should make it easier for Altman to react with grace. I guess given Musk’s own annoyed reaction, maybe Altman was calling the others’ email childish to side with Musks’s dismissive reaction to that same email.
Lastly, this email thread made me wonder what happened between Brockman and Sutskever in the meantime, since it now seems like Brockman no longer holds the same concerns about Altman even though recent events seem to have given a lot of new fire to them.
Do we know anything about why they were concerned about an AGI dictatorship created by Demis?
Presumably it was because Google had just bought DeepMind, back when it was the only game in town?
I get their concerns about Google, but I don’t get why they emphasize Demis. Makes it seem like there’s more to it than “he happens to be DeepMind’s CEO atm”
The fact that Demis is a champion Diplomacy player suggests that there is more to him than meets the eye. Diplomacy is a game won by pretending to be allies with as many people as possible for as long as possible before betraying them at the most optimal time. Infamous for harming friendships when played with friends.
Not that I think this suggests Demis is a bad person, just that there is reason to be extra unsure about his internal stance from examining his public statements.
Edit: @lc gave a ‘skeptical’ react to this comment. I’m not sure which bit is causing the skepticism. Maybe lc is skeptical that being a champion level player in games of strategic misdirection is reason to believe someone is skilled at strategic misdirection? Or maybe the skepticism is about this being relevant to the case at hand? Perhaps the people discussing Demis and ambitions of Singleton-creation and world domination aren’t particularly concerned about specifically Demis, but rather generally about an individual competent and ambitious enough to pull off such a feat?
I dunno. I feel more inclined to put my life in Demis’ hands than Sam’s or Elon’s if forced to make a choice, but I would prefer not to have to. I also would take any of the above having Singleton-based Decisive Strategic Advantage over a nuclear-and-bioweapon-fought-WWIII.
So hard to forsee consequences, and we have only limited power as bystanders. Not no power though.
From Wikipedia: https://en.m.wikipedia.org/wiki/Demis_Hassabis