Or does he merely mean he thinks DeepMind could pause AI development at DeepMind and maybe should once we enter the “gray zone”/”middle zone” (the period before AGI when he says things will start “feeling interesting and strange”)?
Reading his section, I’m concerned that when he talks about hitting pause, he’s secretly thinking that there will be a clear fire alarm for pushing the big red button and that he would just count on the IP-controlling safety committee of DM to stop everything.
Unfortunately, all of the relevant reporting on DM gives a strong impression that the committee may be a rubberstamp, having never actually exerted its power, and that Hassabis has been failing to stop DM from being absorbed into the Borg.
So, if we hit even a Christiano-style slow takeoff of 30% GDP growth a year etc and some real money started to be at stake rather than fun little projects like AlphaGo or AlphaFold, Google would simply ignore the committee and the provisions would be irrelevant. Page & Brin might be transhumanists who take AI risk seriously, but Pichai & the Knife, much less the suits down the line, don’t seem to be. At a certain level, a contract is nothing but a piece of paper stained with ink, lacking any inherent power of its own. (You may recall that WhatsApp had Mark swear to sacred legally-binding contracts with Facebook as part of its acquisition that it would never have advertising as its incredible journey continued, and the founders had hundreds of millions to billions of dollars in stock options vesting while they worked there to help enforce such deeply-important-to-them provisions; you may further recall that WhatsApp has now had advertising for a long time, and the founders are not there.)
Well, I’m being polite—I think they probably were not taking AI risk seriously, because journalists & Elon Musk have attributed quotes to them which are the classic Boomer ‘the AIs will replace us and akshully that is a good thing’ take, but few people are as overt as Hanson or Schmidhuber about that these days. But I don’t want to claim they’re like that without at least digging up & double-checking the various quotes, which would take a while. My point is that even if they are taking it seriously, they don’t matter because they’re long since checked out, and the people actually in charge day-to-day, Pichai & Porat, definitely are not. (Risk/safety is as much about day-to-day implementation as it is about any high-level statements.)
Anyway, relevant update here: post-AI-arms-race, DeepMind/Google Brain have been unceremoniously liquidated by Pichai and merged into ‘Google DeepMind’ (discussion), and Hassabis’s statements about ‘Gemini’ have taken on a capabilities tone. Reading the tea leaves of the new positions and speculating wildly, it looks like GB has been blamed for ‘Xeroxizing’ Google and DM nominally the victor, but at the cost of being pressured into turning into a more product-focused division. No sign of the IP/safety-committee. One thing to keep an eye on here will be the DeepMind Companies House filings (mirrors) - is this a fundamental legal change liquidating the original DeepMind corporation, or a rename+funding+responsibilities?
Reading his section, I’m concerned that when he talks about hitting pause, he’s secretly thinking that there will be a clear fire alarm for pushing the big red button and that he would just count on the IP-controlling safety committee of DM to stop everything.
Unfortunately, all of the relevant reporting on DM gives a strong impression that the committee may be a rubberstamp, having never actually exerted its power, and that Hassabis has been failing to stop DM from being absorbed into the Borg.
So, if we hit even a Christiano-style slow takeoff of 30% GDP growth a year etc and some real money started to be at stake rather than fun little projects like AlphaGo or AlphaFold, Google would simply ignore the committee and the provisions would be irrelevant. Page & Brin might be transhumanists who take AI risk seriously, but Pichai & the Knife, much less the suits down the line, don’t seem to be. At a certain level, a contract is nothing but a piece of paper stained with ink, lacking any inherent power of its own. (You may recall that WhatsApp had Mark swear to sacred legally-binding contracts with Facebook as part of its acquisition that it would never have advertising as its incredible journey continued, and the founders had hundreds of millions to billions of dollars in stock options vesting while they worked there to help enforce such deeply-important-to-them provisions; you may further recall that WhatsApp has now had advertising for a long time, and the founders are not there.)
I wonder how much power Hassabis actually has...
Why do you think this?
Well, I’m being polite—I think they probably were not taking AI risk seriously, because journalists & Elon Musk have attributed quotes to them which are the classic Boomer ‘the AIs will replace us and akshully that is a good thing’ take, but few people are as overt as Hanson or Schmidhuber about that these days. But I don’t want to claim they’re like that without at least digging up & double-checking the various quotes, which would take a while. My point is that even if they are taking it seriously, they don’t matter because they’re long since checked out, and the people actually in charge day-to-day, Pichai & Porat, definitely are not. (Risk/safety is as much about day-to-day implementation as it is about any high-level statements.)
Anyway, relevant update here: post-AI-arms-race, DeepMind/Google Brain have been unceremoniously liquidated by Pichai and merged into ‘Google DeepMind’ (discussion), and Hassabis’s statements about ‘Gemini’ have taken on a capabilities tone. Reading the tea leaves of the new positions and speculating wildly, it looks like GB has been blamed for ‘Xeroxizing’ Google and DM nominally the victor, but at the cost of being pressured into turning into a more product-focused division. No sign of the IP/safety-committee. One thing to keep an eye on here will be the DeepMind Companies House filings (mirrors) - is this a fundamental legal change liquidating the original DeepMind corporation, or a rename+funding+responsibilities?