A month ago, Jeff Dean, lead of Google AI, announced at TED that Google is developing a “general purpose intelligence system”. To give a bit of context, last January, Google already published a paper on a 1.6-trillion parameter model based on the architecture of switch transformers, which improves by on order of magnitude upon GPT-3. Yet, I’ve heard that Google usually only publishes a weaker version of the algorithms they actually develop and deploy at scale.
From a technical viewpoint, and given my understanding of switch transformers and of the challenges of scaling neural networks, I’m guessing that Pathways’ model will now be decentralized on multiple machines (as it does not fit on a single machine), and its computation (both in forward pass and backprop) must optimize its “pathway” by only leveraging some of the machines Pathways is deployed on. Google seems to want to leverage this to build models with 100-trillion+ parameters (note that models have been roughly x10 per year since Bert).
As a comparison, the human brain is estimated to have hundreds of trillions of synapses connecting its 100 billion neurons. In terms of numbers, Google’s algorithms may soon match the human brain. However, as opposed to a human brain, these algorithms crunch billions of data points per second. Moreover, these data are extremely informative, as they track the daily habits, behaviors and beliefs of billions of humans on earth, through their activities on Google Search, YouTube, Google Mail, Google Drive, Google Maps or even Google Smart Keyboard. Perhaps more importantly still, Google’s algorithms leverage this “intelligence” to have a massive-scale impact, by choosing which information will be shown to which human on earth, especially through the YouTube recommendation algorithm and Google Adsense.
I’m curious to know how this news updates your beliefs on the following questions:
What is the probability that Google develops the "first AGI"?What is the probability that there is a human-level AI by 2025?Can a recommendation algorithm (like YouTube's) become AGI?
EDIT: I see that many of you doubt that a recommendation algorithm can be an AGI. Does this mean that you reject the orthogonality thesis?
If YouTube's recommendation algorithm becomes AGI, will it be an existential threat?
(if you believe that any unaligned AGI is an existential threat, then I guess that you should answer yes...)
Should "AGI Safety" focus on the "short term" (say, before 2025)?
Those who are interested in my views can check the Tournesol wiki (currently in staging unchangeable mode, but should come back to normal in a few weeks). Tournesol is a non-profit project colleagues and I have launched to robustly solve AI alignment with a “short-term” agenda. We are searching for funding opportunities.
Pathways: Google’s AGI
A month ago, Jeff Dean, lead of Google AI, announced at TED that Google is developing a “general purpose intelligence system”. To give a bit of context, last January, Google already published a paper on a 1.6-trillion parameter model based on the architecture of switch transformers, which improves by on order of magnitude upon GPT-3. Yet, I’ve heard that Google usually only publishes a weaker version of the algorithms they actually develop and deploy at scale.
From a technical viewpoint, and given my understanding of switch transformers and of the challenges of scaling neural networks, I’m guessing that Pathways’ model will now be decentralized on multiple machines (as it does not fit on a single machine), and its computation (both in forward pass and backprop) must optimize its “pathway” by only leveraging some of the machines Pathways is deployed on. Google seems to want to leverage this to build models with 100-trillion+ parameters (note that models have been roughly x10 per year since Bert).
As a comparison, the human brain is estimated to have hundreds of trillions of synapses connecting its 100 billion neurons. In terms of numbers, Google’s algorithms may soon match the human brain. However, as opposed to a human brain, these algorithms crunch billions of data points per second. Moreover, these data are extremely informative, as they track the daily habits, behaviors and beliefs of billions of humans on earth, through their activities on Google Search, YouTube, Google Mail, Google Drive, Google Maps or even Google Smart Keyboard. Perhaps more importantly still, Google’s algorithms leverage this “intelligence” to have a massive-scale impact, by choosing which information will be shown to which human on earth, especially through the YouTube recommendation algorithm and Google Adsense.
I’m curious to know how this news updates your beliefs on the following questions:
EDIT: I see that many of you doubt that a recommendation algorithm can be an AGI. Does this mean that you reject the orthogonality thesis?
(if you believe that any unaligned AGI is an existential threat, then I guess that you should answer yes...)
Those who are interested in my views can check the Tournesol wiki (currently in staging unchangeable mode, but should come back to normal in a few weeks). Tournesol is a non-profit project colleagues and I have launched to robustly solve AI alignment with a “short-term” agenda. We are searching for funding opportunities.