Another class of routes is for the AI to obtain the resources entirely legitimately, through e.g. running a very successful business where extra intelligence adds significant value. For instance, it’s fun to imagine that Larry Page and Sergey Brin’s first success was not a better search algorithm, but building and/or stumbling on an AI that invented it (and a successful business model) for them; Google now controls a very large proportion of the world’s computing resources. Similarly, if a bit more prosaically, Walmart in the US and Tesco in the UK have grown extremely large, successful businesses based on the smart use of computing resources. For a more directly terrifying scenario, imagine it happening at, say, Lockheed Martin, BAE Systems or Raytheon.
These are not quick, instant takeovers, but I think it is a mistake to imagine that it must happen instantly. An AI that thinks it will be destroyed (or permanently thwarted) if it is discovered would take care to avoid discovery. Scenarios where it can be careful to minimise the risk of discovery until its position is unassailable will look much more appealing than high-risk short-term scenarios with high variance in outcomes. Indeed, it might sensibly seek to build its position in the minds of people-in-general as an invaluable resource for humanity well before its full nature is revealed.
Do you think that human beings will allow a single corporation to control a significant fraction of the world’s resources? How will the company avoid anti-monopoly laws? Does a an AI CEO actually have control over a corporation, or does it only have the freedom to act within the defined social roles of what a “CEO” is allowed to do? I.e. it can negotiate a merger but can’t hire a bunch of scientists and tell them to start mass producing nerve gas.
The U.S. government spends more money in a single year than the combined market capitalization of the 10 largest companies in the world.
In what sense does google “control a very large proportion of the world’s computing resources”? Google maybe has the compute power equivalent to a handful of supercomputers, but even that isn’t organized in a particularly useful way for an AI looking to do something dramatically different from performing millions of internet searches. For the vast majority of problems, I’d rather use ORNL’s Titan than literally every computer google owns.
An AI controlling a company like Google would be able to, say, buy up many of the world’s battle robot manufacturers, or invest a lot of money into human-focused bioengineering), despite those activities being almost entirely unrelated to their core business, and without giving any specific idea of why.
Indeed, on the evidence of the press coverage of Google’s investments, it seems likely that many people would spend a lot of effort inventing plausible cover stories for the AI.
The article you linked to assumes that Google only uses CPUs (and 5-year-old ones at that). It uses this assumption to arrive at a performance estimate which it compares it to GPU-based supercomputers, in a GPU-oriented benchmark.
We can generate similar conclusions in lots of other ways. Intel’s annual revenue is larger than Google’s. The semiconductor industry is an order of magnitude larger than Google. If Google spent literally every dollar they owned on chips, never mind powering them, writing code for them, or even putting them in data centers, then Google might be able to buy 15% of the world’s computer chips. That still wouldn’t be equivalent to “Google now controls a very large proportion of the world’s computing resources.”
And compute power isn’t fungible. GPUs are worthless for a lot of applications. You can’t run a calculation on a million servers spread out across the globe unless you’re doing something very easy like SETI@home. Most algorithms aren’t trivial to split into a million pieces that don’t need to talk to each other.
Okay. I amend my statement to “For every year except for 2013 where Google’s revenue was $55.5 billion while Intel’s was $52.7 billion Intel’s annual revenue has been larger than Google’s.”
At the same time, while Intel is a significant fraction of the semiconductor industry, it’s still almost an order of magnitude smaller than the industry as a whole.
According to your own link that $7B number appears to be Google’s total capital expenses, much of which seems to be devoted to buying land and building new buildings. While many of those buildings may be data centers, $7B in capital expenses is not equivalent to $7B spent on data centers. Google Fiber for instance would be included in “capital expenses” but is not an investment in a data center. Even neglecting non-data center spending, the chips that go inside of the computers in the data center are a small proportion of the total cost of the data center, so that’s not a particularly useful number to throw around without any context.
Do you actually believe the statement “Google now controls a very large proportion of the world’s computing resources”?
We haven’t really nailed down what “a very large proportion” would be; I’m just trying to estimate what the actual fraction is.
Looking at the semiconductor industry market share data that you linked, I notice that numbers 2-11 represent SoCs, DRAM, flash, communication ICs, power ICs, microcontrollers, basically everything except for server CPUs and GPUs. If we look at just the parts that are potentially relevant to AI, the non-mobile CPU market seems to firmly belong to Intel, while the non-mobile GPU market belongs to AMD and nVidia ($5 and $3.6B revenues, respectively).
It’s still not very clear to me how much Google spends directly on computation; the author of the linked article seemed to think the $7B was mostly on datacenters. Even if it’s only a fraction of that, it’s a lot. Compare to the largest supercomputres: Tianhe-2 cost $390M and Titan cost $97M, according to their respective Wikipedia pages.)
While those conclusions appear to be drawn from data that is several years old, it seems reasonable to assume that Google hasn’t grown at a rate substantially different from the industry as a whole.
Maybe they own 4% of the world’s servers, but very probably not 10% and certainly not 40%.
Another class of routes is for the AI to obtain the resources entirely legitimately, through e.g. running a very successful business where extra intelligence adds significant value. For instance, it’s fun to imagine that Larry Page and Sergey Brin’s first success was not a better search algorithm, but building and/or stumbling on an AI that invented it (and a successful business model) for them; Google now controls a very large proportion of the world’s computing resources. Similarly, if a bit more prosaically, Walmart in the US and Tesco in the UK have grown extremely large, successful businesses based on the smart use of computing resources. For a more directly terrifying scenario, imagine it happening at, say, Lockheed Martin, BAE Systems or Raytheon.
These are not quick, instant takeovers, but I think it is a mistake to imagine that it must happen instantly. An AI that thinks it will be destroyed (or permanently thwarted) if it is discovered would take care to avoid discovery. Scenarios where it can be careful to minimise the risk of discovery until its position is unassailable will look much more appealing than high-risk short-term scenarios with high variance in outcomes. Indeed, it might sensibly seek to build its position in the minds of people-in-general as an invaluable resource for humanity well before its full nature is revealed.
Do you think that human beings will allow a single corporation to control a significant fraction of the world’s resources? How will the company avoid anti-monopoly laws? Does a an AI CEO actually have control over a corporation, or does it only have the freedom to act within the defined social roles of what a “CEO” is allowed to do? I.e. it can negotiate a merger but can’t hire a bunch of scientists and tell them to start mass producing nerve gas.
The U.S. government spends more money in a single year than the combined market capitalization of the 10 largest companies in the world.
In what sense does google “control a very large proportion of the world’s computing resources”? Google maybe has the compute power equivalent to a handful of supercomputers, but even that isn’t organized in a particularly useful way for an AI looking to do something dramatically different from performing millions of internet searches. For the vast majority of problems, I’d rather use ORNL’s Titan than literally every computer google owns.
An AI controlling a company like Google would be able to, say, buy up many of the world’s battle robot manufacturers, or invest a lot of money into human-focused bioengineering), despite those activities being almost entirely unrelated to their core business, and without giving any specific idea of why.
Indeed, on the evidence of the press coverage of Google’s investments, it seems likely that many people would spend a lot of effort inventing plausible cover stories for the AI.
This raises interesting questions about who (or what) is really running Google.
I’ll grant that “a very large proportion of the world’s computing resources” was under-specified and over-stated. Sorry.
The article you linked to assumes that Google only uses CPUs (and 5-year-old ones at that). It uses this assumption to arrive at a performance estimate which it compares it to GPU-based supercomputers, in a GPU-oriented benchmark.
We can generate similar conclusions in lots of other ways. Intel’s annual revenue is larger than Google’s. The semiconductor industry is an order of magnitude larger than Google. If Google spent literally every dollar they owned on chips, never mind powering them, writing code for them, or even putting them in data centers, then Google might be able to buy 15% of the world’s computer chips. That still wouldn’t be equivalent to “Google now controls a very large proportion of the world’s computing resources.”
And compute power isn’t fungible. GPUs are worthless for a lot of applications. You can’t run a calculation on a million servers spread out across the globe unless you’re doing something very easy like SETI@home. Most algorithms aren’t trivial to split into a million pieces that don’t need to talk to each other.
From Wikipedia, 2013 revenues:
Google: $60B
Intel: $53B
Qualcomm: $25B
TSMC: $14B
Google spending on datacenters: $7B
GlobalFoundries: $5B
AMD: $5B
ARM Holdings: $0.7B
Okay. I amend my statement to “For every year except for 2013 where Google’s revenue was $55.5 billion while Intel’s was $52.7 billion Intel’s annual revenue has been larger than Google’s.” At the same time, while Intel is a significant fraction of the semiconductor industry, it’s still almost an order of magnitude smaller than the industry as a whole.
According to your own link that $7B number appears to be Google’s total capital expenses, much of which seems to be devoted to buying land and building new buildings. While many of those buildings may be data centers, $7B in capital expenses is not equivalent to $7B spent on data centers. Google Fiber for instance would be included in “capital expenses” but is not an investment in a data center. Even neglecting non-data center spending, the chips that go inside of the computers in the data center are a small proportion of the total cost of the data center, so that’s not a particularly useful number to throw around without any context.
Do you actually believe the statement “Google now controls a very large proportion of the world’s computing resources”?
We haven’t really nailed down what “a very large proportion” would be; I’m just trying to estimate what the actual fraction is.
Looking at the semiconductor industry market share data that you linked, I notice that numbers 2-11 represent SoCs, DRAM, flash, communication ICs, power ICs, microcontrollers, basically everything except for server CPUs and GPUs. If we look at just the parts that are potentially relevant to AI, the non-mobile CPU market seems to firmly belong to Intel, while the non-mobile GPU market belongs to AMD and nVidia ($5 and $3.6B revenues, respectively).
It’s still not very clear to me how much Google spends directly on computation; the author of the linked article seemed to think the $7B was mostly on datacenters. Even if it’s only a fraction of that, it’s a lot. Compare to the largest supercomputres: Tianhe-2 cost $390M and Titan cost $97M, according to their respective Wikipedia pages.)
From what I’ve seen Google probably owns around 2-3% of the world’s servers, which is probably on the order of 2 million machines.
Google claims that their datacenters use 1% of the estimated electricity consumption for data centers world wide and that their data centers are about twice as efficient as the average data center.
While those conclusions appear to be drawn from data that is several years old, it seems reasonable to assume that Google hasn’t grown at a rate substantially different from the industry as a whole.
Maybe they own 4% of the world’s servers, but very probably not 10% and certainly not 40%.