Thanks for your comments. How do you think human intelligence works? Perhaps by doing a massive parallel search to approximate the best solution?
The misunderstanding rests on the implicit assumption that performance scales essentially linearly with hardware resources. Typically, it doesn’t.
I’m confused… if time required is a polynomial or exponential function of your problem size, then hardware that runs twice as fast will still solve your problem twice as fast, won’t it? (How could it not?) And if the algorithm you’re using to solve the problem is perfectly parallelizable (which I grant to AI foom proponents ’cause it seems plausible to me), then throwing twice the hardware at any given problem will solve it twice as fast. (Although yes, it will not solve problems that are twice as big.)
Thanks for your comments. How do you think human intelligence works? Perhaps by doing a massive parallel search to approximate the best solution?
The brain architecture is highly parallel, however, how it forms high-level thoughts is not known. My guess is that’s some sort of parallel Monte Carlo search driven by complex, partially innate and partially learned, heuristics.
m confused… if time required is a polynomial or exponential function of your problem size, then hardware that runs twice as fast will still solve your problem twice as fast, won’t it?
Yes, but it wouldn’t be twice as smart. If you were to speed up a chicken brain by a factor of 10,000 you wouldn’t get a super-human intelligence.
And if the algorithm you’re using to solve the problem is perfectly parallelizable (which I grant to AI foom proponents ’cause it seems plausible to me)
Perfect parallelizability (linear speedup in the number of processors) is physically impossible due to the fact that information propagates at finite speed, though depending on hardware details, as long as your computer doesn’t get too big, you can obtain close to linear speedups on certain problems. NP-complete problems can be solved by brute-force exhaustive search, in principle, which is highly parallelizable. But exhaustive search has a very fast growing exponential complexity, hence it doesn’t get you very far from toy problem instances even on parallel hardware. The more complex heuristics and approximation schemes you use, the less parallelizability you get, in general.
Anyway, 10,000 chickens won’t make a super-human intelligence, even if you found some way to wire their brains togheter.
If you were to speed up a chicken brain by a factor of 10,000 you wouldn’t get a super-human intelligence.
Sure, but if we assume we manage to have a human-level AI, how powerful should we expect it to be if we speed that up by a factor of 10, 100, or more?
Personally, I’m pretty sure such a thing is still powerful enough to take over the world (assuming it is the only such AI), and in any case dangerous enough to lock us all in a future we really don’t want.
At that point, I don’t really care if it’s “superhuman” or not.
It won’t be any smarter at all actually, it will just have more relative time.
Basically, if you take someone, and give them 100 days to do something, they will have 100 times as much time to do it as they would if it takes 1 day, but if it is beyond their capabilities, then it will remain beyond their capabilities, and running at 100x speed is only helpful for projects for which mental time is the major factor—if you have to run experiments and wait for results, all you’re really doing is decreasing the lag time between experiments, and even then only potentially.
Its not even as good as having 100 slaves work on a project (as someone else posited) because you’re really just having ONE slave work on the project for 100 days; copying them 100 times likely won’t help that issue.
This is one of the fundamental problems with the idea of the singularity in the first place; the truth is that designing more intelligent intelligences is probably HARDER than designing simpler ones, possibly by orders of magnitude, and it may not be scalar at all. If you look at rodent brains and human brains, there are numerous differences between them—scaling up a rodent brain to the same EQ as a human brain would NOT give you something as smart as a human, or even sapient.
You are very likely to see declining returns, not accelerating returns, which is exactly what we see in all other fields of technology—the higher you get, the harder it is to go further.
Moreover, it isn’t even clear what a “superhuman” intelligence even means. We don’t even have any way of measuring intelligence absolutely that I am aware of—IQ is a statistical means, as are standardized tests. We can’t say that human A is twice as smart as human B, and without such metrication it may be difficult to determine just how much smarter anything is than a human in the first place. If four geniuses can work together and get the same result as a computer which takes 1000 times as much energy to do the same task, then the computer is inefficient no matter how smart it is.
This efficiency is ANOTHER major barrier as well—human brains run off of cherrios, whereas any AI we build is going to be massively less efficient in terms of energy usage per cycle, at least for the foreseeable futures.
Another question is whether there is some sort of effective cap to intelligence given energy, heat dissipation, proximity of processing centers, ect. Given that we’re only going to see microchips 256 times as dense on a plane as we have presently available, and given the various issues with heat dissipation of 3D chips (not to mention expense), we may well run into some barriers here.
I was looking at some stuff last night and while people claim we may be able to model the brain using an exascale computer, I am actually rather skeptical after reading up on it—while 150 trillion connections between 86 billion neurons doesn’t sound like that much on the exascale, we have a lot of other things, such as glial cells, which appear to play a role in intelligence, and it is not unlikely that their function is completely vital in a proper simulation. Indeed, our utter lack of understanding of how the human brain works is a major barrier for even thinking about how we can make something more intelligent than a human which is not a human—its pretty much pure fantasy at this point. It may be that ridiculous parallelization with low latency is absolutely vital for sapience, and that could very well put a major crimp on silicon-based intelligences at all, due to their more linear nature, even with things like GPUs and multicore processors because the human brain is sending out trillions of signals with each step.
Some possibilities for simulating the human brain could easily take 10^22 FLOPS or more, and given the limitations of transistor-based computing, that looks like it is about the level of supercomputer we’d have in 2030 or so—but I wouldn’t expect much better than that beyond that point because the only way to make better processors at that point is going up or out, and to what extent we can continue doing that… well, we’ll have to see, but it would very likely eat up even more power and I would have to question the ROI at some point. We DO need to figure out how intelligence works, if only because it might make enhancing humans easier—indeed, unless intelligence is highly computationally efficient, organic intelligences may well be the optimal solution from the standpoint of efficiency, and no sort of exponential takeoff is really possible, or even likely, with such.
You are very likely to see declining returns, not accelerating returns, which is exactly what we see in all other fields of technology—the higher you get, the harder it is to go further.
In many fields of technology, we see sigmoid curves, where initial advancements lead to accelerating returns until it becomes difficult to move further ahead without running up against hard problems or fundamental limits, and returns diminish.
Making an artificial intelligence as capable as a human intelligence may be difficult, but that doesn’t mean that if we reach that point, we’ll be facing major barriers to further progression. I would say we don’t have much evidence to suggest humans are even near the ceiling of what’s strictly possible with a purely biological intelligence; we’ve had very little opportunity for further biological development since the point when cultural developments started accounting for most of our environmental viability, plus we face engineering challenges such as only being able to shove so large a cranium through a bipedal pelvis.
We have no way to even measure intelligence, let alone determine how close to capacity we’re at. We could be 90% there, or 1%, and we have no way, presently, of distinguishing between the two.
We are the smartest creatures ever to have lived on the planet Earth as far as we can tell, and given that we have seen no signs of extraterrestrial civilization, we could very well be the most intelligent creatures in the galaxy for all we know.
As for shoving out humans, isn’t the simplest solution to that simply growing them in artificial wombs?
We already have a simpler solution than that, namely the Cesarian section. It hasn’t been a safe option long enough to have had a significant impact as an evolutionary force though. Plus, there hasn’t been a lot of evolutionary pressure for increased intelligence since the advent of agriculture.
We might be the most intelligent creatures in the galaxy, but that’s a very different matter from being near the most intelligent things that could be constructed out of a comparable amount of matter. Natural selection isn’t that great a process for optimizing intelligence, it’s backpedaled on hominids before given the right niche to fill, so while we don’t have a process for measuring how close we are to the ceiling, I think the reasonable prior on our being close to it is pretty low.
Sure, but if we assume we manage to have a human-level AI, how powerful should we expect it to be if we speed that up by a factor of 10, 100, or more?
As powerful as a a team of 10, 100 human slaves, or a little more, but within the same order or magnitude.
Personally, I’m pretty sure such a think is still powerful enough to take over the world (assuming it is the only such AI), and in any case dangerous enough to lock us all in a future we really don’t want.
At first. If the “100 slaves” AI ever gets out of the box, you can multiply the initial number by the amount of hardware it can copy itself to. It can hack computers, earn (or steal) money, buy hardware…
And suddenly we’re talking about a highly coordinated team of millions.
That’s the plot of the Terminator movies, but it doesn’t seem to be a likely scenario.
During their regime, the Nazis locked up, used as slave labor, and eventually killed, millions of people. Most of them were Ashkenazi Jews, perhaps the smartest of all ethnic groups, with a language difficult to comprehend to outsiders, living in close-knit communities with transnational range, and strong inter-community ties. Did they get “out of the box” and take over the Third Reich? Nope.
AIs might have some advantages for being digital, but also disadvantages.
I think you miss the part where the team of millions continues its self-copying until it eats up every available computing power. If there’s any significant computing overhang, the AI could easily seize control of way more computing power than all the human brains put together.
Also, I think you underestimate the “highly coordinated” part. Any copy of the AI will likely share the exact same goals, and the exact same beliefs. Its instances will have common knowledge of this fact. This would creates an unprecedented level of trust. (The only possible exception I can think of are twins. And even so…)
So, let’s recap:
Thinks 100 times faster than a human, though no better.
Can copy itself over many times (the exact amount depends on computing power available).
The resulting team forms a nearly perfectly coordinated group.
Do you at least concede that this is potentially more dangerous than a whole country armed up with nukes? Would you rely on it being less dangerous than that?
When I imagine that I could make my copy which would be identical to me, sharing my goals, able to copy its experiences back to me, and willing to die for me (something like Naruto’s clones), taking over the society seems rather easy. (Assuming that no one else has this ability, and no one suspects me of having it. In real life it would probably help if all the clones looked different, but had an ability to recognize each other.)
Research: For each interesting topic I could make dozen clones which would study the topic in libraries and universities, and discuss their findings with each other. I don’t suppose it would make me an expert on everything, but I could get at least all the university-level education on most things. Resources: If I can make more money than I spend, and if I don’t make too much copies to imbalance the economy, I can let a few dozen clones work and produce the money for the rest of them. At least in the starting phase, until my research groups discover better ways to make money. Contacts: Different clones could go to different places making useful contacts wil different kinds of people. Sometimes you find a person which can help your goals significantly. With many clones I could make contacts in many different social groups, and overcome language or religious barriers (I can have a few clones learn the language or join the religion). Multiple “first impressions”: If I need a help of a given person or organization, I could in many cases gain their trust by sending multiple different clones to them, using different strategies to befriend them, until I find one that works. Taking over democratic organizations: Any organization with low barriers to entry and democratic voting can be taken over by sending enough clones there, and then voting some of the clones as new leaders. A typical non-governmental organization or even a smaller political party could be gained this way. I don’t even need a majority of clones there: two potential leaders competing with each other, half dozen experts openly supporting each of them, and dozen people befriending random voters and explaining them why leader X or leader Y is the perfect choice; then most of the voting would be done by other people. Assassination: If someone is too much of a problem, I can create a clone which kills them and then disappears. This should be used very rarely, not to draw attention to my abilities. Safety: To protect myself, I would send my different clones to different countries over the world. Joining all the winning sides: If there is an important group of people, I could join them, even the groups fighting against each other. Whoever wins, some of my clones are on the winning side.
If that AI runs on expensive or specialized hardware, it can’t necessarily expand much. For instance, if it runs on hardware worth millions of dollars, it can’t exactly copy itself just anywhere yet. Assuming that the first AI of that level will be cutting edge research and won’t be cheap, that gives a certain time window to study it safely.
The AI may be dangerous if it appeared now, but if it appears in, say, fifty years, then it will have to deal with the state of the art fifty years from now. Expanding without getting caught might be considerably more difficult then than it is now—weak AI will be all over the place, for one.
Last, but not least, the AI must have access to its own source code in order to copy it. That’s far from a given, especially if it’s a neural architecture. A human-level AI would not know how it works any more than we know how we work, so if it has no read access to itself or no way to probe its own circuitry, it won’t be able to copy itself at all. I doubt the first AI would actually have fine-grained access to its own inner workings, and I doubt it would have anywhere close to the amount of resources required to reverse engineer itself. Of course, that point is moot if some fool does give it access...
I agree with your first point, though it gets worse for us as hardware gets cheaper and cheaper.
I like your second point even more: it’s actionable. We could work on the security of personal computers.
That last one is incorrect however. The AI only have to access its object code in order to copy itself. That’s something even current computer viruses can do. And we’re back to boxing it.
If the AI is a learning system such as a neural network, and I believe that’s quite likely to be the case, there is no source/object dichotomy at all and the code may very well be unreadable outside of simple local update procedures that are completely out of the AI’s control. In other words, it might be physically impossible for both the AI and ourselves to access the AI’s object code—it would be locked in a hardware box with no physical wires to probe its contents, basically.
I mean, think of a physical hardware circuit implementing a kind of neuron network—in order for the network to be “copiable”, you need to be able to read the values of all neurons. However, that requires a global clock (to ensure synchronization, though AI might tolerate being a bit out of phase) and a large number of extra wires connecting each component to busses going out of the system. Of course, all that extra fluff inflates the cost of the system, makes it bigger, slower and probably less energy efficient. Since the first human-level AI won’t just come out of nowhere, it will probably use off-the-shelf digital neural components, and for cost and speed reasons, these components might not actually offer any way to copy their contents.
This being said, even if the AI runs on conventional hardware, locking it out of its own object code isn’t exactly rocket science. The specification of some programming languages already guarantee that this cannot happen, and type/proof theory is an active research field that may very well be able to prove the conformance of implementation to specification. If the AI is a neural network emulated on conventional hardware, the risks that it can read itself without permission are basically zilch.
There are various notions of intelligence, social intelligence includes the skills for getting in charge. My point is that human-level intelligence, even replicated or sped up, is generally not enough.
Right. You’re definitely gonna be able to get the same solution to the same problem twice as fast. The thing labeled by labels like “NP hard” is that doubling your hardware doesn’t let you solve problems that are twice as complicated in your unit of time. So your dumb robot can do dumb things twice as fast, but it can’t do things twice as smart :P
There’s one more consideration, which is that if you’re approximating and you keep the problem the same, doubling your hardware won’t always let you find a solution that’s twice as good. But I think this can reasonably be either sublinear or superlinear, until you get up to really large amounts of computing power.
Right, the problem is that “twice as fast” doesn’t help you much for most problems. For example, if you are solving the Traveling Salesman Problem, then doubling your hardware will allow you to add one more city to the map (under the worst-case scenario). So, now your AI could solve the problem for 1001 cities, instead of 1000. Yey.
No problem is perfectly parallelizable in a physical sense. If you build a circuit to solve a problem, and that the circuit is one light year across in size, you’re probably not going to solve it in under a year—technically, any decision problem implemented by a circuit is at least O(n) because that’s how the length of the wires scale.
Now, there are a few ways you might want to parallelize intelligence. The first way is by throwing many independent intelligent entities at the problem, but that requires a lot of redundancy, so the returns on that will not be linear. A second way is to build a team of intelligent entities collaborating to solve the problem, each specializing on an aspect—but since each of these specialized intelligent entities is much farther from each other than the respective modules of a single general intelligence, part of the gains will be offset by massive increases in communication costs. A third way would be to grow an AI from within, interleaving various modules so that significant intelligence is available in all locations of the AI’s brain. Unfortunately, doing so requires internal scaffolding (which is going to reduce packing efficiency and slow it down) and it still expands in space, with internal communication costs increasing in proportion of its size.
I mean, ultimately, even if you want to do some kind of parallel search, you’re likely to use some kind of divide and conquer technique with a logarithmic-ish depth. But since you still have to pack data in a 3D space, each level is going to take longer to explore than the previous one, so past a certain point, communication costs might outweigh intelligence gains and parallelization might become somewhat of a pipe dream.
When we talk about the complexity of an algorithm, we have to decide what resources we are going to measure. Time used by a multi-tape Turing machine is the most common measurement, since it’s easy to define and generally matches up with physical time. If you change the model of computation, you can lower (or raise) this to pretty much anything by constructing your clock the right way.
Ah, sorry, I might not have been clear. I was referring to what may be physically feasible, e.g. a 3D circuit in a box with inputs coming in from the top plane and outputs coming out of the bottom plane. If you have one output that depends on all N inputs and pack everything as tightly as possible, the signal would still take Ω(sqrt(N)) time to reach. From all the physically doable models of computation, I think that’s likely as good as it gets.
Oh I see, we want physically possible computers. In that case, I can get it down to log(n) with general relativity, assuming I’m allowed to set up wormholes. (This whole thing is a bit badly defined since it’s not clear what you’re allowed to prepare in advance. Any necessary setup would presumably take Ω(n) time anyways.)
How do you think human intelligence works? Perhaps by doing a massive parallel search to approximate the best solution?
This is just an educated guess, but to me massive parallel search feels very unlikely for human intelligence. To do something “massive parallel”, you need a lot of (almost) identical hardware. If you want to run the same algorithm 100 times in parallel, you need 100 instances of the (almost) same hardware. Otherwise—how can you run that in parallel?
Some parts of human brain work like that, as far as I know. The visual part of the brain, specifically. There are many neurons implementing the same task: scanning an input from a part of retina, detecting lines, edges, and whatever. This is why image recognition is extremely fast and requires a large part of the brain dedicated to this task.
Seems to me (but I am not an expert) that most of the brain functionality is not like this. Especially the parts related to thinking. Thinking is usually slow and needs to be learned—which is the exact opposite of how the massively parallelized parts work.
EDIT: Unless by massive parallel human intelligence you meant multiple people working on the same problem.
Seems to me (but I am not an expert) that most of the brain functionality is not like this. Especially the parts related to thinking. Thinking is usually slow and needs to be learned—which is the exact opposite of how the massively parallelized parts work.
I’m not an expert either, but from what I’ve read on the subject, most of the neocortex does work like this. The architecture used in the visual cortex is largely the same as that used in the rest of the cortex, with some minor variations. This is suggested by the fact that people who lose an area of their neocortex are often able to recover, with another area filling in for it. I’m on a phone, so I can’t go into as much detail as I’d like, but I recommend investigating the work of Mountcastle, and more recently Markram.
Edit: On Intelligence by Jeff Hawkins explains this principle in more depth, it’s an interesting read.
Thanks for your comments. How do you think human intelligence works? Perhaps by doing a massive parallel search to approximate the best solution?
I’m confused… if time required is a polynomial or exponential function of your problem size, then hardware that runs twice as fast will still solve your problem twice as fast, won’t it? (How could it not?) And if the algorithm you’re using to solve the problem is perfectly parallelizable (which I grant to AI foom proponents ’cause it seems plausible to me), then throwing twice the hardware at any given problem will solve it twice as fast. (Although yes, it will not solve problems that are twice as big.)
The brain architecture is highly parallel, however, how it forms high-level thoughts is not known.
My guess is that’s some sort of parallel Monte Carlo search driven by complex, partially innate and partially learned, heuristics.
Yes, but it wouldn’t be twice as smart. If you were to speed up a chicken brain by a factor of 10,000 you wouldn’t get a super-human intelligence.
Perfect parallelizability (linear speedup in the number of processors) is physically impossible due to the fact that information propagates at finite speed, though depending on hardware details, as long as your computer doesn’t get too big, you can obtain close to linear speedups on certain problems.
NP-complete problems can be solved by brute-force exhaustive search, in principle, which is highly parallelizable. But exhaustive search has a very fast growing exponential complexity, hence it doesn’t get you very far from toy problem instances even on parallel hardware. The more complex heuristics and approximation schemes you use, the less parallelizability you get, in general.
Anyway, 10,000 chickens won’t make a super-human intelligence, even if you found some way to wire their brains togheter.
One of the cooler papers I’ve seen connecting MC with thinking is http://www.stanford.edu/~ngoodman/papers/LiederGriffithsGoodman2012NIPS.pdf which claims that MCMC can even explain some cognitive biases. (I don’t know as much about MCMC as I would like, so I can’t evaluate it.)
Sure, but if we assume we manage to have a human-level AI, how powerful should we expect it to be if we speed that up by a factor of 10, 100, or more?
Personally, I’m pretty sure such a thing is still powerful enough to take over the world (assuming it is the only such AI), and in any case dangerous enough to lock us all in a future we really don’t want.
At that point, I don’t really care if it’s “superhuman” or not.
It won’t be any smarter at all actually, it will just have more relative time.
Basically, if you take someone, and give them 100 days to do something, they will have 100 times as much time to do it as they would if it takes 1 day, but if it is beyond their capabilities, then it will remain beyond their capabilities, and running at 100x speed is only helpful for projects for which mental time is the major factor—if you have to run experiments and wait for results, all you’re really doing is decreasing the lag time between experiments, and even then only potentially.
Its not even as good as having 100 slaves work on a project (as someone else posited) because you’re really just having ONE slave work on the project for 100 days; copying them 100 times likely won’t help that issue.
This is one of the fundamental problems with the idea of the singularity in the first place; the truth is that designing more intelligent intelligences is probably HARDER than designing simpler ones, possibly by orders of magnitude, and it may not be scalar at all. If you look at rodent brains and human brains, there are numerous differences between them—scaling up a rodent brain to the same EQ as a human brain would NOT give you something as smart as a human, or even sapient.
You are very likely to see declining returns, not accelerating returns, which is exactly what we see in all other fields of technology—the higher you get, the harder it is to go further.
Moreover, it isn’t even clear what a “superhuman” intelligence even means. We don’t even have any way of measuring intelligence absolutely that I am aware of—IQ is a statistical means, as are standardized tests. We can’t say that human A is twice as smart as human B, and without such metrication it may be difficult to determine just how much smarter anything is than a human in the first place. If four geniuses can work together and get the same result as a computer which takes 1000 times as much energy to do the same task, then the computer is inefficient no matter how smart it is.
This efficiency is ANOTHER major barrier as well—human brains run off of cherrios, whereas any AI we build is going to be massively less efficient in terms of energy usage per cycle, at least for the foreseeable futures.
Another question is whether there is some sort of effective cap to intelligence given energy, heat dissipation, proximity of processing centers, ect. Given that we’re only going to see microchips 256 times as dense on a plane as we have presently available, and given the various issues with heat dissipation of 3D chips (not to mention expense), we may well run into some barriers here.
I was looking at some stuff last night and while people claim we may be able to model the brain using an exascale computer, I am actually rather skeptical after reading up on it—while 150 trillion connections between 86 billion neurons doesn’t sound like that much on the exascale, we have a lot of other things, such as glial cells, which appear to play a role in intelligence, and it is not unlikely that their function is completely vital in a proper simulation. Indeed, our utter lack of understanding of how the human brain works is a major barrier for even thinking about how we can make something more intelligent than a human which is not a human—its pretty much pure fantasy at this point. It may be that ridiculous parallelization with low latency is absolutely vital for sapience, and that could very well put a major crimp on silicon-based intelligences at all, due to their more linear nature, even with things like GPUs and multicore processors because the human brain is sending out trillions of signals with each step.
Some possibilities for simulating the human brain could easily take 10^22 FLOPS or more, and given the limitations of transistor-based computing, that looks like it is about the level of supercomputer we’d have in 2030 or so—but I wouldn’t expect much better than that beyond that point because the only way to make better processors at that point is going up or out, and to what extent we can continue doing that… well, we’ll have to see, but it would very likely eat up even more power and I would have to question the ROI at some point. We DO need to figure out how intelligence works, if only because it might make enhancing humans easier—indeed, unless intelligence is highly computationally efficient, organic intelligences may well be the optimal solution from the standpoint of efficiency, and no sort of exponential takeoff is really possible, or even likely, with such.
In many fields of technology, we see sigmoid curves, where initial advancements lead to accelerating returns until it becomes difficult to move further ahead without running up against hard problems or fundamental limits, and returns diminish.
Making an artificial intelligence as capable as a human intelligence may be difficult, but that doesn’t mean that if we reach that point, we’ll be facing major barriers to further progression. I would say we don’t have much evidence to suggest humans are even near the ceiling of what’s strictly possible with a purely biological intelligence; we’ve had very little opportunity for further biological development since the point when cultural developments started accounting for most of our environmental viability, plus we face engineering challenges such as only being able to shove so large a cranium through a bipedal pelvis.
We have no way to even measure intelligence, let alone determine how close to capacity we’re at. We could be 90% there, or 1%, and we have no way, presently, of distinguishing between the two.
We are the smartest creatures ever to have lived on the planet Earth as far as we can tell, and given that we have seen no signs of extraterrestrial civilization, we could very well be the most intelligent creatures in the galaxy for all we know.
As for shoving out humans, isn’t the simplest solution to that simply growing them in artificial wombs?
We already have a simpler solution than that, namely the Cesarian section. It hasn’t been a safe option long enough to have had a significant impact as an evolutionary force though. Plus, there hasn’t been a lot of evolutionary pressure for increased intelligence since the advent of agriculture.
We might be the most intelligent creatures in the galaxy, but that’s a very different matter from being near the most intelligent things that could be constructed out of a comparable amount of matter. Natural selection isn’t that great a process for optimizing intelligence, it’s backpedaled on hominids before given the right niche to fill, so while we don’t have a process for measuring how close we are to the ceiling, I think the reasonable prior on our being close to it is pretty low.
As powerful as a a team of 10, 100 human slaves, or a little more, but within the same order or magnitude.
100 slaves are not going to take over the world.
One 10,000 year old human might be able to do it, though.
Without any legal protection?
At first. If the “100 slaves” AI ever gets out of the box, you can multiply the initial number by the amount of hardware it can copy itself to. It can hack computers, earn (or steal) money, buy hardware…
And suddenly we’re talking about a highly coordinated team of millions.
That’s the plot of the Terminator movies, but it doesn’t seem to be a likely scenario.
During their regime, the Nazis locked up, used as slave labor, and eventually killed, millions of people. Most of them were Ashkenazi Jews, perhaps the smartest of all ethnic groups, with a language difficult to comprehend to outsiders, living in close-knit communities with transnational range, and strong inter-community ties.
Did they get “out of the box” and take over the Third Reich? Nope.
AIs might have some advantages for being digital, but also disadvantages.
I think you miss the part where the team of millions continues its self-copying until it eats up every available computing power. If there’s any significant computing overhang, the AI could easily seize control of way more computing power than all the human brains put together.
Also, I think you underestimate the “highly coordinated” part. Any copy of the AI will likely share the exact same goals, and the exact same beliefs. Its instances will have common knowledge of this fact. This would creates an unprecedented level of trust. (The only possible exception I can think of are twins. And even so…)
So, let’s recap:
Thinks 100 times faster than a human, though no better.
Can copy itself over many times (the exact amount depends on computing power available).
The resulting team forms a nearly perfectly coordinated group.
Do you at least concede that this is potentially more dangerous than a whole country armed up with nukes? Would you rely on it being less dangerous than that?
When I imagine that I could make my copy which would be identical to me, sharing my goals, able to copy its experiences back to me, and willing to die for me (something like Naruto’s clones), taking over the society seems rather easy. (Assuming that no one else has this ability, and no one suspects me of having it. In real life it would probably help if all the clones looked different, but had an ability to recognize each other.)
Research: For each interesting topic I could make dozen clones which would study the topic in libraries and universities, and discuss their findings with each other. I don’t suppose it would make me an expert on everything, but I could get at least all the university-level education on most things.
Resources: If I can make more money than I spend, and if I don’t make too much copies to imbalance the economy, I can let a few dozen clones work and produce the money for the rest of them. At least in the starting phase, until my research groups discover better ways to make money.
Contacts: Different clones could go to different places making useful contacts wil different kinds of people. Sometimes you find a person which can help your goals significantly. With many clones I could make contacts in many different social groups, and overcome language or religious barriers (I can have a few clones learn the language or join the religion).
Multiple “first impressions”: If I need a help of a given person or organization, I could in many cases gain their trust by sending multiple different clones to them, using different strategies to befriend them, until I find one that works.
Taking over democratic organizations: Any organization with low barriers to entry and democratic voting can be taken over by sending enough clones there, and then voting some of the clones as new leaders. A typical non-governmental organization or even a smaller political party could be gained this way. I don’t even need a majority of clones there: two potential leaders competing with each other, half dozen experts openly supporting each of them, and dozen people befriending random voters and explaining them why leader X or leader Y is the perfect choice; then most of the voting would be done by other people.
Assassination: If someone is too much of a problem, I can create a clone which kills them and then disappears. This should be used very rarely, not to draw attention to my abilities.
Safety: To protect myself, I would send my different clones to different countries over the world.
Joining all the winning sides: If there is an important group of people, I could join them, even the groups fighting against each other. Whoever wins, some of my clones are on the winning side.
There are a lot of “ifs”, though.
If that AI runs on expensive or specialized hardware, it can’t necessarily expand much. For instance, if it runs on hardware worth millions of dollars, it can’t exactly copy itself just anywhere yet. Assuming that the first AI of that level will be cutting edge research and won’t be cheap, that gives a certain time window to study it safely.
The AI may be dangerous if it appeared now, but if it appears in, say, fifty years, then it will have to deal with the state of the art fifty years from now. Expanding without getting caught might be considerably more difficult then than it is now—weak AI will be all over the place, for one.
Last, but not least, the AI must have access to its own source code in order to copy it. That’s far from a given, especially if it’s a neural architecture. A human-level AI would not know how it works any more than we know how we work, so if it has no read access to itself or no way to probe its own circuitry, it won’t be able to copy itself at all. I doubt the first AI would actually have fine-grained access to its own inner workings, and I doubt it would have anywhere close to the amount of resources required to reverse engineer itself. Of course, that point is moot if some fool does give it access...
I agree with your first point, though it gets worse for us as hardware gets cheaper and cheaper.
I like your second point even more: it’s actionable. We could work on the security of personal computers.
That last one is incorrect however. The AI only have to access its object code in order to copy itself. That’s something even current computer viruses can do. And we’re back to boxing it.
If the AI is a learning system such as a neural network, and I believe that’s quite likely to be the case, there is no source/object dichotomy at all and the code may very well be unreadable outside of simple local update procedures that are completely out of the AI’s control. In other words, it might be physically impossible for both the AI and ourselves to access the AI’s object code—it would be locked in a hardware box with no physical wires to probe its contents, basically.
I mean, think of a physical hardware circuit implementing a kind of neuron network—in order for the network to be “copiable”, you need to be able to read the values of all neurons. However, that requires a global clock (to ensure synchronization, though AI might tolerate being a bit out of phase) and a large number of extra wires connecting each component to busses going out of the system. Of course, all that extra fluff inflates the cost of the system, makes it bigger, slower and probably less energy efficient. Since the first human-level AI won’t just come out of nowhere, it will probably use off-the-shelf digital neural components, and for cost and speed reasons, these components might not actually offer any way to copy their contents.
This being said, even if the AI runs on conventional hardware, locking it out of its own object code isn’t exactly rocket science. The specification of some programming languages already guarantee that this cannot happen, and type/proof theory is an active research field that may very well be able to prove the conformance of implementation to specification. If the AI is a neural network emulated on conventional hardware, the risks that it can read itself without permission are basically zilch.
What we usually mean by intelligence doesn’t include the skills necessary for getting to be in charge.
There are various notions of intelligence, social intelligence includes the skills for getting in charge.
My point is that human-level intelligence, even replicated or sped up, is generally not enough.
I’m not sure if that’s a good comparison. Compare the following cases:
A. 1 smart human, given 100 days to solve some problem
B. 100 smart humans, given 1 day to solve some problem.
C. 1,000 smart humans, given 1 day to solve some problem.
A would outperform B on most tasks, and probably even C. Most problems just aren’t that parallelizable.
That’s why I wrote “or a little more, but within the same order or magnitude”
Right. You’re definitely gonna be able to get the same solution to the same problem twice as fast. The thing labeled by labels like “NP hard” is that doubling your hardware doesn’t let you solve problems that are twice as complicated in your unit of time. So your dumb robot can do dumb things twice as fast, but it can’t do things twice as smart :P
There’s one more consideration, which is that if you’re approximating and you keep the problem the same, doubling your hardware won’t always let you find a solution that’s twice as good. But I think this can reasonably be either sublinear or superlinear, until you get up to really large amounts of computing power.
Right, the problem is that “twice as fast” doesn’t help you much for most problems. For example, if you are solving the Traveling Salesman Problem, then doubling your hardware will allow you to add one more city to the map (under the worst-case scenario). So, now your AI could solve the problem for 1001 cities, instead of 1000. Yey.
But given the right approximation algorithm...
No problem is perfectly parallelizable in a physical sense. If you build a circuit to solve a problem, and that the circuit is one light year across in size, you’re probably not going to solve it in under a year—technically, any decision problem implemented by a circuit is at least O(n) because that’s how the length of the wires scale.
Now, there are a few ways you might want to parallelize intelligence. The first way is by throwing many independent intelligent entities at the problem, but that requires a lot of redundancy, so the returns on that will not be linear. A second way is to build a team of intelligent entities collaborating to solve the problem, each specializing on an aspect—but since each of these specialized intelligent entities is much farther from each other than the respective modules of a single general intelligence, part of the gains will be offset by massive increases in communication costs. A third way would be to grow an AI from within, interleaving various modules so that significant intelligence is available in all locations of the AI’s brain. Unfortunately, doing so requires internal scaffolding (which is going to reduce packing efficiency and slow it down) and it still expands in space, with internal communication costs increasing in proportion of its size.
I mean, ultimately, even if you want to do some kind of parallel search, you’re likely to use some kind of divide and conquer technique with a logarithmic-ish depth. But since you still have to pack data in a 3D space, each level is going to take longer to explore than the previous one, so past a certain point, communication costs might outweigh intelligence gains and parallelization might become somewhat of a pipe dream.
That is a pretty cool idea.
There are a few like it. For example: All problems are at least Ω(max(N,M)), in the size of the problem description and output description.
It’s not usually the limiting factor. ;)
Actually, only the output; sometimes you only need the first few bits. Your equation holds if you know you need to read the end of the input.
And technically you can lower that to sqrt(M) if you organize the inputs and outputs on a surface.
When we talk about the complexity of an algorithm, we have to decide what resources we are going to measure. Time used by a multi-tape Turing machine is the most common measurement, since it’s easy to define and generally matches up with physical time. If you change the model of computation, you can lower (or raise) this to pretty much anything by constructing your clock the right way.
Ah, sorry, I might not have been clear. I was referring to what may be physically feasible, e.g. a 3D circuit in a box with inputs coming in from the top plane and outputs coming out of the bottom plane. If you have one output that depends on all N inputs and pack everything as tightly as possible, the signal would still take Ω(sqrt(N)) time to reach. From all the physically doable models of computation, I think that’s likely as good as it gets.
Oh I see, we want physically possible computers. In that case, I can get it down to log(n) with general relativity, assuming I’m allowed to set up wormholes. (This whole thing is a bit badly defined since it’s not clear what you’re allowed to prepare in advance. Any necessary setup would presumably take Ω(n) time anyways.)
This is just an educated guess, but to me massive parallel search feels very unlikely for human intelligence. To do something “massive parallel”, you need a lot of (almost) identical hardware. If you want to run the same algorithm 100 times in parallel, you need 100 instances of the (almost) same hardware. Otherwise—how can you run that in parallel?
Some parts of human brain work like that, as far as I know. The visual part of the brain, specifically. There are many neurons implementing the same task: scanning an input from a part of retina, detecting lines, edges, and whatever. This is why image recognition is extremely fast and requires a large part of the brain dedicated to this task.
Seems to me (but I am not an expert) that most of the brain functionality is not like this. Especially the parts related to thinking. Thinking is usually slow and needs to be learned—which is the exact opposite of how the massively parallelized parts work.
EDIT: Unless by massive parallel human intelligence you meant multiple people working on the same problem.
I’m not an expert either, but from what I’ve read on the subject, most of the neocortex does work like this. The architecture used in the visual cortex is largely the same as that used in the rest of the cortex, with some minor variations. This is suggested by the fact that people who lose an area of their neocortex are often able to recover, with another area filling in for it. I’m on a phone, so I can’t go into as much detail as I’d like, but I recommend investigating the work of Mountcastle, and more recently Markram.
Edit: On Intelligence by Jeff Hawkins explains this principle in more depth, it’s an interesting read.