AGI will have at least 100x faster decision making speed for any given decision, compared to human decision making
AGI will be able to interact with all 9 billion humans at once, in parallel, giving it a massive advantage
Slow motion videos present a helpful analogy
My objection is primarily around the fact that having a 100x faster processing power wouldn’t automatically allow you to do things 100x faster in the physical world:
Any mechanical systems that you control won’t be 100x faster due to limitations of how faster real-world mechanical parts will work. I.e. if you control a drone, you have to deal with the fact that the drone won’t fly/rotate 100x faster just because your processing power is 100x faster. And you’ll probably have to control the drone remotely because you wouldn’t fit the entire AGI on the drone itself, placing a limit on how fast you can make decisions.
Any operations where you rely on human action will run at 1x speed, even if you somewhat streamline them thanks to parallelization and superior decision making
Being 100x faster is useless if you don’t have full information on what the humans are doing/plotting. And they could hide pretty easily by meeting up offline with no electronics in place.
I’d also note that the energy required to speed up a physical action increases with the square of the velocity.
So let’s take a military drone that normally must get confirmation from a human operator before firing at a target. This is the bottleneck for its firing. If an AI takes full control of this drone, the drone is now bottlenecked by things like:
The AI’s processing speed in choosing targets in light of its latest observations and plans
The drone’s speed in aiming at the target
The drone’s speed in moving to a new position
The speed with which the drone can be resupplied with ammunition or fuel
The rate at which it needs to be repaired
If the motion of the drone was speeded up by 100x due to the AI’s processing speed being 100x faster, then this would require at least a 10,000x increase in energy requirements.
Currently existing technology is typically engineered to operate to tolerate demands within the requirements for which it was originally designed. Presently existing drones can’t just be commandeered by an AI and made to move at 100x their normal speed.
This also applies to whatever robots would be necessary for the AI to build a drone army capable of taking full advantage of the AI’s faster processing power. And the AI can’t just pull 10,000x the energy from our present infrastructure. It would have to build an infrastructure capable of supplying that amount of energy using presently existing infrastructure.
It might be that an AGI could achieve a 100x gain in the efficiency in achieving its goals via its superior processing power, constant operation, and ~total self-control. For example, it might be able to figure out a way of attacking using drones that much more efficiently destroys the morale and coordination abilities of its opponent, while still operating at normal drone speed.
2 seems more worrying than reassuring. If you have to rely on human action, you’ll be slowed down. So AI’s who can route around humans, or humans who can delegate more decision-making to AI systems, will have a competitive advantage over AIs that don’t do that. If we’re talking about AGI + decent robotics, there’s in principle nothing that AIs need humans for.
3: “useless without full information” is presumably hyperbole, but I also object to weaker claims like “being 100x faster is less than half as useful as you think, if you haven’t considered that spying is non-trivial”. Random analogy: Consider a conflict (e.g. a war or competition between two firms) except that one side (i) gets only 4 days per year, and (ii) gets a very well-secured room to discuss decisions in. Benefit (ii) doesn’t really seem to help much against the disadvantage from (i)!
Points from this post I agree with:
AGI will have at least 100x faster decision making speed for any given decision, compared to human decision making
AGI will be able to interact with all 9 billion humans at once, in parallel, giving it a massive advantage
Slow motion videos present a helpful analogy
My objection is primarily around the fact that having a 100x faster processing power wouldn’t automatically allow you to do things 100x faster in the physical world:
Any mechanical systems that you control won’t be 100x faster due to limitations of how faster real-world mechanical parts will work. I.e. if you control a drone, you have to deal with the fact that the drone won’t fly/rotate 100x faster just because your processing power is 100x faster. And you’ll probably have to control the drone remotely because you wouldn’t fit the entire AGI on the drone itself, placing a limit on how fast you can make decisions.
Any operations where you rely on human action will run at 1x speed, even if you somewhat streamline them thanks to parallelization and superior decision making
Being 100x faster is useless if you don’t have full information on what the humans are doing/plotting. And they could hide pretty easily by meeting up offline with no electronics in place.
I’d also note that the energy required to speed up a physical action increases with the square of the velocity.
So let’s take a military drone that normally must get confirmation from a human operator before firing at a target. This is the bottleneck for its firing. If an AI takes full control of this drone, the drone is now bottlenecked by things like:
The AI’s processing speed in choosing targets in light of its latest observations and plans
The drone’s speed in aiming at the target
The drone’s speed in moving to a new position
The speed with which the drone can be resupplied with ammunition or fuel
The rate at which it needs to be repaired
If the motion of the drone was speeded up by 100x due to the AI’s processing speed being 100x faster, then this would require at least a 10,000x increase in energy requirements.
Currently existing technology is typically engineered to operate to tolerate demands within the requirements for which it was originally designed. Presently existing drones can’t just be commandeered by an AI and made to move at 100x their normal speed.
This also applies to whatever robots would be necessary for the AI to build a drone army capable of taking full advantage of the AI’s faster processing power. And the AI can’t just pull 10,000x the energy from our present infrastructure. It would have to build an infrastructure capable of supplying that amount of energy using presently existing infrastructure.
It might be that an AGI could achieve a 100x gain in the efficiency in achieving its goals via its superior processing power, constant operation, and ~total self-control. For example, it might be able to figure out a way of attacking using drones that much more efficiently destroys the morale and coordination abilities of its opponent, while still operating at normal drone speed.
2 seems more worrying than reassuring. If you have to rely on human action, you’ll be slowed down. So AI’s who can route around humans, or humans who can delegate more decision-making to AI systems, will have a competitive advantage over AIs that don’t do that. If we’re talking about AGI + decent robotics, there’s in principle nothing that AIs need humans for.
3: “useless without full information” is presumably hyperbole, but I also object to weaker claims like “being 100x faster is less than half as useful as you think, if you haven’t considered that spying is non-trivial”. Random analogy: Consider a conflict (e.g. a war or competition between two firms) except that one side (i) gets only 4 days per year, and (ii) gets a very well-secured room to discuss decisions in. Benefit (ii) doesn’t really seem to help much against the disadvantage from (i)!