A human brain uses as much power as a lightbulb and its size is limited by the birth canal, yet an evolutionary accident gave us John Von Neumann who was far beyond most people. An AI as smart as 1000 Von Neumanns using the power of 1000 lightbulbs could probably figure out how to get more power. Arguments like yours ain’t gonna stop it.
Why would you expect a von Neumann-level AI to be “not aligned” in blatant ways? That’s entirely not obvious to me.
There is a whole spectrum of possibilities between paradise and paperclips. von Neumann was somewhere on that spectrum (assuming the notion of “aligned with humanity’s interests” makes sense for individuals), a human-level AI will also be somewhere on this spectrum. How sure are you of what’s located where?
A videogame AI is constructed with the explicit purpose of losing gracefully to a human. People don’t buy games to be crushed by a perfectly playing opponent.
Have you ever played videogames? Sure AI is often programmed to make certain errors, but there are TONS of things that smash your sense of verisimilitude that would be programmed out if it was easy. You can notice this by comparing the behavior of NPCs in newer vs older games. You can also deduce this by other in-game features: EG tutorials that talk about intended ways to defeat the AI but fail to mention the unintended ways, like putting a bucket over their head so they can’t “see” you sneaking.
Also: plenty of AI also plays PERFECTLY in ways no human is capable of. I gotta ask again, do you play videogames? A lot of game AI is in fact designed to beat you. There are difficulty settings for a reason! The reason is there are a lot of people who actually like being crushed by a more powerful opponent until they figure out how to beat it.
Not only that. AI is generally not a high-value effort in games: pisses off the customers if it’s too good and eats valuable CPU time.
A lot of game AI is in fact designed to beat you.
I disagree. And note that games like Dark Souls are difficult not because the AI is good, but because the player is very very fragile. I would say that game AI is designed to give you a bit of trouble (not too much) and then gracefully lose.
There are difficulty settings for a reason!
Difficulty settings generally don’t make AI smarter, they just give it more advantages (more resources, usually) and simultaneously hit the player with penalties.
A human brain uses as much power as a lightbulb and its size is limited by the birth canal, yet an evolutionary accident gave us John Von Neumann who was far beyond most people. An AI as smart as 1000 Von Neumanns using the power of 1000 lightbulbs could probably figure out how to get more power. Arguments like yours ain’t gonna stop it.
This, and find better ways to optimize power efficiency.
an AI even that was just AS smart as von neumann would still be a very dangerous entity, if it wasn’t aligned with humanity’s interests.
Do you think John von Neumann was “aligned with humanity’s interests”?
Yup. He certainly wasn’t aligned against them in any of the blatant ways I would expect unsafe AI to be.
Why would you expect a von Neumann-level AI to be “not aligned” in blatant ways? That’s entirely not obvious to me.
There is a whole spectrum of possibilities between paradise and paperclips. von Neumann was somewhere on that spectrum (assuming the notion of “aligned with humanity’s interests” makes sense for individuals), a human-level AI will also be somewhere on this spectrum. How sure are you of what’s located where?
For the same reason videogame AI often makes mistakes a human player never would.
A videogame AI is constructed with the explicit purpose of losing gracefully to a human. People don’t buy games to be crushed by a perfectly playing opponent.
Have you ever played videogames? Sure AI is often programmed to make certain errors, but there are TONS of things that smash your sense of verisimilitude that would be programmed out if it was easy. You can notice this by comparing the behavior of NPCs in newer vs older games. You can also deduce this by other in-game features: EG tutorials that talk about intended ways to defeat the AI but fail to mention the unintended ways, like putting a bucket over their head so they can’t “see” you sneaking.
Also: plenty of AI also plays PERFECTLY in ways no human is capable of. I gotta ask again, do you play videogames? A lot of game AI is in fact designed to beat you. There are difficulty settings for a reason! The reason is there are a lot of people who actually like being crushed by a more powerful opponent until they figure out how to beat it.
On occasion :-)
Not only that. AI is generally not a high-value effort in games: pisses off the customers if it’s too good and eats valuable CPU time.
I disagree. And note that games like Dark Souls are difficult not because the AI is good, but because the player is very very fragile. I would say that game AI is designed to give you a bit of trouble (not too much) and then gracefully lose.
Difficulty settings generally don’t make AI smarter, they just give it more advantages (more resources, usually) and simultaneously hit the player with penalties.