Why would you expect a von Neumann-level AI to be “not aligned” in blatant ways? That’s entirely not obvious to me.
There is a whole spectrum of possibilities between paradise and paperclips. von Neumann was somewhere on that spectrum (assuming the notion of “aligned with humanity’s interests” makes sense for individuals), a human-level AI will also be somewhere on this spectrum. How sure are you of what’s located where?
A videogame AI is constructed with the explicit purpose of losing gracefully to a human. People don’t buy games to be crushed by a perfectly playing opponent.
Have you ever played videogames? Sure AI is often programmed to make certain errors, but there are TONS of things that smash your sense of verisimilitude that would be programmed out if it was easy. You can notice this by comparing the behavior of NPCs in newer vs older games. You can also deduce this by other in-game features: EG tutorials that talk about intended ways to defeat the AI but fail to mention the unintended ways, like putting a bucket over their head so they can’t “see” you sneaking.
Also: plenty of AI also plays PERFECTLY in ways no human is capable of. I gotta ask again, do you play videogames? A lot of game AI is in fact designed to beat you. There are difficulty settings for a reason! The reason is there are a lot of people who actually like being crushed by a more powerful opponent until they figure out how to beat it.
Not only that. AI is generally not a high-value effort in games: pisses off the customers if it’s too good and eats valuable CPU time.
A lot of game AI is in fact designed to beat you.
I disagree. And note that games like Dark Souls are difficult not because the AI is good, but because the player is very very fragile. I would say that game AI is designed to give you a bit of trouble (not too much) and then gracefully lose.
There are difficulty settings for a reason!
Difficulty settings generally don’t make AI smarter, they just give it more advantages (more resources, usually) and simultaneously hit the player with penalties.
Yup. He certainly wasn’t aligned against them in any of the blatant ways I would expect unsafe AI to be.
Why would you expect a von Neumann-level AI to be “not aligned” in blatant ways? That’s entirely not obvious to me.
There is a whole spectrum of possibilities between paradise and paperclips. von Neumann was somewhere on that spectrum (assuming the notion of “aligned with humanity’s interests” makes sense for individuals), a human-level AI will also be somewhere on this spectrum. How sure are you of what’s located where?
For the same reason videogame AI often makes mistakes a human player never would.
A videogame AI is constructed with the explicit purpose of losing gracefully to a human. People don’t buy games to be crushed by a perfectly playing opponent.
Have you ever played videogames? Sure AI is often programmed to make certain errors, but there are TONS of things that smash your sense of verisimilitude that would be programmed out if it was easy. You can notice this by comparing the behavior of NPCs in newer vs older games. You can also deduce this by other in-game features: EG tutorials that talk about intended ways to defeat the AI but fail to mention the unintended ways, like putting a bucket over their head so they can’t “see” you sneaking.
Also: plenty of AI also plays PERFECTLY in ways no human is capable of. I gotta ask again, do you play videogames? A lot of game AI is in fact designed to beat you. There are difficulty settings for a reason! The reason is there are a lot of people who actually like being crushed by a more powerful opponent until they figure out how to beat it.
On occasion :-)
Not only that. AI is generally not a high-value effort in games: pisses off the customers if it’s too good and eats valuable CPU time.
I disagree. And note that games like Dark Souls are difficult not because the AI is good, but because the player is very very fragile. I would say that game AI is designed to give you a bit of trouble (not too much) and then gracefully lose.
Difficulty settings generally don’t make AI smarter, they just give it more advantages (more resources, usually) and simultaneously hit the player with penalties.