One of the things we care about most, if there’s a superintelligent AI around, is what it does for, with, and to us. Making a game sufficiently advanced to have accurate models of us in it (1) is really difficult, and (2) is arguably grossly immoral if there’s a real danger that those models of us are going to get eaten or tortured or whatever by an AI. Expanding on #1: it requires human-level AI plus enough capacity to simulate an awful lot of humans, which means also enough capacity to simulate one human very very fast; it is entirely possible that we will have superhumanly capable AI before we have (at least under our control) the ability to simulate millions or billions of humans.
(There is also the issue mentioned by AABoyles and Error, that the AI may well work out it’s in a box. Note that we may not be able to tell whether it’s worked out it’s in a box. And if we’re observing the box and it guesses there’s someone observing the box, it may be able to influence our decisions in ways we would prefer it not to.)
I’m not saying it would solve everything, I’m saying it would be a way to test significant aspects of AI without destroying the world, including significant aspects of their morality. It’s not a “do this magic and morality for AI is solved” as much as a “this doable step helps parts of AI design, probably including preventing the worst classes of paperclip-maximization”.
Yup, maybe. But don’t you think it’s likely that the values we want to impart to an AI are going to be ones that come out really radically differently for a universe without us in it? For instance, we might want the AI to serve us, which of course isn’t even a concept that makes sense if it’s in a simulated universe without us. Or we might want it to value all intelligent life, which is a thing that looks very different if the AI is the only intelligent life in its universe. So: yes, I agree that running the AI in a simulated world might tell us some useful things, but it doesn’t look to me as if the things it could tell us a lot about overlap very much with the things we care most about.
First of all, there is the general problem of ‘does this AI work?’ This includes the general intelligence/rationality-related problems, but possibly also other problems, such as whether it will wirehead itself (whether a box can test that really depends a lot on the implementation).
The morality-stuff is tricky and depends on a lot of stuff, especially on how the AI is implemented. It seems to dangerous to let it play a multiplayer game with humans, even with most restrictions I can think of. However, how to test the morality really depends on how its human-detection system has been implemented. If it just uses some ‘humans generally do these stupid things’ heuristics, you can just plop down a few NPCs. If it uses somewhat smarter heuristics, you might be able to make some animals play the game and let the AI care for them. If it picks something intelligent, you might be able to instantiate other copies of the AI with vastly different utility functions. Basically, there are a lot of approaches to testing morality, but it depends on how the AI is implemented.
One of the things we care about most, if there’s a superintelligent AI around, is what it does for, with, and to us. Making a game sufficiently advanced to have accurate models of us in it (1) is really difficult, and (2) is arguably grossly immoral if there’s a real danger that those models of us are going to get eaten or tortured or whatever by an AI. Expanding on #1: it requires human-level AI plus enough capacity to simulate an awful lot of humans, which means also enough capacity to simulate one human very very fast; it is entirely possible that we will have superhumanly capable AI before we have (at least under our control) the ability to simulate millions or billions of humans.
(There is also the issue mentioned by AABoyles and Error, that the AI may well work out it’s in a box. Note that we may not be able to tell whether it’s worked out it’s in a box. And if we’re observing the box and it guesses there’s someone observing the box, it may be able to influence our decisions in ways we would prefer it not to.)
I’m not saying it would solve everything, I’m saying it would be a way to test significant aspects of AI without destroying the world, including significant aspects of their morality. It’s not a “do this magic and morality for AI is solved” as much as a “this doable step helps parts of AI design, probably including preventing the worst classes of paperclip-maximization”.
Yup, maybe. But don’t you think it’s likely that the values we want to impart to an AI are going to be ones that come out really radically differently for a universe without us in it? For instance, we might want the AI to serve us, which of course isn’t even a concept that makes sense if it’s in a simulated universe without us. Or we might want it to value all intelligent life, which is a thing that looks very different if the AI is the only intelligent life in its universe. So: yes, I agree that running the AI in a simulated world might tell us some useful things, but it doesn’t look to me as if the things it could tell us a lot about overlap very much with the things we care most about.
It would actually tell us a lot of useful things.
First of all, there is the general problem of ‘does this AI work?’ This includes the general intelligence/rationality-related problems, but possibly also other problems, such as whether it will wirehead itself (whether a box can test that really depends a lot on the implementation).
The morality-stuff is tricky and depends on a lot of stuff, especially on how the AI is implemented. It seems to dangerous to let it play a multiplayer game with humans, even with most restrictions I can think of. However, how to test the morality really depends on how its human-detection system has been implemented. If it just uses some ‘humans generally do these stupid things’ heuristics, you can just plop down a few NPCs. If it uses somewhat smarter heuristics, you might be able to make some animals play the game and let the AI care for them. If it picks something intelligent, you might be able to instantiate other copies of the AI with vastly different utility functions. Basically, there are a lot of approaches to testing morality, but it depends on how the AI is implemented.