Due to the lack of details, it is difficult to make a recomendation, but some thoughts.
Both as an AGI challenge and for general human safety, business intelligence datawarehouses are probably a good bet. Any pattern undetected by humans detected by an AI could mean good money, which could feedback into more resources for the AI. Also, the ability of corporations to harm others doesn’t increase significantly with a better business intelligence tool.
Virtual worlds—If the AI is tested in an isolated virtual world, that will be better for us. Test it in a virtual world that is completely unlike ours, a gas giant simulation maybe. Even if it develops extremely capable technology to deal with the gas giant environment within the simulation, it would mean very little in the real world except as a demonstration of intelligence.
Virtual Worlds doesn’t buy you any safety, even if it can’t break out of the simulator.
If you manage to make AI, you’ve got a Really Powerful Optimization Process. If it worked out simulated physics and has access to it’s own source, it’s probably smart enough to ‘foom’, even with the simulation. At which point you have a REALLY powerful optimizer, and no idea how to prove anything about it’s goal system. An untrustable genie.
Also, spending all those cycles on that kind of simulated world would be hugely inefficient.
James, you can’t blame me for responding to the question. Stuart has said that advice on giving up will not be accepted. The question is to minimise the fallout of a lucky stroke moving this guy’s AI forward and fooming. Both of my suggestions were around that.
Virtual worlds—If the AI is tested in an isolated virtual world, that will be better for us. Test it in a virtual world that is completely unlike ours, a gas giant simulation maybe. Even if it develops extremely capable technology to deal with the gas giant environment within the simulation, it would mean very little in the real world except as a demonstration of intelligence.
You are giving a budding superintelligence exposure to a simulation based on our physics? It would work out the physics of the isolated virtual world, deduce from the traces you leave in the design that it is in a simulation and have a good guess on what we believe to be the actual physics of our universe. Maybe even have a hunch about how we have physics wrong. I would not want to bet our existence on it being unable to get out of that box.
My point with the virtual worlds was to put the AI into a simulation sufficiently unlike our world that it wouldn’t be a threat and sufficiently like our world that we would be able to recognise what it does as intelligence. Hence the Gas giant example.
If we were to release an AI into today’s simulations like sims which are much less granular than the one I have proposed in my post, then it would figure out that it is in a simulation much faster.
If we put it into some other kind of universe with weird physics, a magical universe lets say, then we will need to send someone intelligent to do a considerable amount of trials before we release the AI. This is to prove that whatever solutions the AI comes up with are genuinely intelligent and not something that is obvious.
I too agree that we wouldn’t want to bet our existence on it being unable to get out of that box, but what evidence will we leave in the simulation which will point to it that it has to “Press Red for talking to simulator”? Or to put it in even simpler terms, where in our universe is OUR “Press Red to talk to simulator” button?
My point with the virtual worlds was to put the AI into a simulation sufficiently unlike our world that it wouldn’t be a threat and sufficiently like our world that we would be able to recognise what it does as intelligence. Hence the Gas giant example.
I’m not sure I follow. Gas giants run on the same physics as you and me. Do you mean a world with actual different simulated physics?
I too agree that we wouldn’t want to bet our existence on it being unable to get out of that box, but what evidence will we leave in the simulation which will point to it that it has to “Press Red for talking to simulator”?
I don’t know. Who is going to be creating the simulation? How can I be comfortable that he will not either make a bug or design a simulation that a superintelligence cannot deduce that it is artificial? Proving that things way way smarter than me couldn’t know stuff is hard. Possible sometimes but hard.
Or to put it in even simpler terms, where in our universe is OUR “Press Red to talk to simulator” button?
The presence or absence of such a button in our universe provides some evidence about whether we could reliably create a simulation that is undetectable. But not that much evidence.
How would you design such a button? Reciting a fixed verse and afterwards stating what you want from the simulator seems like a good technique.
A majority of the people on this earth believe that such a button exists in form of praying ;)
Additionally the computer on which the virtual world runs shouldn’t be directly connected to other computers to prevent the AGI to escape through some 0day.
Due to the lack of details, it is difficult to make a recomendation, but some thoughts.
Both as an AGI challenge and for general human safety, business intelligence datawarehouses are probably a good bet. Any pattern undetected by humans detected by an AI could mean good money, which could feedback into more resources for the AI. Also, the ability of corporations to harm others doesn’t increase significantly with a better business intelligence tool.
Virtual worlds—If the AI is tested in an isolated virtual world, that will be better for us. Test it in a virtual world that is completely unlike ours, a gas giant simulation maybe. Even if it develops extremely capable technology to deal with the gas giant environment within the simulation, it would mean very little in the real world except as a demonstration of intelligence.
Virtual Worlds doesn’t buy you any safety, even if it can’t break out of the simulator.
If you manage to make AI, you’ve got a Really Powerful Optimization Process. If it worked out simulated physics and has access to it’s own source, it’s probably smart enough to ‘foom’, even with the simulation. At which point you have a REALLY powerful optimizer, and no idea how to prove anything about it’s goal system. An untrustable genie.
Also, spending all those cycles on that kind of simulated world would be hugely inefficient.
James, you can’t blame me for responding to the question. Stuart has said that advice on giving up will not be accepted. The question is to minimise the fallout of a lucky stroke moving this guy’s AI forward and fooming. Both of my suggestions were around that.
You are quite right.
You are giving a budding superintelligence exposure to a simulation based on our physics? It would work out the physics of the isolated virtual world, deduce from the traces you leave in the design that it is in a simulation and have a good guess on what we believe to be the actual physics of our universe. Maybe even have a hunch about how we have physics wrong. I would not want to bet our existence on it being unable to get out of that box.
My point with the virtual worlds was to put the AI into a simulation sufficiently unlike our world that it wouldn’t be a threat and sufficiently like our world that we would be able to recognise what it does as intelligence. Hence the Gas giant example.
If we were to release an AI into today’s simulations like sims which are much less granular than the one I have proposed in my post, then it would figure out that it is in a simulation much faster.
If we put it into some other kind of universe with weird physics, a magical universe lets say, then we will need to send someone intelligent to do a considerable amount of trials before we release the AI. This is to prove that whatever solutions the AI comes up with are genuinely intelligent and not something that is obvious.
I too agree that we wouldn’t want to bet our existence on it being unable to get out of that box, but what evidence will we leave in the simulation which will point to it that it has to “Press Red for talking to simulator”? Or to put it in even simpler terms, where in our universe is OUR “Press Red to talk to simulator” button?
I’m not sure I follow. Gas giants run on the same physics as you and me. Do you mean a world with actual different simulated physics?
I don’t know. Who is going to be creating the simulation? How can I be comfortable that he will not either make a bug or design a simulation that a superintelligence cannot deduce that it is artificial? Proving that things way way smarter than me couldn’t know stuff is hard. Possible sometimes but hard.
The presence or absence of such a button in our universe provides some evidence about whether we could reliably create a simulation that is undetectable. But not that much evidence.
How would you design such a button? Reciting a fixed verse and afterwards stating what you want from the simulator seems like a good technique. A majority of the people on this earth believe that such a button exists in form of praying ;)
Additionally the computer on which the virtual world runs shouldn’t be directly connected to other computers to prevent the AGI to escape through some 0day.