For the same reason that Linus Torvalds is not able to hack every single Linux system in existence, it is reasonable to assume that the probability of ‘backdoor’-type attacks could be reduced or eliminated.
If you’re not talking about a back door, then I’m not sure what you’re trying to say here. Are you implying that the AI will find the ‘reward producing machine’ and somehow use it to produce rewards? It doesn’t work that way because the machine would still need the key to produce a cryptographically-valid reward, and the key would not be stored on the machine. That’s the whole point of using cryptography. For instance, you could do something like http://en.wikipedia.org/wiki/Secret_sharing where the key is divided among several participants and never actually assembled in one place.
Of course any cryptography system has vulnerabilities. The point is not to build a 100% secure system. The point is to make it so that attempting to hack the system has less expected utility than simply doing what the humans say. And if that fails, then the AI will attempt to hack the system using its weakest point: the people controlling the key. Hence my question.
If you’re not talking about a back door, then I’m not sure what you’re trying to say here.
Yeah, we’re talking past each other. I think I understand what you’re saying, and I’ll try to rephrase what I’m saying.
The AI is out. It is free to manipulate the world at its will. Sensors are everywhere. The AI can hear every word you say, feel every keystroke you make, and see everything you see. The only secrets left are the ones in your head.
How do humans reward the AI? You say “cryptographically”, but cryptography requires difficult arithmetic. How do you perform difficult arithmetic on a secret that can’t leave your head?
Too many assumptions are being made here. What is the basis for believing the AI will have sensors everywhere, especially while it’s still under human control? And if it has the ability to put clandestine sensors in even the most secure locations, why couldn’t it plant clandestince brain implants in the people controlling the key?
For the same reason that Linus Torvalds is not able to hack every single Linux system in existence, it is reasonable to assume that the probability of ‘backdoor’-type attacks could be reduced or eliminated.
If you’re not talking about a back door, then I’m not sure what you’re trying to say here. Are you implying that the AI will find the ‘reward producing machine’ and somehow use it to produce rewards? It doesn’t work that way because the machine would still need the key to produce a cryptographically-valid reward, and the key would not be stored on the machine. That’s the whole point of using cryptography. For instance, you could do something like http://en.wikipedia.org/wiki/Secret_sharing where the key is divided among several participants and never actually assembled in one place.
Of course any cryptography system has vulnerabilities. The point is not to build a 100% secure system. The point is to make it so that attempting to hack the system has less expected utility than simply doing what the humans say. And if that fails, then the AI will attempt to hack the system using its weakest point: the people controlling the key. Hence my question.
Yeah, we’re talking past each other. I think I understand what you’re saying, and I’ll try to rephrase what I’m saying.
The AI is out. It is free to manipulate the world at its will. Sensors are everywhere. The AI can hear every word you say, feel every keystroke you make, and see everything you see. The only secrets left are the ones in your head.
How do humans reward the AI? You say “cryptographically”, but cryptography requires difficult arithmetic. How do you perform difficult arithmetic on a secret that can’t leave your head?
Too many assumptions are being made here. What is the basis for believing the AI will have sensors everywhere, especially while it’s still under human control? And if it has the ability to put clandestine sensors in even the most secure locations, why couldn’t it plant clandestince brain implants in the people controlling the key?