It is difficult to constrain the input we give to the AI, but the output can be constrained severely. A smart guy could wake up alone in a room and infer how he evolved, but so long as his only link to the outside world is a light switch that can only be switched once, there is no risk that he will escape.
A man in a room with a light switch isn’t very useful.
An AI can’t optimize over more bits than we allow it as output. If we give a 1 time 32 bit output register then well, we probably could have brute forced it in the first place. If we give it a kilobyte, then it could probably mindhack us.
(And you’re swearing to yourself that you won’t monitor it’s execution? Really? How do you even debug that?)
You have to keep in mind that the point of AI research is to get to something we can let out of the box. If the argument becomes that we can run it on a headless netless 486 which we immediately explode...then yes, you can probably run that. Probably.
Nick Hay, Marcello and I discussed this question a while ago: if you had a halting oracle, how could you use it to help you prove a theorem, such as the Riemann Hypothesis? Let’s say you are only allowed to ask one question; you get one bit of information.
Write a chess program that provably makes only legal moves, iterate as desired to improve it. Or,
Write a chess program. Put it in a sandbox so you only ever see it’s moves. Maybe they’re all legal, or maybe they’re not because you’re having it learn the rules with a big neural net or something. At the end of the round of games, the sandbox clears all the memory that held the chess program except for a list of moves in many games. You keep the source. Anything it learned is gone. Iterate as desired to improve it.
If you’re confident you could work out how it was thinking from the source and move list, what if you only got a sequence of wins and non-wins? (An array of bits)
A sequence of wins and non-wins is enough to tell you whether a given approach can result in intelligent behaviour. That alone is enough to make it a useful experiment.
But as a lone bit, I suspect it’s still pretty useless. It’s not like you can publish it.
Without a proof or some indication of the reasoning, it’s not going to advance the field much. (‘not by one bit.’ ha!)
Sometimes brute forcing is just iterating over the answer space and running some process. We can pretend we got a result indicating P=NP and do math from there, if that were useful. Then try the other way around.
A P?=NP solver would need more than one ouput bit, in case it needed to kick out an error, and isn’t that just asking to be run again? Could you not? With that question, any non-answer is a mindhack.
It is difficult to constrain the input we give to the AI, but the output can be constrained severely. A smart guy could wake up alone in a room and infer how he evolved, but so long as his only link to the outside world is a light switch that can only be switched once, there is no risk that he will escape.
A man in a room with a light switch isn’t very useful. An AI can’t optimize over more bits than we allow it as output. If we give a 1 time 32 bit output register then well, we probably could have brute forced it in the first place. If we give it a kilobyte, then it could probably mindhack us.
(And you’re swearing to yourself that you won’t monitor it’s execution? Really? How do you even debug that?)
You have to keep in mind that the point of AI research is to get to something we can let out of the box. If the argument becomes that we can run it on a headless netless 486 which we immediately explode...then yes, you can probably run that. Probably.
Peter de Blanc wrote a post that seems relevant: What Makes a Hint Good?
P ?= NP is one bit. Good luck brute-forcing that.
FAI is harder.
No it’s not. Look at two simpler cases:
Write a chess program that provably makes only legal moves, iterate as desired to improve it. Or,
Write a chess program. Put it in a sandbox so you only ever see it’s moves. Maybe they’re all legal, or maybe they’re not because you’re having it learn the rules with a big neural net or something. At the end of the round of games, the sandbox clears all the memory that held the chess program except for a list of moves in many games. You keep the source. Anything it learned is gone. Iterate as desired to improve it.
If you’re confident you could work out how it was thinking from the source and move list, what if you only got a sequence of wins and non-wins? (An array of bits)
A sequence of wins and non-wins is enough to tell you whether a given approach can result in intelligent behaviour. That alone is enough to make it a useful experiment.
True, as bits go, that would be a doozy.
But as a lone bit, I suspect it’s still pretty useless. It’s not like you can publish it.
Without a proof or some indication of the reasoning, it’s not going to advance the field much. (‘not by one bit.’ ha!)
Sometimes brute forcing is just iterating over the answer space and running some process. We can pretend we got a result indicating P=NP and do math from there, if that were useful. Then try the other way around.
A P?=NP solver would need more than one ouput bit, in case it needed to kick out an error, and isn’t that just asking to be run again? Could you not? With that question, any non-answer is a mindhack.
You just need to hope the room was made by an infallible carpenter and that you never gave the AI access to MacGyver.
Luckily digital constructs are easier to perfect that wooden ones. Although you wouldn’t think so with the current state of most software.