I think we have a very long track record of embedding our values into law.
I mean you could say that if we haven’t figured out how to do it well in the last 10,000 years, maybe don’t plan on doing it in the next 10. That’s kind of being mean though.
If you have a functioning arbitration process, can’t you just say “don’t do bad things” and leave everything down to the arbitration?
I also kind of feel that adding laws is going in the direction of more complexity. And we really want as simple as possible. (Ie the minimal AI that can sit in a MIRI basement and help them figure out the rest of AI theory or something)
If the human still wants to proceed, they can try to:
I was talking about a scenario where the human has never imagined the possibility, and asking if the AI mentions the possibility to the human (knowing the human may change the law to get it)
The human says “cure my cancer”. The AI reasons that it can
Tell the human of a drug that cures its cancer in the conventional sense.
Tell the human about mind uploading, never mentioning the chemical cure.
If the AI picks 2, the human will change the “law” (which isn’t the actual law, its just some text file the AI wants to obey). Then the AI can upload the human and the human will have a life the AI judges as overall better for them.
You don’t want the AI to never mention a really good idea because it happens to be illegal on a technicality. You also don’t want all the plans to be “persuade humans to make everything legal, then …”
I mean you could say that if we haven’t figured out how to do it well in the last 10,000 years, maybe don’t plan on doing it in the next 10. That’s kind of being mean though.
If you have a functioning arbitration process, can’t you just say “don’t do bad things” and leave everything down to the arbitration?
I also kind of feel that adding laws is going in the direction of more complexity. And we really want as simple as possible. (Ie the minimal AI that can sit in a MIRI basement and help them figure out the rest of AI theory or something)
I was talking about a scenario where the human has never imagined the possibility, and asking if the AI mentions the possibility to the human (knowing the human may change the law to get it)
The human says “cure my cancer”. The AI reasons that it can
Tell the human of a drug that cures its cancer in the conventional sense.
Tell the human about mind uploading, never mentioning the chemical cure.
If the AI picks 2, the human will change the “law” (which isn’t the actual law, its just some text file the AI wants to obey). Then the AI can upload the human and the human will have a life the AI judges as overall better for them.
You don’t want the AI to never mention a really good idea because it happens to be illegal on a technicality. You also don’t want all the plans to be “persuade humans to make everything legal, then …”