in a society of Artificial Intelligences worthy of personhood and lacking any inbuilt tendency to be corrupted by power, it would be right for the AI to murder … I refuse to extend this reply to myself, because the epistemological state you ask me to imagine, can only exist among other kinds of people than human beings.
Interesting reply. But the AIs are programmed by corrupted humans. Do you really expect to be able to check the full source code? That you can outsmart the people who win obfuscated code contests?
How is the epistemological state of human-verified, human-built, non-corrupt AIs, any more possible?
We’re likely to insert our faulty cached wisdom deliberately. We’re unlikely to insert our power-corrupts biases deliberately. We might insert something vaguely analogous accidentally, though.
As for obfuscated source code—we would want programmatic verification of correctness, which would be another huge undertaking on top of solving the AI and FAI problems. Obfuscation doesn’t help you there.
Interesting reply. But the AIs are programmed by corrupted humans. Do you really expect to be able to check the full source code? That you can outsmart the people who win obfuscated code contests?
How is the epistemological state of human-verified, human-built, non-corrupt AIs, any more possible?
We’re likely to insert our faulty cached wisdom deliberately. We’re unlikely to insert our power-corrupts biases deliberately. We might insert something vaguely analogous accidentally, though.
As for obfuscated source code—we would want programmatic verification of correctness, which would be another huge undertaking on top of solving the AI and FAI problems. Obfuscation doesn’t help you there.