I think Christiano’s idea sets us on the right path. One of the things that troubled me in reading Bostrom’s book is the prospect that a superintelligence under human control could be at least as dangerous as an autonomous superintelligence. For example, if superintelligence were hatched by one of the competing military superpowers—Russia, China, or the U.S., let’s say—as an oracle or genie, it might be used for short-sighted or selfish purposes with catastrophic consequences. With superintelligence on his side, the cartoon nightmare of a single man out to rule the world could, um, hardly be ruled out. Our knowledge of human nature gives us every reason to expect abuse.
If, however, a superintelligence could be attuned to seeking approval from not just Hugh but humanity, that might have a very different and laudable result. With the proliferation of cell phones through nearly every part of the world (half of the African population is expected to have one by 2017), this opens the possibility that a superintelligence could seek the approval of a broad swath of humanity. Even better would be approval for decisions whose consequences are hidden behind a Rawlsian veil, so that people expressing an opinion cannot know whether they personally will be a winner or loser by the decision. Under such circumstances, people are, on average, remarkably fair. Still, there may be both practical and theoretical shortcomings to my suggestion. I’d welcome critiques.
The obvious problem is lack of human intelligence. Whether a majority of people on Earth will agree with some question the AI will ask them using the cell phone… it will depend on how specifically the question was framed.
Does it invoke popular applause lights? The majority will say “yes”.
Is it complicated? The majority will say: “I do not understand the question.” Now what? Does the majority of those who answered otherwise decide? Then, for sufficiently complicated questions the answers will be decided by people who are unable to see their own lack of understanding.
I think Christiano’s idea sets us on the right path. One of the things that troubled me in reading Bostrom’s book is the prospect that a superintelligence under human control could be at least as dangerous as an autonomous superintelligence. For example, if superintelligence were hatched by one of the competing military superpowers—Russia, China, or the U.S., let’s say—as an oracle or genie, it might be used for short-sighted or selfish purposes with catastrophic consequences. With superintelligence on his side, the cartoon nightmare of a single man out to rule the world could, um, hardly be ruled out. Our knowledge of human nature gives us every reason to expect abuse. If, however, a superintelligence could be attuned to seeking approval from not just Hugh but humanity, that might have a very different and laudable result. With the proliferation of cell phones through nearly every part of the world (half of the African population is expected to have one by 2017), this opens the possibility that a superintelligence could seek the approval of a broad swath of humanity. Even better would be approval for decisions whose consequences are hidden behind a Rawlsian veil, so that people expressing an opinion cannot know whether they personally will be a winner or loser by the decision. Under such circumstances, people are, on average, remarkably fair. Still, there may be both practical and theoretical shortcomings to my suggestion. I’d welcome critiques.
The obvious problem is lack of human intelligence. Whether a majority of people on Earth will agree with some question the AI will ask them using the cell phone… it will depend on how specifically the question was framed.
Does it invoke popular applause lights? The majority will say “yes”.
Is it complicated? The majority will say: “I do not understand the question.” Now what? Does the majority of those who answered otherwise decide? Then, for sufficiently complicated questions the answers will be decided by people who are unable to see their own lack of understanding.
I agree these are problems with asking people questions, but they like much easier problems to solve than most involved with controlling AI.