Minimal, very imprecise description of Friendly AI:
“A Friendly AI is one that does what we ought to want it to do.”
Unfortunately, we humans have enough trouble figuring out what we do want, let alone what we should want. Hence, even if we could make a Really Powerful Optimization Process that was better than people at achieving goals, we don’t know what goals to make it try to achieve.
Minimal, very imprecise description of Friendly AI:
“A Friendly AI is one that does what we ought to want it to do.”
Unfortunately, we humans have enough trouble figuring out what we do want, let alone what we should want. Hence, even if we could make a Really Powerful Optimization Process that was better than people at achieving goals, we don’t know what goals to make it try to achieve.