My guess is that Friendliness and the first-person perspective are fundamentally entangled. The launching of FAI wouldn’t seem like some outside process booting up and making the world mysteriously good. It’d feel like your own intelligence, compassion, and awareness expanding, and like you are solving the challenges of the world as you joyfully accept full responsibility for all of existence.
I also suspect that the tendency to avoid or merely footnote this point is a major cause of technical hurdles. There’s a reason self-reference (a la the Löbstacle) keeps showing up: That’s the math reflecting the role of consciousness. Or said the other way, consciousness is the evolved solution to problems like the Löbstacle.
These are guesses & intuitions on my part though. I’m not saying this from a place of technical expertise.
I guess I would put it this way, I don’t see a fundamental difference between a human intelligence executing an aligned-version-of-human-ethics and a machine intelligence executing the same aligned-version-of-human-ethics. Even though there may be significant differences in actual implementation.
I lean in a similar direction.
My guess is that Friendliness and the first-person perspective are fundamentally entangled. The launching of FAI wouldn’t seem like some outside process booting up and making the world mysteriously good. It’d feel like your own intelligence, compassion, and awareness expanding, and like you are solving the challenges of the world as you joyfully accept full responsibility for all of existence.
I also suspect that the tendency to avoid or merely footnote this point is a major cause of technical hurdles. There’s a reason self-reference (a la the Löbstacle) keeps showing up: That’s the math reflecting the role of consciousness. Or said the other way, consciousness is the evolved solution to problems like the Löbstacle.
These are guesses & intuitions on my part though. I’m not saying this from a place of technical expertise.
Hmm.
I guess I would put it this way, I don’t see a fundamental difference between a human intelligence executing an aligned-version-of-human-ethics and a machine intelligence executing the same aligned-version-of-human-ethics. Even though there may be significant differences in actual implementation.