I have misgivings about using high level concepts to constrain an AI (be it friendliness or approval). I suspect we may well not share many concepts at all unless there is some form of lower level constraint system that makes our ontologies similar. If we must program the ontology in and it is not capable of drift, I have doubts it will be able to come up with vastly novel ways of seeing the world, limiting its potential power.
My favourite question is why build systems that are separate from us anyway? Or to put another way, how can we build a computational system that interacts with our brains as if it was part of us. Assume that we are multi-‘sort of agent’ systems that (mostly) pull in the same direction, how can we get computers to be part of that system.
I think some of the ideas of approval directed agents might be relevant, I suspect parts of our brain monitoring other parts and giving approval of their actions is part of the reason for consciousness (and also the dopamine system).
I have misgivings about using high level concepts to constrain an AI (be it friendliness or approval). I suspect we may well not share many concepts at all unless there is some form of lower level constraint system that makes our ontologies similar. If we must program the ontology in and it is not capable of drift, I have doubts it will be able to come up with vastly novel ways of seeing the world, limiting its potential power.
My favourite question is why build systems that are separate from us anyway? Or to put another way, how can we build a computational system that interacts with our brains as if it was part of us. Assume that we are multi-‘sort of agent’ systems that (mostly) pull in the same direction, how can we get computers to be part of that system.
I think some of the ideas of approval directed agents might be relevant, I suspect parts of our brain monitoring other parts and giving approval of their actions is part of the reason for consciousness (and also the dopamine system).