On point 1- OK, I screwed up slightly. Neither individual is allowed to argue with the other in a manner which the other one would see as brainwashing or unfair manipulation if in possesion of all the facts. The system rules out anything deceptive by correcting both parties on anything that is a question of fact.
On point 2- Then they both argue using their own rules of argument. Presumably, the individual at State 1 is unconvinced.
Neither individual is allowed to argue with the other in a manner which the other one would see as brainwashing or unfair manipulation if in possesion of all the facts.
Presumably this means “all the morally relevant facts,” since giving State 1 “all the facts” would be isomorphic to presenting him with the argument-simulation. But determining all the morally relevant facts is a big part of the problem statement. If the AI could determine which aspects of which actions were morally relevant, and to what the degree and sign of that moral valence was, it wouldn’t need CEV.
We could lock down the argument more, just to be safe.
I’m not sure whether a text-only channel between State 1 and State 7, allowing only if-then type statements with a probability attached, would allow brainwashing or hypnosis. But I’m also not sure how many State 1 racists would be convinced that racism is unethical, over such a channel.
How about the individual versions at State 1 and State 7 both get all the facts that they consider relevant themselves? And maybe a State 1 racist really wouldn’t have CEV towards non-racism- we just have to accept that.
On point 1- OK, I screwed up slightly. Neither individual is allowed to argue with the other in a manner which the other one would see as brainwashing or unfair manipulation if in possesion of all the facts. The system rules out anything deceptive by correcting both parties on anything that is a question of fact. On point 2- Then they both argue using their own rules of argument. Presumably, the individual at State 1 is unconvinced.
Presumably this means “all the morally relevant facts,” since giving State 1 “all the facts” would be isomorphic to presenting him with the argument-simulation. But determining all the morally relevant facts is a big part of the problem statement. If the AI could determine which aspects of which actions were morally relevant, and to what the degree and sign of that moral valence was, it wouldn’t need CEV.
We could lock down the argument more, just to be safe.
I’m not sure whether a text-only channel between State 1 and State 7, allowing only if-then type statements with a probability attached, would allow brainwashing or hypnosis. But I’m also not sure how many State 1 racists would be convinced that racism is unethical, over such a channel.
How about the individual versions at State 1 and State 7 both get all the facts that they consider relevant themselves? And maybe a State 1 racist really wouldn’t have CEV towards non-racism- we just have to accept that.