That’s not an accurate parallel. The fact that you thought it was a good idea to use the average human utility function proves that you expect it to have a result almost identical to an FAI using your own utility function. If the average human wants you not to kill the orphans, and you also want not to kill the orphans, it doesn’t matter which algorithm you use to decide not to kill the orphans.
I think that you’re looking too deeply into this; what I’m trying to say is that accepting excuses of the form “I was tempted to do ~x before doing x, so clearly I have properties characteristic of someone who does ~x” is a slippery slope.
If you killed the orphans because otherwise Dr. Evil would have converted the orphans into clones of himself, and taken over the world, then your destruction of the orphanage is more indicative of a desire for Dr. Evil not to take over the world than any opinion on orphanages.
The fact you were tempted not to destroy the orphanage (despite the issue of Dr. Evil) is indicative of the fact you don’t want to kill orphans.
I don’t see how it is slippery at all. Instead, it seems that you have simply jumped off the slope.
If you were tempted to save the orphans you have some properties that lead to not killing orphans. You likely share some properties with compassionate, moral people.
That doesn’t make you compassionate or moral. I’m often tempted to murder people by cutting out their heart and shoving it into their mouth.
This doesn’t make me a murderer, but it does mean I have some properties characteristic of murderers.
“I was tempted not to kill all those orphans, so clearly, I’m a compassionate and moral person.”
That’s not an accurate parallel. The fact that you thought it was a good idea to use the average human utility function proves that you expect it to have a result almost identical to an FAI using your own utility function. If the average human wants you not to kill the orphans, and you also want not to kill the orphans, it doesn’t matter which algorithm you use to decide not to kill the orphans.
I think that you’re looking too deeply into this; what I’m trying to say is that accepting excuses of the form “I was tempted to do ~x before doing x, so clearly I have properties characteristic of someone who does ~x” is a slippery slope.
If you killed the orphans because otherwise Dr. Evil would have converted the orphans into clones of himself, and taken over the world, then your destruction of the orphanage is more indicative of a desire for Dr. Evil not to take over the world than any opinion on orphanages.
The fact you were tempted not to destroy the orphanage (despite the issue of Dr. Evil) is indicative of the fact you don’t want to kill orphans.
I don’t see how it is slippery at all. Instead, it seems that you have simply jumped off the slope.
If you were tempted to save the orphans you have some properties that lead to not killing orphans. You likely share some properties with compassionate, moral people.
That doesn’t make you compassionate or moral. I’m often tempted to murder people by cutting out their heart and shoving it into their mouth.
This doesn’t make me a murderer, but it does mean I have some properties characteristic of murderers.