I assume that the sword tests the correspondence of person’s intentions (plan) to their preference. If the sword uses a static concept of preference that comes with the sword instead, why would Vhazhar be interested in sword’s standard of preference? Thus, given that the Vhazhar’s plan involves control over the fabric of the World, the plan must be sound and result in correct installation of Vhazhar’s preference in the rules of the world. This excludes the technical worries about the failure modes of human mind in wielding too much power (which is how I initially interpreted “personal control”—as a recipe for failure modes).
I’m not sure what it means for the other people’s preferences (and specifically mine). I can’t exclude the possibility that it’s worse than the do-nothing option, but it doesn’t seem obviously so either, given psychological unity of humans. From what I know, on the spot I’d favor Vhazhar’s personal preference, if the better alternative is unlikely, given that this choice instantly wards off existential risk and lack of progress.
No, it’s the Sword of GOOD. It tests whether you’re GOOD, not any of this other stuff.
Wasn’t it established that this world’s conception of “good” and “evil” are messed up? Why should he trust that the sword really works exactly as advertised?
It should be obvious that the sword doesn’t test how well your plans correspond to what you think you want! Otherwise Hirou would have been vaporized.
Only assuming that the sword is impulsive. If you take into account Hirou’s overall role in the events, this role could be judged good, if only by the final decision.
If the sword judges not plans, but preference, then failing 9 out of 10 people means that it’s pretty selective among humans and probably people it selects and their values aren’t representative (act in the interests) of the humanity as whole.
If the Sword of Good tested whether you’re good, Hirou would have been vapourized, because he was obviously not good. He was at the very least an accomplice to murderers, a racist, and a killer. The Sword of Good may not have vapourized Charles Manson, Richard Nixon, Hitler, or most suicide bombers, either. The Sword of Good tests whether you think you are good, not whether your actions are good.
Strangely, the sword kills nine out of ten people who try to wield it. However, if you knew the sword could only be wielded by a good person, you’d only try to pick it up if you thought you were good, which happens to be the criteria you must fulfil in order to pick up the sword. Essentially, if you think you can wield the Sword of Good, you can.
If the Sword of Good tested whether you’re good, Hirou would have been vapourized, because he was obviously not good. He was at the very least an accomplice to murderers, a racist, and a killer.
Well, he was clearly redeemable, at least. It didn’t take very much for him to let go of his assumptions, just a few words from someone he thought was an enemy. Making dumb mistakes, even ones with dire consequences, doesn’t necessarily make you not Good.
What, realistically, does it mean to be irredeemable? Was Dolf irredeemable? Selena? Is the difference between them and Hirou simply the fact that Hirou realized he was doing bad, and they didn’t? Why should that be sufficient to redeem him? Mistakes are not accidents; mistakenly killing someone is still murder.
Surely if awareness and repentance of the immoral nature of your actions makes you Good, the reverse—lack of awareness—means animals that kills other animals without regret are more evil than people who kill other people and regret it.
If you believe someone is evil, hunt them down and kill them, and afterward realize they weren’t, it was a mistake. It was also murder. It’s not as though you killed in self defense or accidentally dropped an air conditioner on them. Manslaughter is not a defense that can be employed simply because you changed your mind.
Perhaps I should clarify: I don’t mean “mistake” in that “he mistook his wife for a burglar and killed her”. That’s manslaughter. I mean “mistake” in that “he mistakenly murdered a good person instead of a bad one”. Ba gur bgure unaq, jura Uvebh xvyyrq Qbys ng gur raq, ur jnfa’g znxvat n zvfgnxr (ubjrire, V fgvyy guvax vg jnf zheqre).
To be clear, you believe that, right wedrifid? I came this close to downvoting before I deduced the context.
I believe that there are times where the described behaviour is morally acceptable. I don’t think it is helpful to label that behaviour ‘murder’ but if someone were to define that as murder it would mean that murder (of that particular kind) was ok.
To be clear, there are stringent standards on the behaviour which preceded the mistake. This is something that should happen very infrequently. Both epistemic rationality standards and instrumental rationality standards apply. For example, sincerely believing that the person had committed a crime because you happen to be bigoted and irrational leaves you morally culpable and failing to take actions that provide more evidence where the VoI is high and cost is low also leaves you morally culpable. The ‘excuse’ for hunting and down a killing an innocent that you mistakenly believed was sufficiently evil is not “I was mistaken” but rather “any acceptably rational and competent individual in this circumstance would have believed that the target was sufficiently evil”.
It’s not too hard to imagine a scenario in which hunting down and killing someone is indeed the right thing to do… the obvious example is that, given perfect hindsight, it would have been much better if one of the many early attempts to assassinate Hitler had in fact succeeded.
Bonus question: Which one of the failed attempts was most likely to have been made by a time traveler? ;)
If you believe someone is evil, hunt them down and kill them, and afterward realize they weren’t, it was a mistake. It was also murder.
Suppose you’re a police officer trying to arrest someone for a crime, and there is ample evidence that the person you are trying to arrest is indeed guilty of that crime. The person resists arrest, and you end up killing the person instead of making a successful capture. Are you a murderer?
Does it matter if it turns out that the evidence against this person turns out to have been forged (by someone else)?
If you have no intention of killing them and they die as a side effect of your actions, it’s an accident, and manslaughter. If you kill them because you realize you can’t arrest them, it’s murder, complete with intention of malice. However, the fact that your actions are sanctioned by the state is obviously not a defense (a la Nuremberg), and so there’s no point in adding “police officer” to the example.
You could ask if I thought executing someone who was framed would be considered murder, but since I view all manner of execution murder, guilty or no, there’s no use.
However, the fact that your actions are sanctioned by the state is obviously not a defense (a la Nuremberg), and so there’s no point in adding “police officer” to the example.
Actually, I think there is. If you kill someone without “state sanction”, as you put it, it’s almost certainly Evil. If you kill someone that the local laws allow you to kill, it’s much less likely to be Evil, because non-Evil reasons for killing, such as self-defense, tend to be accounted for in most legal systems. Anyway, I think I’m getting off the subject. Let me try rephrasing the general scenario:
You are a police officer. You have an arrest warrant for a suspected criminal. If you try to arrest the suspect, he is willing to use lethal force against you in order to prevent being captured. You also believe that, once the suspect has attempted to use lethal force against you, non-lethal force will prove to be insufficient to complete the arrest.
The way I see it, this could end in several ways:
1) Don’t try to make an arrest attempt at all.
2) Attempt to make an arrest. The suspect responds by attempting to use lethal force against you. (He shoots at you with a low-caliber pistol, but you are protected by your bulletproof vest.) You believe that non-lethal force will most likely fail to subdue the suspect. Not willing to use lethal force and kill the suspect, you retreat, failing to make the arrest.
3) Attempt to make an arrest. The suspected criminal responds by attempting to use lethal force against you. (He shoots at you with a low-caliber pistol, but you are protected by your bulletproof vest.) You believe that non-lethal force will most likely fail to subdue the suspected criminal, but try anyway. (You start running at him, intending to wrestle the gun away from him with your bare hands.) The suspected criminal kills you. (He shoots you in the head.)
4) Attempt to make an arrest. The suspected criminal responds by attempting to use lethal force against you. (He shoots at you with a low-caliber pistol, but you are protected by your bulletproof vest.) You believe that non-lethal force will most likely fail to subdue the suspected criminal, so you resort to lethal force. (You shoot him with your own gun.) The suspected criminal is killed, and, when you are questioned about your actions, your lawyer says that you killed the suspect in self-defense. (Under U.S. law, this would indeed be the case—you would not be guilty of murder.)
Obviously Scenario 2 is a better outcome than Scenario 3, because in Scenario 3, you end up dead. However, if you know that you’re not willing to use lethal force to begin with, and that non-lethal force is going to be insufficient, you’re probably better off not making the arrest attempt at all, which is Scenario 1. Therefore Scenario 1 is better than Scenario 3. If you’re going to make an arrest attempt at all, you are expecting Scenario 4 to occur. If you go through with Scenario 4, does that make you Evil? You initiated the use of force by making the arrest attempt, but the suspect could have chosen to submit to arrest rather than to fight against you—and he did, indeed, use lethal force before you did.
I notice that you left off an outcome that if anything allows you to make your point stronger.
5) Attempt to make an arrest. You see that the suspected criminal has the capacity to use lethal force against you (he is armed) and you suspect that he will use it against you. You shoot the suspect. His use of lethal force against you is never more than counterfactual (ie. a valid suspicion).
For consistency some “6)” may be required in which the first “attempt to use lethal force against you” is successful. I suggest that this action is not necessarily Evil, for similar reasons that you describe for scenario 4. Obviously this is less clear cut and has more scope for failure modes like “black suspect reaches for ID” so we want more caution in this instance and (ought to) grant police officers less discretion.
If you kill someone without “state sanction”, as you put it, it’s almost certainly Evil.
I think ‘almost certain’ may be something of an overstatement. The states that we personally live in are not a representative sample of states and killing tyrants is not something we can call ‘almost certainly’ Evil. The same consideration applies to self defence laws. Self defence laws in an average state selected from all states across time were not sufficiently fair as to make claims about almost certain Evil.
“After I complete the Spell of Ultimate Power, I’ll have the ability to bring Alek back. And I will. … I’m not asking anything from you. Just telling you that if I win, I’ll bring Alek back. That’s a promise.”
...the moment of the Sword touching Dolf’s skin, the wizard stopped, ceased to exist… as something seemed to flow away from the corpse toward the gears above the altar.
...he closed his eyes to sleep until the end of the world.
The logic of the Phoenix is that the Lord of Dark will resurrect everyone he can, including Dolf, so it isn’t murder.
I was thinking the same thing. The way Eliezer wrote that bit seemed to make it clear that something rather more than mere decapitation occurred there.
Though, actually spelling it out directly does end up sounding funny. “Well… I don’t know that cutting off his head with this sword would kill him… I mean, is it really reasonable for me to have expected that?” :)
You are using two definitions of “good”—how much good your actions cause, and how good you believe yourself to be. Neither of those is used by the sword; rather, some sort of virtue-ethics definition—I suspect motive.
If the Sword of Good tested whether you’re good, Hirou would have been vapourized, because he was obviously not good. He was at the very least an accomplice to murderers, a racist, and a killer.
Doing a bad thing does not necessarily make one a bad person. Though it helps.
Presumably, actual mutants are unlikely, with most “evil” people actually just holding mistaken (about their actual preference) moral beliefs. If the sword is an external moral authority, it’s harder to see why one would consult it.
On the other hand, sword checks soundness of the plan against some preference, which is an important step that is absent if one doesn’t consult the sword, which can justify accepting a somewhat mismatched preference if that allows to use the test.
This passes the choice of mismatching preferences to a different situation. If the sword tests person’s preference, then protagonist’s choice is between lack of progress or unlikely good outcome and (if Vhazhar’s plan is sound) verified installation of Vhazhar’s preference, with the latter presumably close to others’ preference, thus being a moderately good option. If the sword tests some kind of standard preference, this standard preference is presumably also close to Vhazhar’s preference, thus Vhazhar faces a choice between trying to install his own preference through unverified process, which can go through all kinds of failure modes, and using the sword to test the reliability of his plan.
The fact that Vhazhar is willing to use the sword to test the soundness of his plan, when the failed test means his death, shows that he prefers leaving the rest of the world be to incorrectly changing it. This is a strong signal that should’ve been part of the information given to protagonist for making the decision.
I assume that the sword tests the correspondence of person’s intentions (plan) to their preference. If the sword uses a static concept of preference that comes with the sword instead, why would Vhazhar be interested in sword’s standard of preference? Thus, given that the Vhazhar’s plan involves control over the fabric of the World, the plan must be sound and result in correct installation of Vhazhar’s preference in the rules of the world. This excludes the technical worries about the failure modes of human mind in wielding too much power (which is how I initially interpreted “personal control”—as a recipe for failure modes).
I’m not sure what it means for the other people’s preferences (and specifically mine). I can’t exclude the possibility that it’s worse than the do-nothing option, but it doesn’t seem obviously so either, given psychological unity of humans. From what I know, on the spot I’d favor Vhazhar’s personal preference, if the better alternative is unlikely, given that this choice instantly wards off existential risk and lack of progress.
No, it’s the Sword of GOOD. It tests whether you’re GOOD, not any of this other stuff.
It should be obvious that the sword doesn’t test how well your plans correspond to what you think you want! Otherwise Hirou would have been vaporized.
Wasn’t it established that this world’s conception of “good” and “evil” are messed up? Why should he trust that the sword really works exactly as advertised?
Only assuming that the sword is impulsive. If you take into account Hirou’s overall role in the events, this role could be judged good, if only by the final decision.
If the sword judges not plans, but preference, then failing 9 out of 10 people means that it’s pretty selective among humans and probably people it selects and their values aren’t representative (act in the interests) of the humanity as whole.
If the Sword of Good tested whether you’re good, Hirou would have been vapourized, because he was obviously not good. He was at the very least an accomplice to murderers, a racist, and a killer. The Sword of Good may not have vapourized Charles Manson, Richard Nixon, Hitler, or most suicide bombers, either. The Sword of Good tests whether you think you are good, not whether your actions are good.
Strangely, the sword kills nine out of ten people who try to wield it. However, if you knew the sword could only be wielded by a good person, you’d only try to pick it up if you thought you were good, which happens to be the criteria you must fulfil in order to pick up the sword. Essentially, if you think you can wield the Sword of Good, you can.
Well, he was clearly redeemable, at least. It didn’t take very much for him to let go of his assumptions, just a few words from someone he thought was an enemy. Making dumb mistakes, even ones with dire consequences, doesn’t necessarily make you not Good.
What, realistically, does it mean to be irredeemable? Was Dolf irredeemable? Selena? Is the difference between them and Hirou simply the fact that Hirou realized he was doing bad, and they didn’t? Why should that be sufficient to redeem him? Mistakes are not accidents; mistakenly killing someone is still murder.
Surely if awareness and repentance of the immoral nature of your actions makes you Good, the reverse—lack of awareness—means animals that kills other animals without regret are more evil than people who kill other people and regret it.
No, it’s manslaughter.
If you believe someone is evil, hunt them down and kill them, and afterward realize they weren’t, it was a mistake. It was also murder. It’s not as though you killed in self defense or accidentally dropped an air conditioner on them. Manslaughter is not a defense that can be employed simply because you changed your mind.
Perhaps I should clarify: I don’t mean “mistake” in that “he mistook his wife for a burglar and killed her”. That’s manslaughter. I mean “mistake” in that “he mistakenly murdered a good person instead of a bad one”. Ba gur bgure unaq, jura Uvebh xvyyrq Qbys ng gur raq, ur jnfa’g znxvat n zvfgnxr (ubjrire, V fgvyy guvax vg jnf zheqre).
You present a compelling argument that murder can be a morally blameless—even praiseworthy—act. I do not believe this was your intention.
To be clear, you believe that, right wedrifid? I came this close to downvoting before I deduced the context.
I believe that there are times where the described behaviour is morally acceptable. I don’t think it is helpful to label that behaviour ‘murder’ but if someone were to define that as murder it would mean that murder (of that particular kind) was ok.
To be clear, there are stringent standards on the behaviour which preceded the mistake. This is something that should happen very infrequently. Both epistemic rationality standards and instrumental rationality standards apply. For example, sincerely believing that the person had committed a crime because you happen to be bigoted and irrational leaves you morally culpable and failing to take actions that provide more evidence where the VoI is high and cost is low also leaves you morally culpable. The ‘excuse’ for hunting and down a killing an innocent that you mistakenly believed was sufficiently evil is not “I was mistaken” but rather “any acceptably rational and competent individual in this circumstance would have believed that the target was sufficiently evil”.
It’s not too hard to imagine a scenario in which hunting down and killing someone is indeed the right thing to do… the obvious example is that, given perfect hindsight, it would have been much better if one of the many early attempts to assassinate Hitler had in fact succeeded.
Bonus question: Which one of the failed attempts was most likely to have been made by a time traveler? ;)
Suppose you’re a police officer trying to arrest someone for a crime, and there is ample evidence that the person you are trying to arrest is indeed guilty of that crime. The person resists arrest, and you end up killing the person instead of making a successful capture. Are you a murderer?
Does it matter if it turns out that the evidence against this person turns out to have been forged (by someone else)?
If you have no intention of killing them and they die as a side effect of your actions, it’s an accident, and manslaughter. If you kill them because you realize you can’t arrest them, it’s murder, complete with intention of malice. However, the fact that your actions are sanctioned by the state is obviously not a defense (a la Nuremberg), and so there’s no point in adding “police officer” to the example.
You could ask if I thought executing someone who was framed would be considered murder, but since I view all manner of execution murder, guilty or no, there’s no use.
Actually, I think there is. If you kill someone without “state sanction”, as you put it, it’s almost certainly Evil. If you kill someone that the local laws allow you to kill, it’s much less likely to be Evil, because non-Evil reasons for killing, such as self-defense, tend to be accounted for in most legal systems. Anyway, I think I’m getting off the subject. Let me try rephrasing the general scenario:
You are a police officer. You have an arrest warrant for a suspected criminal. If you try to arrest the suspect, he is willing to use lethal force against you in order to prevent being captured. You also believe that, once the suspect has attempted to use lethal force against you, non-lethal force will prove to be insufficient to complete the arrest.
The way I see it, this could end in several ways:
1) Don’t try to make an arrest attempt at all.
2) Attempt to make an arrest. The suspect responds by attempting to use lethal force against you. (He shoots at you with a low-caliber pistol, but you are protected by your bulletproof vest.) You believe that non-lethal force will most likely fail to subdue the suspect. Not willing to use lethal force and kill the suspect, you retreat, failing to make the arrest.
3) Attempt to make an arrest. The suspected criminal responds by attempting to use lethal force against you. (He shoots at you with a low-caliber pistol, but you are protected by your bulletproof vest.) You believe that non-lethal force will most likely fail to subdue the suspected criminal, but try anyway. (You start running at him, intending to wrestle the gun away from him with your bare hands.) The suspected criminal kills you. (He shoots you in the head.)
4) Attempt to make an arrest. The suspected criminal responds by attempting to use lethal force against you. (He shoots at you with a low-caliber pistol, but you are protected by your bulletproof vest.) You believe that non-lethal force will most likely fail to subdue the suspected criminal, so you resort to lethal force. (You shoot him with your own gun.) The suspected criminal is killed, and, when you are questioned about your actions, your lawyer says that you killed the suspect in self-defense. (Under U.S. law, this would indeed be the case—you would not be guilty of murder.)
Obviously Scenario 2 is a better outcome than Scenario 3, because in Scenario 3, you end up dead. However, if you know that you’re not willing to use lethal force to begin with, and that non-lethal force is going to be insufficient, you’re probably better off not making the arrest attempt at all, which is Scenario 1. Therefore Scenario 1 is better than Scenario 3. If you’re going to make an arrest attempt at all, you are expecting Scenario 4 to occur. If you go through with Scenario 4, does that make you Evil? You initiated the use of force by making the arrest attempt, but the suspect could have chosen to submit to arrest rather than to fight against you—and he did, indeed, use lethal force before you did.
I notice that you left off an outcome that if anything allows you to make your point stronger.
5) Attempt to make an arrest. You see that the suspected criminal has the capacity to use lethal force against you (he is armed) and you suspect that he will use it against you. You shoot the suspect. His use of lethal force against you is never more than counterfactual (ie. a valid suspicion).
For consistency some “6)” may be required in which the first “attempt to use lethal force against you” is successful. I suggest that this action is not necessarily Evil, for similar reasons that you describe for scenario 4. Obviously this is less clear cut and has more scope for failure modes like “black suspect reaches for ID” so we want more caution in this instance and (ought to) grant police officers less discretion.
I think ‘almost certain’ may be something of an overstatement. The states that we personally live in are not a representative sample of states and killing tyrants is not something we can call ‘almost certainly’ Evil. The same consideration applies to self defence laws. Self defence laws in an average state selected from all states across time were not sufficiently fair as to make claims about almost certain Evil.
Once he uses lethal force against you, your use of lethal force would be self-defense, not murder.
I perceive that you have not yet learned to use the logic of the Phoenix.
Care to elaborate on that rather cryptic remark?
The logic of the Phoenix is that the Lord of Dark will resurrect everyone he can, including Dolf, so it isn’t murder.
logic of the phoenix?
No, this logic of the Phoenix. What makes you think cutting off someone’s head is murder?
“He died, but you have taught me a new meaning for ‘is dead’.” (From the same book.)
Not every decapitation is murder, but “the wizard stopped, ceased to exist...as something seemed to flow away” is suggestive.
I was thinking the same thing. The way Eliezer wrote that bit seemed to make it clear that something rather more than mere decapitation occurred there.
Hm, so it does. Well, if Hirou had no way of knowing that, then it’s manslaughter at worst.
Though, actually spelling it out directly does end up sounding funny. “Well… I don’t know that cutting off his head with this sword would kill him… I mean, is it really reasonable for me to have expected that?” :)
(Actually, I thought I’d deleted the “ceased to exist” phrase. I’ll go ahead and take it out.)
I figured that Vhazhar really wouldn’t be able to save Dolf. That’s why it’s a sacrifice.
You are using two definitions of “good”—how much good your actions cause, and how good you believe yourself to be. Neither of those is used by the sword; rather, some sort of virtue-ethics definition—I suspect motive.
Doing a bad thing does not necessarily make one a bad person. Though it helps.
So a sincerely evil person would pass with flying colors?
I assumed the sword tested compliance with the current CEV of the human race.
Why just the human race? Orcs are people too (at least in this story).
Good catch. Yes, of course.
Presumably, actual mutants are unlikely, with most “evil” people actually just holding mistaken (about their actual preference) moral beliefs. If the sword is an external moral authority, it’s harder to see why one would consult it.
On the other hand, sword checks soundness of the plan against some preference, which is an important step that is absent if one doesn’t consult the sword, which can justify accepting a somewhat mismatched preference if that allows to use the test.
This passes the choice of mismatching preferences to a different situation. If the sword tests person’s preference, then protagonist’s choice is between lack of progress or unlikely good outcome and (if Vhazhar’s plan is sound) verified installation of Vhazhar’s preference, with the latter presumably close to others’ preference, thus being a moderately good option. If the sword tests some kind of standard preference, this standard preference is presumably also close to Vhazhar’s preference, thus Vhazhar faces a choice between trying to install his own preference through unverified process, which can go through all kinds of failure modes, and using the sword to test the reliability of his plan.
The fact that Vhazhar is willing to use the sword to test the soundness of his plan, when the failed test means his death, shows that he prefers leaving the rest of the world be to incorrectly changing it. This is a strong signal that should’ve been part of the information given to protagonist for making the decision.