No, because they each have their own incompatible definitions of good. A conversation beforehand is only helpful if they have a means of enforcing agreements.
If they have limited information on the good, wouldn’t a conversation invoke a kind of ethical Aumann’s Agreement Theorem?
In general, if everyone agrees about some morality and disagrees about what it entails, that’s a disagreement over facts, and confusion over facts will cause problems in any decision theory.
wouldn’t a conversation invoke a kind of ethical Aumann’s Agreement Theorem?
Yes, if there is time for a polite conversation before making an ethical decision. Too bad that the manufacturers of trolley problems usually don’t allow enough time for idle chit-chat.
Still, it is an interesting conjecture. The eAAT conjecture. Can we find a proof? A counter-example?
Here is an attempt at a counter-example. I strongly prefer to keep my sexual orientation secret from you. You only mildly prefer to know my sexual orientation. Thus, it might seem that my orientation should remain secret. But then we risk that I will receive inappropriate birthday gifts from you. Or, what if I prefer to keep secret the fact that I have been diagnosed with an incurable fatal disease? What if I wish to keep this a secret only to spare your feelings?
Of course, we can avoid this kind of problem by supplementing our utility maximization principle with a second moral axiom—No Secrets. Can we add this axiom and still call ourselves pure utilitarians? Can we be mathematically consistent utilitarians without this axiom? I’ll leave this debate to others.
It is an interesting exercise, though, to revisit the von Neumann/Savage/Aumann-Anscombe algorithms for constructing utility functions when agents are allowed to keep some of their preferences secret. Agents still would know their own utilities exactly, but would only have a range (or a pdf?) for the utilities of other agents. It might be illuminating to reconstruct game theory and utilitarian ethics incorporating this twist.
The TDT users sees the problem as being that if he fights for a cause, that others may also fight for some less-important cause that they think is more important, leading to both causes being harmed. He responds by reducing his willingness to fight.
Someone who is morally uncertain (because he’s not omniscient) realizes that the cause he is fighting for might not be the most important one, and that other’s causes may actually be correct, which should reduce his willingness to fight by the same amount.
If we assume that all agents believe in the same complicated process for calculating the utilities, but are unsure how it works out in practice, then what they lack is a totally physical knowledge that should follow all the agreement theorems. If agent’s extrapolated volitions are not coherent, this is false.
Really? Is there a single good value in the universe? Happiness, comfort, fun, freedom, you can’t even conceive someone who weighs the worth of these values slightly differently than someone else and yet both can remain non-evil?
Fair point. If they’re slightly different, it should be a slight problem, and TDT would help that. If they’re significantly different, it would be a significant problem, and you might be able to make a case that one is evil.
If you can call someone “evil” even though they may altruistically work for the increase of the well-being of others, as they perceive it to be, then what’s the word you’d use to describe people who are sadists and actively seek to hurt others, or people who would sacrifice the wellbeing of millions people for their own selfish benefit?
Your labelling scheme doesn’t serve me in treating people appropriately, realizing which people I ought consider enemies and which I ought treat as potential allies—nor which people strive to increase total (or average) utility and which people strive to decrease it.
So what’s its point? Why consider these people “evil”? It almost seems to me as if you’re working backwards from a conclusion, starting with the assumption that all good people must have the same goals, and therefore someone who differs must be evil.
It depends on if you interpret “good” and “evil” as words derived from “should,” as I was doing. Good people are those that act as they should behave, and evil people as those that act as they shouldn’t behave. There is only one right thing to do.
But if you want to define evil another way, honestly, you’re probably right. I would note that I think “might be able to make the case that” is enough qualification.
So, more clearly:
If everyone’s extrapolated values are in accordance with my extrapolated values, information is our only problem, which we don’t need moral and decision theories to deal with.
If our extrapolated values differ, then they may differ a bit, in which case we have a small problem, or a medium amount, in which case there’s a big problem, or a lot, in which case there’s a huge problem. I can rate them on a continuous scale as to how well they accord with my extrapolated values. The ones at the top, I can work with, and those at the bottom, I can work against. However TDT states that we should be nicer to those at the bottom so that they’ll be nicer than us, whereas CDT does not, and therein lies the difference.
No, because they each have their own incompatible definitions of good. A conversation beforehand is only helpful if they have a means of enforcing agreements.
If everyone’s an altruistic consequentialist, they have the same definition of good. If not, they’re evil.
If everyone is an omniscient altruistic consequentialist, that is.
If they have limited information on the good, wouldn’t a conversation invoke a kind of ethical Aumann’s Agreement Theorem?
In general, if everyone agrees about some morality and disagrees about what it entails, that’s a disagreement over facts, and confusion over facts will cause problems in any decision theory.
Yes, if there is time for a polite conversation before making an ethical decision. Too bad that the manufacturers of trolley problems usually don’t allow enough time for idle chit-chat.
Still, it is an interesting conjecture. The eAAT conjecture. Can we find a proof? A counter-example?
Here is an attempt at a counter-example. I strongly prefer to keep my sexual orientation secret from you. You only mildly prefer to know my sexual orientation. Thus, it might seem that my orientation should remain secret. But then we risk that I will receive inappropriate birthday gifts from you. Or, what if I prefer to keep secret the fact that I have been diagnosed with an incurable fatal disease? What if I wish to keep this a secret only to spare your feelings?
Of course, we can avoid this kind of problem by supplementing our utility maximization principle with a second moral axiom—No Secrets. Can we add this axiom and still call ourselves pure utilitarians? Can we be mathematically consistent utilitarians without this axiom? I’ll leave this debate to others.
It is an interesting exercise, though, to revisit the von Neumann/Savage/Aumann-Anscombe algorithms for constructing utility functions when agents are allowed to keep some of their preferences secret. Agents still would know their own utilities exactly, but would only have a range (or a pdf?) for the utilities of other agents. It might be illuminating to reconstruct game theory and utilitarian ethics incorporating this twist.
The TDT users sees the problem as being that if he fights for a cause, that others may also fight for some less-important cause that they think is more important, leading to both causes being harmed. He responds by reducing his willingness to fight.
Someone who is morally uncertain (because he’s not omniscient) realizes that the cause he is fighting for might not be the most important one, and that other’s causes may actually be correct, which should reduce his willingness to fight by the same amount.
If we assume that all agents believe in the same complicated process for calculating the utilities, but are unsure how it works out in practice, then what they lack is a totally physical knowledge that should follow all the agreement theorems. If agent’s extrapolated volitions are not coherent, this is false.
Really? Is there a single good value in the universe? Happiness, comfort, fun, freedom, you can’t even conceive someone who weighs the worth of these values slightly differently than someone else and yet both can remain non-evil?
Fair point. If they’re slightly different, it should be a slight problem, and TDT would help that. If they’re significantly different, it would be a significant problem, and you might be able to make a case that one is evil.
If you can call someone “evil” even though they may altruistically work for the increase of the well-being of others, as they perceive it to be, then what’s the word you’d use to describe people who are sadists and actively seek to hurt others, or people who would sacrifice the wellbeing of millions people for their own selfish benefit?
Your labelling scheme doesn’t serve me in treating people appropriately, realizing which people I ought consider enemies and which I ought treat as potential allies—nor which people strive to increase total (or average) utility and which people strive to decrease it.
So what’s its point? Why consider these people “evil”? It almost seems to me as if you’re working backwards from a conclusion, starting with the assumption that all good people must have the same goals, and therefore someone who differs must be evil.
It depends on if you interpret “good” and “evil” as words derived from “should,” as I was doing. Good people are those that act as they should behave, and evil people as those that act as they shouldn’t behave. There is only one right thing to do.
But if you want to define evil another way, honestly, you’re probably right. I would note that I think “might be able to make the case that” is enough qualification.
So, more clearly:
If everyone’s extrapolated values are in accordance with my extrapolated values, information is our only problem, which we don’t need moral and decision theories to deal with.
If our extrapolated values differ, then they may differ a bit, in which case we have a small problem, or a medium amount, in which case there’s a big problem, or a lot, in which case there’s a huge problem. I can rate them on a continuous scale as to how well they accord with my extrapolated values. The ones at the top, I can work with, and those at the bottom, I can work against. However TDT states that we should be nicer to those at the bottom so that they’ll be nicer than us, whereas CDT does not, and therein lies the difference.