Well, he didn’t actually identify dust mote disutility as zero; he says that dust motes register as zero on his torture scale. He goes on to mention that torture isn’t on his dust-mote scale, so he isn’t just using “torture scale” as a synonym for “disutility scale”; rather, he is emphasizing that there is more than just a single “(dis)utility scale” involved. I believe his contention is that the events (torture and dust-mote-in-the-eye) are fundamentally different in terms of “how the mind experiences and deals with [them]”, such that no amount of dust motes can add up to the experience of torture… even if they (the motes) have a nonzero amount of disutility.
I believe I am making much the same distinction with my separation of disutility into trivial and non-trivial categories, where no amount of trivial disutility across multiple people can sum to the experience of non-trivial disutility. There is a fundamental gap in the scale (or different scales altogether, à la Jones), a difference in how different amounts of disutility work for humans. For a more concrete example of how this might work, suppose I steal one cent each from one billion different people, and Eliezer steals $100,000 from one person. The total amount of money I have stolen is greater than the amount that Eliezer has stolen; yet my victims will probably never even realize their loss, whereas the loss of $100,000 for one individual is significant. A cent does have a nonzero amount of purchasing power, but none of my victims have actually lost the ability to purchase anything; whereas Eliezer’s, on the other hand, has lost the ability to purchase many, many things.
I believe utility for humans works in the same manner. Another thought experiment I found helpful is to imagine a certain amount of disutility, x, being experienced by one person. Let’s suppose x is “being brutally tortured for a week straight”. Call this situation A. Now divide this disutility among people until we have y people all experiencing (1/y)*x disutility—say, a dust speck in the eye each. Call this situation B. If we can add up disutility like Eliezer supposes in the main article, the total amount of disutility in either situation is the same. But now, ask yourself: which situation would you choose to bring about, if you were forced to pick one?
Would you just flip a coin?
I believe few, if any, would choose situation A. This brings me to a final point I’ve been wanting to make about this article, but have never gotten around to doing. Mr. Yudkowsky often defines rationality as winning—a reasonable definition, I think. But with this dust speck scenario, if we accept Mr. Yudkowsky’s reasoning and choose the one-person-being-tortured option, we end up with a situation in which every participant would rather that the other option had been chosen! Certainly the individual being tortured would prefer that, and each potentially dust-specked individual* would gladly agree to experience an instant of dust-speckiness in order to save the former individual.
I don’t think this is winning; no one is happier with this situation. Like Eliezer says in reference to Newcomb’s problem, if rationality seems to be telling us to go with the choice that results in losing, perhaps we need to take another look at what we’re calling rationality.
*Well, assuming a population like our own, not every single individual would agree to experience a dust speck in the eye to save the to-be-tortured individual; but I think it is clear that the vast majority would.
Thank you for trying to address this problem, as it’s important and still bothers me.
But I don’t find your idea of two different scales convincing. Consider electric shocks. We start with an imperceptibly low voltage and turn up the dial until the first level at which the victim is able to perceive slight discomfort (let’s say one volt). Suppose we survey people and find that a one volt shock is about as unpleasant as a dust speck in the eye, and most people are indifferent between them.
Then we turn the dial up further, and by some level, let’s say two hundred volts, the victim is in excruciating pain. We can survey people and find that a two hundred volt shock is equivalent to whatever kind of torture was being used in the original problem.
So one volt is equivalent to a dust speck (and so on the “trivial scale”), but two hundred volts is equivalent to torture (and so on the “nontrivial scale”). But this implies either that triviality exists only in degree (which ruins the entire argument, since enough triviality aggregated equals nontriviality) or that there must be a sharp discontinuity somewhere (eg a 21.32 volt shock is trivial, but a 21.33 volt shock is nontrivial). But the latter is absurd. Therefore there should not be separate trivial and nontrivial utility scales.
Except perception doesn’t work like that. We can have two qualitatively different perceptions arising from quantities of the same stimulus. We know that irritation and pain use different nerve endings, for instance; and electric shock in different quantities could turn on irritation at a lower threshold than pain. Similarly, a dim colored light is perceived as color on the cone cells, while a very bright light of the same frequency is perceived as brightness on the rod cells. A baby wailing may be perceived as unpleasant; turn it up to jet-engine volume and it will be perceived as painful.
Okay, good point. But if we change the argument slightly to the smallest perceivable amount of pain it’s still biting a pretty big bullet to say 3^^^3 of those is worse than 50 years of torture.
(the theory would also imply that an infinite amount of irritation is not as bad as a tiny amount of pain, which doesn’t seem to be true)
(the theory would also imply that an infinite amount of irritation is not as bad as a tiny amount of pain, which doesn’t seem to be true)
Hmm not sure. It seems quite plausible to me that for any n, an instance of real harm to one person is worse than n instances of completely harmless irritation to n people. Especially if we consider a bounded utility function; the n instances of irritation have to flatten out at some finite level of disutility, and there is no a priori reason to exclude torture to one person having a worse disutility than that asymptote.
Having said all that, I’m not sure I buy into the concept of completely harmless irritation. I doubt we’d perceive a dust speck as a disutility at all except for the fact that it has small probability of causing big harm (loss of life or offspring) somewhere down the line. A difficulty with the whole problem is the stipulation that the dust specks do nothing except cause slight irritation… no major harm results to any individual. However, throwing a dust speck in someone’s eye would in practice have a very small probability of very real harm, such as distraction while operating dangerous machinery (driving, flying etc), starting an eye infection which leads to months of agony and loss of sight, a slight shock causing a stumble and broken limbs or leading to a bigger shock and heart attack. Even the very mild irritation may be enough to send an irritable person “over the edge” into punching a neighbour, or a gun rampage, or a borderline suicidal person into suicide. All these are spectacularly unlikely for each individual, but if you multiply by 3^^^3 people you still get order 3^^^3 instances of major harm.
With that many instances, it’s even highly likely that at least one of the specs in the eye will offer a rare opportunity for some poor prisoner to escape his captors, who had intended to subject him to 50 years of torture.
the theory would also imply that an infinite amount of irritation is not as bad as a tiny amount of pain, which doesn’t seem to be true)
I’m increasingly convinced that the whole Torture vs. Dust Specks scenario is sparking way more heat than light, but...
I can imagine situations where an infinite amount of some type of irritation integrated to something equivalent to some finite but non-tiny amount of pain. I can even imagine situations where that amount was a matter of preference: if you asked someone what finite level of pain they’d accept to prevent some permanent and annoying but non-painful condition, I’d expect the answers to differ significantly. Granted, “lifelong” is not “infinite”, and there’s hyperbolic discounting and various other issues to correct for, but even after these corrections a finite answer doesn’t seem obviously wrong.
Well, for one thing, pain is not negative utility ….
Pain is a specific set of physiological processes. Recent discoveries suggest that it shares some brain-space with other phenomena such as social rejection and math anxiety, which are phenomenologically distinct.
It is also phenomenologically distinct from the sensations of disgust, grief, shame, or dread — which are all unpleasant and inspire us to avoid their causes. Irritation, anxiety, and many other unpleasant sensations can take away from our ability to experience pleasure; many of them can also make us less effective at achieving our own goals.
In place of an individual experiencing “50 years of torture” in terms of physiological pain, we might consider 50 years of frustration, akin to the myth of Sisyphus or Tantalus; or 50 years of nightmare, akin to that inflicted on Alex Burgess by Morpheus in The Sandman ….
First of all, you might benefit from looking up the beard fallacy.
To address the issue at hand directly, though:
Of course there are sharp discontinuities. Not just one sharp discontinuity, but countless. However, there is not particular voltage at which there is a discontinuity. Rather, increasing the voltage increases the probability of a discontinuity.
I will list a few discontinuities established by torture.
Nightmares. As the electrocution experience becomes more severe, the probability that it will result in a nightmare increases. After 50 years of high voltage, hundreds or even thousands of such nightmares are likely to have occurred. However, 1 second of 1V is unlikely to result in even a single nightmare. The first nightmare is a sharp discontinuity. But furthermore, each additional nightmare is another sharp discontinuity.
Stress responses to associational triggers. The first such stress response is a sharp discontinuity, but so is every other one. But please note that there is a discontinuity for each instance of stress response that follows in your life: each one is its own discontinuity. So, if you will experience 10,500 stress responses, that is 10,500 discontinuities. It’s impossible to say beforehand what voltage or how many seconds will make the difference between 10,499 and 10,500, but in theory a probability could be assigned. I think there are already actual studies that have measured the increased stress response after electroshock, over short periods.
Flashbacks. Again, the first flashback is a discontinuity; as is every other flashback. Every time you start crying during a flashback is another discontinuity.
Social problems. The first relationship that fails (e.g., first woman that leaves you) because of the social ramifications of damage to your psyche is a discontinuity. Every time you flee from a social event: another discontinuity. Every fight that you have with your parents as a result of your torture (and the fact that you have become unrecognizable to them) is a discontinuity. Every time you fail to make eye contact is a discontinuity. If not for the torture, you would have made the eye contact, and every failure represents a forked path in your entire future social life.
I could go on, but you can look up the symptoms of PTSD yourself. I hope, however, that I have impressed upon you the fact that life constitutes a series of discrete events, not a continuous plane of quantifiable and summable utility lines. It’s “sharp discontinuities” all the way down to elementary particles. Be careful with mathematical models involving a continuum.
Please note that flashbacks, nightmares, stress responses to triggers, and social problems do not result from specs of dust in the eye.
A better metaphor: What if we replaced “getting a dust speck in your eye” with “being horribly tortured for one second”? Ignore the practical problems of the latter, just say the person experiences the exact same (average) pain as being horribly tortured, but for one second.
That allows us to directly compare the two experiences much better, and it seems to me it eliminates the “you can’t compare the two experiences”- except of course with long term effects of torture, I suppose; to get a perfect comparison we’d need a torture machine that not only does no physical damage, but no psychological damage either.
On the other hand, it does leave in OnTheOtherHandle’s argument about “fairness” (specifically in the “sharing of burdens” definition, since otherwise we could just say the person tortured is selected at random). Which to me as a utilitarian makes perfect sense; I’m not sure if I agree or disagree with him on that.
A cent does have a nonzero amount of purchasing power, but none of my victims have actually lost the ability to purchase anything
Assuming that none of them end up one cent short for something they would otherwise have been able to pay for, which out of a billion people is probably going to happen. It doesn’t have to be their next purchase.
But this is analogous to saying some tiny percentage of the people who got dust specks would be driving a car at that moment and lose control, resulting in an accident. That would be an entirely different ballgame, even if the percent of people this happened to was unimaginably tiny, because in an unimaginably vast population, lots of people are bound to die of gruesome dust-speck related accidents.
But Eliezer explicitly denied any externalities at all; in our hypothetical the chance of accidents, blindness, etc are literally zero. So the chances of not being able to afford a vital heart transplant or whatever for want of a penny must also be literally zero in the analogous hypothetical, no matter how ridiculously large the population gets.
Not being able to pay for something due to the loss of money isn’t an externality, it’s the only kind of direct consequence you’re going to get. If you took a hundred thousand dollars from an individual, they might still be able to make their next purchase, but the direct consequence would be their being unable to pay for things they could previously have afforded.
You might be right. I’ll have to think about this, and reconsider my stance. One billion is obviously far less than 3^^^3, but you are right in that the 10 million dollars stolen by you would be preferable to me than the 100,000 dollars stolen by Eliezer. I also consider losing 100,000 dollars less than or equal to 100,000 times as bad as losing one dollar. This indicates one of two things:
A) My utility system is deeply flawed.
B) My utility system includes some sort of ‘diffiusion factor’ wherein a disutility of X becomes <X when divided among several people, and the disutility becomes lower the more people it’s divided among. In essence, there is some disutility for one person suffering a lot of disutility, that isn’t there when it’s divided among a lot of people.
Of this, B seems more likely, and I didn’t take it into account when considering torture vs. dust specks. In any case, some introspection on this should help me further define my utility function, so thanks for giving me something to think about.
For a more concrete example of how this might work, suppose I steal one cent each from one billion different people, and Eliezer steals $100,000 from one person. The total amount of money I have stolen is greater than the amount that Eliezer has stolen; yet my victims will probably never even realize their loss, whereas the loss of $100,000 for one individual is significant. A cent does have a nonzero amount of purchasing power, but none of my victims have actually lost the ability to purchase anything; whereas Eliezer’s, on the other hand, has lost the ability to purchase many, many things.
Isn’t this a reductio of your argument? Stealing $10,000,000 has less economic effect than stealing $100,000, really? Well, why don’t we just do it over and over, then, since it has no effect each time? If I repeated it enough times, you would suddenly decide that the average effect of each $10,000,000 theft, all told, had been much larger than the average effect of the $100,000 theft. So where is the point at which, suddenly, stealing 1 more cent from everyone has a much larger and disproportionate effect, enough to make up for all the “negligible” effects earlier?
Money is not a linear function of utility. A certain amount is necessary to existance (enough to obtain food, shelter, etc.) A person’s first dollar is thus a good deal more valuable than a person’s millionth dollar, which is in turn more valuable than their billionth dollar. There is clearly some additional utility from each additional dollar, but I suspect that the total utility may well be asymptotic.
The total disutility of stealing an amount of money, $X, from a person with total wealth $Y, is (at least approcximately) equal to the difference in utility between $Y and $(Y-X). (There may be some additional disutility from the fact that a theft occurred—people may worry about being the next victim or falsely accuse someone else or so forth—but that should be roughly equivalent for any theft, and thus I shall disregard it).
So. Stealing one dollar from a person who will starve without that dollar is therefore worse than stealing one dollar from a person who has a billion more dollars in the bank.
Stealing one dollar from each of one billion people, who will each starve without that dollar, is far, far worse than stealing $100 000 from one person who has another $1e100 in the bank.
Stealing $100 000 from a person who only had $100 000 to start with is worse than stealing $1 from each of one billion people, each of whom have another billion dollars in savings.
Now, if we assume a level playing field—that is, that every single person starts with the same amount of money (say, $1 000 000) and no-one will starve if they lose $100 000, then it begins to depend on the exact function used to find the utility of money.
There are functions such that a million thefts of $1 each results in less disutility that a single theft of $100 000. (If asked to find an example, I will take a simple exponential function and fiddle with the parameters until this is true). However, if you continue adding additional thefts of $1 each from the same million people, an interesting effect takes place; each additional theft of $1 each from the same million people is worse than the previous one. By the time you hit the hundred-thousandth theft of $1 each from the same million people, that last theft is substantially more than ten times worse than a single theft of $100 000 from one person.
Yeah, but also keep in mind that people’s utility functions cannot be very concave. (My rephrasing is pretty misleading but I can’t think of a better one, do read the linked post.)
Hmmm. The linked post talks about the perceived utility of money; that is, what the owner of the money thinks it is worth. This is not the same as the actual utility of money, which is what I am trying to use in the grandparent post.
I apologise if that was not clear, and I hope that this has cleared up any lingering misunderstandings.
It seems like you and Hul-Gil are using different formulae for evaluating utility (or, rather, disutility); and, therefore, you are talking past each other.
While Hul-Gil is looking solely at the immediate purchasing power of each individual, you are considering ripple effects affecting the economy as a whole. Thus, while stealing a single penny from a single individual may have negligible disutility, removing 1e9 such pennies from 1e9 individuals will have a strong negative effect on the economy, thus reducing the effective purchasing power of everyone, your victims included.
This is a valid point, but it doesn’t really lend any support to either side in your argument with Hul-Gil, since you’re comparing apples and oranges.
I’m pretty sure Eliezer’s point holds even if you only consider the immediate purchasing power of each individual.
Let us define thefts A and B:
A : Steal 1 cent from each of 1e9 individuals.
B : Steal 1e7 cents from 1 individual.
The claim here is that A has negligible disutility compared to B. However, we can define a new theft C as follows:
C: Steal 1e7 cents from each of 1e9 individuals.
Now, I don’t discount the possibility that there are arguments to the contrary, but naively it seems that a C theft is 1e9 times as bad as a B theft. But a C theft is equivalent to 1e7 A thefts. So, necessarily, one of those A thefts must have been worse than a B theft—substantially worse. Eliezer’s question is: if the first one is negligible, at what point do they become so much worse?
I think this is a question of ongoing collateral effects (not sure if “externalities” is the right word to use here). The examples that speak of money are additionally complicated by the fact that the purchasing power of money does not scale linearly with the amount of money you have.
Consider the following two scenarios:
A). Inflict −1e-3 utility on 1e9 individuals with negligible consequences over time, or B). Inflict a −1e7 utility on a single individual, with further −1e7 consequences in the future.
vs.
C). Inflict a −1e-3 utility on 1e9 individuals leading to an additional −1e9 utility over time, or B). Inflict a one-time −1e7 utility on a single individual, with no additional consequences.
Which one would you pick, A or B, and C or D ? Of course, we can play with the numbers to make A and C more or less attractive.
I think the problem with Eliezer’s “dust speck” scenario is that his disutility of option A—i.e., the dust specs—is basically epsilon, and since it has no additional costs, you might as well pick A. The alternative is a rather solid chunk of disutility—the torture—that will further add up even after the initial torture is over (due to ongoing physical and mental health problems).
The “grand theft penny” scenario can be seen as AB or CD, depending on how you think about money; and the right answer in either case might change depending on how much you think a penny is actually worth.
As a rather firm speck-ist, I’d like to say that this is the best attempt at a formal explanation of speckism that I’ve read so far! I’m grateful for this, and pleased that I no longer need to use muddier and vaguer justifications.
The loss of $100,000 (or one cent) is more or less significant depending on the individual. Which is worse: stealing a cent from 100,000,000 people, or stealing $100,000 from a billionaire? What if the 100,000,000 people are very poor and the cent would buy half a slice of bread and they were hungry to start with? (Tiny dust specks, at least, have a comparable annoyance effect on almost everyone.)
Eliezer’s main gaffe here is choosing a “googolplex” people with dust specks when humans do not even have an intuition for googols. So let’s scale the problem down to a level a human can understand: instead of a googolplex dust specks versus 50 years of torture, let’s take “50 years of torture versus a googol (1 followed by 100 zeros) dust specks”, and scale it down linearly to “1 second of torture verses “6.33 x 10^90 dust specks, one per person”—which is still far more people than have ever lived, so let’s make it “a dust speck once per minute for every person on Earth for their entire lives (while awake) and make it retroactive for all of our human ancestors too” (let’s pretend for a moment that humans won’t evolve a resistance to dust specks as a result). By doing this we are still eliminating virtually all of the dust specks.
So now we have one second of torture versus roughly 2 billion billions of dust specks, which is nothing at all compared to a googol of dust specks. Once the numbers are scaled down to a level that ordinary college graduates can begin to comprehend, I think many of them would change their answer. Indeed, some people might volunteer for one second of torture just to save themselves from getting a tiny dust speck in their eye every minute for the rest of their lives.
The fact that humans can’t feel these numbers isn’t something you teach by just saying it. You teach it by creating a tension between the feeling brain and the thinking brain. Due to your ego, I would guess your brain can better imagine feeling a tiny dust speck in its eye once per minute for your entire life − 20 million specks—than 20 million people getting a tiny dust speck in their eye once, but how is it any different morally? For most people also, 20 billion people with a dust speck feels just the same as 20 million. They both feel like “really big numbers”, but in reality one number is a thousand times worse, and your thinking brain can see that. In this way, I hope you learn to trust your thinking brain more than your feeling one.
This argument does not show that putting dust specks in the eyes of 3^^^3 people is better than torturing one person for 50 years. It shows that putting dust specks in the eyes of 3^^^3 people and then telling them they helped save someone from torture is better than torturing one person for 50 years.
“But with this dust speck scenario, if we accept Mr. Yudkowsky’s reasoning and choose the one-person-being-tortured option, we end up with a situation in which every participant would rather that the other option had been chosen! Certainly the individual being tortured would prefer that, and each potentially dust-specked individual* would gladly agree to experience an instant of dust-speckiness in order to save the former individual.”
A question for comparison: would you rather have a 1/Googolplex chance of being tortured for 50 years, or lose 1 cent?
(A better comparison in this case would be if you replaced “tortured for 50 years” with “death”.)
Also: for the original metaphor, imagine that you aren’t the only person being offered this choice, and that the people suffering the consequences are out of the same pool- which is how real life works, although in this world we have a population of 1 googolplex rather than 7 billion. If we replace “dust speck” with “horribly tortured for 1 second”, and we give 1.5 billion people the same choice and presume they all make the same decision, then the choice is between 1.5 billion people being horribly tortured for 50 years, and 1 googolplex people begin horribly tortured for 50 years.
A question for comparison: would you rather have a 1/Googolplex chance of being tortured for 50 years, or lose 1 cent?
Whenever I drive, I have a greater than a 1/googlolplex chance of getting into an accident which would leave me suffering for 50 years, and I still drive. I’m not sure how to measure the benefit I get from driving, but there are at least some cases where it’s pretty small, even if it’s not exactly a cent.
Whenever one bends down to pick up a dropped penny, one has more than a 1/Googolplex chance of a slip-and-fall accident which would leave one suffering for 50 years.
Another thing that seems to be a factor, at least for me, is that there’s a term in my utility function for “fairness,” which usually translates to something roughly similar to “sharing of burdens.” (I also have a term for “freedom,” which is in conflict with fairness but is on the same scale and can be traded off against it.)
Why wouldn’t this be a situation in which “the complexity of human value” comes into play? Why is it wrong to think something along the lines of, “I would be willing to make everyone a tiny bit worse off so that no one person has to suffer obscenely”? It’s the rationale behind taxation, and while it’s up for debate many Less Wrongers support moderate taxation if it would help a few people a lot while hurting a bunch of people a little bit.
Think about it: the exact number of dollars taken from people in taxes don’t go directly toward feeding the hungry. Some of it gets eaten up in bureaucratic inefficiencies, some of it goes to bribery and embezzlement, some of it goes to the military. This means if you taxed 1,000,000 well-off people $1 each, but only ended up giving 100 hungry people $1000 each to stave of a painful death from starvation, we as utilitarians would be absolutely, 100% obligated to oppose this taxation system, not because it’s inefficient, but because doing nothing would be better. There is to be no room for debate; it’s $100,000 - $1,000,000 = net loss; let the 100 starving peasants die.
Note that you may be a libertarian and oppose taxation on other grounds, but most libertarians wouldn’t say you are literally doing morality wrong if you think it’s better to take $1 each from a million people, even if only $100,000 of it gets used to help the poor.
I could easily be finding ways to rationalize my own faulty intuitions—but I managed to change my mind about Newcomb’s problem and about the first example given in the above post despite powerful initial intuitions, and I managed to work the latter out for myself. So I think, if I’m expected to change my mind here, I’m justified in holding out for an explanation or formulation that clicks with me.
That makes no sense. Just because one thing cost $1, and another thing cost $1000, does not mean that the first thing happening 1001 times is better than the second one happening once.
Preferences logically precede prices. If they didn’t, nobody would be able to decide what they were willing to spend on anything in the first place. If utilitarianism requires that you decide the value of things based on their prices, then utilitarians are conformists without values of their own, who derive all of their value judgments from non-utilitarian market participants who actually have values.
(Besides, money that is spent on “overhead” does not magically disappear from the economy. Someone is still being paid to do something with that money, who in turn buys things with the money, and so on. And even if the money does disappear—say, dollar bills are burnt in a furnace—it still would not represent a loss of productive capacity in the economy. Taxing money and then completely destroying the money (shrinking the money supply) is sound monetary policy, and it occurs on a regular (cyclical) basis. Your whole argument here is a complete non-starter.)
Well, he didn’t actually identify dust mote disutility as zero; he says that dust motes register as zero on his torture scale. He goes on to mention that torture isn’t on his dust-mote scale, so he isn’t just using “torture scale” as a synonym for “disutility scale”; rather, he is emphasizing that there is more than just a single “(dis)utility scale” involved. I believe his contention is that the events (torture and dust-mote-in-the-eye) are fundamentally different in terms of “how the mind experiences and deals with [them]”, such that no amount of dust motes can add up to the experience of torture… even if they (the motes) have a nonzero amount of disutility.
I believe I am making much the same distinction with my separation of disutility into trivial and non-trivial categories, where no amount of trivial disutility across multiple people can sum to the experience of non-trivial disutility. There is a fundamental gap in the scale (or different scales altogether, à la Jones), a difference in how different amounts of disutility work for humans. For a more concrete example of how this might work, suppose I steal one cent each from one billion different people, and Eliezer steals $100,000 from one person. The total amount of money I have stolen is greater than the amount that Eliezer has stolen; yet my victims will probably never even realize their loss, whereas the loss of $100,000 for one individual is significant. A cent does have a nonzero amount of purchasing power, but none of my victims have actually lost the ability to purchase anything; whereas Eliezer’s, on the other hand, has lost the ability to purchase many, many things.
I believe utility for humans works in the same manner. Another thought experiment I found helpful is to imagine a certain amount of disutility, x, being experienced by one person. Let’s suppose x is “being brutally tortured for a week straight”. Call this situation A. Now divide this disutility among people until we have y people all experiencing (1/y)*x disutility—say, a dust speck in the eye each. Call this situation B. If we can add up disutility like Eliezer supposes in the main article, the total amount of disutility in either situation is the same. But now, ask yourself: which situation would you choose to bring about, if you were forced to pick one?
Would you just flip a coin?
I believe few, if any, would choose situation A. This brings me to a final point I’ve been wanting to make about this article, but have never gotten around to doing. Mr. Yudkowsky often defines rationality as winning—a reasonable definition, I think. But with this dust speck scenario, if we accept Mr. Yudkowsky’s reasoning and choose the one-person-being-tortured option, we end up with a situation in which every participant would rather that the other option had been chosen! Certainly the individual being tortured would prefer that, and each potentially dust-specked individual* would gladly agree to experience an instant of dust-speckiness in order to save the former individual.
I don’t think this is winning; no one is happier with this situation. Like Eliezer says in reference to Newcomb’s problem, if rationality seems to be telling us to go with the choice that results in losing, perhaps we need to take another look at what we’re calling rationality.
*Well, assuming a population like our own, not every single individual would agree to experience a dust speck in the eye to save the to-be-tortured individual; but I think it is clear that the vast majority would.
Thank you for trying to address this problem, as it’s important and still bothers me.
But I don’t find your idea of two different scales convincing. Consider electric shocks. We start with an imperceptibly low voltage and turn up the dial until the first level at which the victim is able to perceive slight discomfort (let’s say one volt). Suppose we survey people and find that a one volt shock is about as unpleasant as a dust speck in the eye, and most people are indifferent between them.
Then we turn the dial up further, and by some level, let’s say two hundred volts, the victim is in excruciating pain. We can survey people and find that a two hundred volt shock is equivalent to whatever kind of torture was being used in the original problem.
So one volt is equivalent to a dust speck (and so on the “trivial scale”), but two hundred volts is equivalent to torture (and so on the “nontrivial scale”). But this implies either that triviality exists only in degree (which ruins the entire argument, since enough triviality aggregated equals nontriviality) or that there must be a sharp discontinuity somewhere (eg a 21.32 volt shock is trivial, but a 21.33 volt shock is nontrivial). But the latter is absurd. Therefore there should not be separate trivial and nontrivial utility scales.
Except perception doesn’t work like that. We can have two qualitatively different perceptions arising from quantities of the same stimulus. We know that irritation and pain use different nerve endings, for instance; and electric shock in different quantities could turn on irritation at a lower threshold than pain. Similarly, a dim colored light is perceived as color on the cone cells, while a very bright light of the same frequency is perceived as brightness on the rod cells. A baby wailing may be perceived as unpleasant; turn it up to jet-engine volume and it will be perceived as painful.
Okay, good point. But if we change the argument slightly to the smallest perceivable amount of pain it’s still biting a pretty big bullet to say 3^^^3 of those is worse than 50 years of torture.
(the theory would also imply that an infinite amount of irritation is not as bad as a tiny amount of pain, which doesn’t seem to be true)
Hmm not sure. It seems quite plausible to me that for any n, an instance of real harm to one person is worse than n instances of completely harmless irritation to n people. Especially if we consider a bounded utility function; the n instances of irritation have to flatten out at some finite level of disutility, and there is no a priori reason to exclude torture to one person having a worse disutility than that asymptote.
Having said all that, I’m not sure I buy into the concept of completely harmless irritation. I doubt we’d perceive a dust speck as a disutility at all except for the fact that it has small probability of causing big harm (loss of life or offspring) somewhere down the line. A difficulty with the whole problem is the stipulation that the dust specks do nothing except cause slight irritation… no major harm results to any individual. However, throwing a dust speck in someone’s eye would in practice have a very small probability of very real harm, such as distraction while operating dangerous machinery (driving, flying etc), starting an eye infection which leads to months of agony and loss of sight, a slight shock causing a stumble and broken limbs or leading to a bigger shock and heart attack. Even the very mild irritation may be enough to send an irritable person “over the edge” into punching a neighbour, or a gun rampage, or a borderline suicidal person into suicide. All these are spectacularly unlikely for each individual, but if you multiply by 3^^^3 people you still get order 3^^^3 instances of major harm.
With that many instances, it’s even highly likely that at least one of the specs in the eye will offer a rare opportunity for some poor prisoner to escape his captors, who had intended to subject him to 50 years of torture.
I’m increasingly convinced that the whole Torture vs. Dust Specks scenario is sparking way more heat than light, but...
I can imagine situations where an infinite amount of some type of irritation integrated to something equivalent to some finite but non-tiny amount of pain. I can even imagine situations where that amount was a matter of preference: if you asked someone what finite level of pain they’d accept to prevent some permanent and annoying but non-painful condition, I’d expect the answers to differ significantly. Granted, “lifelong” is not “infinite”, and there’s hyperbolic discounting and various other issues to correct for, but even after these corrections a finite answer doesn’t seem obviously wrong.
Well, for one thing, pain is not negative utility ….
Pain is a specific set of physiological processes. Recent discoveries suggest that it shares some brain-space with other phenomena such as social rejection and math anxiety, which are phenomenologically distinct.
It is also phenomenologically distinct from the sensations of disgust, grief, shame, or dread — which are all unpleasant and inspire us to avoid their causes. Irritation, anxiety, and many other unpleasant sensations can take away from our ability to experience pleasure; many of them can also make us less effective at achieving our own goals.
In place of an individual experiencing “50 years of torture” in terms of physiological pain, we might consider 50 years of frustration, akin to the myth of Sisyphus or Tantalus; or 50 years of nightmare, akin to that inflicted on Alex Burgess by Morpheus in The Sandman ….
First of all, you might benefit from looking up the beard fallacy.
To address the issue at hand directly, though:
Of course there are sharp discontinuities. Not just one sharp discontinuity, but countless. However, there is not particular voltage at which there is a discontinuity. Rather, increasing the voltage increases the probability of a discontinuity.
I will list a few discontinuities established by torture.
Nightmares. As the electrocution experience becomes more severe, the probability that it will result in a nightmare increases. After 50 years of high voltage, hundreds or even thousands of such nightmares are likely to have occurred. However, 1 second of 1V is unlikely to result in even a single nightmare. The first nightmare is a sharp discontinuity. But furthermore, each additional nightmare is another sharp discontinuity.
Stress responses to associational triggers. The first such stress response is a sharp discontinuity, but so is every other one. But please note that there is a discontinuity for each instance of stress response that follows in your life: each one is its own discontinuity. So, if you will experience 10,500 stress responses, that is 10,500 discontinuities. It’s impossible to say beforehand what voltage or how many seconds will make the difference between 10,499 and 10,500, but in theory a probability could be assigned. I think there are already actual studies that have measured the increased stress response after electroshock, over short periods.
Flashbacks. Again, the first flashback is a discontinuity; as is every other flashback. Every time you start crying during a flashback is another discontinuity.
Social problems. The first relationship that fails (e.g., first woman that leaves you) because of the social ramifications of damage to your psyche is a discontinuity. Every time you flee from a social event: another discontinuity. Every fight that you have with your parents as a result of your torture (and the fact that you have become unrecognizable to them) is a discontinuity. Every time you fail to make eye contact is a discontinuity. If not for the torture, you would have made the eye contact, and every failure represents a forked path in your entire future social life.
I could go on, but you can look up the symptoms of PTSD yourself. I hope, however, that I have impressed upon you the fact that life constitutes a series of discrete events, not a continuous plane of quantifiable and summable utility lines. It’s “sharp discontinuities” all the way down to elementary particles. Be careful with mathematical models involving a continuum.
Please note that flashbacks, nightmares, stress responses to triggers, and social problems do not result from specs of dust in the eye.
A better metaphor: What if we replaced “getting a dust speck in your eye” with “being horribly tortured for one second”? Ignore the practical problems of the latter, just say the person experiences the exact same (average) pain as being horribly tortured, but for one second.
That allows us to directly compare the two experiences much better, and it seems to me it eliminates the “you can’t compare the two experiences”- except of course with long term effects of torture, I suppose; to get a perfect comparison we’d need a torture machine that not only does no physical damage, but no psychological damage either.
On the other hand, it does leave in OnTheOtherHandle’s argument about “fairness” (specifically in the “sharing of burdens” definition, since otherwise we could just say the person tortured is selected at random). Which to me as a utilitarian makes perfect sense; I’m not sure if I agree or disagree with him on that.
Assuming that none of them end up one cent short for something they would otherwise have been able to pay for, which out of a billion people is probably going to happen. It doesn’t have to be their next purchase.
But this is analogous to saying some tiny percentage of the people who got dust specks would be driving a car at that moment and lose control, resulting in an accident. That would be an entirely different ballgame, even if the percent of people this happened to was unimaginably tiny, because in an unimaginably vast population, lots of people are bound to die of gruesome dust-speck related accidents.
But Eliezer explicitly denied any externalities at all; in our hypothetical the chance of accidents, blindness, etc are literally zero. So the chances of not being able to afford a vital heart transplant or whatever for want of a penny must also be literally zero in the analogous hypothetical, no matter how ridiculously large the population gets.
Not being able to pay for something due to the loss of money isn’t an externality, it’s the only kind of direct consequence you’re going to get. If you took a hundred thousand dollars from an individual, they might still be able to make their next purchase, but the direct consequence would be their being unable to pay for things they could previously have afforded.
You might be right. I’ll have to think about this, and reconsider my stance. One billion is obviously far less than 3^^^3, but you are right in that the 10 million dollars stolen by you would be preferable to me than the 100,000 dollars stolen by Eliezer. I also consider losing 100,000 dollars less than or equal to 100,000 times as bad as losing one dollar. This indicates one of two things:
A) My utility system is deeply flawed. B) My utility system includes some sort of ‘diffiusion factor’ wherein a disutility of X becomes <X when divided among several people, and the disutility becomes lower the more people it’s divided among. In essence, there is some disutility for one person suffering a lot of disutility, that isn’t there when it’s divided among a lot of people.
Of this, B seems more likely, and I didn’t take it into account when considering torture vs. dust specks. In any case, some introspection on this should help me further define my utility function, so thanks for giving me something to think about.
Isn’t this a reductio of your argument? Stealing $10,000,000 has less economic effect than stealing $100,000, really? Well, why don’t we just do it over and over, then, since it has no effect each time? If I repeated it enough times, you would suddenly decide that the average effect of each $10,000,000 theft, all told, had been much larger than the average effect of the $100,000 theft. So where is the point at which, suddenly, stealing 1 more cent from everyone has a much larger and disproportionate effect, enough to make up for all the “negligible” effects earlier?
See also: http://lesswrong.com/lw/n3/circular_altruism/
Money is not a linear function of utility. A certain amount is necessary to existance (enough to obtain food, shelter, etc.) A person’s first dollar is thus a good deal more valuable than a person’s millionth dollar, which is in turn more valuable than their billionth dollar. There is clearly some additional utility from each additional dollar, but I suspect that the total utility may well be asymptotic.
The total disutility of stealing an amount of money, $X, from a person with total wealth $Y, is (at least approcximately) equal to the difference in utility between $Y and $(Y-X). (There may be some additional disutility from the fact that a theft occurred—people may worry about being the next victim or falsely accuse someone else or so forth—but that should be roughly equivalent for any theft, and thus I shall disregard it).
So. Stealing one dollar from a person who will starve without that dollar is therefore worse than stealing one dollar from a person who has a billion more dollars in the bank.
Stealing one dollar from each of one billion people, who will each starve without that dollar, is far, far worse than stealing $100 000 from one person who has another $1e100 in the bank.
Stealing $100 000 from a person who only had $100 000 to start with is worse than stealing $1 from each of one billion people, each of whom have another billion dollars in savings.
Now, if we assume a level playing field—that is, that every single person starts with the same amount of money (say, $1 000 000) and no-one will starve if they lose $100 000, then it begins to depend on the exact function used to find the utility of money.
There are functions such that a million thefts of $1 each results in less disutility that a single theft of $100 000. (If asked to find an example, I will take a simple exponential function and fiddle with the parameters until this is true). However, if you continue adding additional thefts of $1 each from the same million people, an interesting effect takes place; each additional theft of $1 each from the same million people is worse than the previous one. By the time you hit the hundred-thousandth theft of $1 each from the same million people, that last theft is substantially more than ten times worse than a single theft of $100 000 from one person.
Yeah, but also keep in mind that people’s utility functions cannot be very concave. (My rephrasing is pretty misleading but I can’t think of a better one, do read the linked post.)
Hmmm. The linked post talks about the perceived utility of money; that is, what the owner of the money thinks it is worth. This is not the same as the actual utility of money, which is what I am trying to use in the grandparent post.
I apologise if that was not clear, and I hope that this has cleared up any lingering misunderstandings.
It seems like you and Hul-Gil are using different formulae for evaluating utility (or, rather, disutility); and, therefore, you are talking past each other.
While Hul-Gil is looking solely at the immediate purchasing power of each individual, you are considering ripple effects affecting the economy as a whole. Thus, while stealing a single penny from a single individual may have negligible disutility, removing 1e9 such pennies from 1e9 individuals will have a strong negative effect on the economy, thus reducing the effective purchasing power of everyone, your victims included.
This is a valid point, but it doesn’t really lend any support to either side in your argument with Hul-Gil, since you’re comparing apples and oranges.
I’m pretty sure Eliezer’s point holds even if you only consider the immediate purchasing power of each individual.
Let us define thefts A and B:
A : Steal 1 cent from each of 1e9 individuals. B : Steal 1e7 cents from 1 individual.
The claim here is that A has negligible disutility compared to B. However, we can define a new theft C as follows:
C: Steal 1e7 cents from each of 1e9 individuals.
Now, I don’t discount the possibility that there are arguments to the contrary, but naively it seems that a C theft is 1e9 times as bad as a B theft. But a C theft is equivalent to 1e7 A thefts. So, necessarily, one of those A thefts must have been worse than a B theft—substantially worse. Eliezer’s question is: if the first one is negligible, at what point do they become so much worse?
I think this is a question of ongoing collateral effects (not sure if “externalities” is the right word to use here). The examples that speak of money are additionally complicated by the fact that the purchasing power of money does not scale linearly with the amount of money you have.
Consider the following two scenarios:
A). Inflict −1e-3 utility on 1e9 individuals with negligible consequences over time, or B). Inflict a −1e7 utility on a single individual, with further −1e7 consequences in the future.
vs.
C). Inflict a −1e-3 utility on 1e9 individuals leading to an additional −1e9 utility over time, or B). Inflict a one-time −1e7 utility on a single individual, with no additional consequences.
Which one would you pick, A or B, and C or D ? Of course, we can play with the numbers to make A and C more or less attractive.
I think the problem with Eliezer’s “dust speck” scenario is that his disutility of option A—i.e., the dust specs—is basically epsilon, and since it has no additional costs, you might as well pick A. The alternative is a rather solid chunk of disutility—the torture—that will further add up even after the initial torture is over (due to ongoing physical and mental health problems).
The “grand theft penny” scenario can be seen as AB or CD, depending on how you think about money; and the right answer in either case might change depending on how much you think a penny is actually worth.
As a rather firm speck-ist, I’d like to say that this is the best attempt at a formal explanation of speckism that I’ve read so far! I’m grateful for this, and pleased that I no longer need to use muddier and vaguer justifications.
The loss of $100,000 (or one cent) is more or less significant depending on the individual. Which is worse: stealing a cent from 100,000,000 people, or stealing $100,000 from a billionaire? What if the 100,000,000 people are very poor and the cent would buy half a slice of bread and they were hungry to start with? (Tiny dust specks, at least, have a comparable annoyance effect on almost everyone.)
Eliezer’s main gaffe here is choosing a “googolplex” people with dust specks when humans do not even have an intuition for googols. So let’s scale the problem down to a level a human can understand: instead of a googolplex dust specks versus 50 years of torture, let’s take “50 years of torture versus a googol (1 followed by 100 zeros) dust specks”, and scale it down linearly to “1 second of torture verses “6.33 x 10^90 dust specks, one per person”—which is still far more people than have ever lived, so let’s make it “a dust speck once per minute for every person on Earth for their entire lives (while awake) and make it retroactive for all of our human ancestors too” (let’s pretend for a moment that humans won’t evolve a resistance to dust specks as a result). By doing this we are still eliminating virtually all of the dust specks.
So now we have one second of torture versus roughly 2 billion billions of dust specks, which is nothing at all compared to a googol of dust specks. Once the numbers are scaled down to a level that ordinary college graduates can begin to comprehend, I think many of them would change their answer. Indeed, some people might volunteer for one second of torture just to save themselves from getting a tiny dust speck in their eye every minute for the rest of their lives.
The fact that humans can’t feel these numbers isn’t something you teach by just saying it. You teach it by creating a tension between the feeling brain and the thinking brain. Due to your ego, I would guess your brain can better imagine feeling a tiny dust speck in its eye once per minute for your entire life − 20 million specks—than 20 million people getting a tiny dust speck in their eye once, but how is it any different morally? For most people also, 20 billion people with a dust speck feels just the same as 20 million. They both feel like “really big numbers”, but in reality one number is a thousand times worse, and your thinking brain can see that. In this way, I hope you learn to trust your thinking brain more than your feeling one.
This argument does not show that putting dust specks in the eyes of 3^^^3 people is better than torturing one person for 50 years. It shows that putting dust specks in the eyes of 3^^^3 people and then telling them they helped save someone from torture is better than torturing one person for 50 years.
Yes—though it does mean Eliezer has to assume that the reader’s implausible state of knowledge is not and will not be shared by many of the 3^^^3.
Dust, it turns out, is not naturally occurring, but is only produced as a byproduct of thought experiments.
“But with this dust speck scenario, if we accept Mr. Yudkowsky’s reasoning and choose the one-person-being-tortured option, we end up with a situation in which every participant would rather that the other option had been chosen! Certainly the individual being tortured would prefer that, and each potentially dust-specked individual* would gladly agree to experience an instant of dust-speckiness in order to save the former individual.”
A question for comparison: would you rather have a 1/Googolplex chance of being tortured for 50 years, or lose 1 cent? (A better comparison in this case would be if you replaced “tortured for 50 years” with “death”.)
Also: for the original metaphor, imagine that you aren’t the only person being offered this choice, and that the people suffering the consequences are out of the same pool- which is how real life works, although in this world we have a population of 1 googolplex rather than 7 billion. If we replace “dust speck” with “horribly tortured for 1 second”, and we give 1.5 billion people the same choice and presume they all make the same decision, then the choice is between 1.5 billion people being horribly tortured for 50 years, and 1 googolplex people begin horribly tortured for 50 years.
Whenever I drive, I have a greater than a 1/googlolplex chance of getting into an accident which would leave me suffering for 50 years, and I still drive. I’m not sure how to measure the benefit I get from driving, but there are at least some cases where it’s pretty small, even if it’s not exactly a cent.
Whenever one bends down to pick up a dropped penny, one has more than a 1/Googolplex chance of a slip-and-fall accident which would leave one suffering for 50 years.
But you also slightly improve your physical fitness which might reduce the probability of an accident further down the line by more than 1/10^10^100.
Another thing that seems to be a factor, at least for me, is that there’s a term in my utility function for “fairness,” which usually translates to something roughly similar to “sharing of burdens.” (I also have a term for “freedom,” which is in conflict with fairness but is on the same scale and can be traded off against it.)
Why wouldn’t this be a situation in which “the complexity of human value” comes into play? Why is it wrong to think something along the lines of, “I would be willing to make everyone a tiny bit worse off so that no one person has to suffer obscenely”? It’s the rationale behind taxation, and while it’s up for debate many Less Wrongers support moderate taxation if it would help a few people a lot while hurting a bunch of people a little bit.
Think about it: the exact number of dollars taken from people in taxes don’t go directly toward feeding the hungry. Some of it gets eaten up in bureaucratic inefficiencies, some of it goes to bribery and embezzlement, some of it goes to the military. This means if you taxed 1,000,000 well-off people $1 each, but only ended up giving 100 hungry people $1000 each to stave of a painful death from starvation, we as utilitarians would be absolutely, 100% obligated to oppose this taxation system, not because it’s inefficient, but because doing nothing would be better. There is to be no room for debate; it’s $100,000 - $1,000,000 = net loss; let the 100 starving peasants die.
Note that you may be a libertarian and oppose taxation on other grounds, but most libertarians wouldn’t say you are literally doing morality wrong if you think it’s better to take $1 each from a million people, even if only $100,000 of it gets used to help the poor.
I could easily be finding ways to rationalize my own faulty intuitions—but I managed to change my mind about Newcomb’s problem and about the first example given in the above post despite powerful initial intuitions, and I managed to work the latter out for myself. So I think, if I’m expected to change my mind here, I’m justified in holding out for an explanation or formulation that clicks with me.
That makes no sense. Just because one thing cost $1, and another thing cost $1000, does not mean that the first thing happening 1001 times is better than the second one happening once.
Preferences logically precede prices. If they didn’t, nobody would be able to decide what they were willing to spend on anything in the first place. If utilitarianism requires that you decide the value of things based on their prices, then utilitarians are conformists without values of their own, who derive all of their value judgments from non-utilitarian market participants who actually have values.
(Besides, money that is spent on “overhead” does not magically disappear from the economy. Someone is still being paid to do something with that money, who in turn buys things with the money, and so on. And even if the money does disappear—say, dollar bills are burnt in a furnace—it still would not represent a loss of productive capacity in the economy. Taxing money and then completely destroying the money (shrinking the money supply) is sound monetary policy, and it occurs on a regular (cyclical) basis. Your whole argument here is a complete non-starter.)