The problem is the line of reasoning, where a “50 years of torture” is better than 3^^^3 years with a dust speck in the eye every so often.
That’s not even the dilemma you linked to. The dilemma you linked to “one person be horribly tortured for fifty years without hope or rest, or that 3^^^3 people get dust specks in their eyes”.
What is then the torture of all the Humanity, against the super happy Felix with 3^^^3 pyramids. Nothing. By the same line of reasoning.
It’s probably bad practice to say two lines of reasoning are the same line of reasoning, if you don’t believe in either of them.
For starters I don’t need to have a positive factor for Felix’s further happiness in my utilty function. That alone is a significant difference.
Look. You have one person, under terrible torture for 50 years on one side and a gazillion of people with a slight discomfort every year or so on the other side.
It is claimed that the first is better.
Now, you have a small humanity as is, only enslaved for pyramid building for Felix. He has eons of subjective time to enjoy this pyramids and he is unbelievably happy. More happy than any man, woman or child could ever be. The amount of happiness of Felix outweights the misery of billion of people by a factor of a million.
What’s the fundamental difference between those two cases? I don’t see it, do you?
The only similarity between those cases is that they involve utility calculations you disagree with. Otherwise every single detail is completely different. (e. g. the sort of utility considered, two negative utilities being traded against each other vs. trading utility elsewhere (positive and negative) for positive utility, which side of the trade the single person with the large individual utility difference is on, the presence of perverse incentives, etc, etc).
If anything it would be more logical to equate Felix with the tortured person and treat this as a reductio ad absurdum of your position on the dust speck problem. (But that would be wrong too, since the numbers aren’t actually the problem with Felix, the fact that there’s an incentive to manipulate your own utility function that way is (among other things).)
You aren’t seeing forest for the trees… the thing that is identical is that you are trading utilities across people, which is fundamentally problematic and leads to either tortured child or utility monster, or both.
Omelas is a goddamned paradise. Omelas without the tortured child would be better, yeah, but Omelas as described is still better than any human civilization that has ever existed. (For one thing, it only contains one miserable child.)
Well it seems to me they are trading N dust specks vs torture in Omelas. edit: Actually, I don’t like Omelas [as example]. I think that miserable child would only make the society way worse, with the people just opting to e.g. kill someone when it ever so slightly results in increase in their personal expected utility. This child in Omelas puts them straight on the slippery slope, and making everyone aware of slippage makes people slide down for fun and profit.
Our ‘civilization’ though, of course, is a god damn jungle and so its pretty damn bad. It’s pretty hard to beat on the moral wrongness scale, from first principles; you have to take our current status quo and modify it to get to something worse (or take our earlier status quo).
Your edit demonstrates that you really don’t get consequentialism at all. Why would making a good tradeoff (one miserable child in exchange for paradise for everyone else) lead to making a terrible one (a tiny bit of happiness for one person in exchange for death for someone else)?
People are individual survival machines, that’s why. Each bastard in the Omelas knows at the gut level (not in some abstract way) that there’s a child being miserable specifically for a tiny bit of his happiness. His personally. He will then kill for larger bit of his happiness. He isn’t society. He’s an individual. It is all between him and that child. At very best, between him&his family, and that child. The society ain’t part of equation. (And if it is, the communism should of worked perfectly in that universe) [assuming that the individual believes he won’t be caught]
edit: also i think you don’t understand the story. They didn’t take the child apart for much needed organs to save other folks in Omelas. The child is miserable for the purpose of bringing sense of unity into the commune, for the purpose of making them value their happiness. That is already very irrational, and not only that but also entirely contrary to how homo sapiens behave when exposed to gross injustice.
edit: To explain my use of language. We are not talking about rational agents and what they ought to decide. We are taking of irrational agents that are supposedly (premise of the story) made more well behaved by participation in a pointless and evil ritual, which is the opposite of the known effect of direct participation in that sort of ritual, on populace. That’s why the story makes a poor case against utilitarianism. Because the consequence is grossly invalid.
What ever. The reason why I don’t like that story too much is, I do not believe that, given the way homo sapiens are, demonstrating them that child in the Omelas would have consequence stated in the story, even if they are instructed that this is the consequence. It’s too much of a stretch. The effect of such on H. Sapiens, that I would forecast, would be entirely opposite. The Omelas is doing something more similar to how you break in the soldiers for effective Holocaust death squad—the soldiers that later kill others or themselves outside the orders. You make the soldiers participate all together in something like that. That’s why I don’t like this as example. I’m arguing against my own point of bringing it up as example. Because the reason we don’t like Omelas is because keeping child like this won’t have positive consequence. (and for it to have stated positive consequence, the people already have to have a grossly irrational reaction to exposure to that child)
the thing that is identical is that you are trading utilities across people,
This is either wrong (the utility functions of the people involved aren’t queried in the dust speck problem) or so generic as to be encompassed in the concept of “utility calculation”.
Aggregating utility functions across different people is an unsolved problem, but not necessarily an unsolvable one. One way of avoiding utility monsters would be to normalize utility functions. The obvious way to do that leads to problems such as arachnophobes getting less cake even if they like cake equally much, but IMO that’s better than utility monsters.
This is either wrong (the utility functions of the people involved aren’t queried in the dust speck problem) or so generic as to be encompassed in the concept of “utility calculation”.
The utilities of many people are a vector, you are to map it to a scalar value, that loses a lot of information in process, and it seems to me however you do it, leads to some sort of objectionable outcomes. edit: I have a feeling one could define it reasonably with some sort of Kolmogorov complexity like metric that would grow incredibly slowly for the dust specks and would never equate what ever hideously clever thing does our brain do to most of the neurons when we suffer; the suffering beating the dust specks on the complexity (you’d have to write down the largest number you can write down in as many bits as the bits being tortured in the brain; then that number of dust specks starts getting to the torture level). We need to understand how pain works before we can start comparing pain vs dust specks.
Really? Every use of utilities I have seen either uses a real world measure (such as money) with a notation that it isn’t really utilities or they go directly for the unfalsifiable handwaving. So far I haven’t seen anything to suggest “aggregating utility functions” is even theoretically possible. For that matter most of what I have read suggests that even an individual’s “utility function” is usually unmanageably fuzzy, or even unfalsifiable, itself.
Felix is essentially a Utility Monster: a thought experiment that’s been addressed here before. As that family of examples shows, happiness-maximization breaks down rather spectacularly when you start considering self- or other-modification or any seriously unusual agents. You can bite that bullet, if you want, but not many people here do; fortunately, there are a few other ways you can tackle this if you’re interested in a formalization of humanlike ethics. The “Value Stability and Aggregation” post linked above touches on the problem, for example, as does Eliezer’s Fun Theory sequence.
You don’t need any self-modifying or non-humanlike agents to run into problems related to “Torture vs. Dust Specks”, though; all you need is to be maximizing over the welfare of a lot of ordinary agents. 3^^^3 is an absurdly huge number and leads you to a correspondingly counterintuitive conclusion (one which, incidentally, I’d estimate has led to more angry debate than anything else on this site), but lesser versions of the same tradeoff are quite realistic; unless you start invoking sacred vs. profane values or otherwise define the problem away, it differs only in scale from the same utilitarian calculations you make when, say, assigning chores.
In one case, (Torture to avoid the specks) the larger portion of people is better off if you pick the single person. In the other case, (Build pyramids to please Felix) the larger portion of people is worse off if you pick the single person.
So if my position was “The majority should win” It would be right to torture the person and it would be right to depose Felix.
I’m not sure if it’s a fundamental difference or a good difference, but I think that means I can lay out the following 4 distinct answer pairs:
Depose Felix, Torture Man: Majority wins.
Adore Felix, Speck people: Minority wins.
Adore Felix, Torture Man: Mean Happiness wins.
Depose Felix, Speck People: Minimum happiness wins. (Assuming either Felix is happier about being deposed than an average person with a dust speck in their eye, or dead, and no longer counted for minimum happiness.)
So I think I can see all 4 distinct positions, if I’m not missing something.
In one case, (Torture to avoid the specks) the larger portion of people is better off if you pick the single person. In the other case, (Build pyramids to please Felix) the larger portion of people is worse off if you pick the single person.
Imagine that there is one tortured for 50 years and then free of any dust speck for the next 3^^^3 years.
Then we don’t have “the larger portion of people” anymore. Is anything different in such a case?
Imagine that there is one tortured for 50 years and then free of any dust speck for the next 3^^^3 years.
If I understand the dilemma, in your most recent phrasing, it’s this: A person who lives 3^^^3 years either: a) has to suffer a dustspeck per year b) has to suffer 50 years of torture at some point in that time, then I assume gets the memory of that torture deleted from his mind and his mind’s state restored to what it was before the torture (so that he doesn’t suffer further disutility from that memory or the broken mind-state, he only has to suffer the torture itself), He lives the remaining 3^^^3 years dustspeck-free.
If we don’t know what his own preferences are, and have no way of asking him, what should we choose on his behalf?
Can we have one dilemma at a time, please, Thomas? You said something about 3^^^3 years—therefore you’re not talking about the dilemma as stated in the original sequence, as that dilemma doesn’t say anything about 3^^^3 years.
Which preferences are in question now?
The preferences relating to the original dilemma, are the preferences of the person who presumably prefers not to get tortured, vs the preferences of 3^^^3 people who presumably prefer not to get a dust speck in the eye.
Well, first of all, I’m assuming that you’re doing that to both groupings (since otherwise I could say “Well, one has only one person and one has a massive number of people, which is a difference.” but that seems like a trivial point)
So if you apply it to both, then it’s just one person considering tradeoff A, (pay torture to go speck free for eons)
And another person considering tradeoff B(personally build pyramids for eons to get to live in your own collection of pyramids for some years.)
I could say that in once case the pain is relatively dense (torture, condensed to 50 years) and the pleasure is relatively sparse,(speck free, over 3^^^3 years) and that in the other case the pain relatively sparse (slave labor, spread out over a long time) and the pleasure is relatively dense (Incomprehensible pyramidgasm.).
I’m not sure if that matters or in what ways that difference matters. I’m really not up to date on how your brain handles that specifically and would probably need to look it up further.
personally build pyramids for eons to get to live in your own collection of pyramids for some years.)
No. Building pyramids as humans. And enjoying them much, much longer as they stand there, for Felix. Enjoyed by Felix.
Maybe the amount of our pleasure with Giza pyramids already exceeded the pain invested to build them. I don’t know.
Can all the pains of a slave be justified by all the pleasures of the tourist, visiting the hole in the rock, he was forced carving for 50 years?
Or can a large group of sick sadists are entitled to slowly torture someone, since their pleasure sum will be greater than the pain of the unlucky one?
Maybe the amount of our pleasure with Giza pyramids already exceeded the pain invested to build them. I don’t know. Can all the pains of a slave be justified by all the pleasures of the tourist, visiting the hole in the rock, he was forced carving for 50 years?
Was it that much pain? I read in National Geographic, IIRC, that the modern archaeological conception was that the pyramids were mostly or entirely built by paid labor—Nile farmers killing time during the dry season. This may even be a good thing, depending on whether it diverted imperial tax revenue from foreign adventurism into monument/tomb-building.
Well, it’s still a fun Fermi calculation problem, anyway.
Let’s see, the Pyramids have been the targets of tourism since at least the original catalogue of wonders of the ancient world, Antipater of Sidon ~140 BC which includes “the great man-made mountains of the lofty pyramids”. So that’s ~2150 years of tourism (2012+140). Quickly checking, Wikipedia says 12.8 million people visited Egypt for tourism in 2008, but surely not all of them visited the pyramids? Let’s halve it to 6 million.
Let’s pretend Egyptian tourism followed a linear growth between 140 BC with one visitor (Antipater) and 6 million in 2012 (yes, world population & wealth has grown and so you’d expect tourism to grow a lot, but Egypt has been pretty chaotic recently), over 2150 years. We can just average that to 3 million a year, which gives us a silly total number of tourists of 2150 * 3 million or 6.45 billion visitors.
There are 138 pyramids, WP says, with the Great Pyramid estimated at 100,000 workers. Let’s halve it (again with the assumptions!) at 50k workers a pyramid, 50,000 * 138 = 6.9m workers total.
This gives us the visitor:worker ratio of 6.45b:6.9m, or 21,500:23, or 934.8:1.
And of course the pyramids are still there, so whatever the real ratio, it’s getting better (modulo issues of maintenance and restoration).
You’d need a heck of a lot more tourism than for Egypt… although apparently there’s quite a range of estimates of deaths, from less than 20,000 a year to more than 200,000 a year. Given the substantially less tourism to the Aztec pyramids (inasmuch as apparently only 2 small unimpressive Aztec pyramids survive, with all the impressive ones like Tenochtitlan destroyed), it’s safe to say that the utilitarian calculus will never work out for them.
It seems to me that any historical event that was both painful to the participants, and interesting to read and learn about after the fact, creates the same dilemma that’s been discussed here. Will World War Two have been a net good if 10,000 years from now trillions of people have gotten incredible enjoyment from watching movies, reading books, and playing videogames that involve WWII as a setting in some way?
The first solution to this dilemma that comes to mind is that ready substitutes exist for most of the entertainments associated with these unpleasant events. If the Aztecs had built their pyramids and then never sacrificed anyone on them it probably wouldn’t hurt the modern tourist trade that much. And if WWII had never happened and thus caused the Call of Duty videogame franchise to never exist, it wouldn’t have a big impact on utility because some cognates of the Doom, Unreal, and similar franchises would still exist (those franchises are based on fictional events, so no one got hurt inspiring them).
In fact, if I was to imagine an alternate human history where no war, slavery, or similar conflict had ever happened, and the inhabitants got all their enjoyment from entertainment media based on fictional conflicts, I think such a world would have a much higher net utility than our own.
Sure—but can you offhand fit an exponential curve and calculate its summation? I’m sure it’s doable with the specified endpoints and # of periods (just steal a simple interest formula), but it’s more work than halving and multiplying.
Well… integral from t0 to t1 of exp(at+b) dt = (exp(at1+b)-exp(a*t2+b))/a i.e. the difference between the endpoints times the time needed to increase by a factor of e… a 6-million-fold increase is about 22.5 doublings (knowing 2^20 = 1 million), hence about 15 factors of e (knowing that ln 2 = 0.7) i.e. about one in 150… hence the total number of tourists is about 1 billion (about six times less than Rhwawn’s estimate—my eyeballs had told me it would be about one third… close enough!)
Being very very outraged isn’t really an argument.
Give us your own (non-utilitarian I assume) decision theory that you consider encapsulating all that is good and moral, if you please.
If you can’t, please stop being outraged as those of us who try to solve the problem, even if you feel we’ve taken wrong turns in the path towards the solution.
I don’t know, 3^^^3 is a pretty long time to fix brain trauma. Or are you offering complete restoration after the torture? In that case, I might just take it.
I am not offering anything at all. I strongly advice you NOT to substitute the slight discomfort over long time period with a horrible torture for a shorter period.
What’s the fundamental difference between those two cases? I don’t see it, do you?
One fundamental difference is that I don’t care about Felix’s further happiness. After some point, I may even resent it, which would make his additional happiness of negative utility to me.
Another difference is that happiness may be best represented as a percentage with an upper bound of e.g. 100% happy, rather than be an integer you can keep adding to without end.
I think Felix’s case may be an interesting additional scenario to consider, in order to be sure that AIs don’t fall victims to it (e.g. by creating a superintelligence and making it super-happy, to the expense of normal human happiness). But it’s not the same scenario as the specks.
The FAI should make a drug which will make you happy for Felix. edit: to clarify. The two choices here are not happy naturally vs happy via wireheading. The two choices are intense AI-induced ‘natural’ unhappiness, vs drug induced happiness. It’s similar to having your hand amputated, with or without ‘wireheading’, err, painkillers. I think it is pretty clear that if you have someone’s hand amputated, it is better if they can’t feel it and see it. Be careful with non-wireheading FAIs, ’less all surgery will be without anaesthesia (perhaps with only the muscle relaxant).
Well, in some sense, achieving happiness by anything other than reproduction, is already wireheading. Doesn’t need to be with a wire; what if I make a video which evokes intense feeling of pleasure? How far you can go before it is a mind hack?
edit: actually, I think the AI could raise people to be very empathetic for Felix, and very happy for him. Is it not good to raise your kids so that they can be happy in the world the way it is (when they can’t change anything anyway) ?
“achieving happiness by anything other than [subgoals of] reproduction” is wireheading from the perspective of my genes, and if they want to object I’m not stopping them. Happiness via drugs is wireheading from the perspective of me, and I object myself.
What if there’s double rainbow ? What if you have lower than ‘normal’ level of some neurotransmitter and under-appreciate the double rainbow without drugs? What if higher than ‘normal’?
I’m not advocating drugs, by the way, just pointing out the difficulty in making any binary distinction here. The natural happy should be preferred to wire-headed happy, but the society does think that some people should take anti-depressants. If you are to labour in the name of the utility monster anyway, you could as well be happy. You object to happiness via drug as substitute for happiness without drugs, but if the happiness without drugs is not going to happen—then what?
Well, in some sense, achieving happiness by anything other than reproduction, is already wireheading.
No. This reduces the words to the point of meaninglessness. Human beings have values other than reproduction, values that make them happy when satisfied—art, pride, personal achievement, understanding, etc. Wireheading is about being made happy directly, regardless of the satisfaction of the various values.
The scenario previously discussed about Felix is that he was happy and everyone else suffered.
Now you’re posing a scenario where everyone is happy, but they’re made happy by having their values rewritten to place extremelty value on Felix’s happiness instead.
At this point, I hope we’re not pretending it’s the same scenario with only minor modifications, right? Your scenario is about the AI rewriting our values, it’s not about trading our collective suffering for Felix’s happiness.
Your scenario can effectively remove the person of Felix from the situation altogether, and the AI could just make us all very happy that the laws of physics keep on working.
You say art… what if I am a musician and I am making a song? That’s good, right? What if I get 100 experimental subjects to sit in MRI, as they listen to test music, and using my intelligence and some software tools, make very pleasurable song? What if I know that it works by activating such and such connections here and there which end up activating the reward system? What if I don’t use MRI, but use internal data available in my own brain, to achieve same result?
I know that this is arriving at meaninglessness, I just don’t see it as reducing the words anywhere; the words already only seem meaningful in the context of limited depth of inference, but it all falls apart if you are to make more steps (like an axiomatic system that leads to self contradiction). Making people happy [as terminal goal], this way or that, just leads to some form of really objectionable behaviour if done by something more intelligent than human.
You say art… what if I am a musician and I am making a song? That’s good, right? What if I get 100 experimental subjects to sit in MRI, as they listen to test music, and using my intelligence and some software tools, make very pleasurable song? What if I know that it works by activating such and such connections here and there which end up activating the reward system? What if I don’t use MRI, but use internal data available in my own brain, to achieve same result?
Be specific about what you are asking, please. What does the “what if” mean here? Whether these thing should be considered good? Whether such things should be considered “wireheading”? Whether we want an AI to do such things? What?
Making people happy, this way or that, just leads to some form of really objectionable behaviour if done by something more intelligent than human.
This claim doesn’t seem to make much sense to me. I’ve already been made non-objectionably happy by people more intelligent than me from time to time. My parents, when I was child. Good writers and funny entertainers, as an adult. How does it become authomatically “really objectionable” if it’s “something more intelligent than human” as opposed to “something more intelligent than you, personally?”
Be specific about what you are asking, please. What does the “what if” mean here? Whether these thing should be considered good? Whether such things should be considered “wireheading”? Whether we want an AI to do such things? What?
I’m trying to make you think a little deeper about your distinction between wireheading and non-wireheading. The point is that your choice of the dividing line is entirely arbitrary (and most people don’t agree where to put dividing line). I don’t know where you put the dividing line, and frankly I don’t care; i just want you to realize that you’re drawing arbitrary line on the beach, to the left of it is the land, to the right is the ocean. edit: That’s how maps work, not how territory works, btw.
This claim doesn’t seem to make much sense to me. I’ve already been made non-objectionably happy by people more intelligent than me from time to time. My parents, when I was child. Good writers and funny entertainers, as an adult. How does it become authomatically “really objectionable” if it’s “something more intelligent than human” as opposed to “something more intelligent than you, personally?”
I’d say, they had a goal to achieve something other than happiness , and the happiness was incidental.
I’m trying to make you think a little deeper about your distinction between wireheading and non-wireheading.
Don’t assume you know how deeply I think about it. The only thing I’ve effectively communicated to you so far that I consider it ludicrous to say that “achieving happiness by anything other than reproduction, is already wireheading”
We can agree Yes/No, that this discussion doesn’t have much of anything to do with the Felix scenario, right? Please answer this question.
The point is that your choice of the dividing line is entirely arbitrary (and most people don’t agree where to put dividing line).
Perhaps people don’t have to agree, and the people whose coherent extrapolated volition allows a situation “W” to be done to them, should so have it done to them, regardless of whether you label W to be ‘wireheading’ or ‘wellbeing’.
Or perhaps not. After all, it’s not as if I ever declared Friendliness to be a solved problem, so I don’t know why you keep talking to me as if I claimed it’s easy to arrive at a conclusion.
“Whether such things should be considered “wireheading”?” is what i want you to consider, yes.
I don’t have a binary classifier, absolute wireheading vs non-wireheading. I have the wireheadedness quantity. Connecting a wire straight into your pleasure centre will have wireheadedness of (very close to) 1, reproduction (maximization of expected number of each gene) will have wireheadedness of 0, taking heroin will be close to 1, taking LSD will be lower, the wireheadedness of the art varies depending on how much of your brain is involved in making pleasure out of art (how much involved is the art), and perhaps to how much of a hack the art is, though ultimately all of art is to greater or lesser extent a hack. edit: and i am actually earning my living sort of making art (i make CGI software, but also do CGI myself).
I don’t consider the low wireheadedness to be necessarily good. That’s the christian moral connotations, which I do not share as an atheist grown in non religious family.
Happiness, as a state of mind in humans, seems less to me about how strong the “orgasms” are than how frequently they occur without lessening the probability they will continue to occur. So what problems might there be with maximizing total future happy seconds experienced in humans, including emulations thereof (other than describing with sufficient accuracy the concepts of ‘human’ and ‘happiness’ to a computer)?
I think doing so would extrapolate to increasing population and longevity to within resource constraints and diminishing returns on improving average happiness uptime and existential risk mitigation, which seem to me to be the crux of people’s intuitions about the Felix and Wireheading problems.
That’s not even the dilemma you linked to. The dilemma you linked to “one person be horribly tortured for fifty years without hope or rest, or that 3^^^3 people get dust specks in their eyes”.
It’s probably bad practice to say two lines of reasoning are the same line of reasoning, if you don’t believe in either of them.
For starters I don’t need to have a positive factor for Felix’s further happiness in my utilty function. That alone is a significant difference.
Look. You have one person, under terrible torture for 50 years on one side and a gazillion of people with a slight discomfort every year or so on the other side.
It is claimed that the first is better.
Now, you have a small humanity as is, only enslaved for pyramid building for Felix. He has eons of subjective time to enjoy this pyramids and he is unbelievably happy. More happy than any man, woman or child could ever be. The amount of happiness of Felix outweights the misery of billion of people by a factor of a million.
What’s the fundamental difference between those two cases? I don’t see it, do you?
The only similarity between those cases is that they involve utility calculations you disagree with. Otherwise every single detail is completely different. (e. g. the sort of utility considered, two negative utilities being traded against each other vs. trading utility elsewhere (positive and negative) for positive utility, which side of the trade the single person with the large individual utility difference is on, the presence of perverse incentives, etc, etc).
If anything it would be more logical to equate Felix with the tortured person and treat this as a reductio ad absurdum of your position on the dust speck problem. (But that would be wrong too, since the numbers aren’t actually the problem with Felix, the fact that there’s an incentive to manipulate your own utility function that way is (among other things).)
You aren’t seeing forest for the trees… the thing that is identical is that you are trading utilities across people, which is fundamentally problematic and leads to either tortured child or utility monster, or both.
Omelas is a goddamned paradise. Omelas without the tortured child would be better, yeah, but Omelas as described is still better than any human civilization that has ever existed. (For one thing, it only contains one miserable child.)
Well it seems to me they are trading N dust specks vs torture in Omelas. edit: Actually, I don’t like Omelas [as example]. I think that miserable child would only make the society way worse, with the people just opting to e.g. kill someone when it ever so slightly results in increase in their personal expected utility. This child in Omelas puts them straight on the slippery slope, and making everyone aware of slippage makes people slide down for fun and profit.
Our ‘civilization’ though, of course, is a god damn jungle and so its pretty damn bad. It’s pretty hard to beat on the moral wrongness scale, from first principles; you have to take our current status quo and modify it to get to something worse (or take our earlier status quo).
Your edit demonstrates that you really don’t get consequentialism at all. Why would making a good tradeoff (one miserable child in exchange for paradise for everyone else) lead to making a terrible one (a tiny bit of happiness for one person in exchange for death for someone else)?
People are individual survival machines, that’s why. Each bastard in the Omelas knows at the gut level (not in some abstract way) that there’s a child being miserable specifically for a tiny bit of his happiness. His personally. He will then kill for larger bit of his happiness. He isn’t society. He’s an individual. It is all between him and that child. At very best, between him&his family, and that child. The society ain’t part of equation. (And if it is, the communism should of worked perfectly in that universe) [assuming that the individual believes he won’t be caught]
edit: also i think you don’t understand the story. They didn’t take the child apart for much needed organs to save other folks in Omelas. The child is miserable for the purpose of bringing sense of unity into the commune, for the purpose of making them value their happiness. That is already very irrational, and not only that but also entirely contrary to how homo sapiens behave when exposed to gross injustice.
edit: To explain my use of language. We are not talking about rational agents and what they ought to decide. We are taking of irrational agents that are supposedly (premise of the story) made more well behaved by participation in a pointless and evil ritual, which is the opposite of the known effect of direct participation in that sort of ritual, on populace. That’s why the story makes a poor case against utilitarianism. Because the consequence is grossly invalid.
Tapping out, inferential distance too wide.
What ever. The reason why I don’t like that story too much is, I do not believe that, given the way homo sapiens are, demonstrating them that child in the Omelas would have consequence stated in the story, even if they are instructed that this is the consequence. It’s too much of a stretch. The effect of such on H. Sapiens, that I would forecast, would be entirely opposite. The Omelas is doing something more similar to how you break in the soldiers for effective Holocaust death squad—the soldiers that later kill others or themselves outside the orders. You make the soldiers participate all together in something like that. That’s why I don’t like this as example. I’m arguing against my own point of bringing it up as example. Because the reason we don’t like Omelas is because keeping child like this won’t have positive consequence. (and for it to have stated positive consequence, the people already have to have a grossly irrational reaction to exposure to that child)
This is either wrong (the utility functions of the people involved aren’t queried in the dust speck problem) or so generic as to be encompassed in the concept of “utility calculation”.
Aggregating utility functions across different people is an unsolved problem, but not necessarily an unsolvable one. One way of avoiding utility monsters would be to normalize utility functions. The obvious way to do that leads to problems such as arachnophobes getting less cake even if they like cake equally much, but IMO that’s better than utility monsters.
The utilities of many people are a vector, you are to map it to a scalar value, that loses a lot of information in process, and it seems to me however you do it, leads to some sort of objectionable outcomes. edit: I have a feeling one could define it reasonably with some sort of Kolmogorov complexity like metric that would grow incredibly slowly for the dust specks and would never equate what ever hideously clever thing does our brain do to most of the neurons when we suffer; the suffering beating the dust specks on the complexity (you’d have to write down the largest number you can write down in as many bits as the bits being tortured in the brain; then that number of dust specks starts getting to the torture level). We need to understand how pain works before we can start comparing pain vs dust specks.
Really? Every use of utilities I have seen either uses a real world measure (such as money) with a notation that it isn’t really utilities or they go directly for the unfalsifiable handwaving. So far I haven’t seen anything to suggest “aggregating utility functions” is even theoretically possible. For that matter most of what I have read suggests that even an individual’s “utility function” is usually unmanageably fuzzy, or even unfalsifiable, itself.
You must know, that every pyramid Felix does not get, causes him a dust speck size pain.
The Humanity’s suffering is only several billion times as much as a single 50 years torture of one individual.
Nothing compared to 3^^^3 “dust specks”??!
Felix is essentially a Utility Monster: a thought experiment that’s been addressed here before. As that family of examples shows, happiness-maximization breaks down rather spectacularly when you start considering self- or other-modification or any seriously unusual agents. You can bite that bullet, if you want, but not many people here do; fortunately, there are a few other ways you can tackle this if you’re interested in a formalization of humanlike ethics. The “Value Stability and Aggregation” post linked above touches on the problem, for example, as does Eliezer’s Fun Theory sequence.
You don’t need any self-modifying or non-humanlike agents to run into problems related to “Torture vs. Dust Specks”, though; all you need is to be maximizing over the welfare of a lot of ordinary agents. 3^^^3 is an absurdly huge number and leads you to a correspondingly counterintuitive conclusion (one which, incidentally, I’d estimate has led to more angry debate than anything else on this site), but lesser versions of the same tradeoff are quite realistic; unless you start invoking sacred vs. profane values or otherwise define the problem away, it differs only in scale from the same utilitarian calculations you make when, say, assigning chores.
In one case, (Torture to avoid the specks) the larger portion of people is better off if you pick the single person. In the other case, (Build pyramids to please Felix) the larger portion of people is worse off if you pick the single person.
So if my position was “The majority should win” It would be right to torture the person and it would be right to depose Felix.
I’m not sure if it’s a fundamental difference or a good difference, but I think that means I can lay out the following 4 distinct answer pairs:
Depose Felix, Torture Man: Majority wins.
Adore Felix, Speck people: Minority wins.
Adore Felix, Torture Man: Mean Happiness wins.
Depose Felix, Speck People: Minimum happiness wins. (Assuming either Felix is happier about being deposed than an average person with a dust speck in their eye, or dead, and no longer counted for minimum happiness.)
So I think I can see all 4 distinct positions, if I’m not missing something.
Edit: Fixed spacing.
Imagine that there is one tortured for 50 years and then free of any dust speck for the next 3^^^3 years.
Then we don’t have “the larger portion of people” anymore. Is anything different in such a case?
If I understand the dilemma, in your most recent phrasing, it’s this: A person who lives 3^^^3 years either:
a) has to suffer a dustspeck per year
b) has to suffer 50 years of torture at some point in that time, then I assume gets the memory of that torture deleted from his mind and his mind’s state restored to what it was before the torture (so that he doesn’t suffer further disutility from that memory or the broken mind-state, he only has to suffer the torture itself), He lives the remaining 3^^^3 years dustspeck-free.
If we don’t know what his own preferences are, and have no way of asking him, what should we choose on his behalf?
But what does this have to do with Felix?
It is argued in the said sequence, how much better is to have 1 tortured for 50 years, than 3^^^3 people having slight discomfort.
Which preferences are in question now?
Can we have one dilemma at a time, please, Thomas? You said something about 3^^^3 years—therefore you’re not talking about the dilemma as stated in the original sequence, as that dilemma doesn’t say anything about 3^^^3 years.
The preferences relating to the original dilemma, are the preferences of the person who presumably prefers not to get tortured, vs the preferences of 3^^^3 people who presumably prefer not to get a dust speck in the eye.
Well, first of all, I’m assuming that you’re doing that to both groupings (since otherwise I could say “Well, one has only one person and one has a massive number of people, which is a difference.” but that seems like a trivial point)
So if you apply it to both, then it’s just one person considering tradeoff A, (pay torture to go speck free for eons)
And another person considering tradeoff B(personally build pyramids for eons to get to live in your own collection of pyramids for some years.)
I could say that in once case the pain is relatively dense (torture, condensed to 50 years) and the pleasure is relatively sparse,(speck free, over 3^^^3 years) and that in the other case the pain relatively sparse (slave labor, spread out over a long time) and the pleasure is relatively dense (Incomprehensible pyramidgasm.).
I’m not sure if that matters or in what ways that difference matters. I’m really not up to date on how your brain handles that specifically and would probably need to look it up further.
No. Building pyramids as humans. And enjoying them much, much longer as they stand there, for Felix. Enjoyed by Felix.
Maybe the amount of our pleasure with Giza pyramids already exceeded the pain invested to build them. I don’t know.
Can all the pains of a slave be justified by all the pleasures of the tourist, visiting the hole in the rock, he was forced carving for 50 years?
Or can a large group of sick sadists are entitled to slowly torture someone, since their pleasure sum will be greater than the pain of the unlucky one?
I don’t think so.
I’ve heard that the labourers who made the pyramids were actually quite well paid.
Was it that much pain? I read in National Geographic, IIRC, that the modern archaeological conception was that the pyramids were mostly or entirely built by paid labor—Nile farmers killing time during the dry season. This may even be a good thing, depending on whether it diverted imperial tax revenue from foreign adventurism into monument/tomb-building.
I saw it, too. Had to use other example. Mayan or Aztec pyramids maybe.
Well, it’s still a fun Fermi calculation problem, anyway.
Let’s see, the Pyramids have been the targets of tourism since at least the original catalogue of wonders of the ancient world, Antipater of Sidon ~140 BC which includes “the great man-made mountains of the lofty pyramids”. So that’s ~2150 years of tourism (2012+140). Quickly checking, Wikipedia says 12.8 million people visited Egypt for tourism in 2008, but surely not all of them visited the pyramids? Let’s halve it to 6 million.
Let’s pretend Egyptian tourism followed a linear growth between 140 BC with one visitor (Antipater) and 6 million in 2012 (yes, world population & wealth has grown and so you’d expect tourism to grow a lot, but Egypt has been pretty chaotic recently), over 2150 years. We can just average that to 3 million a year, which gives us a silly total number of tourists of 2150 * 3 million or 6.45 billion visitors.
There are 138 pyramids, WP says, with the Great Pyramid estimated at 100,000 workers. Let’s halve it (again with the assumptions!) at 50k workers a pyramid, 50,000 * 138 = 6.9m workers total.
This gives us the visitor:worker ratio of 6.45b:6.9m, or 21,500:23, or 934.8:1.
And of course the pyramids are still there, so whatever the real ratio, it’s getting better (modulo issues of maintenance and restoration).
Maybe those pyramids in Egypt are not so bad, after all.
But with how much tourism you can justify Aztec pyramids, where people were slaughtered?
How many billion tourist should come to be worth to start with them all over again?
You’d need a heck of a lot more tourism than for Egypt… although apparently there’s quite a range of estimates of deaths, from less than 20,000 a year to more than 200,000 a year. Given the substantially less tourism to the Aztec pyramids (inasmuch as apparently only 2 small unimpressive Aztec pyramids survive, with all the impressive ones like Tenochtitlan destroyed), it’s safe to say that the utilitarian calculus will never work out for them.
It seems to me that any historical event that was both painful to the participants, and interesting to read and learn about after the fact, creates the same dilemma that’s been discussed here. Will World War Two have been a net good if 10,000 years from now trillions of people have gotten incredible enjoyment from watching movies, reading books, and playing videogames that involve WWII as a setting in some way?
The first solution to this dilemma that comes to mind is that ready substitutes exist for most of the entertainments associated with these unpleasant events. If the Aztecs had built their pyramids and then never sacrificed anyone on them it probably wouldn’t hurt the modern tourist trade that much. And if WWII had never happened and thus caused the Call of Duty videogame franchise to never exist, it wouldn’t have a big impact on utility because some cognates of the Doom, Unreal, and similar franchises would still exist (those franchises are based on fictional events, so no one got hurt inspiring them).
In fact, if I was to imagine an alternate human history where no war, slavery, or similar conflict had ever happened, and the inhabitants got all their enjoyment from entertainment media based on fictional conflicts, I think such a world would have a much higher net utility than our own.
Big romances have been inspired by much smaller events, it should be noted.
The first approximation which springs in my mind would be an exponential growth rather than a linear one.
Sure—but can you offhand fit an exponential curve and calculate its summation? I’m sure it’s doable with the specified endpoints and # of periods (just steal a simple interest formula), but it’s more work than halving and multiplying.
Well… integral from t0 to t1 of exp(at+b) dt = (exp(at1+b)-exp(a*t2+b))/a i.e. the difference between the endpoints times the time needed to increase by a factor of e… a 6-million-fold increase is about 22.5 doublings (knowing 2^20 = 1 million), hence about 15 factors of e (knowing that ln 2 = 0.7) i.e. about one in 150… hence the total number of tourists is about 1 billion (about six times less than Rhwawn’s estimate—my eyeballs had told me it would be about one third… close enough!)
I’m actually a little surprised that his such gross approximation puts it off by only 6x. For a Fermi estimate that’s perfectly acceptable.
Being very very outraged isn’t really an argument.
Give us your own (non-utilitarian I assume) decision theory that you consider encapsulating all that is good and moral, if you please.
If you can’t, please stop being outraged as those of us who try to solve the problem, even if you feel we’ve taken wrong turns in the path towards the solution.
Found this by random clicking around, I expect no one’s still reading this, but maybe we’ll catch each other via Inbox:
How about “optimize the worst case” from in game theory? It settles both the dust speck vs. torture and the the Utility Monster Felix problems neatly.
I don’t know, 3^^^3 is a pretty long time to fix brain trauma. Or are you offering complete restoration after the torture? In that case, I might just take it.
I am not offering anything at all. I strongly advice you NOT to substitute the slight discomfort over long time period with a horrible torture for a shorter period.
One fundamental difference is that I don’t care about Felix’s further happiness. After some point, I may even resent it, which would make his additional happiness of negative utility to me.
Another difference is that happiness may be best represented as a percentage with an upper bound of e.g. 100% happy, rather than be an integer you can keep adding to without end.
I think Felix’s case may be an interesting additional scenario to consider, in order to be sure that AIs don’t fall victims to it (e.g. by creating a superintelligence and making it super-happy, to the expense of normal human happiness). But it’s not the same scenario as the specks.
The FAI should make a drug which will make you happy for Felix. edit: to clarify. The two choices here are not happy naturally vs happy via wireheading. The two choices are intense AI-induced ‘natural’ unhappiness, vs drug induced happiness. It’s similar to having your hand amputated, with or without ‘wireheading’, err, painkillers. I think it is pretty clear that if you have someone’s hand amputated, it is better if they can’t feel it and see it. Be careful with non-wireheading FAIs, ’less all surgery will be without anaesthesia (perhaps with only the muscle relaxant).
Cute, but that’s effectively the well-known scenario of Wireheading where the complexity of human value is replaced by mere ‘happiness’.
Well, in some sense, achieving happiness by anything other than reproduction, is already wireheading. Doesn’t need to be with a wire; what if I make a video which evokes intense feeling of pleasure? How far you can go before it is a mind hack?
edit: actually, I think the AI could raise people to be very empathetic for Felix, and very happy for him. Is it not good to raise your kids so that they can be happy in the world the way it is (when they can’t change anything anyway) ?
“achieving happiness by anything other than [subgoals of] reproduction” is wireheading from the perspective of my genes, and if they want to object I’m not stopping them. Happiness via drugs is wireheading from the perspective of me, and I object myself.
What if there’s double rainbow ? What if you have lower than ‘normal’ level of some neurotransmitter and under-appreciate the double rainbow without drugs? What if higher than ‘normal’?
I’m not advocating drugs, by the way, just pointing out the difficulty in making any binary distinction here. The natural happy should be preferred to wire-headed happy, but the society does think that some people should take anti-depressants. If you are to labour in the name of the utility monster anyway, you could as well be happy. You object to happiness via drug as substitute for happiness without drugs, but if the happiness without drugs is not going to happen—then what?
No. This reduces the words to the point of meaninglessness. Human beings have values other than reproduction, values that make them happy when satisfied—art, pride, personal achievement, understanding, etc. Wireheading is about being made happy directly, regardless of the satisfaction of the various values.
The scenario previously discussed about Felix is that he was happy and everyone else suffered. Now you’re posing a scenario where everyone is happy, but they’re made happy by having their values rewritten to place extremelty value on Felix’s happiness instead.
At this point, I hope we’re not pretending it’s the same scenario with only minor modifications, right? Your scenario is about the AI rewriting our values, it’s not about trading our collective suffering for Felix’s happiness.
Your scenario can effectively remove the person of Felix from the situation altogether, and the AI could just make us all very happy that the laws of physics keep on working.
You say art… what if I am a musician and I am making a song? That’s good, right? What if I get 100 experimental subjects to sit in MRI, as they listen to test music, and using my intelligence and some software tools, make very pleasurable song? What if I know that it works by activating such and such connections here and there which end up activating the reward system? What if I don’t use MRI, but use internal data available in my own brain, to achieve same result?
I know that this is arriving at meaninglessness, I just don’t see it as reducing the words anywhere; the words already only seem meaningful in the context of limited depth of inference, but it all falls apart if you are to make more steps (like an axiomatic system that leads to self contradiction). Making people happy [as terminal goal], this way or that, just leads to some form of really objectionable behaviour if done by something more intelligent than human.
Be specific about what you are asking, please. What does the “what if” mean here? Whether these thing should be considered good? Whether such things should be considered “wireheading”? Whether we want an AI to do such things? What?
This claim doesn’t seem to make much sense to me. I’ve already been made non-objectionably happy by people more intelligent than me from time to time. My parents, when I was child. Good writers and funny entertainers, as an adult. How does it become authomatically “really objectionable” if it’s “something more intelligent than human” as opposed to “something more intelligent than you, personally?”
I’m trying to make you think a little deeper about your distinction between wireheading and non-wireheading. The point is that your choice of the dividing line is entirely arbitrary (and most people don’t agree where to put dividing line). I don’t know where you put the dividing line, and frankly I don’t care; i just want you to realize that you’re drawing arbitrary line on the beach, to the left of it is the land, to the right is the ocean. edit: That’s how maps work, not how territory works, btw.
I’d say, they had a goal to achieve something other than happiness , and the happiness was incidental.
Don’t assume you know how deeply I think about it. The only thing I’ve effectively communicated to you so far that I consider it ludicrous to say that “achieving happiness by anything other than reproduction, is already wireheading”
We can agree Yes/No, that this discussion doesn’t have much of anything to do with the Felix scenario, right? Please answer this question.
Perhaps people don’t have to agree, and the people whose coherent extrapolated volition allows a situation “W” to be done to them, should so have it done to them, regardless of whether you label W to be ‘wireheading’ or ‘wellbeing’.
Or perhaps not. After all, it’s not as if I ever declared Friendliness to be a solved problem, so I don’t know why you keep talking to me as if I claimed it’s easy to arrive at a conclusion.
“Whether such things should be considered “wireheading”?” is what i want you to consider, yes.
I don’t have a binary classifier, absolute wireheading vs non-wireheading. I have the wireheadedness quantity. Connecting a wire straight into your pleasure centre will have wireheadedness of (very close to) 1, reproduction (maximization of expected number of each gene) will have wireheadedness of 0, taking heroin will be close to 1, taking LSD will be lower, the wireheadedness of the art varies depending on how much of your brain is involved in making pleasure out of art (how much involved is the art), and perhaps to how much of a hack the art is, though ultimately all of art is to greater or lesser extent a hack. edit: and i am actually earning my living sort of making art (i make CGI software, but also do CGI myself).
I don’t consider the low wireheadedness to be necessarily good. That’s the christian moral connotations, which I do not share as an atheist grown in non religious family.
Happiness, as a state of mind in humans, seems less to me about how strong the “orgasms” are than how frequently they occur without lessening the probability they will continue to occur. So what problems might there be with maximizing total future happy seconds experienced in humans, including emulations thereof (other than describing with sufficient accuracy the concepts of ‘human’ and ‘happiness’ to a computer)?
I think doing so would extrapolate to increasing population and longevity to within resource constraints and diminishing returns on improving average happiness uptime and existential risk mitigation, which seem to me to be the crux of people’s intuitions about the Felix and Wireheading problems.