You can get mind states that are ambiguous mixes of awake and asleep.
I am having trouble parsing this statement. Does it mean that when simulating a mind you could also simulate ambiguous awake/asleep in addition to simulating sleep and wakefulness? Or does it mean that a stored, unsimulated mind is ambiguously neither awake or asleep?
That makes a lot of sense, thank you.
Thanks for the reply. It sounds like maybe my mistake was assuming that unsimulated brain data was functionally and morally equivalent to an unconscious brain. From what you are saying it sounds like the data would need to be simulated even to generate unconsciousness.
And much like Vaniver below (above? earlier!), I am unsure how to translate these sorts of claims into anything testable
One thing I consider very suspicious is that deaf people often don’t just deny the terminal value of hearing. They also deny its instrumental value. The instrumental values of hearing are obvious. This indicates to me that they are denying it for self-esteem reasons and group loyalty reasons, the same way I have occasionally heard multiculturalists claim behaviors of obvious instrumental value (like being on time) are merely the subjective values of Western culture.
The typical defense of this denial (and other disability-rights type claims) is hearing only has instrumental value because society is structured in a way that makes use of it. But this is obviously false, hearing would be useful on a desert island, and there are some disabilities that society is not technologically capable of solving (there’s no way to translate instrumental music into sign language). Plus, structuring society around disabilities is essentially having society pay to enable a person instead of having biology do it for free. Obviously it’s better than not accommodating them, but it;s even better to have biology do the accommodation for free if that is possible.
I think another factor is simply my knowledge of the human brain structure, and the psychological unity of humankind. It seems like it would be a much smaller departure from standard brain design to switch the specific target of the “romance” module of the brain, than it would be to completely erase all desire to enjoy the pleasures that a sense of hearing can provide us, and to assign terminal value to being inconvenienced by things like not being able to talk to people who aren’t in your visual range.
I think another thing that supports my intuitions is Bostrom’s Reversal test. Imagine instead of discussing giving a preexisting sense to people who lack it, we were considering giving people a new sense that no human being has ever had before. Should we do that? If there were no side effects, I would say yes! As I told Vaniver in my reply to them, I really want to be able sense magnetic fields. Seeing infrared and ultraviolet would also be fun. The fact that my intuitions are the same in the Reversal Test provides evidence that they are not based on the Status Quo Bias.
it may be clearer to consider counterfactual mes of every possible sexual orientation, and comparing the justifications they can come up with for why it’s egosyntonic to have the orientation that they have.
I think that maybe all of them would be perfectly justifying in saying that their sexual orientation is a terminal value and the buck stops there.
On the other hand, I’m nowhere near 100% sure I wouldn’t take a pill to make me bisexual.
If you kept all of my values the same and deleted my sexual orientation, what would regrow?
I think a way to help tease out your intuition would be Bostrom’s reversal test. If transhumanist scientists invented a new kind of sexual orientation, a new kind of sexual experiences, and so on, would you want to be modified to be able to enjoy this new, never before seen type of sex. I don’t know how you’d reply, for me it would probably be yes or no, depending on specific details of the new kind of sex.
I think the reason I would sometimes say yes is that I have a strong preexisting preference for novelty and novel experiences. So my desire for new sexual preferences would grow out of that.
Incidentally, Bostrom’s reversal test also supports my intuitions about deafness. If transhumanists invented new senses that no human has ever had before, would I want to have them if there were no side effects? Of course I would! I especially want to be able to sense magnetic fields like sharks can.
It seems to me that most people lack the ability to be aroused by people—typically, their ability is seriously limited, to half of the population at most.
When I was talking about being queer I wasn’t just talking about the experience of being aroused, I was talking about the desire to have that experience, and that experience being egosyntonic. It’s fairly easy to rephrase any preference a person has to sound like an ability or lack thereof. For instance, you could say that I lack the ability to enjoy skinning people alive. But that’s because I don’t want to skin people alive, or to enjoy it! That’s a terminal value, the buck stops there.
Some other factors to consider:
Even if I was to define “being aroused” as an ability, that doesn’t perfectly map onto the discuss. In the case of removing deafness we are adding an ability. In the case of changing queerness to heterosexuality, we are either removing an ability to find some people arousing and replacing it with a different one (in the case of homosexuals) or removing an ability and replacing it with nothing (in the case of bisexuals).
Arousal has very little instrumental value compared to hearing. Even if someone with the power of hearing took no pleasure from music or people’s voices they would still benefit from being able to hear people talk outside of their visual range, and to hear cars coming when they cross the street. I can see deaf people denying the terminal benefits of hearing, but denying the instrumental ones seems obviously crazy.
I can’t claim that I would choose to be gay for that reason, starting from emptiness.
Starting from emptiness you would be completely indifferent to everything, including changes to your future preferences. To paraphrase Eliezer, you would be a rock, not a perfectly impartial being.
At some point you just have to say “These are my terminal values, the buck stops here.”
Now, while I would not say it is impossible to create a creature that assigns a terminal value to deafness, I find it unlikely that humans are such creatures. The way human psychology works makes me assign a much higher probability to their being self-deceived for group status purposes.
But since then, you’ve concluded that being queer isn’t actually something (at least some people, like me) differentially approve of.
I’m not sure what I wrote that gave you this idea. I do think that queer people approve of being queer. What I’m talking about when I say “approval” is preferences that are ego-syntonic, that are line with the kind of person they want to be. Most queer people consider their preference to be ego-syntonic. Being queer is the kind of person they want to be and they would not change it if they could. Those who do not are usually motivated by mistaken religious ideas, rather than clearly reasoned disapproval.
What I am trying to say is that being queer is a statement about what people want to do. When we say that someone is queer that means that they have a desire to engage in romantic and sexual relationships that are different from the heterosexual norm. This desire is ego-syntonic, it is approved of.
Being deaf, by contrast, is a statement about what people are able do. They lack the ability to hear things.
If you removed someone’s deafness, none of their desires would change. They would still want everything they wanted before they were deaf. If they were really attached to their current lifestyle they could buy earplugs. By contrast, if you changed a queer person into a straight person, they would stop wanting to have non-heteronormative relationships. They’d be able to continue their current lifestyle (or at least, as able as anyone is in a heteronormative society), but they wouldn’t want to.
There are some people who claim that they prefer being deaf to being able to hear, and that being deaf is ego-syntonic. I believe that they are confused. I think what they really value isn’t being deaf, it’s the community that they have built with other deaf people. They are confusing their preference to display loyalty to their community with with a preference to not be able to hear. In addition I think they are confused for some other reasons:
Sour grapes. When people are unable to do something, they often convince themselves they didn’t want to do it anyway in order to assuage their ego.
Confusing “life could be better” with “life is not worth living.” As I said before, a lot of disability rights advocates seem to think that if you admit that their disability makes their life even slightly worse, that means their life is not worth living at all and they should be euthanized. This is not true.
If people got hit in the head with a baseball bat every day.....
Happy death spirals around their community. They love their community and want to say more and more good things about it. So they say that their community is so awesome that living in it is worth being significantly less good at sensing one’s surroundings.
To sum it up, I believe that being queer is an ego-syntonic desire. I believe that being deaf is not ego-syntonic, but people say it is out of a desire to have self-esteem and be proud of and loyal to the deaf community.
Rereading your original comment keeping in mind that you’re talking mostly about approval rather than desire or preference… so, would you say that Deaf people necessarily disapprove of deafness?
I’d say that a good portion of them do approve of it. There seem to be a lot of disability rights activists who seem to think that being disabled and making more disabled people is okay.
I should also mention, however, that I do think it is possible to mistakenly approve or disapprove of something. For instance I used to disapprove of pornography and voluntary prostitution. However, I eventually realized that the arguments for why those things were bad were wrong/incoherent, and realized that I should never have disapproved of those things. Disapproval of pornography and voluntary prostitution was never my CEV.
I think a large portion of disability-rights activists are also confused in their thinking, and would have different views if their thinking was clearer. For instance, many disability rights activists seem to think that any suggestion that disability is bad implies that the lives of disabled people aren’t worth living and that they should all be involuntarily euthanized, which is obviously false. It’s possible to believe your life is worth living while simultaneously believing it could be better.
It’s not clear to me how this difference justifies the distinction in my thinking I was describing.
I believe the difference is that in the case of deaf people, you are improving their lives by giving them more abilities to achieve the values they have (in this case, an extra sense). By contrast, with queerness you are erasing a value a person has and replacing it with a different value that is easier to achieve. I believe that helping a person achieve their existing values is a laudable goal, but that changing a person’s values is usually morally problematic, even if their new values are easier to achieve than their old ones.
Now, keep in mind that I am speaking in principle, not in practice. In the real-life case of deafness this issue is more complicated than the way I just described it. There are other issues, for instance, the value of an extra sense is to some extent tied to the support mechanisms society has developed for it. I think that the deaf community may be voicing a valid concern that society has good set of support mechanism for people who are fully deaf and fully hearing, but not as good mechanisms for people with the kind of mid-range hearing that cochlear implants provide.
But those are concerns of practice, not principle. In principle having extra senses should make it easier to achieve your values. I mean, wouldn’t you want super-hearing, microscopic vision, etc if you could get them without any side-effects.
How do we tell whether what I value is to find a mate of Type A, or to find a mate I find attractive?
I think the fact that you are unwilling to have your criteria for attractiveness be modified is good evidence that it is the former and not the latter.
Is this simply a semantic disagreement—that is, do we just have different understandings of what the phrase “who I am” refers to? Or is there something we’d expect to observe differently in the world were you correct and I mistaken about this?
I think there are two issues, one is semantic, the other is that I did completely understand what you meant by being changed into someone who isn’t queer.
First, the semantic issue. I have been trying to approach the issue of Personal Identity by righting a wrong question. Instead of asking “Am I the same person as him?” I instead ask “How desirable would it be for me to change into that person?” I find that this approache generates the same intuitive results as traditional approaches to personal identity (for instance both approaches identify being killed and being wireheaded as very undesirable outcomes) but doesn’t get bogged down by the issues of what exactly it means to be “the same.”
Saying that you literally wouldn’t be the same person was hyperbolic of me. I was trying to draw attention to the fact that our values are an important part of who we are, and that changing our values can change our identity. It would be more accurate to say something like “the new you is only 90% the same person as the previous you.”
The other issue is that I don’t think I quite understood what you meant when you talked about being changed. To give a framework to what I mean, I call your attention to Yvain’s famous post on Wanting, Liking, and Approving. When you talked about being changed to not be queer, I assumed you meant that your Wanting, Liking, and Approving stats had all been changed. You had been changed so that you wanted to not be queer, liked it, and deeply approved of this fact.
However, this does not match your description of what you imagine the subjective experience of being modified to not be attracted to men would be like. You say:
If I woke up tomorrow morning and I was no longer sexually attracted to men, that would be startling, and it would be decidedly inconvenient in terms of my existing marriage, but I wouldn’t be someone else, any more than if I stopped being sexually attracted to anyone, or stopped liking the taste of beef, or lost my arm.
That sounds to me like your Wanting and Liking stats have been modified, but your Approving stat has stayed the same.
I consider the “Approving” portion of your personality to be a much bigger part of your personal identity than “Wanting” and “Liking.” So if the change left the “Approving” portion of your personality intact, I would completely agree with you that you are still the same person that you were before, regardless of what personal-identity framework I am using.
I acknowledge that life is more difficult in certain readily quantifiable ways for queer people than for straight people, but it doesn’t follow that I would use a reliable therapy for making me straight if such a thing existed… and in fact I wouldn’t. Nor would I encourage the development of such a therapy, particularly, and indeed the notion of anyone designing such a therapy makes me more than faintly queasy. And if it existed, I’d be reluctant to expose my children to it. And I would be sympathetic to claims that developers and promoters of such a technology are in some way acting ‘against’ queer folk.
I think there is a fundamental difference between being queer and being deaf. Being queer means you have different values from other people. You are attracted to different types of people than is typical. Being deaf means you have different abilities from other people. You can’t hear things a typical person can. If you are struck deaf your fundamental values haven’t changed. If your sexual orientation has changed, they have.
And that’s not because I want the difficulties themselves; I don’t. I want those differential difficulties to disappear; I just don’t like the idea of having them disappear by making everyone straight. I want them to disappear by having the culture treat queers and straights in ways that don’t create differential difficulties.
If you weren’t queer you would have more difficulties fulfilling your values, not less. If your value is to find a mate of Type A, and you modify yourself to like Type B instead, you will be even less likely to find a mate of Type A than you were before, since your new self will be pursuing Type B people. In other words, if you weren’t queer you wouldn’t be better off because you wouldn’t be you, you’d be somebody else.
Now, you might argue that the fact that the new you can fulfill his values more easily can compensate for this. I would disagree. I am not a negative utilitarian, but I do believe there are many circumstances where creating and fulfilling new values/people cannot fully compensate for the destruction of old values/people, even if the new values are easier to fulfill than the old ones. And I believe that changing one’s sexual orientation is usually one of those circumstances.
You are, in this very post, questing and saying that your utility function PROBABLY this and that you dont think there’s uncertainty about it… That is, you display uncertainty about your utility function. Check mate.
Even if I was uncertain about my utility function, you’re still wrong. The factor you are forgetting about is uncertainty. With a bounded utility function infinite utility scores the same as a smaller amount of utility. So you should always assume a bounded utility function, because unbounded utility functions don’t offer any more utility than bounded ones and bounded ones outperform unbounded ones in situations like Pascal’s Mugging. There’s really no point to believing you have an unbounded function.
I just used the same logic you did. But the difference is that I assumed a bounded utility function was the default standard for comparison, whereas you assumed, for no good reason, that the unbounded one was.
I don’t know what the proper way to calculate utility when you are uncertain about your utility function. But I know darn well that doing an expected-utility calculation about what utility each function will yield and using one of the two functions that are currently in dispute to calculate that utility is a crime against logic. If you do that you’re effectively assigning “having an unbounded function” a probability of 1. And 1 isn’t a probability.
Your formulation of “unbounded utility function always scores infinity so it always wins” is not the correct way to compare two utility functions under uncertainty. You could just as easily say “unbounded and bounded both score the same, except in Pascal’s mugging where bounded scores higher, so bounded always wins.”
I think that using expected utility calculation might be valid for things like deciding whether you assign any utility at all to object or consequence. But for big meta-level questions about what your utility function even is attempting to use them is a huge violation of logic.
It seems to me that the project of transhumanism in general is actually the project of creating artificial utility monsters. If we consider a utility monster a creature that can transmute resources into results more efficiently that’s essentially what a transhuman is.
In a world where all humans have severe cognitive and physical disabilities and die at the age of 30 a baseline human would be a utility monster. They would be able to achieve far more of their life goals and desires than all other humans would. Similarly, a transhuman with superhuman cognitive abilities, physical abilities, and indefinite lifespan would be a utility monster from the point of view of modern people.
So to answer the opening question about whether or not effective altruists have ever considered building artificial utility monsters: Any effective altruist who has donated any money to the SIAI, FHI, or other organization has already started doing this. We’ve been working towards creating artificial utility monsters for decade now.
Now, you might have been meaning something slightly different than that. Maybe you meant to create some creature with an inhuman psychology, like orgasmium. To answer that question I’d have to delve deeper and more personally into my understanding of ethics.
Long story short, I think that would be a terrible idea. My population ethics only considers the creation of entities with complex values that somewhat resemble human ones to be positive. For all other types of creatures I am a negative preference utilitarian, I consider their addition to be a bad thing and that we should make sacrifices to prevent it. And that’s even assuming that it is possible to compare their utility functions with ours. I don’t think interpersonal utility comparison between two human-like creatures is hard at all. But a creature with a totally alien set of values is likely impossible.
I suspect that calling your utility function itself into question like that isn’t valid in terms of expected utility calculations.
I think what you’re suggesting is that on top of our utility function we have some sort of meta-utility function that just says “maximize your utility function, whatever it is.” That would fall into your uncertainty trap, but I don’t think that is the case, I don’t think we have a meta-function like that, I think we just have our utility function.
If you were allowed to cast your entire utility function into doubt you would be completely paralyzed. How do you know you don’t have an unbounded utility function for paperclips? How do you know you don’t have an unbounded utility function for, and assign infinite utility to, the universe being exactly the way it would be if you never made a fully rational decision again and just went around your life on autopilot? The end result is that there are a number of possible courses of action that would all generate infinity utility and no way to choose between them because infinity=infinity. The only reason your argument sounds logical is because you are allowing the questioning of the boundedness of the utility function, but not its contents.
I think that knowledge of your utility function is probably a basic, prerational thing, like deciding to use expected utility maximization and Bayesian updating in the first place. Attempting to insert your utility function itself into your calculations seems like a basic logical error.
This tends to imply the Sadistic Conclusion: that it is better to create some lives that aren’t worth living than it is to create a large number of lives that are barely worth living.
I think that the Sadistic Conclusion is correct. I argue here that it is far more in line with typical human moral intuitions than the repugnant one.
There are several “impossibility” theorems that show it is impossible to come up with a way to order populations that satisfies all of a group of intuitively appealing conditions.
If you take the underlying principle of the Sadistic Conclusion, but change the concrete example to something smaller scale and less melodramatic than “Create lives not worth living to stop the addition of lives barely worth living,” you will find that it is very intuitively appealing.
For instance, if you ask people if they should practice responsible family planning or spend money combating overpopulation they agree. But (if we assume that the time and money spent on these efforts could have been devoted to something more fun) this is the same principle. The only difference is that instead creating a new life not worth living we are instead subtracting an equivalent amount of utility from existing people.
It’s worth noting that the question of what is a better way of evaluating such prospects is distinct from the question of how I in fact evaluate them.
Good point. What I meant was closer to “which method of evaluation does the best job of capturing how you intuitively assign value” rather than which way is better in some sort of objective sense. For me #1 seems to describe how I assign value and disvalue to repeating copies better than #2 does, but I’m far from certain.
So I think that from my point of view Omega offering to extend the length of a repeated event so it contains a more even mixture of good and bad is the same as Omega offering to not repeat a bad event and repeat a good event instead. Both options contain zero value, I would rather Omega leave me alone and let me go do new things. But they’re better than him repeating a bad event.
I think I understand your viewpoint. I do have an additional question though, which is what you think about how to to evaluate moments that have a combination of good and bad.
For instance, let’s suppose you have the best day ever, except that you had a mild pain in your leg for the most of the day. All the awesome stuff you did during the day more than made up for that mild pain though.
Now let’s suppose you are offered the prospect of having a copy of you repeat that day exactly. We both agree that doing this would add no additional value, the question is whether it would be valueless, or add disvalue?
There are two possible ways I see to evaluate this:
You could add up all the events of the day and decide they contain more good than bad, therefore this was a “good” day. “Good” things have no value when repeated, so you would assign zero value to having a copy relive this day. You would not pay to have it happen, but you also wouldn’t exert a great effort to stop it.
You could assign value to the events first before adding them up, assigning zero value to all the good things and a slight negative value to the pain in your leg. Therefore you would assign negative value to having a copy relive this day and would pay to stop it from happening.
To me (1) seems to be an intuitively better way of evaluating the prospect of a copy reliving the day than (2). It also lines up with my intuition that it wouldn’t be bad news if MWI was true. But I wonder if you would think differently?
For my own part, I share your #1 and #2, don’t share your #3 (that is, I’d rather Omega not reproduce the bad stuff, but if they’re going to do so, it makes no real difference to me whether they reproduce the good stuff as well)
One thing that makes me inclined towards #3 is the possibility that the multiverse is constantly reproducing my life over and over again, good and bad. I do not think that I would consider it devastatingly bad news if it turns out that the Many-Worlds interpretation is correct.
If I really believed that repeated bad experiences could not ever be compensated for by repeated good ones, I would consider the Many Worlds Interpretation to be the worst news ever, since there were tons of me out in the multiverse having a mix of good and bad experiences, but the good ones “don’t count” because they already happened somewhere else. But I don’t consider it bad news. I don’t think that if there was a machine that could stop the multiverse from splitting that I would pay to have it built.
One way to explain my preferences in this regard would be that I believe that repeated “good stuff” can compensate for repeated “bad stuff,” but that it can’t compensate for losing brand new “good stuff” or experiencing brand new “bad stuff.”
However, I am not certain about this. There may be some other explanation for my preferences. Another possibility that I think is likely is that I think that repeated “good stuff” only loses its value for copies that have a strong causal connection to the current me. Other mes who exist somewhere out in the multiverse have no connection to this version of me whatsoever, so my positive experiences don’t detract from their identical ones. But copies that I pay to have created (or to not be) are connected to me in such a fashion, so I (and they) do feel that their repeated experiences are less valuable.
This second explanation seems a strong contender as well, since I already have other moral intuitions in regards to causal connection (for instance, if there was a Matrioshka brain full of quintillions environmentalists in a part of the multiverse so far off they will never interact with us, I would not consider their preferences to be relevant when forming environmental policy, but I would consider the preferences of environmentalists here on Earth right now to be relevant). This relates to that “separability” concept we discussed a while ago.
Or maybe both of these explanations are true. I’m not sure.
Also, I’m curious, why are you indifferent in case 4? I think I might not have explained it clearly. What I was going for was that Omega say “I’m making a copy of you in a bad time of your life. I can either not do it at all, or extend the copy’s lifespan so that it is now a copy of a portion of your life that had both good and bad moments. Both options cost $10.” I am saying that I think I might be indifferent about what I spend $10 on in that case.
I don’t see anything inconsistent about believing that a good life loses values with repetition, but a bad life does not lose disvalue. It’s consistent with the Value of Boredom, which I thoroughly endorse.
Now, there’s a similar question where I think my thoughts on the subject might get a little weird. Imagine you have some period of your life that started out bad, but then turned around and then became good later so that in the end that period of life was positive on the net. I have the following preferences in regards to duplicating it:
I would not pay to have a simulation that perfectly relived that portion of my life.
If Omega threatened to simulate the bad first portion of that period of life, but not the good parts that turned it around later, I would pay him not to.
If Omega threatened to simulate the bad first portion of that period of life, but not the good parts that turned it around later, I would probably pay him to extend the length of the simulation so that it also encompassed the compensating good part of that period of life.
If the cost of 2 and 3 was identical I think would probably be indifferent. I would not care whether the simulation never occurred, or if it was extended.
So it seems like I think that repeated good experiences can sometimes “make up for” repeated bad ones, at least if they occur in the same instance of simulation. But all they can do is change the value I give to the simulation from “negative” to “zero.” They can’t make it positive.
These preferences I have do strike me as kind of weird. But on the other hand, the whole situation is kind of weird, so maybe any preferences I have about it will end up seeming weird no matter what they are.
It’s been a long time since you posted this, but if you see my comment, I’d be curious about what some others patches one could apply are. I have pretty severe scrupulosity issues around population ethics and often have trouble functioning because I can’t stop thinking about them. I dislike pure total utilitarianism, but I have trouble rejecting it precisely because of “galaxy far far away” type issues. I spend a lot of time worrying about the idea that I am forced to choose between two alternatives:
1) That (to paraphrase what you said in your critique of total utilitarianism) it is a morally neutral act to kill someone if you replace them with someone whose lifetime utility is equal to the first person’s remaining lifetime utility (and on a larger scale, the Repugnant Conclusion), or
2.That the human race might be obligated to go extinct if it turns out there is some utopia in some other branch of the multiverse, or the Andromeda Galaxy, or in some ancient, undiscovered fallen civilization in the past. Or that if the Earth was going to explode and I could press a button to save it, but it would result in future generations living slightly lower quality lives than present generations, I shouldn’t push the button.
I’d really like to know some ways that I can reject both 1 and 2. I really admire your work on population ethics and find that your thinking on the subject is really closely aligned with my own, except that you’re better at it than me :)