Ghatanathoah

• Ghatanathoah 12 Nov 2021 7:12 UTC

  1 point

  in reply to: JBlack’s comment on: How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

  My main objection for the simplified utility functions is that they are presented as depending only upon the current external state of the world in some vaguely linear and stable way. Every adjective in there corresponds to discarding a lot of useful information about preferences that people actually have.

  The main argument I’ve heard for this kind of simplification is that your altruistic, morality-type preferences ought to be about the state of the external world because their subject is the wellbeing of other people, and the external world is where other people live. The linearity part is sort of an extension of the principle of treating people equally. I might be steelmanning it a little, a lot of times the argument is less that and more that having preferences that are in any way weird or complex is “arbitrary.” I think this is based on the mistaken notion that “arbitrary” is a synonym for “picky” or “complicated.”

  I find this argument unpersuasive because altruism is also about respecting the preferences of others, and the preferences of others are, as you point out, extremely complicated and about all sorts of things other than the current state of the external world. I am also not sure that having nonlinear altruistic preferences is the same thing as not valuing people equally. And I think that our preferences about the welfare of others are often some of the most path-dependent preferences that we have.

  EDIT: I have sense found this post, which discusses some similar arguments and refutes them more coherently than I do.

  Second EDIT: I still find myself haunted by the “scary situation” I linked to and find myself wishing there was a way to tweak a utility function a little to avoid it, or at least get a better “exchange rate” than “double tiny good thing and more-than doubling horrible thing while keeping probability the same.” I suppose there must be a way since the article I linked to said it would not work on all bounded utility functions.

• Ghatanathoah 11 Nov 2021 2:04 UTC

  1 point

  in reply to: JBlack’s comment on: How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

  In general though, this consideration is likely to be irrelevant. Most universes will be nowhere near the upper or lower bounds, and the chance of any individual’s decision being single-handedly responsible for doing a universe scale shifts toward a utility bound is so tiny that even estimating orders of magnitude of the unlikelihood is difficult. These are angels-on-head-of-pin quibbles.

  That makes sense. So it sounds like the Egyptology Objection is almost a form of Pascal’s Mugging in and of itself. If you are confronted by a Mugger (or some other, slightly less stupid scenario where there is a tiny probability of vast utility or disutility) the odds that you are at a “place” on the utility function that would affect the credibility threshold for the Mugger one way or another are just as astronomical as the odds that the Mugger is giving you. So an agent with a bounded utility function is never obligated to research how much utility the rest of the universe has before rejecting the mugger’s offer. They can just dismiss it as not credible and move on.

  And Mugging-type scenarios are the only scenarios where this Egyptology stuff would really come up, because in normal situations with normal probabilities of normal amounts of (dis)utility, the rescaling and reshifting effect makes your “proximity to the bound” irrelevant to your behavior. That makes sense!

  I also wanted to ask about something you said in an earlier comment:

  I suspect most of the “scary situations” in these sorts of theories are artefacts of trying to formulate simplified situations to test specific principles, but accidentally throw out all the things that make utility functions a reasonable approximation to preference ordering. The quoted example definitely fits that description.

  I am not sure I understand exactly what you mean by that. How do simplified hypotheticals for testing specific principles make utility functions fail to approximate preference ordering? I have a lot of difficulty with this, where I worry that if I do not have the perfect answer to various simplified hypotheticals it means that I do not understand anything about anything. But I also understand that simplified hypotheticals often causes errors like removing important details and reifying concepts.

• Ghatanathoah 9 Nov 2021 6:52 UTC

  1 point

  in reply to: JBlack’s comment on: How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

  The main “protection” of bounded utility is that at every point on the curve, the marginal utility of money is nonzero, and the threat of disutility is bounded. So there always exists some threshold credibility below which no threat (no matter how bad) makes expected utility positive for paying them.

  That makes sense. What I am trying to figure out is, does that threshold credibility change depending on “where you are on the curve.” To illustrate this, imagine two altruistic agents, A and B, who have the same bounded utility function. A lives in a horrifying hell world full of misery. B lives in a happy utopia. So A is a lot “closer” to the lower bound than B. Both A and B are confronted by a Pascal’s Mugger who threatens them with an arbitrarily huge disutility.

  Does the fact that agent B is “farther” from lower bound than agent A mean that the two agents have different credibility thresholds for rejecting the mugger? Because the amount of disutility that B needs to receive to get close to the lower bound is larger than the amount that A needs to receive? Or will their utility functions have the same credibility threshold because they have the same lower and upper bounds, regardless of “how much” utility or disutility they happen to “possess” at the moment? Again, I do not know if this is a coherent question or if it is born out of confusion about how utility functions work.

  It seems to me that an agent with a bounded utility function shouldn’t need to do any research about the state of the rest of the universe before dismissing Pascal’s Mugging and other tiny probabilities of vast utilities as bad deals. That is why this question concerns me.

  One continuous example of this is an exponential discounter, where the decisions are time-invariant but from a global view the space of potential future utility is exponentially shrinking.

  Thanks, that example made it a lot easier to get my head around the idea! I think understand it better now. This might not be technically accurate, but to me having a uniform rescaling and reshifting of utility that preserves future decisions like that doesn’t even feel like I am truly “valuing” future utility less. I know that in some sense I am, but it feels more like I am merely adjusting and recalibrating some technical details of my utility function in order to avoid “bugs” like Pascal’s Mugging. It feels similar to making sure that all my preferences are transitive to avoid money pumps, the goal is to have a functional decision theory, rather to to change my fundamental values.

• Ghatanathoah 8 Nov 2021 7:26 UTC

  1 point

  in reply to: JBlack’s comment on: How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

  TLDR: What I really want to know is:

  1. Is an agent with a bounded utility function justified (because of their bounded function) in rejecting any “Pascal’s Mugging” type scenario with tiny probabilities of vast utilities, regardless of how much utility or disutility they happen to “have” at the moment? Does everything just rescale so that the Mugging is an equally bad deal no matter what the relative scale of future utility is?

  2. If you have a bounded utility function, are your choices going to be the same regardless of how much utility various unchangeable events in the past generated for you? Does everything just rescale when you gain or lose a lot of utility so that the relative value of everything is the same? I expect the answer is going to be “yes” based on our previous discussion, but am a little uncertain because of the various confused thoughts on the subject that I have been having lately.

  Full length Comment:

  I don’t think I explained my issue clearly. Those arguments about Pascal’s Mugging are addressing it from the perspective of its unlikeliness, rather than using a bounded utility function against it.

  I am trying to understand bounded utility functions and I think I am still very confused. What I am confused about right now is how a bounded utility function protects from Pascal’s Mugging at different “points” along the function.

  Imagine we have a bounded utility function that has a “S” curve shape. The function goes up and down from 0 and flattens as it approaches the upper and lower bounds.

  If someone has utility at around 0, I see how they resist Pascal’s Mugging. Regardless of whether the Mugging is a threat or a reward, it approaches their upper or lower bound and then diminishes. So utility can never “outrace” probability.

  But what if they have a level of utility that is close to the upper bound and a Mugger offers a horrible threat? If the Mugger offered a threat that would reduce their utility to 0, would they respond differently than they would to one that would send it all the way to the lower bound? Would the threat get worse as the utility being cancelled out by the disutility got further from the bound and closer to 0? Or is the idea that in order for a threat/​reward to qualify as a Pascal’s Mugging it has to be so huge that it goes all the way down to a bound?

  And if someone has a level of utility or disutility close to the bound, does that mean disutility matters more so they become a negative utilitarian close to the upper bound and a positive utilitarian close to the lower one? I don’t think that is the case, I think that, as you said, “the relative scale of future utility makes no difference in short-term decisions.” But I am confused about how.

  I think I am probably just very confused in general about utility functions and about bounded utility functions. While some people have criticized bounded utility functions, I have never come across this specific type of criticism before. It seems far more likely that I am confused than that I am the first person to notice an obvious flaw.

• Ghatanathoah 7 Nov 2021 20:34 UTC

  1 point

  in reply to: JBlack’s comment on: How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

  Hi, one other problem occurred to me in regards to short term decisions and bounded utility.

  Suppose you are in a situation where you have a bounded utility function, plus a truly tremendous amount of utility. Maybe you’re an immortal altruist who has helped quadrillions of people, maybe you’re an immortal egoist who has lived an immensely long and happy life. You are very certain that all of that was real, and it is in the past and can’t be changed.

  You then confront a Pascal’s Mugger who threatens to inflict a tremendous amount of disutility unless you give the $5. If you’re an altruist they threaten to torture quintillions of people, if you are an egoist they threaten to torture you for a quintillion years, something like that. As with standard Pascal’s mugging, the odds of them be able to carry this threat out are astronomically unlikely.

  In this case, it still fells like you ought to ignore the mugger. Does that make sense considering that, even though your bounded utility function assigns less disvalue to such a threat, it also assigns less value to the $5 because you have so much utility already? Plus, if they are able to carry out their threat, they would be able to significantly lower your utility so that it is much “further away from the bound” than it was before. Does it matter that as they push your utility further and further “down” away from the bound, utility becomes “more valuable.”

  Or am I completely misunderstanding how bounded utility is calculated? I’ve never seen this specific criticism of bounded utility functions before, and much smarter people than me have studied this issue, so I imagine that I must be? I am not sure exactly how adding utility and subtracting disutility is calculated. It seems like if the immortal altruist whose helped quadrillions of people has a choice between gaining 3 utilons, or inflicting 2 disutilons to gain 5 utilitons, that they should be indifferent between the two, even if they have a ton of utility and very little disutility in their past.

• Ghatanathoah 18 Oct 2021 5:09 UTC

  1 point

  in reply to: JBlack’s comment on: How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

  Thanks, again for your help :) That makes me feel a lot better. I have the twin difficulties of having severe OCD-related anxiety about weird decision theory problems, and being rather poor at the math required to understand them.

  The case of the immortal who becomes uncertain of the reality of their experiences is I think what that “Pascal’s Mugging for Bounded Utilities” article I linked to the the OP was getting at. But it’s a relief to see that it’s just a subset of decisions under uncertainty, rather than a special weird problem.

• Ghatanathoah 18 Oct 2021 1:05 UTC

  1 point

  in reply to: JBlack’s comment on: How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

  the importance to the immortal of the welfare of one particular region of any randomly selected planet of those 10^30 might be less than that of Ancient Egypt. Even if they’re very altruistic.

  Ok, thanks, I get that now, I appreciate your help. The thing I am really wondering is, does this make any difference at all to how that immortal would make decisions once Ancient Egypt is in the past and cannot be changed? Assuming that they have one of those bounded utility functions where their utility is asymptotic to the bound, but never actually reaches it, I don’t feel like it necessarily would.

  If Ancient Egypt is in the past and can’t be changed, the immortal might, in some kind of abstract sense, value that randomly selected planet of those 10^30 worlds less than they valued Egypt. But if they are actually in a situation where they are on that random planet, and need to make altruistic decisions about helping the people on that planet, then their decisions shouldn’t really be affected. Even if the welfare of that planet is less valuable to them than the welfare of Ancient Egypt, that shouldn’t matter if their decisions don’t affect Ancient Egypt and only affect the planet. They would be trading less valuable welfare off against other less valuable welfare, so it would even out. Since their utility function is asymptotic to the bound, they would still act to increase their utility, even if the amount of utility they can generate is very small.

  I am totally willing to accept the Egyptology argument if all it is saying is that past events that cannot be changed might affect the value of present-day events in some abstract sense (at least if you have a bounded utility function). Where I have trouble accepting it is if those same unchangeable past events might significantly affect what choices you have to make about future events that you can change. If future welfare is only 0.1x as valuable as past welfare, that doesn’t really matter, because future welfare is the only welfare you are able to affect. If it’s only possible to make a tiny difference, then you might as well try, because a tiny difference is better than no difference. The only time when the tininess seems relevant to decisions is Pascal’s Mugging type scenarios where one decision can generate tiny possibilities of huge utility.

• Ghatanathoah 16 Oct 2021 7:35 UTC

  1 point

  in reply to: JBlack’s comment on: How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

  The phrasing here seems to be a confused form of decision making under uncertainty. Instead of the agent saying “I don’t know what the distribution of outcomes will be”, it’s phrased as “I don’t know what my utility function is”.

  I think part of it is that I am conflating two different parts of the Egyptology problem. One part is uncertainty: it isn’t possible to know certain facts about the welfare of Ancient Egyptians that might affect how “close to the bound” you are. The other part is that most people have a strong intuition that those facts aren’t relevant to our decisions, whether we are certain of them or not. But there’s this argument that those facts are relevant if you have an altruistic bounded utility function because they affect how much diminishing returns your function has.

  For example, I can imagine that if I was an altruistic immortal who was alive during ancient Egypt, I might be unwilling to trade a certainty of a good outcome in ancient Egypt for an uncertain amazingly terrific outcome in the far future because of my bounded utility function. That’s all good, it should help me avoid Pascal’s Mugging. But once I’ve lived until the present day, it feels like I should continue acting the same way I did in the past, continue to be altruistic, but in a bounded fashion. It doesn’t feel like I should conclude that, because of my achievements as an altruist in Ancient Egypt, that there is less value to being an altruist in the present day.

  In the case of the immortal, I do have all the facts about Ancient Egypt, but they don’t seem relevant to what I am doing now. But in the past, in Egypt, I was unwilling to trade certain good outcomes for uncertain terrific ones because my bounded utility function meant I didn’t value the larger ones linearly. Now that the events of Egypt are in the past and can’t be changed, does that mean I value everything less? Does it matter if I do, if the decrease in value is proportionate? If I treat altruism in the present day as valuable, does that contradict the fact that I discounted that same value back in Ancient Egypt?

  I think that’s why I’m phrasing it as being uncertain of what my utility function is. It feels like if I have a bounded utility function, I should be unwilling (within limits) to trade a sure thing for a small possibility of vast utility, thereby avoiding Pascal’s Mugging and similar problems. But it also feels like, once I have that sure thing, and the fact that I have it cannot be changed, I should be able to continue seeking more utility, and how many sure things I have accumulated in the past should not change that.

• Ghatanathoah 15 Oct 2021 22:34 UTC

  1 point

  in reply to: JBlack’s comment on: How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

  I really still don’t know what you mean by “knowing how close to the bound you are”.

  What I mean is, if I have a bounded utility function where there is some value, X, and (because the function is bounded) X diminishes in value the more of it there is, what if I don’t know how much X there is?

  For example, suppose I have a strong altruistic preference that the universe have lots of happy people. This preference is not restricted by time and space, it counts the existence of happy people as a good thing regardless of where or when they exist. This preference is also agent neutral, it does not matter whether I, personally, am responsible for those people existing and being happy, it is good regardless. This preference is part of a bounded utility function, so adding more happy people starts to have diminishing returns the closer one gets to a certain bound. This allows me to avoid Pascal’s Mugging.

  However, if adding more people has diminishing returns because the function is bounded, and my preference is not restricted by time, space, or agency, that means that I have no way of knowing what those diminishing returns are unless I know how many happy people have ever existed in the universe. If there are diminishing returns based on how many people there are, total, in the universe, then the value of adding more people in the future might change depending on how many people existed in the past.

  That is what I mean by “knowing how close to the bound” I am. If I value some “X”, what if it isn’t possible to know how much X there is? (like I said before, a version of this for egoistic preferences might be if the X is happiness over your lifetime, and you don’t know how much X there is because you have amnesia or something).

  I was hoping that I might be able to fix this issue by making a bounded utility function where X diminishes in value smoothly and proportionately. So a million happy people in ancient Egypt has proportional diminishing returns to a billion and so on. So when I am making choices about maximizing X in the present, the amount of X I get is diminished in value, but it is proportionately diminished, so the decisions that I make remain the same. If there was a vast population in the past, the amount of X I can generate has very small value according to a bounded utility function. But that doesn’t matter because it’s all that I can do.

  That way, even if X decreases in value the more of it there is, it will not effect any choices I make where I need to choose between different probabilities of getting different amounts of X in the future.

  I suppose I could also solve it by making all of my preferences agent-relative instead of agent-neutral, but I would like to avoid that. Like most people I have a strong moral intuition that my altruistic preferences should be agent-neutral. I suppose it might also get me into conflict with other agents with bounded agent-relative utility functions if we value the same act differently.

  If I am explaining this idea poorly, let me try directing you to some of the papers I am referencing. Besides the one I mentioned in the OP, there is this one by Beckstead and Thomas (pages 16, 17, and 18 are where it discusses it).

• Ghatanathoah 10 Oct 2021 7:07 UTC

  1 point

  in reply to: JBlack’s comment on: How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

  REA doesn’t help at all there, though. You’re still computing U(2X days of torture) - U(X days of torture)

  I think I see my mistake now, I was treating a bounded utility function using REA as subtracting the “unbounded” utilities of the two choices and then comparing the post-subtraction results using the bounded utility function. It looks like you are supposed to judge each one’s utility by the bounded function before subtracting them.

  Unfortunately REA doesn’t change anything at all for bounded utility functions. It only makes any difference for unbounded ones.

  That’s unfortunate. I was really hoping that it could deal with the Egyptology scenario by subtracting the unknown utility value of Ancient Egypt and only comparing the difference in utility between the two scenarios. That way the total utilitarian (or some other type of altruist) with a bounded utility function would not need to research how much utility the people of Ancient Egypt had in order to know how good adding happy people to the present day world is. That just seems insanely counterintuitive.

  I suppose there might be some other way around the Egyptology issue. Maybe if you have a bounded or nonlinear utility function that is sloped at the correct rate it will give the same answer regardless of how happy the Ancient Egyptians were. If they were super happy then the value of whatever good you do in the present is in some sense reduced. But the value of whatever resources you would sacrifice in order to do good is reduced as well, so it all evens out. Similarly, if they weren’t that happy, the value of the good you do is increased, but the value of whatever you sacrifice in order to do that good is increased proportionately. So a utilitarian can go ahead and ignore how happy the ancient Egyptians were when doing their calculations.

  It seems like this might work if the bounded function has adding happy lives have diminishing returns at a reasonably steady and proportional rate (but not so steady that it is effectively unbounded and can be Pascal’s Mugged).

  With the “long lived egoist” example I was trying to come up with a personal equivalent to the Egyptology problem. In the Egyptology problem, a utilitarian does not know how close they are to the “bound” of their bounded utility function because they do not know how happy the ancient Egyptians were. In the long lived egoist example, they do not know how close to the bound they are because they don’t know exactly how happy and long lived their past self was. It also seems insanely counterintuitive to say that, if you have a bounded utility function, you need to figure out exactly how happy you were as a child in order to figure out how good it is for you to be happy in the future. Again, I wonder if a solution might be to have a bounded utility function with returns that diminish at a steady and proportional rate.

• Ghatanathoah 9 Oct 2021 5:10 UTC

  1 point

  in reply to: qwertyasdef’s comment on: How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

  Thank you for your reply. That was extremely helpful to have someone crunch the numbers. I am always afraid of transitivity problems when considering ideas like this, and I am glad it might be possible to avoid the Egyptology objection without introducing any.

• Ghatanathoah 9 Oct 2021 5:08 UTC

  1 point

  in reply to: JBlack’s comment on: How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

  Thanks a lot for the reply. That makes a lot of sense and puts my mind more at ease.

  To me this sounds more like any non-linear utility, not specifically bounded utility.

  You’re probably right, a lot of my math is shaky. Let me try to explain the genesis of the example I used. I was trying to test REA for transitivity problems because I thought that it might have some further advantages to conventional theories. In particular, it seemed to me that by subtracting before averaging, REA could avoid the two examples those articles I references:

  1. The total utilitarian with a bounded utility function who needs to research how many happy people lived in ancient Egypt to establish how “close to the bound” they were and therefore how much they should discount future utility.

  2. The very long lived egoist with a bounded utility function who vulnerable to Pascal’s mugging because they are unsure of how many happy years they have lived already (and therefore how “close to the bound” they were).

  It seemed like REA, by subtracting past utility that they cannot change before doing the calculation, could avoid both those problems. I do not know if those are real problems or if a non-linear/​bounded utility with a correctly calibrated discount rate could avoid them anyway, but it seemed worthwhile to find ways around them. But I was really worried that REA might create intransitivity issues with bounded utility functions, the lottery example I was using was an example of the kind of intransitivity problem that I was thinking of.

  It also occurred to me that REA might avoid another peril of bounded utility functions that I read about in this article. Here is the relevant quote:

  “if you have a bounded utility function and were presented with the following scary situation: “Heads, 1 day of happiness for you, tails, everyone is tortured for a trillion days” you would (if given the opportunity) increase the stakes, preferring the following situation: “Heads, 2 days of happiness for you, tails, everyone is tortured forever. (This particular example wouldn’t work for all bounded utility functions, of course, but something of similar structure would.)”

  It seems like REA might be able to avoid that. If we imagine that the person is given a choice between two coins, since they have to pick one, the “one day of happiness+trillion days of torture” is subtracted beforehand, so all the person needs to do is weigh the difference. Even if we get rid of the additional complications of computing infinity that “tortured forever” creates, by replacing it with some larger number like “2 trillion days”, I think it might avoid it.

  But I might be wrong about that, especially if REA always gives the same answers in finite situations. If that’s the case it just might be better to find a formulation of an unbounded utility function that does its best to avoid Pascal’s Mugging and also the “scary situations” from the article, even if it does it imperfectly.

• Ghatanathoah 7 Oct 2021 6:17 UTC

  1 point

  in reply to: JBlack’s comment on: How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

  That aside, relative expected value is purely a patch that works around some specific problems with infinite expected values, and gives exactly the same results in all cases with finite expected values.

  That’s what I thought as well. But then it occurred to me that REA might not give exactly the same results in all cases with finite expected values if one has a bounded utility function. If I am right, this could result in scenarios where someone could have circular values or end up the victim of a money pump.

  For example, imagine there is a lottery that costs $1 to for a ticket and generates x utility for odds of y. The value for x is very large, the value for y is quite small, like in Pascal’s mugging. A person with a bounded utility function does not enter it. However, imagine that there is another lottery that costs a penny for a ticket, and generates 0.01x utility for odds of y. Because this person’s utility function is bounded, y odds of 0.01x utility is worth a penny to them, even though y odds of x utility is not worth a dollar to them. The person buys a ticket for a penny. Then they are offered a chance to buy another. Because they are using REA, they only count the difference in utility from buying the new ticket, and do not count the ticket they already have, so they buy another. Eventually they buy 100 tickets. Then someone offers to buy the tickets from them for a dollar. Because they have a bounded utility function, y odds of winning x are less valuable than a dollar, so they take the trade. They are now back where they started.

  Does that make sense? Or have I made some sort of error somewhere? (maybe pennies become more valuable the less you have, which would interrupt the cycle?) It seems like with a bounded utility function, REA might have transitivity problems like that. Or have I made a mistake and misunderstood how to discount using REA?

  I am really concerned about this, because REA seems like a nice way to address the Egyptology objection to bounded utility functions. You don’t need to determine how much utility already exists in the world by studying Ancient Egypt, because you only take into account the difference in utility, not the total utility, when calculating where the bound is in your utility function. Ditto for the Pascal’s mugging example. So I really want there to be a way to discount the Egyptology stuff, without also generating intransitive preferences.

[Question] How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

• Ghatanathoah 27 Sep 2021 19:28 UTC

  1 point

  in reply to: Kenny ’s comment on: Dem­i­na­tal­ist To­tal Utilitarianism

  That can be said about any period in life. It’s just a matter of perspective and circumstances. The best years are never the same for different people.

  That’s true, but I think that for the overwhelming majority of people, their childhoods and young adulthoods were at the very least good years, even if they’re not always the best. They are years that contain significantly more good than bad for most people. So if you create a new adult who never had a childhood, and whose lifespan is proportionately shorter, they will have a lower total amount of wellbeing over their lifetime than someone who had a full-length life that included a childhood.

• Ghatanathoah 27 Sep 2021 4:36 UTC

  1 point

  in reply to: Isnasene’s comment on: Dem­i­na­tal­ist To­tal Utilitarianism

  I took a crack at figuring it out here.

  I basically take a similar approach to you. I give animals a smaller -u0 penalty if they are less self-aware and less capable of forming the sort of complex eudaimonic preferences that human beings can. I also treat complex eudaimonic preferences as generating greater moral value when satisfied in order to avoid incentivizing creating animals over creating humans.

• Ghatanathoah 27 Sep 2021 4:26 UTC

  1 point

  in reply to: Tom Penn’s comment on: Dem­i­na­tal­ist To­tal Utilitarianism

  I think another good way to look at u0 that compliments yours is to look at it as the “penalty for dying with many preferences left unsatisfied.” Pretty much everyone dies with some things that they wanted to do left undone. I think most people have a strong moral intuition that being unable to fulfill major life desires and projects is tragic, and think a major reason death is bad is that it makes us unable to do even more of what we want to do with our lives. I think we could have u0 represent that intuition.

  If we go back to Peter Singer’s original formulation of this topic, we can think of unsatisfied preferences as “debts” that are unpaid. So if we have a choice between creating two people who live x years, or 1 person who lives 2x years, assuming their total lifetime happiness is otherwise the same, we should prefer the one person for 2x years. This is because the two people living x years generate the same amount of happiness, but twice the amount of “debt” from unfulfilled preferences. Everyone will die with some unfulfilled preferences because everyone will always want more, and that’s fine and part of being human.

  Obviously we need to calibrate this idea delicately in order to avoid any counterintuitive conclusions. If we treat creating a preference as a “debt” and satisfying it as merely “paying the debt” to “break even” then we get anti-natalism. We need to treat the “debt” that creating a preference generates as an “investment” that can “pay off” by creating tremendous happiness/​life satisfaction when it is satisfied, but occasionally fails to “pay off” if its satisfaction is thwarted by death or something else.

  I think that this approach could also address Isnasene’s question below of figuring out the -u0 penatly for nonhuman animals. Drawing from Singer again, since nonhuman animals are not mentally capable of having complex preferences for the future, they generate a smaller u0 penalty. The preferences that they die without having satisfied are not as strong or complex. This fits nicely with the human intuition that animals are more “replaceable” than humans and are of lesser (although nonzero) moral value. It also fits the intuition that animals with more advanced, human-like minds are of greater moral value.

  Using that approach for animals also underscores the importance of treating creating preferences as an “investment” that can “pay off.” Otherwise it generates the counterintuitive conclusion that we should often favor creating animals over humans, since they have a lower u0 penalty. Treating complex preference creation as an “investment” means that humans are capable of generating far greater happiness/​satisfaction than animals, which more than outweighs our greater u0 penalty.

  We would also need some sort of way to avoid incentivizing the creation of intelligent creatures with weird preferences that are extremely easy to satisfy, or a strong preference for living a short life as an end in itself. This is a problem pretty much all forms of utilitarianism suffer from. I’m comfortable with just adding some kind of hack massively penalizing the creation of creatures with preferences that do not somehow fit with some broad human idea of eudaimonia.

  What links here?

  • Ghatanathoah's comment on Dem­i­na­tal­ist To­tal Utilitarianism by Vanessa Kosoy (27 Sep 2021 4:36 UTC; 1 point)

• Ghatanathoah 27 Sep 2021 4:04 UTC

  1 point

  in reply to: Kenny ’s comment on: Dem­i­na­tal­ist To­tal Utilitarianism

  Point taken, but for the average person, the time period of growing up isn’t just a joyless period where they do nothing but train and invest in the future. Most people remember their childhoods as a period of joy and their college years as some of the best of their lives. Growing and learning isn’t just preparation for the future, people find large portions of it to be fun. So the “existing” person would be deprived of all that, whereas the new person would not be.

• Ghatanathoah 19 Sep 2021 21:17 UTC

  1 point

  in reply to: Decius’s comment on: Dem­i­na­tal­ist To­tal Utilitarianism

  If someone is in a rut and could either commit suicide or take the reprogramming drug (and expects to have to take it four times before randomizing to a personality that is better than rerolling a new one), why is that worse than killing them and allowing a new human to be created?

  If such a drug is so powerful that the new personality is essentially a new person, then you have created a new person whose lifespan will be a normal human lifespan minus however long the original person lived before they got in a rut. By contrast, if they commit suicide and you create a new human, you have created a new person who will likely live a normal human lifespan. So taking the drug even once is clearly worse than suicide + replacement since, all else being equal, it is better to create a someone with a longer lifespan than a shorter one (assuming their lifespan is positive, overall, of course).

• Ghatanathoah 15 Sep 2021 0:39 UTC

  1 point

  in reply to: Stuart_Armstrong’s comment on: Ex­am­ple pop­u­la­tion ethics: or­dered dis­counted utility

  So if they don’t want to be killed, that counts as a negative if we do that, even if we replace them with someone happier.

  
  I have that idea as my “line of retreat.” My issue with it is that it is hard to calibrate it so that it leaves as big a birth-death asymmetry as I want without degenerating into full-blown anti-natalism. There needs to be some way to say that the new happy person’s happiness can’t compensate for the original person’s death without saying that the original person’s own happiness can’t compensate for their own death, which is hard. If I calibrate it to avoid anti-natalism it becomes such a small negative that it seems like it could easily be overcome by adding more people with only a little more welfare.

  There’s also the two step “kill and replace” method, where in step one you add a new life barely worth living without affecting anyone. Since the new person exists now, they count the same as everyone else, so then in the second step you kill someone and transfer their resources to the new person. If this process gives the new person the same amount of utility as the old one, it seems neutral under total utilitarianism. I suppose under total preference utilitarianism its somewhat worse, since you now have two people dying with unsatisfied preferences instead of one, but it doesn’t seem like a big enough asymmetry for me.

  I feel like in order to reject the two step process, and to have as big an asymmetry as I want, I need to be able to reject “mere addition” and accept the Sadistic Conclusion. But that in turn leads to “galaxy far far away issues” where it becomes wrong to have children because of happy people in some far off place. Or “Egyptology” issues where its better for the world to end than for it to decline so future people have somewhat worse lives, and we are obligated to make sure the Ancient Egyptians didn’t have way better lives than ours before we decide on having children. I just don’t know. I want it to stop hurting my brain so badly, but I keep worrying about how there’s no solution that isn’t horrible or ridiculous.

  This has degenerate solutions too—it incentivises producing beings that are very easy to satisfy and that don’t mind being killed.

  For this one, I am just willing to just decree that creating creatures with a diverse variety of complex human-like psychologies is good, and creating creatures with weird minmaxing unambitious creatures is bad (or at least massively sub-optimal). To put it another way, Human Nature is morally valuable and needs to be protected.

  Another resource that helped me on this was Derek Parfit’s essay “What Makes Someone’s Life Go Best.” You might find it helpful, it parallels some of your own work on personal identity and preferences. The essay describes which of our preferences we feel count as part of our “self interest” and which do not. It helped me understand things, like why people general feel obligated to respect people’s “self interest” preferences (i.e. being happy, not dying), but not their “moral preferences” (i.e. making the country a theocracy, executing heretics).

  Parfit’s “Success Theory,” as he calls it, basically argues that only preferences that are “about your own life” count as “welfare” or “self interest.” So that means that we would not be making the world a better place by adding lives who prefer that the speed of light stay constant, or that electrons keep having negative charges. That doesn’t defuse the problem entirely, you could still imagine creating creatures with super unambitious life goals. But it gets it part of the way, the rest, again, I deal with by “defending Human Nature.”

  I’m more wanting to automate the construction of values from human preferences

  I had a question about that. It is probably a silly question since my understanding of decision and game theory is poor. When you were working on that you said that there was no independence of irrelevant alternatives. I’ve noticed that IIA is something that trips me up a lot when I think about population ethics. I want to be able to say something like “Adding more lives might be bad if there is still the option to improve existing ones instead, but might be good if the existing ones have already died and that option is foreclosed.” This violates IIA because I am conditioning whether adding more lives is good on whether there is another alternative or not.

  I was wondering if my brain might be doing the thing you described in your post on no IIA, where it is smashing two different values together and getting different results if there are more alternatives. It probably isn’t I am probably just being irrational, but reading that post just felt familiar.