Chantiel

Karma: 88

Chantiel Nov 12, 2021, 8:09 PM
1 point
in reply to: conchis’s comment on: A system of infinite ethics
Thanks for the response.

Third, the average view prefers arbitrarily small populations over very large populations, as long as the average wellbeing was higher. For example, a world with a single, extremely happy individual would be favored to a world with ten billion people, all of whom are extremely happy but just ever-so-slightly less happy than that single person.

In an infinite universe, there’s already infinitely-many people, so I don’t think this applies to my infinite ethical system.

First, consider a world inhabited by a single person enduring excruciating suffering. The average view entails that we could improve this world by creating a million new people whose lives were also filled with excruciating suffering if the suffering of the new people was ever-so-slightly less bad than the suffering of the original person.

Second, the average view entails the sadistic conclusion: It can sometimes be better to create lives with negative wellbeing than to create lives with positive wellbeing from the same starting point, all else equal.

In a finite universe, I can see why those verdicts would be undesirable. But in an infinite universe, there’s already infinitely-many people at all levels of suffering. So, according to my own moral intuition at least, it doesn’t seem that these are bad verdicts.

You might have differing moral intuitions, and that’s fine. If you do have an issue with this, you could potentially modify my ethical system to make it an analogue of total utilitarianism. Specifically, consider the probability distribution something would have if it conditions on it ending up somewhere in this universe, but doesn’t even know if it will be an actual agent with preferences or not.That is, it uses some prior that allows for the possibility that of ending up as a preference-free rock or something. Also, make sure the measure of life satisfaction treats existences with neutral welfare and the existences of things without preferences as zero. Now, simply modify my system to maximize the expected value of life satisfaction, given this prior. That’s my total-utilitarianism-infinite-analog ethical system.

So, to give an example of how this works, consider the situation in which you can torture one person to avoid creating a large number of people with pretty decent lives. Well, the large number of people with pretty decent lives would increase the moral value of the world, because creating those people makes it more likely that a prior that something would end up as an agent with positive life satisfaction rather than some inanimate object, conditioning only on being something in this universe. But adding a tortured creature would only decrease the moral value of the universe. Thus, this total-utilitarian-infinite-analogue ethical system would prefer create the large number of people with decent lives than to tortured one creature.

Of course, if you accept this system, then you have to a way to deal with the repugnant conclusion, just like you need to find a way to deal with it using regular total utilitarian in a finite universe. I’ve yet to see any satisfactory solution to the repugnant conclusion. But if there is one, I bet you could extend it to this total-utilitarian-infinite-analogue ethical system. This is because because this ethical system is a lot like regular total utilitarian, except it replaces, “total number of creatures with satisfaction x” with “total probability mass of ending up as a creature with satisfaction x”.

Given the lack of a satisfactory solution to the repugnant conclusion, I prefer the idea of just sticking with my average-utilitarianism-like infinite ethical system. But I can see why you might have different preferences.

Chantiel Nov 10, 2021, 11:31 PM
1 point
in reply to: Slider’s comment on: A system of infinite ethics

Under my eror model you run into trouble when you treat any transfininte amount the same. From that perspective recognising two transfinite amounts that could be different is progress.

I guess this is the part I don’t really understand. My infinite ethical system doesn’t even think about transfinite quantities. It only considers the prior probability over ending up in situations, which is always real-valued. I’m not saying you’re wrong, of course, but I still can’t see any clear problem.

Another attempt to throw a situation you might not be able to handle. Instead of having 2 infinite groups of unknown relative size all receiving the same bad thing as compensation for the abuse 1 slice of cake for one gorup and 2 slices of cake for the second group. Could there be a difference in the group size that perfectly balances the cake slice difference in order to keep cake expectation constant?

Are you asking if there is a way to simultaneously change the group size as well as change the relative amount of cake for each group so the expected number of cakes received is constant?

If this is what you mean, then my system can deal with this. First off, remember that my system doesn’t worry about the number of agents in a group, but instead merely cares about the probability of an agent ending up in that group, conditioning only on being in this universe.

By changing the group size, however you define it, you can affect the probability of you ending up in that group. To see why, suppose you can do something to add any agents in a certain situation-description into the group. Well, as long as this situation has a finite description length, the probability of ending up in that situation is non-zero, so thus stopping them from being in that situation can decrease the probability of you ending up in that group.

So, currently, the expected value of cake received from these situations is P(in first group) * 1 + P(in second group) * 2. (For simplicity, I’m assuming no one else in the universe gets cake.) So, if you increase the number of cakes received by the second group by u, you just need to decrease P(in the first group) by 2u to keep the expectation constant.

Additional challenging situation. Instead of giving 1 or 2 slices of cake say that each slice is 3 cm wide so the original choices are between 3 cm of cake and 6 cm of cake. Now take some custom amount of cake slice (say 2.7 cm) then determine what would be group size to keep the world cake expectation the same. Then add 1 person to that group. Then convert that back to a cake slice width that keeps cake expectation the same. How wide is the slice?.

If literally only one more person gets cake, even considering acaucal effects, then this would in general not affect the expected value of cake. So the slice would still be 2.7cm.

Now, perhaps you meant that you directly cause one more person to get cake, resulting acausally in infinitely-many others getting cake. If so, then here’s my reasoning:

Previously, the expected value of cake received from these situations was P(in first group) * 1 + P(in second group) * 2. Since cake size in non-constant, let’s add a variable to this. So let’s use P(in first group) * u + P(in second group) * 2. I’m assuming only the 1-slice group gets its cake amount adjusted; you can generalize beyond this. u represents the amount of cake the first group gets, with one 3cm slice being represented as 1.

Suppose adding the extra person acausally results in an increase in the probability of ending up in the first group by $ϵ$ . So then, to avoid changing the expected value of cake, we need P(old probability of being in first group) * 1 = (P(old probability of being in first group) + $\epsilon) * u.

Solve that, and you get u = P(old probability of being in first group) / (P(old probability of being in first group) + $\epsilon). Just plug in the exact numbers of how much adding the person changes the probability of of ending up in the group, and you can get an exact slice width.

Another formulation of the same challenge: Define a real number r for which converting that to a group size would get you a group of 5 people.

I’m not sure what you mean here. What does it mean to convert a real number to a group size? One trivial way to interpret this is that the answer is 5: if you convert 5 to a group size, I guess(?) that means a group of five people. So, there you go, the answer would be 5. I take it this isn’t what you meant, though.

Did you get on board about the difference between “help all the stars” and “all the stars as they could have been”?

No, I’m still not sure what you mean by this.

Chantiel Nov 10, 2021, 10:02 PM
1 point
in reply to: conchis’s comment on: A system of infinite ethics

My point was more that, even if you can calculate the expectation, standard versions of average utilitarianism are usually rejected for non-infinitarian reasons (e.g. the repugnant conclusion) that seem like they would plausibly carry over to this proposal as well.

If I understand correctly, average utilitarianism isn’t rejected due to the repugnant conclusion. In fact, it’s the opposite: the repugnant conclusion is a problem for total utilitarianism, and average utilitarianism is one way to avoid the problem. I’m just going off what I read on The Stanford Encyclopedia of Philosophy, but I don’t have particular reason to doubt what it says.

Separately, while I understand the technical reasons for imposing boundedness on the utility function, I think you probably also need a substantive argument for why boundedness makes sense, or at least is morally acceptable. Boundedness below risks having some pretty unappealing properties, I think.

Yes, I do think boundedness is essential for a utility function. The issue unbounded utility functions is that the expected value according to some probability distributions will be undefined. For example, if your utility follows a Cauchy distribution, then the expected utility is undefined.

Your actual probability distribution over utilities in an unbounded utility function wouldn’t exactly follow a Cauchy distribution. However, I think that for whatever reasonable probability distribution you would use in real life, an unbounded utility function have still have an undefined expected value.

To see why, note that there is a non-zero probability probability that your utility really will be sampled from a Cauchy distribution. For example, suppose you’re in some simulation run by aliens, and to determine your utility in your life after the simulation ends, they sample from the Cauchy distribution. (This is supposing that they’re powerful enough to give you any utility). I don’t have any completely conclusive evidence to rule out this possibility, so it has non-zero probability. It’s not clear to me why an alien would do the above, or that they would even have the power to, but I still have no way to rule it out with infinite confidence. So your expected utility, conditioning on being in this situation, would be undefined. As a result, you can prove that your total expected utility would also be undefined.

So it seems to me that the only way you can actually have your expected values be robustly well-defined is by having a bounded utility function.

Because the sigmoid function essentially saturates at very low levels of welfare, at some point you seem to end up in a perverse version of Torture vs. dust specks where you think it’s ok (or indeed required) to have 3^^^3 people (whose lives are already sufficiently terrible) horribly tortured for fifty years without hope or rest, to avoid someone in the middle of the welfare distribution getting a dust speck in their eye.

In principle, I do think this could occur. I agree that at first it intuitively seems undesirable. However, I’m not convinced it is, and I’m not convinced that there is a value system that avoids this without having even more undesirable results.

It’s important to note that the sufficiently terrible lives need to be really, really, really bad already. So much so that being horribly tortured for fifty years does almost exactly nothing to affect their overall satisfaction. For example, maybe they’re already being tortured for more than 3^^^^3 years, so adding fifty more years does almost exactly nothing to their life satisfaction.

Maybe it still seems to you that getting tortured for 50 more years would still be worse than getting a dust speck in the eye of an average person. However, if so, consider this scenario. You know you have a 50% chance of being tortured for more than 3^^^^3 years, and a 50% chance not being tortured and living in a regular world. However, you have have a choice: you can agree to get a very minor form of discomfort, like a dust speck in your eye in the case in which you aren’t tortured, and you will as a result tortured for 50 fewer years if you don’t end up in the situation in which you get tortured. So I suppose, given what you say, you would take it. But suppose your were given this opportuinty again. Well, you’d again be able to subtract 50 years of torture and get just a dust speck, so I guess you’d take it.

Imagine you’re allowed to repeat this process for an extremely long time. If you think that getting one dust speck is worth it to avoid 50 years of torture, then I think you would keep accepting one more dust speck until your eyes have as much dust in them as they possibly could. And then, once you’re done this this, you could go on to accepting some other extremely minor form of discomfort to avoid another 50 years of torture. Maybe you you start accepting an almost-exactly-imperceptible amount of back pain for another 50 years of torture reduction. And then continue this until your back, and the rest of your body parts, hurt quite a lot.

Here’s the result of your deals: you have a 50% chance of being incredibly uncomfortable. Your eyes are constantly blinded and heavily irritated by dust specs, and you feel a lot of pain all over your body. And you have a 50% chance of being horribly tortured for more than 3^^^^3 years. Note that even though you get your tortured sentence reduced by 50 * <number extremely minor discomforts you get> years, this results in the amount of time your spend tortured would decrease by a very, very, very, almost infinitesimal proportion.

Personally, I much rather have a 50% chance of being able to have a life that actually decent, even if it means that I won’t get to decrease the amount of time I’d spend possibly getting tortured by a near-infinitesimal proportion.

What if you still refuse? Well, the only way I can think of justifying your refusal is by having an unbounded utility function, so getting an extra 50 years of torture is around as bad as getting the first 50 years of torture. But as I’ve said, the expected values of unbounded utility functions seem to be undefined in reality, so this doesn’t seem like a good idea.

My point from the above is that getting one more dust speck in someone’s eye could in principle be better than having someone be tortured for 50 years, provided the tortured person would already have been tortured by a super-ultra-virtually-infinitely long time anyways.

Chantiel Nov 9, 2021, 2:05 AM
1 point
AF
in reply to: abramdemski’s comment on: Troll Bridge
Oh, I’m sorry; you’re right. I messed up on step two of my proposed proof that your technique would be vulnerable to the same problem.

However, it still seems to me that agents using your technique would also be concerning likely to fail to cross, or otherwise suffer from other problems. Like last time, suppose $⊢ (A =^{'} C r o s s^{'} ⟹ U = - 10)$ and that $A =^{'} C r o s s^{'}$ . So if the agent decides to cross, it’s either because of the chicken rule, because not crossing counterfactually results in utility $\leq$ −10, or because crossing counterfactually results in utility greater than −10.

If the agent crosses because of the chicken rule, then this is a bad reason, so the bridge will blow up.

I had already assumed that not crossing counterfactually results in utility greater than −10, so it can’t be the middle case.

Suppose instead the crossing counterfactual results in a utility greater than −10 utility. This seems very strange. By assumption, it’s provable using the AI’s proof system that $(A =^{'} C r o s s ⟹ U = - 10)$ . And the AI’s counterfactual environment is supposed to line up with reality.

So, in other words, the AI has decided to cross and has already proven that crossing entails it will get −10 utility. And if the counterfactual environment assigns greater than −10 utility, then that counterfactual environment provably, within the agent’s proof system, doesn’t line up with reality. So how do you get an AI to both believe it will cross, believe crossing entails −10 utility, and still counterfactually thinks that crossing will result in greater than −10 utility?

In this situation, the AI can prove, within its own proof system, that the counterfactual environment of getting > −10 utility is wrong. So I guess we need an agent that allows itself to use a certain counterfactual environment even though the AI already proved that it’s wrong. I’m concerned about the functionality of such an agent. If it already ignores clear evidence that it’s counterfactual environment is wrong in reality, then that would really make me question that agent’s ability to use counterfactual environments in other situations that line up in reality.

So it seems to me that for an agent using your take on counterfactuals to cross, it would need to either think that not crossing counterfactually results in utility $\leq - 10$ , or to ignore conclusive evidence that the counterfactual environment it’s using for its chosen action would in fact not line up with reality. Both of these options seem rather concerning to me.

Also, even if you do decide to let the AI ignore conclusive evidence (to the AI) that crossing makes utility be −10, I’m concerned the bridge would get blown up anyways. I know we haven’t formalized “a bad reason”, but we’ve taken it to mean something like, “something that seems like a bad reason to the AI”. If the AI wants its counterfactual environments to line up with reality, and it can clearly see that, for the action it decides to take, it doesn’t line up with reality, then this seems like a “bad” reason to me.

Chantiel Nov 8, 2021, 8:49 PM
1 point
in reply to: Slider’s comment on: A system of infinite ethics
Thanks for clearing some things up. There are still some things I don’t follow, though.

You said my system would be ambivalent between between sand and insult. I just wanted to make sure I understand what you’re saying here. Is insult specifically throwing sand at the same people that get it thrown at in dust, and get the sand amount of sand thrown at them at the same throwing speed? If so, then it seems to me that my system would clearly prefer sand to insult. This is because there in some non-zero chance of an agent, conditioning only on being in this universe, being punched due to people like me choosing insult. This would make their satisfaction lower than it otherwise would be, thus decreasing the moral value of the universe if I chose insult over sand.

On the other hand, perhaps the people harmed by sand from “insult” would be lower than the number harmed by sand in “dust”. In this situation, my ethical system could potentially prefer insult over dust. This doesn’t seem like a bad thing to me, though, if it means you save some agents in certain agent-situation-descriptions from getting sand thrown at them.

Also, I’m wondering about your paragraph starting with, “The basic sitatuino is that I have intuitions which I can’t formulate that well. I will try another route.” If I’m understanding it correctly, I think I more or less agree with what you said in that paragraph. But I’m having a hard time understanding the significance of it. Are you intending to show a potential problem with my ethical system using it? The paragraph after it makes it seem like you were, but I’m not really sure.

Chantiel Nov 7, 2021, 7:39 PM
1 point
in reply to: gjm’s comment on: A system of infinite ethics

The fact that it’s lavishly uncomputable is a problem for using it in practice, of course :-).

Yep. To be fair, though, I suspect any ethical system that respects agents’ arbitrary preferences would also be incomputable. As a silly example, consider an agent whose terminal values are, “If Turing machine T halts, I want nothing more than to jump up and down. However, if it doesn’t halt, then it is of the utmost importance to me that I never jump up and down and instead sit down and frown.” Then any ethical system that cares about those preferences is incomputable.

Now this is pretty silly example, but I wouldn’t be surprised if there were more realistic ones. For one, it’s important to respect other agents’ moral preferences, and I wouldn’t be surprised if their ideal moral-preferences-on-infinite-reflection would be incomputable. I seems to me that morall philosophers act as some approximation of, “Find the simplest model of morality that mostly agrees with my moral intuitions”. If they include incomputable models, or arbitrary Turing machines that may or may not halt, then the moral value of the world to them would in fact be incomputable, so any ethical system that cares about preferences-given-infinite-reflection would also be incomputable.

I have some other concerns, but haven’t given the matter enough thought to be confident about how much they matter. For instance: if the fundamental thing we are considering probability distributions over is programs specifying a universe and an experience-subject within that universe, then it seems like maybe physically bigger experience subjects get treated as more important because they’re “easier to locate”, and that seems pretty silly. But (1) I think this effect may be fairly small, and (2) perhaps physically bigger experience-subjects should on average matter more because size probably correlates with some sort of depth-of-experience?

I’m not that worried about agents that are physically bigger, but it’s true that there may be some agents or agents descriptions in situations that are easier to pick out (in terms of having a short description length) then others. Maybe there’s something really special about the agent that makes it easy to pin down.

I’m not entirely sure if this would be a bug or a feature. But if it’s a bug, I think it could be dealt with by just choosing the right prior over agents-situations. Specifically, for any description of an environment with finitely-many agents A, make the probability of ending up as $a \in A$ , conditioned only on being one of the agents in that environment, should be constant for all $a \in A$ . This way, the prior isn’t biased in favor of the agents that are easy to pick out.

Chantiel Nov 7, 2021, 12:03 AM
3 points
AF
in reply to: abramdemski’s comment on: Troll Bridge
If we define “bad reasoning” as “crossing when there is a proof that crossing is bad” in general, this begs the question of how to evaluate actions. Of course the troll will punish counterfactual reasoning which doesn’t line up with this principle, in that case. The only surprising thing in the proof, then, is that the troll also punishes reasoners whose counterfactuals respect proofs (EG, EDT).

I’m concerned that may not realize that your own current take on counterfactuals respects logical to some extent, and that, if I’m reasoning correctly, could result in agents using it to fail the troll bridge problem.

You said in “My current take on counterfactuals”, that counterfactual should line up with reality. That is, the action the agent actually takes should in the utility it was said to have in its counterfactual environment.

You say that a “bad reason” is one such that the agents the procedure would think is bad. The counterfactuals in your approach are supposed to line up with reality, so if an AI’s counterfactuals don’t line up in reality, then this seems like this is a “bad” reason according to the definition you gave. Now, if you let your agent think “I’ll get < −10 utility if I don’t cross”, then it could potentially cross and not get blown up. But this seems like a very unintuitive and seemingly ridiculous counterfactual environment. Because of this, I’m pretty worried it could result in an AI with such counterfactual environments malfunctioning somehow. So I’ll assume the AI doesn’t have such a counterfactual environment.

Suppose acting using a counterfactual environment that doesn’t line up with reality counts as a “bad” reason for agents using your counterfactuals. Also suppose that in the counterfactual environment in which the agent doesn’t cross, the agent counterfactually gets more than −10 utility. Then:
1. Suppose $⊢ A =^{'} C r o s s^{'} ⟹ U = - 10$
2. Suppose $A =^{'} C r o s s^{'}$ . Then if the agent crosses it must be because either it used the chicken rule or because its counterfactual environment doesn’t line up with reality in this case. Either way, this is a bad reason for crossing, so the bridge gets blown up. Thus, the AI gets −10 utility.
3. Thus, $⊢ (⊢ A =^{'} C r o s s^{'} ⟹ U = - 10) ⟹ U = - 10$
4. Thus, by Lob’s theorem, $⊢ A =^{'} C r o s s^{'} ⟹ U = - 10$
Thus, either the agent doesn’t cross the bridge or it does and the bridge explodes. You might just decide to get around this by saying it’s okay for the agent to think it would get less than −10 utility if it didn’t cross. But I’m rather worried that this would cause other problems.

You seem to be assuming that the agent’s architecture has solved the problem of logical updatelessness, IE, of applying reasoning only to the (precise) extent to which it is beneficial to do so. But this is one of the problems we would like to solve! So I object to the “stop thinking about it” step w/o more details of the decision theory which allows you to do so.

I’ll talk about some ways I thought of potentially formalizing, “stop thinking if it’s bad”.

One simple way to try to do so is to have an agent using regular evidential decision theory but have a special, “stop thinking about this thing” action that it can take. Every so often, the agent considers taking this action using regular evidential decision theory. So, in the troll bridge case, it could potentially see that the path of reasoning it’s following is potentially dangerous, and thus decide to stop. Also, the agent needs to avoid thinking too many thoughts before considering to take the “stop thinking about this thing” action. Otherwise, it could think all sorts of problematic thoughts before being able to stop itself.

This simple technique might actually be enough to solve the problem, especially if the AI has the ability to choose its own inference algorithm to find one that makes the AI able to realize, “thinking about this is bad” before it finds the concrete bad thing. And, for what it’s worth, it’s enough for me personally to get across the bridge.

Now, this might not work reliably, because the AI might stumble upon a problematic thought before realizing it’s bad. So, instead, consider this. The AI has an ability to “rollback” its belief state and then adjust its search path. For example, suppose the AI thinks through the Troll Bridge proof and then realizes that it really messed up in doing so. Then it could have a hard-coded way rolling back its belief state to what it was before it thought about the troll bridge thing, mark the nodes it expanded as verboten, and then go expand some other node.

When should it rollback? Well, it shouldn’t rollback every time it comes across any evidence that decreases its expected utility. Doing so would result in the AI being very poorly calibrated by ignoring all negative evidence.

One option to do this is to actually manually come up with a rollback method that both rollbacks whenever doing so increases expected utility without resulting in poor calibration.

However, this sounds sort of hard. One cheap trick get around doing this is to just give the AI a special action to modify its rollback method, and then to select the new rollback method using its regular decision procedure. For example, I think it’s clear, using evidential decision theory, that choosing the rollback method, “Rollback any thought that decreases my expected utility estimate” would result in low utility. Because that would just mean the AI would delude itself. But I bet a rollback method that says, “Roll back if you find any troll bridge proofs” would work okay.

This trick might not be perfect, since the AI could potentially think about problematic before getting a rollback procedure good enough to roll it back. But as long as the AI is smart enough to realize that it should try to get a really good rollback function before doing much of anything else, then I bet it would work okay.

Also, don’t forget that we still need to do something about the agent-simulates-predictor problem. In the agent-simulates-predictor problem, agents are penalized for thinking about things in too much detail. And in whatever counterfactual environment you use, you’ll need a way to deal with the agent-simulates-predictor problem. I think the most obvious approach is by controlling what the AI things about. And if you’ve already done that, then you can pass the troll bridge problem for free.

Also, I think it’s important to note that just the fact the AI is trying to avoid thinking of crossing-is-bad proofs makes the proofs (potentially) not go through. For example, in the proof you originally gave, you supposed there is a proof the crossing results in −10 utility, and thus says the agent must have crossed from the chicken rule. But if the AI is trying to avoid these sorts of “proofs”, then if it does cross, it simply could have been because the AI decided to avoid following whatever train of thought would prove that it would get −10 utility. This is considered a reasonable thing to do by the AI, so it doesn’t seem like a “bad” reason.

There may be possible alternative proofs that apply to an AI that tries to steer its reasoning away from problematic areas. I’m not sure, though. I also suspect that any such proofs would be more complicated and thus harder to find.

Chantiel Nov 5, 2021, 10:18 PM
1 point
in reply to: gjm’s comment on: A system of infinite ethics

So let’s try again. The key thing in your system is not a program that outputs a hypothetical being’s stream of experiences, it’s a program that outputs a complete description of a (possibly infinite) universe and also an unambiguous specification of a particular experience-subject within that universe. This is only possible if there are at most countably many experience-subjects in said universe, but that’s probably OK.

That’s closer to what I meant. By “experience-subject”, I think you mean a specific agent at a specific time. If so, my system doesn’t require an unambiguous specification of an experience-subject.

My system doesn’t require you to pinpoint the exact agent. Instead, it only requires you to specify a (reasonably-precise) description of an agent and its circumstances. This doesn’t mean picking out a single agent, as there many be infinitely-many agents that satisfy such a description.

As an example, a description could be something like, “Someone named gjm in an 2021-Earth-like world with personality <insert a description of your personality and thoughts> who has <insert description of my life experiences> and is currently <insert description of how your life is currently>”

This doesn’t pick out a single individual. There are probably infinitely-many gjms out there. But as long as the description is precise enough, you can still infer your probable eventual life satisfaction.

But other than that, your description seems pretty much correct.

It’s now stupid-o’-clock where I am and I need to get some sleep.

I feel you. I also posted something at stupid-o’-clock and then woke up a 5am, realized I messed up, and then edited a comment and hoped no one saw the previous error.

Chantiel Nov 5, 2021, 9:48 PM
1 point
in reply to: Slider’s comment on: A system of infinite ethics

The integactions are all supposed to be negative in peace, punch, dust, insult. The surprising thing to me would be that the system would be ambivalent between sand and insult being a bad idea. If we don’t necceasrily prefer D to C when helping does it matter if we torture our people a lot or a little as its going to get infinity saturated anyway.

Could you explain what insult is supposed to do? You didn’t say what in the previous comment. Does it causally hurt infinitely-many people?

Anyways, it seems to me that my system would not be ambivalent about whether you torture people a little or a lot. Let C be the class of finite descriptions of circumstances of agents in the universe that would get hurt a little or a lot if you decide to hurt them. The probability of an agent ending up in class C is non-zero. But if you decide to torture them a lot their expected life-satisfaction would be much lower than if you decide to torture them a little. Thus, the total moral value of the universe would be lower if you decide to torture a lot rather than a little.

When I say “world has finite or infinite people” that is “within description” say that there are infinite people because I believe there are infinitely many stars. Then all the acausal copies of sol are going to have their own “out there” stars. Acts that “help all the stars” and “all the stars as they could have been” are different. Atleast until we consider that any agent that decides to “help all the stars” will have acausal shadows “that could have been”. But still this consideration increases the impact on the multiverse (or keeps it the same if moving from a monoverse to a multiverse in the same step).

I can’t say I’m following you here. Specifically, how do you consider, “help all the stars” and “all the stars as they could have been” to be different? I thought, “help” meant, “make it better than it otherwise could have been”. I’m also not sure what counts as acausal shadows. I, alas, couldn’t find this phrase used anywhere else online.

If I have real, non-zero impacts for infinite amount of people naively that would add up to a more than finite aggregate.

Remember that my ethical system doesn’t aggregate anything across all agents in the universe. Instead, it merely considers finite descriptions of situations an agent could be in the universe, and then aggregates the expected value of satisfaction in these situations, weighted by probability conditioning only on being in this universe.

There’s no way for this to be infinite. The probabilities of all the situations sum to 1 (they are assumed to be disjoint), and the measure of life satisfaction was said to be bounded.

And remember, my system doesn’t first find your causal impact on moral value of the universe and then somehow use this to find the acausal impact. Because in our universe, I think the causal impact will always be zero. Instead, just directly worry about acausal impacts. And your acausal impact on the moral value of the universe will always be finite and non-infinitesimal.

Chantiel Nov 5, 2021, 9:11 PM
1 point
in reply to: Richard_Kennaway’s comment on: A system of infinite ethics
Thanks for responding. As I said, the measure of satisfaction is bounded. And all bounded random variables have a well-defined expected value. Source: Stack Exchange.

Chantiel Nov 5, 2021, 12:36 PM
1 point
in reply to: gjm’s comment on: A system of infinite ethics
Oh, I’m sorry; I misunderstood you. When you said the average of utilities, I thought you meant the utility averaged among all the different agents in the world. Instead, it’s just, roughly, an average among probability density function of utility. I say roughly because I guess integration isn’t exactly an average.

Chantiel Nov 5, 2021, 3:59 AM
1 point
in reply to: tivelen’s comment on: A system of infinite ethics
Please see this comment for an explanation.

Chantiel Nov 5, 2021, 3:57 AM
1 point
in reply to: gjm’s comment on: A system of infinite ethics
RE: scenario one:

All these worlds come out exactly the same, so “infinitely many happy, one unhappy” is indistinguishable from “infinitely many unhappy, one happy”

It’s not clear to me how they are indistinguishable. As long as the agent that’s unhappy can have itself and its circumstances described with a finite description length, then it would have non-zero probability of an agent ending up as that one. Thus, making the agent unhappy would decrease the moral value of the world.

I’m not sure what would happen if the single unhappy agent has infinite complexity and 0 probability. But I suspect that this could be dealt with if you expanded the system to also consider non-real probabilities. I’m no expert on non-real probabilities, but I bet you the probability of being unhappy given there is an unhappy agent would be infinitesimally more probable than the probability in the world in which there’s no unhappy agents.

RE: scenario two: It’s not clear to me how this is crazy. For example, consider this situation: when agents are born, an AI flips a biased coin to determine what will happen to them. Each coin has a 99.999% chance of landing on heads and a 0.001% chance of landing on tails. If the coin lands on heads, the AI will give the agent some very pleasant experience stream, and all such agents will get the same pleasant experience stream. But if it lands on tails, the AI will give the agent some unpleasant experience stream that is also very different from the other unpleasant ones.

This sounds like a pretty good situation to me. It’s not clear to me why it wouldn’t be. I mean, I don’t see why the diversity of the positive experiences matters. And if you do care about the diversity of positive experiences, this would have unintuitive results. For example, suppose all agents have identical preferences and they satisfaction is maximized by experience stream S. Well, if you have a problem with the satisfied agents having just one experience stream, then you would be incentivized to coerce the agents to instead have a variety of different experience streams, even if they didn’t like these experience streams as much.

RE: scenario three:

The CotU has computed that with the switch in the “Nice” position, the expected utility of an experience-subject in the resulting universe is large and positive; with the switch in the “Nasty” position, it’s large and negative. But in both cases every possible experience-subject has a nonzero probability of being generated at any time.

I don’t follow your reasoning. You just said in the “Nice” position, the expected value of this is large and positive and in the “Nasty” it’s large and negative. And since my ethical system seeks to maximize the expected value of life satisfaction, it seems trivial to me that it would prefer the “nice” button.

Whether or not you switch it to the “Nice” position won’t rule out any possible outcomes for an agent, but it seems pretty clear that it would change the probabilities of them.

RE: scenario four: My ethical system would prefer the “Nice” position for the same reason described in scenario three.

RE: scenario five:

So far as I can tell, there is no difference in the set of possible experience-subjects in the world where you do and the world where you don’t. Both the tortured-to-death and the not-tortured-to-death versions of me are apparently possibilities, so it seems that with probability 1 each of them will occur somewhere in this universe, so neither of them is removed from our set of possible experience-streams when we condition on occurrence in our universe.

Though none of the experience streams are impossible, the probability of you getting tortured is still higher conditioning on me deciding the torture you. To see why, note the situation, “Is someone just like Slider who is vulnerable to being tortured by demon lord Chantiel”. This has finite description length, and thus non-zero probability. And if I decide to torture you, then the probability of you getting tortured if you end up in this situation is high. Thus, the total expected value of life satisfaction would be lower if I decided to torture you. So my ethical system would recommend not torturing you.

In general, don’t worry about if an experience stream is possible or not. In an infinite universe with quantum noise, I think pretty much all experience streams would occur with non-zero probability. But you can still adjust the probabilities of an agent ending up with the different streams.

Chantiel Nov 5, 2021, 2:58 AM
1 point
in reply to: Slider’s comment on: A system of infinite ethics

By one logic because we prefer B to A then if we “acausalize” this we should still preserve this preference (because “the amount of copies granted” would seem to be even handed), so we would expect to prefer D to C. However in a system where all infinites are of equal size then C=D and we become ambivalent between the options.

We shouldn’t necessarily prefer D to C. Remember that one of the main things you can do to increase the moral value of the universe is to try to causally help other creatures so that other people who are in sufficiently similar circumstances to you you will also help, so you acausally make them help others. Suppose you instead have the option to instead causally help all of the agents that would have been acausally helped if you just causally help one agent. Then the AI shouldn’t prefer D to C, because the results are identical.

Here an adhoc analysis might choose a clear winner. Then consider the combined problem where you have all the options, peace, punch, dust and insult. Having 1 analysis that gets applied to all options equally will run into trouble. If the analysis is somehow “turn options into real probablities” then problem with infinidesimals are likely to crop up.

Could you explain how this would cause problems? If those are the options, it seems like a clear-but case of my ethical system recommending peace, unless there is some benefit to punching, insulting, or throwing sand you haven’t mentioned.

To see why, if you decide to throw sand, you’re decreasing the satisfaction of agents in situations of the form “Can get sand thrown at them from someone just like Slider”. This would in general decrease the moral value of the world, so my system wouldn’t recommend it. The same reasoning can show that the system wouldn’t recommend punching or insulting.

There could be the “modulo infinity” failure mode where peace and dust get the same number. “One class only” would fail to give numbers for one of the subproblems.

Interesting. Could you elaborate?

I’m not really clear the reason for you are worried about these different classes. Remember that any action you will do will, at least acausally, help a countably infinite number of agents. Similarly, I think all your actions will have some real-valued affect on the moral value of the universe. To see why, just note that as long as you help one agent, then the expected satisfaction of agents in situations of the form, “<description of the circumstances of the above agent> who can be helped by someone just like Slider”. This has finite complexity, and thus real and non-zero probability. And the moral value of the universe is capped at whatever the domain of the life satisfaction measure is, so you can’t have infinite increases to the moral value of the universe, either.

Chantiel Nov 4, 2021, 9:03 PM
1 point
in reply to: gjm’s comment on: A system of infinite ethics

I’ll begin at the end: What is “the expected value of utility” if it isn’t an average of utilities?

I’m just using the regular notion of expected value. That is, let P(u) be the probability density you get utility u. Then, the expected value of utility is $\int_{[a, b]} u P (u) d u$ , where $\int$ uses Lebesgue integration for greater generality. Above, I take utility to be in $[a, b]$ .

Also note that my system cares about a measure of satisfaction, rather than specifically utility. In this case, just replace P(u) to be that measure of life satisfaction instead of a utility.

Also, of course, P(u) is calculated conditioning on being an agent in this universe, and nothing else.

And how do you calculate P(u) given the above? Well, one way is to first start with some disjoint prior probability distribution over universes and situations you could be in, where the situations are concrete enough to determine your eventual life satisfaction. Then just do a Bayes update on “is an agent in this universe and get utility u” by setting the probabilities of hypothesis in which the agent isn’t in this universe or doesn’t have preferences. Then just renormalize the probabilities so they sum to 1. After that, you can just use this probability distribution of possible worlds W to calculate P(u) in a straightforward manner. E.g. $\int_{W} P (u t i l i t y = U | W) d P (w)$ .

(I know I pretty much mentioned the above calculation before, but I thought rephrasing it might help.)
What links here?
- Chantiel's comment on A system of infinite ethics by Chantiel (Nov 5, 2021, 3:59 AM; 1 point)

Chantiel Nov 4, 2021, 12:53 AM
1 point
in reply to: Slider’s comment on: A system of infinite ethics

Post is pretty long winded,a bit wall fo texty in a lot of text which seems like fixed amount of content while being very claimy and less showy about the properties.

Yeah, I see what you mean. I have a hard time balancing between being succinct and providing sufficient support and detail. It actually used to be shorter, but I lengthened it to address concerns brought up a review.

My suspicion is that the acausal impact ends up being infinidesimal anyway. Even if one would get finite probability impact for probabilties concerning a infinite universe for claims like “should I help this one person” then claims like “should I help these infinite persons” would still have an infinity class jump between the statements (even if both need to have an infinite kick into the universe to make a dent there is an additional level to one of these statements and not all infinities are equal).

Could you elaborate what you mean by a class jump?

Remember that if you ask, “should I help this one person”, that is another way of saying, “should I (acausally) help this infinite class of people in similar circumstances”. And I think in general the cardinality of this infinity would be the same as the cardinality of people helped by considering “should I help these infinitely-many persons”

Most likely the number of people in this universe is countably infinite, and all situations are repeated infinitely-many times. Thus, asking, “should I help this one person” would acausally help $ℵ_{0}$ people, and so would causally helping the infinitely-many people.

I am going to anticipate that your scheme will try to rule out statements like “should I help these infinite persons” for a reason like “its not of finite complexity”. I am not convinced that finite complexity descriptions are good guarantees that the described condition makes for a finite proportion of possibility space. I think “Getting a perfect bullseye” is a description of finite complexity but it describes and outcome of (real) 0 probabaility. Being positive is of no guarantee of finitude, infinidesimal chances would spell trouble for the theory. And if statements like “Slider or (near equivalent) gets a perfect bullseye” are disallowed for not being finitely groundable then most references to infinite objects are ruled out anyway. Its not exactly an infinite ethic if it is not allowed to refer to infinite things.

No, my system doesn’t rule out statements of the form, “should I help these infinitely-many persons”. This can have finite complexity, after all, provided there is sufficient regularity in who will be helped. Also, don’t forget, even if you’re just causally helping a single person, you’re still acausally helping infinitely-many people. So, in a sense, ruling out helping infinitely-many people would rule out helping anyone.

I am also slightly worried that “description cuts” will allow “doubling the ball” kind of events where total probability doesn’t get preserved. That phenomenon gets around the theorethical problems by designating some sets non-measurable. But then being a a set doesn’t mean its measurable. I am worried that “descriptions always have a usable probablity” is too lax and will bleed from the edges like a naive assumption that all sets are measurable would.

I’m not sure what specifically you have in mind with respect to doubling the sphere-esque issues. But if your system of probabilistic reasoning doesn’t preserve the total probability when partitioning an event into multiple events, that sounds like a serious problem with your probabilistic reasoning system. I mean, if your reasoning system does this, then it’s not even a probability measure.

If you can prove $U ⟺ V \lor W$ , but the system still says $P (U) \neq P (V \lor W)$ , then you aren’t satisfying one of the basic desiderata that motivated Bayesian probability theory: asking the same question in two different ways should result in the same probability. And $V \lor W$ is just another way of asking $U$ .

Chantiel Nov 2, 2021, 3:32 AM
1 point
in reply to: gjm’s comment on: A system of infinite ethics

Of course you can make moral decisions without going through such calculations. We all do that all the time. But the whole issue with infinite ethics—the thing that a purported system for handling infinite ethics needs to deal with—is that the usual ways of formalizing moral decision processes produce ill-defined results in many imaginable infinite universes. So when you propose a system of infinite ethics and I say “look, it produces ill-defined results in many imaginable infinite universes”, you don’t get to just say “bah, who cares about the details?” If you don’t deal with the details you aren’t addressing the problems of infinite ethics at all!

Well, I can’t say I exactly disagree with you here.

However, I want to note that this isn’t a problem specific to my ethical system. It’s true that in order to use my ethical system to make precise moral verdicts, you need to more fully formalize probability theory. However, the same is also true with effectively every other ethical theory.

For example, consider someone learning about classical utilitarianism and its applications in a finite world. Then they could argue:

Okay, I see your ethical system says to make the balance of happiness to unhappiness as high as possible. But how am I supposed to know what the world is actually like and what the effects of my actions are? Do other animals feel happiness and unhappiness? Is there actually a heaven and Hell that would influence moral choices? This ethical system doesn’t answer any of this. You can’t just handwave this away! If you don’t deal with the details you aren’t addressing the problems of ethics at all!

Also, I just want to note that my system as described seems to be unique among the infinite ethical systems I’ve seen in that it doesn’t make obviously ridiculous moral verdicts. Every other one I know of makes some recommendations that seem really silly. So, despite not providing a rigorous formalization of probability theory, I think my ethical system has value.

But what you actually want (I think) isn’t quite a probability distribution over universes; you want a distribution over experiences-in-universes, and not your experiences but those of hypothetical other beings in the same universe as you. So now think of the programs you’re working with as describing not your experiences necessarily but those of some being in the universe, so that each update is weighted not by Pr(I have experience X | my experiences are generated by program P) but by Pr(some subject-of-experience has experience X | my experiences are generated by program P), with the constraint that it’s meant to be the same subject-of-experience for each update. Or maybe by Pr(a randomly chosen subject-of-experience has experience X | my experiences are generated by program P) with the same constraint.

Actually, no, I really do want a probability distribution over what I would experience, or more generally, the situations I’d end up being in. The alternatives you mentioned, Pr(some subject-of-experience has experience X | my experiences are generated by program P) and Pr(a randomly chosen subject-of-experience has experience X | my experiences are generated by program P), both lead to problems for the reasons you’ve already described.

I’m not sure what made you think I didn’t mean, P(I have experience x | …). Could you explain?

We’re concerned about infinitarian paralysis, where we somehow fail to deliver a definite answer because we’re trying to balance an infinite amount of good against an infinite amount of bad. So far as I can see, your system still has this problem. E.g., if I know there are infinitely many people with various degrees of (un)happiness, and I am wondering whether to torture 1000 of them, your system is trying to calculate the average utility in an infinite population, and that simply isn’t defined.

My system doesn’t compute the average utility of anything. Instead, it tries to compute the expected value of utility (or life satisfaction). I’m sorry if this was somehow unclear. I didn’t think I ever mentioned I was dealing with averages anywhere, though. I’m trying to get better at writing clearly, so if you remember what made you think this, I’d appreciate hearing.

Chantiel Nov 2, 2021, 3:28 AM
1 point
in reply to: gjm’s comment on: A system of infinite ethics

You say, “There must be some reasonable way to calculate this.”

(where “this” is Pr(I’m satisfied | I’m some being in such-and-such a universe)) Why must there be? I agree that it would be nice if there were, of course, but there is no guarantee that what we find nice matches how the world actually is.

To use probability theory to form accurate beliefs, we need a prior. I didn’t think this was controversial. And if you have a prior, as far as I can tell, you can then compute Pr(I’m satisfied | I’m some being in such-and-such a universe) by simply updating on “I’m some being in such-and-such a universe” using Bayes’ theorem.

That is, you need to have some prior probability distribution over concrete specifications of the universe you’re in and your situation in it. Now, to update on “I’m some being in such-and-such a universe”, just look at each concrete possible situation-and-universe and assign P(“I’m some being in such-and-such a universe” | some concrete hypothesis) to 0 if the hypothesis specifies you’re in some universe other than the such-and-such universe. And set this probability is 1 if it does specify you are in such a universe. As long as the possible universes are specified sufficiently precisely, then I don’t see why you couldn’t do this.

Chantiel Nov 2, 2021, 1:42 AM
3 points
in reply to: Slider’s comment on: A system of infinite ethics

Kind of hard to ge a handle.

Are you referring to it being hard to understand? If so, I appreciate the feedback and am interested in the specifics what is difficult to understand. Clarity is a top priority for me.

If I have a choice of (finitely) helping a single human and I believe there to be infinite humans then the probability of a human being helped in my world will nudge less than a real number. And if we want to stick with probabilties being real then the rounding will make infinitarian paralysis.

You are correct that a single human would have 0 or infinitistimal causal impact on the moral value of the world or the satisfaction of an arbitrary human. However, it’s important to note that my system requires you to use a decision theory that considers not just your causal impacts, but also your acausal ones.

Remember that if you decide to take a certain action, that implies that other agents who are sufficiently similar to you and in sufficiently similar circumstances also take that action. Thus, you can acausally have non-infinitesimal impact on the satisfaction of agents in situations of the form, “An agent in a world with someone just like Slider who is also in very similar circumstances to Slider’s.” The above scenario is of finite complexity and isn’t ruled out by evidence. Thus, the probability of an agent ending up in such a situation, conditioning only only on being some agent in this universe, is nonzero.

Another scenario raises the possibility of the specter of fanatism. Say by doing murder I can create an AI that will make all future agents happy but being murdered is not happy times. Comparing agents before and after the singularity might make sense. And so might killing different finite amounts of people. but mixing them gets tricky or favours the “wider class”. One could think of a distribution where for values between 0 and 4 you up the utility by 1 except for pi (or any single real (or any set of measure 0)) for which you lower it by X. Any finite value for X will not be able to nudge the expectation value anywhere. Real ranges vs real ranges makes sense, discrete sets vs discrete sets makes sense, but when you cross transfinite archimedean classes one is in trouble.

I’m not really following what you see as the problem here. Perhaps by above explanation clears things up. If not, would you be willing to elaborate on how transfinite archimedean classes could potentially lead to trouble?

Also, to be clear, my system only considers finite probabilities and finite changes to the moral value of the world. Perhaps there’s some way to extend it beyond this, but as far as I know it’s not necessary.

Chantiel Nov 1, 2021, 12:02 AM
1 point
in reply to: gjm’s comment on: A system of infinite ethics
(Assuming you’re read my other response you this comment):

I think it might help if I give a more general explanation of how my moral system can be used to determine what to do. This is mostly taken from the article, but it’s important enough that I think it should be restated.

Suppose you’re considering taking some action that would benefit our world or future life cone. You want to see what my ethical system recommends.

Well, for almost possible circumstances an agent could end up in in this universe, I think your action would have effectively no causal or acausal effect on them. There’s nothing you can do about them, so don’t worry about them in your moral deliberation.

Instead, consider agents of the form, “some agent in an Earth-like world (or in the future light-cone of one) with someone just like <insert detailed description of yourself and circumstances>”. These are agents you can potentially (acausally) affect. If you take an action to make the world a better place, that means the other people in the universe who are very similar to you and in very similar circumstances would also take that action.

So if you take that action, then you’d improve the world, so the expected value of life satisfaction of an agent in the above circumstances would be higher. Such circumstances are of finite complexity and not ruled out by evidence, so the probability of an agent ending up in such a situation, conditioning only on being in this universe, in non-zero. Thus, taking that action would increase the moral value of the universe and my ethical system would thus be liable to recommend taking that action.

To see it another way, moral deliberation with my ethical system works as follows:

I’m trying to make the universe a better place. Most agents are in situations in which I can’t do anything to affect them, whether causally or acausally. But there are some agents in situations that that I can (acausally) affect. So I’m going to focus on making the universe as satisfying as possible for those agents, using some impartial weighting over those possible circumstances.