Here’s what I think is true and important about this post: some people will try to explicitly estimate expected values in ways that don’t track the real expected values, and when they do this, they’ll make bad decisions. We should avoid these mistakes, which may be easy to fall into, and we can avoid some of them by using regressions of the kind described above in the case of charity cost-effectiveness estimates. As Toby points out, this is common ground between GiveWell and GWWC. Let me list a what I take to be a few points of disagreement.
I think that after making an appropriate attempt to gather evidence, the result of doing the best expected value calculation that you can is by far the most important input into a large scale philanthropic decision. We should think about the result of the calculation makes sense, we should worry if it is wildly counterintuitive, and we should try hard to avoid mistakes. But the result of this calculation will matter more than most kinds of informal reasoning, especially if the differences in expected value are great. I think this will be true for people who are competent with thinking in terms of subjective probabilities and expected values, which will rule out a lot of people, but will include a lot of the people who would consider whether to make important philanthropic decisions on the basic of expected value calculations.
I think this argument unfairly tangles up making decisions explicitly on the basis of expected value calculations with Pascal’s Mugging. It’s not too hard to choose a bounded utility function that doesn’t tell you to pay the mugger, and there are independent (though not clearly decisive) reasons to use a bounded utility function for decision-making, even when the probabilities are stable. Since the unbounded utility function assumption can shoulder the blame, the invocation of Pascal’s Mugging doesn’t seem all that telling. (Also, for reasons Wei Dai gestures at I don’t accept Holden’s conjecture that making regression adjustments will get us out of the Pascal’s Mugging problem, even if we have unbounded utility functions.)
Though I agree that intuition can be a valuable tool when trying to sanity check an expected value calculation, I am hesitant to rely too heavily on it. Things like scope insensitivity and ambiguity aversion could easily make me unreasonably queasy about relying a perfectly reasonable expected value calculation.
Finally, I classify several of the arguments in this post as “perfect world” arguments because they involve thinking a lot about what would happen if everyone behaved in a certain kind of way. I don’t want to rest too much weight on these arguments because my behavior doesn’t causally or acausally affect the way enough people would behave in order for these arguments to be directly relevant to my decisions. Even if I accepted perfect world arguments, some of these arguments appear not to work. For example, if all donors were rational altruists, and that was common knowledge, then charities that were effective would have a strong incentive to provide evidence of their effectiveness. If some charity refused to share information, that would be very strong evidence that the charity was not effective. So it doesn’t seem to be true, as Holden claims, that if everyone was totally reliant on explicit expected value calculations, we’d all give to charities about which we have very little information. (Deciding not to be totally transparent is not such good evidence now, since donors are far from being rational altruists.)
Though I have expressed mostly disagreement, I think Holden’s post is very good and I’m glad that he made it.
On a more positive note, Holden and Dario’s research on this issue gave me a much better understanding about how the regression would work and how it could suggest that GWWC should be putting more emphasis on evaluating individual charities, relative to relying on (appropriately adjusted) cost-effectiveness estimates.
Demanding a bounded utility function seems like a horrible idea. For starters it violates our expectation of independence for causally isolated universes. In other words the utility of both universe A and B existing isn’t the sum of the utility of A existing and B existing.
Besides, intuitively if you were really offered with perfect confidence a bet with a 99.99...90% chance of winning you a dollar but if you lose all the souls in the world will burn in hell forever do you really think it would be a worthwhile bet to take?
What is bad about Pascal’s mugging isn’t the existence of arbitrarily bad outcomes, it’s the existence of arbitrarily bad expectations. What we really want is that the integral of utility relative to the probability measure is never infinite. This is a much more reasonable condition. In other words utility should be L1 integrabale with respect to your prior probability distribution (and as long as you never condition on events of probability 0 this property persists).
Besides, intuitively if you were really offered with perfect confidence a bet with a 99.99...90% chance of winning you a dollar but if you lose all the souls in the world will burn in hell forever do you really think it would be a worthwhile bet to take?
Just how many nines do those ellipses represent? That’s kind of important! I mean, if there is enough nines then the value of the dollar easily outweighs the risk and your intuition is deceiving you. Consider for example:
Given the information I currently have there is some non zero chance that I am wrong about the origin of the universe. It could be that there really is a God that will punish us with eternity in hell if we have the wrong belief about Him. A friendly superintelligence has a reasonable chance of figuring this out for us and telling us the Good News. Me having an extra dollar produces a non-zero increase in the probability that we create an AI. Multiplying non-zero chances together gives other nonzero chances. For any given probability of eternities in hell there is a number of nines such that (100 − 99.99#{n * “9”}90)% is a lower probability of eternities in hell.
Moral of this story: Be careful when throwing not-quite-infinities around to prove a point. Not-quite-infinities @#$% things up almost as much as infinities.
Besides, intuitively if you were really offered with perfect confidence a bet with a 99.99...90% chance of winning you a dollar but if you lose all the souls in the world will burn in hell forever do you really think it would be a worthwhile bet to take?
While I genuinely appreciate how implausible it is to consider that bet worthwhile, there is a flipside. See the The LIfespan Dilemma. The point is that people have inconsistent preferences here, so it is easy to find intuitive counterexamples to any position one could take. I find the bounded approach to be the least of all evils.
What is bad about Pascal’s mugging isn’t the existence of arbitrarily bad outcomes, it’s the existence of arbitrarily bad expectations. What we really want is that the integral of utility relative to the probability measure is never infinite.
It is actually pretty hard to achieve this. Consider a sequence of lives L1, L2, etc. such that (i) the temporary quality of each life increases over time in each life, and increases at the same rate, (ii) the duration of the lives in the sequence is increasing and approaching infinity, (iii) the utility of each lives in the sequence is approaching infinity. And now consider an infinitely long life L whose temporary quality increases at the same rate as L1, L2, etc. Plausibly, L is better than L1, L2, etc. This means L has a value which exceeds any finite value, and it means you will take any chance of L, no matter how small, to any of the Li, no matter how certain it would be. When choosing among different available actions which may lead to some of these lives, you would, in practice, consider only the chance of getting an infinitely long life, turning to other considerations only to break ties. Upshot: weak background assumptions + unbounded utility function --> obsessing over infinities, neglecting finite considerations except to break ties.
To be clear, the background assumptions are: that L, L1, L2, etc. are alternatives you could reasonably have non-zero probability in; that L is better than L1, L2, etc.; normal axioms of decision theory.
(You may also consider a variation of this argument involving histories of human civilization, rather than lives.)
Here’s what I think is true and important about this post: some people will try to explicitly estimate expected values in ways that don’t track the real expected values, and when they do this, they’ll make bad decisions. We should avoid these mistakes, which may be easy to fall into, and we can avoid some of them by using regressions of the kind described above in the case of charity cost-effectiveness estimates. As Toby points out, this is common ground between GiveWell and GWWC. Let me list a what I take to be a few points of disagreement.
I think that after making an appropriate attempt to gather evidence, the result of doing the best expected value calculation that you can is by far the most important input into a large scale philanthropic decision. We should think about the result of the calculation makes sense, we should worry if it is wildly counterintuitive, and we should try hard to avoid mistakes. But the result of this calculation will matter more than most kinds of informal reasoning, especially if the differences in expected value are great. I think this will be true for people who are competent with thinking in terms of subjective probabilities and expected values, which will rule out a lot of people, but will include a lot of the people who would consider whether to make important philanthropic decisions on the basic of expected value calculations.
I think this argument unfairly tangles up making decisions explicitly on the basis of expected value calculations with Pascal’s Mugging. It’s not too hard to choose a bounded utility function that doesn’t tell you to pay the mugger, and there are independent (though not clearly decisive) reasons to use a bounded utility function for decision-making, even when the probabilities are stable. Since the unbounded utility function assumption can shoulder the blame, the invocation of Pascal’s Mugging doesn’t seem all that telling. (Also, for reasons Wei Dai gestures at I don’t accept Holden’s conjecture that making regression adjustments will get us out of the Pascal’s Mugging problem, even if we have unbounded utility functions.)
Though I agree that intuition can be a valuable tool when trying to sanity check an expected value calculation, I am hesitant to rely too heavily on it. Things like scope insensitivity and ambiguity aversion could easily make me unreasonably queasy about relying a perfectly reasonable expected value calculation.
Finally, I classify several of the arguments in this post as “perfect world” arguments because they involve thinking a lot about what would happen if everyone behaved in a certain kind of way. I don’t want to rest too much weight on these arguments because my behavior doesn’t causally or acausally affect the way enough people would behave in order for these arguments to be directly relevant to my decisions. Even if I accepted perfect world arguments, some of these arguments appear not to work. For example, if all donors were rational altruists, and that was common knowledge, then charities that were effective would have a strong incentive to provide evidence of their effectiveness. If some charity refused to share information, that would be very strong evidence that the charity was not effective. So it doesn’t seem to be true, as Holden claims, that if everyone was totally reliant on explicit expected value calculations, we’d all give to charities about which we have very little information. (Deciding not to be totally transparent is not such good evidence now, since donors are far from being rational altruists.)
Though I have expressed mostly disagreement, I think Holden’s post is very good and I’m glad that he made it.
On a more positive note, Holden and Dario’s research on this issue gave me a much better understanding about how the regression would work and how it could suggest that GWWC should be putting more emphasis on evaluating individual charities, relative to relying on (appropriately adjusted) cost-effectiveness estimates.
Demanding a bounded utility function seems like a horrible idea. For starters it violates our expectation of independence for causally isolated universes. In other words the utility of both universe A and B existing isn’t the sum of the utility of A existing and B existing.
Besides, intuitively if you were really offered with perfect confidence a bet with a 99.99...90% chance of winning you a dollar but if you lose all the souls in the world will burn in hell forever do you really think it would be a worthwhile bet to take?
What is bad about Pascal’s mugging isn’t the existence of arbitrarily bad outcomes, it’s the existence of arbitrarily bad expectations. What we really want is that the integral of utility relative to the probability measure is never infinite. This is a much more reasonable condition. In other words utility should be L1 integrabale with respect to your prior probability distribution (and as long as you never condition on events of probability 0 this property persists).
Just how many nines do those ellipses represent? That’s kind of important! I mean, if there is enough nines then the value of the dollar easily outweighs the risk and your intuition is deceiving you. Consider for example:
Given the information I currently have there is some non zero chance that I am wrong about the origin of the universe. It could be that there really is a God that will punish us with eternity in hell if we have the wrong belief about Him. A friendly superintelligence has a reasonable chance of figuring this out for us and telling us the Good News. Me having an extra dollar produces a non-zero increase in the probability that we create an AI. Multiplying non-zero chances together gives other nonzero chances. For any given probability of eternities in hell there is a number of nines such that (100 − 99.99#{n * “9”}90)% is a lower probability of eternities in hell.
Moral of this story: Be careful when throwing not-quite-infinities around to prove a point. Not-quite-infinities @#$% things up almost as much as infinities.
While I genuinely appreciate how implausible it is to consider that bet worthwhile, there is a flipside. See the The LIfespan Dilemma. The point is that people have inconsistent preferences here, so it is easy to find intuitive counterexamples to any position one could take. I find the bounded approach to be the least of all evils.
It is actually pretty hard to achieve this. Consider a sequence of lives L1, L2, etc. such that (i) the temporary quality of each life increases over time in each life, and increases at the same rate, (ii) the duration of the lives in the sequence is increasing and approaching infinity, (iii) the utility of each lives in the sequence is approaching infinity. And now consider an infinitely long life L whose temporary quality increases at the same rate as L1, L2, etc. Plausibly, L is better than L1, L2, etc. This means L has a value which exceeds any finite value, and it means you will take any chance of L, no matter how small, to any of the Li, no matter how certain it would be. When choosing among different available actions which may lead to some of these lives, you would, in practice, consider only the chance of getting an infinitely long life, turning to other considerations only to break ties. Upshot: weak background assumptions + unbounded utility function --> obsessing over infinities, neglecting finite considerations except to break ties.
To be clear, the background assumptions are: that L, L1, L2, etc. are alternatives you could reasonably have non-zero probability in; that L is better than L1, L2, etc.; normal axioms of decision theory.
(You may also consider a variation of this argument involving histories of human civilization, rather than lives.)