Besides, intuitively if you were really offered with perfect confidence a bet with a 99.99...90% chance of winning you a dollar but if you lose all the souls in the world will burn in hell forever do you really think it would be a worthwhile bet to take?
Just how many nines do those ellipses represent? That’s kind of important! I mean, if there is enough nines then the value of the dollar easily outweighs the risk and your intuition is deceiving you. Consider for example:
Given the information I currently have there is some non zero chance that I am wrong about the origin of the universe. It could be that there really is a God that will punish us with eternity in hell if we have the wrong belief about Him. A friendly superintelligence has a reasonable chance of figuring this out for us and telling us the Good News. Me having an extra dollar produces a non-zero increase in the probability that we create an AI. Multiplying non-zero chances together gives other nonzero chances. For any given probability of eternities in hell there is a number of nines such that (100 − 99.99#{n * “9”}90)% is a lower probability of eternities in hell.
Moral of this story: Be careful when throwing not-quite-infinities around to prove a point. Not-quite-infinities @#$% things up almost as much as infinities.
Just how many nines do those ellipses represent? That’s kind of important! I mean, if there is enough nines then the value of the dollar easily outweighs the risk and your intuition is deceiving you. Consider for example:
Given the information I currently have there is some non zero chance that I am wrong about the origin of the universe. It could be that there really is a God that will punish us with eternity in hell if we have the wrong belief about Him. A friendly superintelligence has a reasonable chance of figuring this out for us and telling us the Good News. Me having an extra dollar produces a non-zero increase in the probability that we create an AI. Multiplying non-zero chances together gives other nonzero chances. For any given probability of eternities in hell there is a number of nines such that (100 − 99.99#{n * “9”}90)% is a lower probability of eternities in hell.
Moral of this story: Be careful when throwing not-quite-infinities around to prove a point. Not-quite-infinities @#$% things up almost as much as infinities.