Probability of success if you continue: small. Probability of success if you give up: zero.
Doug, that’s exactly what people say to me when I challenge them on why they buy lottery tickets. “The chance of winning is tiny, but if I don’t buy a ticket, the chance is zero.”
I can’t think of one single case in my experience when the argument “It has a small probability of success, but we should pursue it, because the probability if we don’t try is zero” turned out to be a good idea. Typically it is an excuse not to confront the flaws of a plan that is just plain unripe. You know what happens when you try a strategy with a tiny probability of success? It fails, that’s what happens.
Doug was right, Eliezer was wrong. At least as the quote is stated. That is not to say that implementing a heuristic avoidance of low probability plans is not usually a good idea.
I think there is a distinction. In this case literally the entire argument is “It has a small probability of success, but we should pursue it, because the probability if we don’t try is zero”. It would be valid to justify a low probability with a high utility, but sometimes people just ignore or refuse to calculate probabilities because they believe that all alternatives are futile, even in the face of repeated counterevidence pushing the probability ever-lower. While such a situation is possible, beliefs of this type are far more likely to be caused by rationalization.
Shut up and do the impossible!
Doug was right, Eliezer was wrong. At least as the quote is stated. That is not to say that implementing a heuristic avoidance of low probability plans is not usually a good idea.
I think there is a distinction. In this case literally the entire argument is “It has a small probability of success, but we should pursue it, because the probability if we don’t try is zero”. It would be valid to justify a low probability with a high utility, but sometimes people just ignore or refuse to calculate probabilities because they believe that all alternatives are futile, even in the face of repeated counterevidence pushing the probability ever-lower. While such a situation is possible, beliefs of this type are far more likely to be caused by rationalization.