“The point is that given this information, rationality picks choice 2.”—Posted by: GreedyAlgorithm
Sorry, no. Given this information, rationality says that there is not enough information to make an appropriate decision, and demands to know the context. If contextual information isn’t available, rationality will say that either option 1 or 2 may be right, depending on circumstances.
Rationality never dismisses context as irrelevant just because it isn’t known. If unknown factors make the right answer uncertain, then you must accept that it is uncertain.
Context can change what you’re trying to achieve. Many people seem to assume that the point (re Circular Altruism problems) is to save as many lives as possible, but this might have to be balanced with other goals—e.g. setting a limit to acceptable risk (as in not risking destruction of the entire human population, whatever their number), or spreading risk instead of marking certain people for death (as in putting the last few people from a sinking ship in the lifeboat, not leaving them behind to make a crowded lifeboat safer).
Making assumptions is one of the danger pitfalls for rational thinkers. So is a reluctance to say “I don’t know the answer” when appropriate.
“The point is that given this information, rationality picks choice 2.”—Posted by: GreedyAlgorithm
Sorry, no. Given this information, rationality says that there is not enough information to make an appropriate decision, and demands to know the context. If contextual information isn’t available, rationality will say that either option 1 or 2 may be right, depending on circumstances.
Rationality never dismisses context as irrelevant just because it isn’t known. If unknown factors make the right answer uncertain, then you must accept that it is uncertain.
Context can change what you’re trying to achieve. Many people seem to assume that the point (re Circular Altruism problems) is to save as many lives as possible, but this might have to be balanced with other goals—e.g. setting a limit to acceptable risk (as in not risking destruction of the entire human population, whatever their number), or spreading risk instead of marking certain people for death (as in putting the last few people from a sinking ship in the lifeboat, not leaving them behind to make a crowded lifeboat safer).
Making assumptions is one of the danger pitfalls for rational thinkers. So is a reluctance to say “I don’t know the answer” when appropriate.