Our brains obviously have things like the sunk cost fallacy and all the various heuristics built in for a reason. We don’t have an explicit utility function in our brain, and so we have all kinds of hacked on solutions.
For example the sunk cost fallacy in our brain is there to prevent us from giving up on tasks prematurely. Our brains need this tendency, because we also have a boredom tendency built in. An AI would simply calculate the expected value of working on each of the things it could be doing. The boredom tendency would be countered by an understanding of diminishing marginal returns (or not at all, for an unconscious AI), and the sunk cost fallacy would be unnecessary.
You need to be explicit that all these heuristics and biases are not overall good things, but simply hacked on aspects of human psychology.
I personally feel that learning about rationality has greatly improved my life, but 1. so do New Age advocates and 2. that’s just one data point.
Our brains obviously have things like the sunk cost fallacy and all the various heuristics built in for a reason. We don’t have an explicit utility function in our brain, and so we have all kinds of hacked on solutions.
For example the sunk cost fallacy in our brain is there to prevent us from giving up on tasks prematurely. Our brains need this tendency, because we also have a boredom tendency built in. An AI would simply calculate the expected value of working on each of the things it could be doing. The boredom tendency would be countered by an understanding of diminishing marginal returns (or not at all, for an unconscious AI), and the sunk cost fallacy would be unnecessary.
You need to be explicit that all these heuristics and biases are not overall good things, but simply hacked on aspects of human psychology.
I personally feel that learning about rationality has greatly improved my life, but 1. so do New Age advocates and 2. that’s just one data point.