As a bounded agent, you have to be aware that it’s physically impossible to consider all the hypotheses. When you encounter new evidence, you might think of a new hypothesis to promote that you hadn’t thought of before—in fact, this is an unavoidable part of being a good bounded agent. So don’t worry about coming up with the One True Prior ahead of time and then updating it—instead, try to plan for the most likely outcomes, but leave a “something else” category and be ready to change your mind.
And given that we’re biased, when we make plans we’re probably going to get some probabilities wrong—in this case, future events contain information about how one was biased. Try to learn about your own biases, which often means being more influenced by evidence than an unbiased agent.
If you still want to try reasoning probabilistically, I’d look into Tetlock’s good judgment project and start planning how to practice my probability estimation. Oh, and check out the calibration game.
Some random thoughts:
As a bounded agent, you have to be aware that it’s physically impossible to consider all the hypotheses. When you encounter new evidence, you might think of a new hypothesis to promote that you hadn’t thought of before—in fact, this is an unavoidable part of being a good bounded agent. So don’t worry about coming up with the One True Prior ahead of time and then updating it—instead, try to plan for the most likely outcomes, but leave a “something else” category and be ready to change your mind.
And given that we’re biased, when we make plans we’re probably going to get some probabilities wrong—in this case, future events contain information about how one was biased. Try to learn about your own biases, which often means being more influenced by evidence than an unbiased agent.
If you still want to try reasoning probabilistically, I’d look into Tetlock’s good judgment project and start planning how to practice my probability estimation. Oh, and check out the calibration game.