I think his point is that you are still entirely unable to even enumerate, let alone process, all the relevant hypotheses, nor does the formula inform you of those, nor does it inform you how to deal with cyclic updates (or even that those are a complicated case), etc.
It’s particularly bad when it comes to what rationalists describe as “expected utility calculations”. The ideal expected utility is a sum of the differential effect of the actions being compared, over all hypotheses, multiplied with their probabilities… a single component of the sum provides very little or no information about the value of the sum, especially when picked by someone with a financial interest as strong as “if i don’t convince those people I can’t pay my rent”. Then, the actions themselves have an impact on the future decision making, which makes the expected value sum grow and branch out like some crazy googol-headed fractal hydra. Mostly when someone’s talking much about Bayes they have some simple and invalid expected value calculation that they want you to perform and act upon, so that you’ll be worse off in the end and they’ll be better off in the end.
I think his point is that you are still entirely unable to even enumerate, let alone process, all the relevant hypotheses, nor does the formula inform you of those, nor does it inform you how to deal with cyclic updates (or even that those are a complicated case), etc.
It’s particularly bad when it comes to what rationalists describe as “expected utility calculations”. The ideal expected utility is a sum of the differential effect of the actions being compared, over all hypotheses, multiplied with their probabilities… a single component of the sum provides very little or no information about the value of the sum, especially when picked by someone with a financial interest as strong as “if i don’t convince those people I can’t pay my rent”. Then, the actions themselves have an impact on the future decision making, which makes the expected value sum grow and branch out like some crazy googol-headed fractal hydra. Mostly when someone’s talking much about Bayes they have some simple and invalid expected value calculation that they want you to perform and act upon, so that you’ll be worse off in the end and they’ll be better off in the end.