Actually, here’s a much simpler, more intuitive way to think about probabilistically specified goals.
Visualize a probability distribution as a heat map of the possibility space. Specifying a probabilistic goal then just says, “Here’s where I want the heat to concentrate”, and submitting it to active inference just uses the available inferential machinery to actually squeeze the heat into that exact concentration as best you can.
When our heat-map takes the form of “heat” over dynamical trajectories, possible “timelines” of something that can move, “squeezing the heat into your desired concentration” means exactly “squeezing the future towards desired regions”. All you’re changing is how you specify desired regions: from giving them an “absolute” value (that can actually undergo any linear transformation and be isomorphic) to giving them a purely “relative” value (relative to disjoint events in your sample space).
This is fine, because after all, it’s not like you could really have an “infinite” desire for something finite-sized in the first place. If you choose to think of utilities in terms of money, the “goal probabilities” are just the relative prices you’re willing to pay for a certain outcome: you start with odds, the number of apples you’ll trade for an orange, and convert from odds to probabilities to get your numbers. It’s just using “barter” among disjoint random events instead of “currency”.
I’m confused so I’ll comment a dumb question hoping my cognitive algorithms are sufficiently similar to other LW:ers, such that they’ll be thinking but not writing this question.
“If I value apples at 3 units and oranges at 1 unit, I don’t want at 75%/25% split. I only want apples, because they’re better! (I have no diminishing returns.)”
>”If I value apples at 3 units and oranges at 1 unit, I don’t want at 75%/25% split. I only want apples, because they’re better! (I have no diminishing returns.)”
I think what I’d have to ask here is: if you only want apples, why are you spending your money on oranges? If you will not actually pay me 1 unit for an orange, why do you claim you value oranges at 1 unit?
Another construal: you value oranges at 1 orange per 1 unit because if I offer you a lottery over those and let you set the odds yourself, you will choose to set them to 50⁄50. You’re indifferent to which one you receive, so you value them equally. We do the same trick with apples and find you value them at 3 units per 1 apple.
I now offer you a lottery between receiving 3 apples and 1 orange, and I’ll let you pay 3 units to tilt the odds by one expected apple. Since the starting point was 1.5 expected apples and 0.5 expected oranges, and you insist you want only 3 expected apples and 0 expected oranges, I believe I can make you end up paying more than 3 units per apple now, despite our having established that as your “price”.
The lesson is, I think, don’t offer to pay finite amounts of money for outcomes you want literally zero of, as someone may in fact try to take you up on it.
Actually, here’s a much simpler, more intuitive way to think about probabilistically specified goals.
Visualize a probability distribution as a heat map of the possibility space. Specifying a probabilistic goal then just says, “Here’s where I want the heat to concentrate”, and submitting it to active inference just uses the available inferential machinery to actually squeeze the heat into that exact concentration as best you can.
When our heat-map takes the form of “heat” over dynamical trajectories, possible “timelines” of something that can move, “squeezing the heat into your desired concentration” means exactly “squeezing the future towards desired regions”. All you’re changing is how you specify desired regions: from giving them an “absolute” value (that can actually undergo any linear transformation and be isomorphic) to giving them a purely “relative” value (relative to disjoint events in your sample space).
This is fine, because after all, it’s not like you could really have an “infinite” desire for something finite-sized in the first place. If you choose to think of utilities in terms of money, the “goal probabilities” are just the relative prices you’re willing to pay for a certain outcome: you start with odds, the number of apples you’ll trade for an orange, and convert from odds to probabilities to get your numbers. It’s just using “barter” among disjoint random events instead of “currency”.
I’m confused so I’ll comment a dumb question hoping my cognitive algorithms are sufficiently similar to other LW:ers, such that they’ll be thinking but not writing this question.
“If I value apples at 3 units and oranges at 1 unit, I don’t want at 75%/25% split. I only want apples, because they’re better! (I have no diminishing returns.)”
Where does this reasoning go wrong?
>”If I value apples at 3 units and oranges at 1 unit, I don’t want at 75%/25% split. I only want apples, because they’re better! (I have no diminishing returns.)”
I think what I’d have to ask here is: if you only want apples, why are you spending your money on oranges? If you will not actually pay me 1 unit for an orange, why do you claim you value oranges at 1 unit?
Another construal: you value oranges at 1 orange per 1 unit because if I offer you a lottery over those and let you set the odds yourself, you will choose to set them to 50⁄50. You’re indifferent to which one you receive, so you value them equally. We do the same trick with apples and find you value them at 3 units per 1 apple.
I now offer you a lottery between receiving 3 apples and 1 orange, and I’ll let you pay 3 units to tilt the odds by one expected apple. Since the starting point was 1.5 expected apples and 0.5 expected oranges, and you insist you want only 3 expected apples and 0 expected oranges, I believe I can make you end up paying more than 3 units per apple now, despite our having established that as your “price”.
The lesson is, I think, don’t offer to pay finite amounts of money for outcomes you want literally zero of, as someone may in fact try to take you up on it.
The problem with the typeface on LW comments is that I, l and 1 look really damn similar.