Active inference is an extension of predictive coding in which some beliefs are so rigid that, when they conflict with observations, it’s easier to act to change future observations than it is to update those beliefs. We can call these hard-to-change beliefs “goals”, thereby unifying beliefs and goals in a way that EUM doesn’t.
You’re probably aware of it but it makes sense to explicitize that this move also puts in the goal category many biases, addictions, and maladaptive/disendorsed behaviors.
EUM treats goals and beliefs as totally separate. But in practice, agents represent both of these in terms of the same underlying concepts. When those concepts change, both beliefs and goals change.
Active inference is one framework that attempts to address it. Jeffrey-Bolker is another one, though I haven’t dipped my toes into it deep enough to have an informed opinion on whether it’s more promising than active inference for the thing you want to do.
Based on similar reasoning, Scott Garrabrant rejects the independence axiom. He argues that the axiom is unjustified because rational agents should be able to lock in values like fairness based on prior agreements (or even hypothetical agreements).
I first thought that this introduces epistemic instability because vNM EU theory rests on the independence axiom (so it looked like: to unify EU theory with active inference you wanted to reject one of the things defining EU theory qua EU theory) but then I realized that you hadn’t assumed vNM as a foundation for EU theory, so maybe it’s irrelevant. But still, as far as I remember, different foundations of EU theory give you slightly different implications (and many of them have some equivalent of the independence axiom; at least Savage does), so it might be good for you to think explicitly about what kind of EU foundation you’re assuming. But it also might be irrelevant. I don’t know. I’m leaving this thought-train-dump in case it might be useful.
You’re probably aware of it but it makes sense to explicitize that this move also puts in the goal category many biases, addictions, and maladaptive/disendorsed behaviors.
Active inference is one framework that attempts to address it. Jeffrey-Bolker is another one, though I haven’t dipped my toes into it deep enough to have an informed opinion on whether it’s more promising than active inference for the thing you want to do.
I first thought that this introduces epistemic instability because vNM EU theory rests on the independence axiom (so it looked like: to unify EU theory with active inference you wanted to reject one of the things defining EU theory qua EU theory) but then I realized that you hadn’t assumed vNM as a foundation for EU theory, so maybe it’s irrelevant. But still, as far as I remember, different foundations of EU theory give you slightly different implications (and many of them have some equivalent of the independence axiom; at least Savage does), so it might be good for you to think explicitly about what kind of EU foundation you’re assuming. But it also might be irrelevant. I don’t know. I’m leaving this thought-train-dump in case it might be useful.