Yup, that’s the direction I want. If the distributions are exponential family, then that dramatically narrows down the space of distributions which need to be represented in order to represent abstractions in general. That means much simpler data structures—e.g. feature functions and Lagrange multipliers, rather than whole distributions.
So, your thesis is, only exponential models give rise to nice abstractions? And, since it’s important to have abstractions, we might just as well have our agents reason exclusively in terms of exponential models?
More like: exponential family distributions are a universal property of information-at-a-distance in large complex systems. So, we can use exponential models without any loss of generality when working with information-at-a-distance in large complex systems.
Yup, that’s the direction I want. If the distributions are exponential family, then that dramatically narrows down the space of distributions which need to be represented in order to represent abstractions in general. That means much simpler data structures—e.g. feature functions and Lagrange multipliers, rather than whole distributions.
So, your thesis is, only exponential models give rise to nice abstractions? And, since it’s important to have abstractions, we might just as well have our agents reason exclusively in terms of exponential models?
More like: exponential family distributions are a universal property of information-at-a-distance in large complex systems. So, we can use exponential models without any loss of generality when working with information-at-a-distance in large complex systems.
That’s what I hope to show, anyway.