If they have mean/sd in common (as in e.g. a Gaussian clustering problem), then the mean/sd are exactly the abstract information. If they’re all completely independent, without any latents (like mean/sd) at all, then the blob itself is not a natural abstraction, at least if we’re staying within an information-theoretic playground.
I do expect this will eventually need to be extended beyond mutual information, especially to handle the kinds of abstractions we use in math (like “groups”, for instance). My guess is that most of the structure will carry over; Bayes nets and mutual information have pretty natural category-theoretic extensions as I understand it, and I expect that roughly the same approach and techniques I use here will extend to that setting. I don’t personally have enough expertise there to do it myself, though.
If they have mean/sd in common (as in e.g. a Gaussian clustering problem), then the mean/sd are exactly the abstract information. If they’re all completely independent, without any latents (like mean/sd) at all, then the blob itself is not a natural abstraction, at least if we’re staying within an information-theoretic playground.
I do expect this will eventually need to be extended beyond mutual information, especially to handle the kinds of abstractions we use in math (like “groups”, for instance). My guess is that most of the structure will carry over; Bayes nets and mutual information have pretty natural category-theoretic extensions as I understand it, and I expect that roughly the same approach and techniques I use here will extend to that setting. I don’t personally have enough expertise there to do it myself, though.