It seems like “generators” should just be simple functions over natural abstractions? But I see two different ways to go with this, inspired either by the minimal latents approach, or by the redundant-information one.
First, suppose I want to figure out a high-level model of some city, say Berlin. I already have a “city” abstraction, let’s call it P(Λ), which summarizes my general knowledge about cities in terms of a probability distribution over possible structures. I also know a bunch of facts about Berlin specifically, let’s call their sum F. Then my probability distribution over Berlin’ structure is just P(XBerlin)=P(Λ|F).
Alternatively, suppose I want to model the low-level dynamics of some object I have an abstract representation for. In this case, suppose it’s the business scene of Berlin. I condition my abstraction of a business P(B) on everything I know about Berlin, P(B|XBerlin), then sample from the resulting distribution several times until I get a “representative set”. Then I model its behavior directly.
It seems like “generators” should just be simple functions over natural abstractions? But I see two different ways to go with this, inspired either by the minimal latents approach, or by the redundant-information one.
First, suppose I want to figure out a high-level model of some city, say Berlin. I already have a “city” abstraction, let’s call it P(Λ), which summarizes my general knowledge about cities in terms of a probability distribution over possible structures. I also know a bunch of facts about Berlin specifically, let’s call their sum F. Then my probability distribution over Berlin’ structure is just P(XBerlin)=P(Λ|F).
Alternatively, suppose I want to model the low-level dynamics of some object I have an abstract representation for. In this case, suppose it’s the business scene of Berlin. I condition my abstraction of a business P(B) on everything I know about Berlin, P(B|XBerlin), then sample from the resulting distribution several times until I get a “representative set”. Then I model its behavior directly.
This doesn’t seem quite right, though.