It makes sense that to address the challenges of the agent being embedded one needs to start at the very foundations. I suspect that there is a fair bit of work before even addressing embedding the agents. For example, in a basic map-territory correspondence the map is a part of the territory. So, a question arises, what does it mean for a part of the territory be a coarse-grained representation of the territory? What restrictions it places on the type of territories that are internally mappable to begin with? For example, it has to admit lossy compression of some kind, yet not be completely fractal. Anyway, my point is that focusing on the agency maybe a wrong place to start, there are more basic questions of embeddings that need to be addressed first. And even figuring out what those questions might be would count as progress.
It makes sense that to address the challenges of the agent being embedded one needs to start at the very foundations. I suspect that there is a fair bit of work before even addressing embedding the agents. For example, in a basic map-territory correspondence the map is a part of the territory. So, a question arises, what does it mean for a part of the territory be a coarse-grained representation of the territory? What restrictions it places on the type of territories that are internally mappable to begin with? For example, it has to admit lossy compression of some kind, yet not be completely fractal. Anyway, my point is that focusing on the agency maybe a wrong place to start, there are more basic questions of embeddings that need to be addressed first. And even figuring out what those questions might be would count as progress.
I strongly agree with this. Those sorts of questions are exactly what I see as the main objective of my own research right now.