So when people say “the universal prior says that Maxwell’s equations are simple and Zeus is complex”, what they actually mean is that in their experience mathematical descriptions of natural phenomena have proved more fruitful than descriptions that involve agents; the universal prior has nothing to do with this, and invoking it is dangerous as it encourages double-counting of evidence
Attempting to work the dependence of my epistemology on my experience into my epistemology itself creates a cycle in the definitions of types, and wrecks the whole thing. I suspect that reformalizing as a fixpoint thing would fix the problem, but I suspect even more strongly that the point I’m already at would be a unique fixpoint and that I’d be wrecking its elegance for the sake of generalizing to hypothetical agents that I’m not and may never encounter. (Or that all such fixpoints can be encoded as prefixes, which I too feel like sweeping under the rug.)
Attempting to work the dependence of my epistemology on my experience into my epistemology itself creates a cycle in the definitions of types, and wrecks the whole thing. I suspect that reformalizing as a fixpoint thing would fix the problem, but I suspect even more strongly that the point I’m already at would be a unique fixpoint and that I’d be wrecking its elegance for the sake of generalizing to hypothetical agents that I’m not and may never encounter. (Or that all such fixpoints can be encoded as prefixes, which I too feel like sweeping under the rug.)