TurnTrout comments on Has anybody used quantification over utility functions to define “how good a model is”?

[TurnTrout]

I ask because I already have a result that says this in MDPs: you can compute all optimal value functions iff you know the environment dynamics up to isomorphism.