I find the existing MDP isomorphisms/equivalences to be pretty lacking.
I have a paper on equivalences (and counterfactual equivalences, which is stronger) for POMDPs: https://arxiv.org/abs/1801.03737
I have a paper on equivalences (and counterfactual equivalences, which is stronger) for POMDPs: https://arxiv.org/abs/1801.03737