We have some recent stuff on generalizing MDPs to have a causal model inside every state (‘path dependent structural equation models’, to appear in UAI this year).
Thanks Ilya for those links, in particular the second one looks quite relevant to something we’ve been working on in a rather different context (that’s the benefit of speaking the same language!)
We would also be curious to see a draft of the MDP-generalization once you have something ready to share!
Pretty interesting.
Since you are interested in policies that operate along some paths only, you might find these of interest:
https://pubmed.ncbi.nlm.nih.gov/31565035/
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6330047/
We have some recent stuff on generalizing MDPs to have a causal model inside every state (‘path dependent structural equation models’, to appear in UAI this year).
Thanks Ilya for those links, in particular the second one looks quite relevant to something we’ve been working on in a rather different context (that’s the benefit of speaking the same language!)
We would also be curious to see a draft of the MDP-generalization once you have something ready to share!
https://auai.org/uai2021/pdf/uai2021.89.preliminary.pdf (this really is preliminary, e.g. they have not yet uploaded a newer version that incorporates peer review suggestions).
---
Can’t do stuff in the second paper without worrying about stuff in the first (unless your model is very simple).