My understanding has improved since writing this post.
Generative and predictive models can indeed be substantially different—but as you point out the reason we give is unsatisfying.
The better thing to point towards is there are finite generative models such that the optimal predictive model is infinite.
See this paper for more.
My understanding has improved since writing this post.
Generative and predictive models can indeed be substantially different—but as you point out the reason we give is unsatisfying.
The better thing to point towards is there are finite generative models such that the optimal predictive model is infinite.
See this paper for more.