However, the critic is learning to predict; it’s just that all we need to predict is expected value.
See also muZero. Also note that predicting optimal value formally ties in with predicting a model. In MDPs, you can reconstruct all of the dynamics using just |S| optimal value functions.
See also muZero. Also note that predicting optimal value formally ties in with predicting a model. In MDPs, you can reconstruct all of the dynamics using just |S| optimal value functions.