I think people are somewhat talking past each other, and the following basically summarizes everyone’s position:
1) When dealing with the future, we have to make use of the best models available—we can’t base decisions now on data we don’t have yet.
2) New data should be used both to evaluate and improve models.
2a) It is important to test models against data that were not used in formulating the model, to avoid over-fitting. This can be new data as it becomes available, but should also be existing data reserved for the purpose.
I think people are somewhat talking past each other, and the following basically summarizes everyone’s position:
1) When dealing with the future, we have to make use of the best models available—we can’t base decisions now on data we don’t have yet.
2) New data should be used both to evaluate and improve models.
2a) It is important to test models against data that were not used in formulating the model, to avoid over-fitting. This can be new data as it becomes available, but should also be existing data reserved for the purpose.