Bingo! That’s exactly what I was trying to say. Thanks for listening. :-)
My jargon mostly comes from political science. We’d say the meteorologists are using an overly complicated model, or seizing on spurious correlations, or that they have a low pseudo-R-squared. I’m not sure any of those are helpful. Personally, I think your words—the meteorologists are too ignorant for us to applaud their calibration—are more elegant.
The only other thing I would add is that the reason why it doesn’t make sense to applaud the meteorologists’ guess-level calibration is because they have such poor model-level calibration. In other words, while their confidence about any given guess seems accurate, their implicit confidence about the accuracy of their model as a whole is too high. If your (complex) model does not beat a naive predictor, social science (and, frankly, Occam’s Razor) says you ought to abandon it in favor of a simpler model. By sticking to their complex models in the face of weak predictive power, the meteorologists suggest that either (1) they don’t know or care about Occam’s Razor, or (2) they actually think their model has strong predictive power.
Here’s a really crude indicator of improvement in weather forecasting: I can remember when jokes about forecasts being wrong were a cliche. I haven’t heard a joke about weather forecasts for years, probably decades, which suggests that forecasts have actually gotten fairly good, even if they’re not as accurate as the probabilities in the forecasts suggest.
Does anyone remember when weatherman jokes went away?
Can we conclude that the prevalence of the cliche dropping is related to the quality of weather forecasting? All else being equal I expect a culture to develop a resistance to any given cliche over time. For example the cliche “It’s not you it’s me” has dropped in use and been somewhat relegated to ‘second order cliche’ . But it is true now at least as much as it has been in the past.
A fair point, though if a cliche has lasted for a very long time, I think it’s more plausible that its end is about changed conditions rather than boredom.
Bingo! That’s exactly what I was trying to say. Thanks for listening. :-)
My jargon mostly comes from political science. We’d say the meteorologists are using an overly complicated model, or seizing on spurious correlations, or that they have a low pseudo-R-squared. I’m not sure any of those are helpful. Personally, I think your words—the meteorologists are too ignorant for us to applaud their calibration—are more elegant.
The only other thing I would add is that the reason why it doesn’t make sense to applaud the meteorologists’ guess-level calibration is because they have such poor model-level calibration. In other words, while their confidence about any given guess seems accurate, their implicit confidence about the accuracy of their model as a whole is too high. If your (complex) model does not beat a naive predictor, social science (and, frankly, Occam’s Razor) says you ought to abandon it in favor of a simpler model. By sticking to their complex models in the face of weak predictive power, the meteorologists suggest that either (1) they don’t know or care about Occam’s Razor, or (2) they actually think their model has strong predictive power.
Here’s a really crude indicator of improvement in weather forecasting: I can remember when jokes about forecasts being wrong were a cliche. I haven’t heard a joke about weather forecasts for years, probably decades, which suggests that forecasts have actually gotten fairly good, even if they’re not as accurate as the probabilities in the forecasts suggest.
Does anyone remember when weatherman jokes went away?
Can we conclude that the prevalence of the cliche dropping is related to the quality of weather forecasting? All else being equal I expect a culture to develop a resistance to any given cliche over time. For example the cliche “It’s not you it’s me” has dropped in use and been somewhat relegated to ‘second order cliche’ . But it is true now at least as much as it has been in the past.
A fair point, though if a cliche has lasted for a very long time, I think it’s more plausible that its end is about changed conditions rather than boredom.
Gotcha. Thanks for the explanation, it’s been very clarifying. =)