Is there a general way to answer questions like this, which often occur in economics and the social sciences:
“Does institution X play a part in keeping parameter Y stable? It looks like parameter Y has been really stable for awhile now. Is institution X doing a good job, or is it completely useless?”
Well, to go ahead and state the incredibly obvious: in cases where institution X is not equally well-established globally, one thing to look at is variations in X among different nations, geographic regions, populations, etc. (depending on the kind of thing X is). If Y remains equally stable across the board while X varies, that’s evidence that X doesn’t have much to do with Y.
Part of the problem is that changes in X might be aimed at keeping Y stable when some other factor Z varies. See Milton Friedman’s Thermostat:
If the driver is doing his job right, and correctly adjusting the gas pedal to the hills, you should find zero correlation between gas pedal and speed, and zero correlation between hills and speed.
For example, it’s hard to answer the question “do armies stop invasions?” by using correlations, because the rulers can adjust the strength of the army in response to the risk of getting invaded, so the resulting risk depends mostly on the risk tolerance of the rulers.
Is there a general way to answer questions like this, which often occur in economics and the social sciences:
“Does institution X play a part in keeping parameter Y stable? It looks like parameter Y has been really stable for awhile now. Is institution X doing a good job, or is it completely useless?”
Well, to go ahead and state the incredibly obvious: in cases where institution X is not equally well-established globally, one thing to look at is variations in X among different nations, geographic regions, populations, etc. (depending on the kind of thing X is). If Y remains equally stable across the board while X varies, that’s evidence that X doesn’t have much to do with Y.
Part of the problem is that changes in X might be aimed at keeping Y stable when some other factor Z varies. See Milton Friedman’s Thermostat:
For example, it’s hard to answer the question “do armies stop invasions?” by using correlations, because the rulers can adjust the strength of the army in response to the risk of getting invaded, so the resulting risk depends mostly on the risk tolerance of the rulers.