if we want economics to be a science, we have to recognize that it is not ok for macroeconomists to hole up in separate camps, one that supports its version of the geocentric model of the solar system and another that supports the heliocentric model. As scientists, we have to hold ourselves to a standard that requires us to reach a consensus about which model is right, and then to move on to other questions.
The alternative to science is academic politics, where persistent disagreement is encouraged as a way to create distinctive sub-group identities.
What if everyone knows that all the models are flawed, but the geocentric model makes the best predictions in one sub-domain, and the heliocentric model in another?
Then the most important question for any model would be what domains it’s good at.
For example: one model approximates the population as infinite, so it gets decent predictions when the number of agents in each category exceeds five (this is rare).
These requirements to apply the model should be the first thing taught about the model.
Well, if you replace “geocentric model” and “heliocentric model” with “general relativity” and “quantum field theory” that’s pretty much the situation that obtains in present-day theoretical physics.
In physics general relativity and quantum field theory are applied to different domains and at least one, possibly both, are widely recognized as mere approximations to the ultimate theory that subsumes them.
I’ll defer to Dr. Miller on this if he cares to weigh in, or any other professional economist, but my outsider’s impression is that in economics as discussed by Romer the situation is more that contradictory theories are being applied to the same domain, without a serious effort to determine experimentally which (if either) is correct.
I’ll defer to Dr. Miller on this if he cares to weigh in, or any other professional economist, but my outsider’s impression is that in economics as discussed by Romer the situation is more that contradictory theories are being applied to the same domain, without a serious effort to determine experimentally which (if either) is correct.
Sometimes, yes. And at least one school loudly insists that they don’t need no stinkin’ experimental evidence, because they’re actually doing a deductive formal science. In a sign of uncertain health for economics, they are considered heterodox, but not yet laughed out of polite academia.
Rapid, yes, but extremely noisy. If you make some decision and lose a ton of money, that doesn’t mean it was a bad decision; maybe you just got unlucky.
What’s noisy is the information you get rapidly, not necessarily the outcomes. If you (stupidly, of course) buy and short-sell the same amount of some asset, your outcome is noiseless but the information you get from watching what actually happens to the asset is still noisy.
Sure. If, for instance, you decide that the information you want to get from watching your investments is the value of pi, then provided you compute it in the right way (e.g., by ignoring your investments and summing a rapidly convergent series) it won’t be noisy at all.
I assume you mean something distinctly less trivial than that, but I’m not sure what. I’d have thought that in a typical case the feedback you’re seeking is something like “is this trading strategy I’m using a good one?” Meaning: “if I continue to use it, will I make money?”. I have never worked at an investment bank or hedge fund, but my impression is that usually it takes some time before the answer to that question becomes clear, if it ever does. (The fact that the market is changing beneath your feet as you observe it doesn’t make that any easier, of course.)
I’d have thought that in a typical case the feedback you’re seeking is something like “is this trading strategy I’m using a good one?”
Yes, that’s a fair example.
my impression is that usually it takes some time before the answer to that question becomes clear, if it ever does
Nope, people who can’t figure out the answer to this question fairly quickly go out of business on about the same time scale :-)
How quickly actually depends on the trading strategy. One of the big advantages of high-frequency trading, for example, is that the trader will know there is something wrong with the strategy in a matter of hours. He may fine-tune it for months, but whether it works or not is clear very quickly. On the other end of the spectrum are long-term illiquid investments like private equity. In those cases it can, indeed, take years before you know whether your choices were good ones.
I would say that typically a few weeks to a couple of months of losses (or a few months of neither making nor losing money) is enough to subject a trading strategy to scrutiny.
First, we’re talking about typically complex strategies that are designed to run for a long time. Usually they accept short-term variation, but think that they’ll come out ahead after the noise diversifies away. But trading ideas usually have much faster feedback cycles. For example, if you think that at the moment some asset XYZ is trading cheap because of a temporary imbalance in supply and demand and it will revert to its normal valuation in a few hours—you’ll find out if that idea was correct in a few hours.
Second, the strategies I mentioned have already passed a very high bar—they convinced a sufficient number of people that they are good enough to commit real money to. At the research stage, figuring out if a strategy looks decent takes a few minutes—you feed it to a trading simulator based on historical data and look at the results. The strategies in the grandparent post are the equivalent of clinical trials in pharma—there are already a lot of people convinced they would work.
Finance, though, is a sub-field that provides one with rapid and unambiguous feedback most of the time :-)
It’s also a field in which one can get “positive feedback” (ie: make a profit) by taking completely randomized actions and then just waiting for the world to reward you anyway. Most available studies show that most professional money-managers don’t beat the market most of the time. On the whole, making money off macroeconomic growth doesn’t yield scientific bona-fides.
Market-tracking index funds are decidedly non-random. Lumifer’s point is that you’re confusing the professional money-manager’s ability to make money off people who don’t want to manage their own money (i.e. marketing) with the professional money-manager’s ability to make money off picking individual stocks (i.e. finance), which on average does not exist. The money-management field does not represent ‘free money’ in that you need to actively market for clients, and there is not free money to be had in doing mediocre money management with your own money.
The point that you were trying to make, that there’s money to made simply by having capital and investing it in the entire economy as a whole, doesn’t seem like a knock-down objection to Lumifer’s point; by watching how my index fund holdings are doing, I am getting rapid and unambiguous feedback about how the overall economy is doing relative to various segments or other holdings.
Let me also point out three different things and note that they are different.
Thing one is financial services. They are services—they provide you with something in exchange for some sort of a fee. For example, providing an index-tracking mutual fund is a service. If you want to use this service, you pay for it and if the service is sufficiently popular, the provider makes a profit. This is not different in principle from buying the services of, say, a gardener or a car mechanic. Of course, some providers make inflated claims about their services, but that’s hardly limited to finance.
Thing two is investment/trading where you are trying to, basically, extract (more) money out of a market.
Thing three is putting capital to work which, strictly speaking, doesn’t even require markets. If you have some value and you put that value to productive use, you can expect (subject to a large number of caveats) to get some profit. This is not even finance, but basic economics.
Note that in none of these cases anyone is “taking completely randomized actions” or is guaranteed a profit.
As scientists, we have to hold ourselves to a standard that requires us to reach a consensus about which model is right, and then to move on to other questions.
This is an example Goodhart’s law. Real sciences of course ultimately reach a consensus around the truth, but trying for consensus for the sake of consensus is likely to result in a consensus around a false belief being reached.
but trying for consensus for the sake of consensus
I think the aim is not consensus, but consistency. If two camps hold irreconcilable views, one of them is wrong and it’s highly useful to know which one. The fact that both views have some domains where they seem to work better than the other is not a good excuse.
I think the aim is not consensus, but consistency.
Consistency with what. If you mean consistency with the truth, I agree. However, the context talks about consistency among expert positions, which is what is normally called consensus. Downvoted for introducing terminology for seemingly no purpose other than to confuse the issue.
If two camps hold irreconcilable views, one of them is wrong and it’s highly useful to know which one.
True, however, pushing for consensus or consistency will not tell you which one is wrong. Rather it will result in one of them being declared “wrong”, not necessarily the one that actually is.
The fact that both views have some domains where they seem to work better than the other is not a good excuse.
Ok, did you mean to write this in reply to some other comment?
Persistent disagreement is a sign that some of the participants in a discussion are not committed to the norms of science.
He should reread Kuhn. Kuhn says that the cause of persistent disagreement is usually the lack of a relevant and workable scientific paradigm which can identify important problems, resolve disputes, and thereby mandate researchers to come to consensus. Romer’s use of the phrase “the norms of science” indicates that he believes in a singular, universal, monolithic set of principles which is valid for all types of scientific inquiry. But economists obviously cannot use the same principles as physicists, simply because they cannot run experiments. What Romer is really complaining about is that there is no good paradigm for economics, but that’s not anyone’s fault—the discovery and articulation of a paradigm is as difficult as doing the science that the paradigm supports. A more valid criticism of the field would be “We are trying to do science without a strong enough paradigm, and the weakness of the paradigm is preventing us from resolving our disagreements definitively. Instead of trying to do more research along the same old lines, we should go back to the philosophical foundations and re-examine what it means to do economics.”
I’ve been wondering lately whether it is possible for economics to get a more empirical foundation. Clearly, a serious difficulty in the field is our lack of having a way for doing controlled trials. Does anyone know if anyone has tried bribing people to live in small-towns/enclaves (one to serve as control) for a time to see if we can isolate some effects at small levels that may or may not scale up? Or is this just too ridiculously impractical? (Or just too expensive?)
That’s not true. Economics (in particular, microeconomics) can and actually does do a lot of controlled trials. I don’t think that’s the problem. Consider psychology—it does a LOT of controlled trials and generates a very impressive amount of garbage.
microeconomics) can and actually does do a lot of controlled trials.
Do you happen to know anywhere I can read simplified (layman-readable) results of some of these?
Psychology has recently been implicated in the “can’t reproduce your results” scandal, suggesting that a lot of the garbage they generate is due, more or less, to pressure to publish, bias towards confirming expectations, and insufficient safeguards. Do microeconomics trials suffer the same problems?
--Paul Romer, NYU, “My Paper “Mathiness in the Theory of Economic Growth”
What if everyone knows that all the models are flawed, but the geocentric model makes the best predictions in one sub-domain, and the heliocentric model in another?
Then the most important question for any model would be what domains it’s good at.
For example: one model approximates the population as infinite, so it gets decent predictions when the number of agents in each category exceeds five (this is rare).
These requirements to apply the model should be the first thing taught about the model.
Well, if you replace “geocentric model” and “heliocentric model” with “general relativity” and “quantum field theory” that’s pretty much the situation that obtains in present-day theoretical physics.
In physics general relativity and quantum field theory are applied to different domains and at least one, possibly both, are widely recognized as mere approximations to the ultimate theory that subsumes them.
I’ll defer to Dr. Miller on this if he cares to weigh in, or any other professional economist, but my outsider’s impression is that in economics as discussed by Romer the situation is more that contradictory theories are being applied to the same domain, without a serious effort to determine experimentally which (if either) is correct.
Hanson once said that testablity is not a useful notion in the social sciences. That seems kind of crazy to me, but I am not an economist.
Sometimes, yes. And at least one school loudly insists that they don’t need no stinkin’ experimental evidence, because they’re actually doing a deductive formal science. In a sign of uncertain health for economics, they are considered heterodox, but not yet laughed out of polite academia.
I have no idea which one you are talking about.
Does it actually? Economists rarely make unhedged predictions and then rigorously test how did they turn out.
Finance, though, is a sub-field that provides one with rapid and unambiguous feedback most of the time :-)
Rapid, yes, but extremely noisy. If you make some decision and lose a ton of money, that doesn’t mean it was a bad decision; maybe you just got unlucky.
No, not usually, at least for common to me values of “extremely” :-) Finance does not require you to select high-variance bets :-)
True, but if your actual outcomes are “extremely noisy” you need better risk management which happens to be part of finance.
What’s noisy is the information you get rapidly, not necessarily the outcomes. If you (stupidly, of course) buy and short-sell the same amount of some asset, your outcome is noiseless but the information you get from watching what actually happens to the asset is still noisy.
Doesn’t that entirely depend on what kind of information you are interested in?
Sure. If, for instance, you decide that the information you want to get from watching your investments is the value of pi, then provided you compute it in the right way (e.g., by ignoring your investments and summing a rapidly convergent series) it won’t be noisy at all.
I assume you mean something distinctly less trivial than that, but I’m not sure what. I’d have thought that in a typical case the feedback you’re seeking is something like “is this trading strategy I’m using a good one?” Meaning: “if I continue to use it, will I make money?”. I have never worked at an investment bank or hedge fund, but my impression is that usually it takes some time before the answer to that question becomes clear, if it ever does. (The fact that the market is changing beneath your feet as you observe it doesn’t make that any easier, of course.)
Yes, that’s a fair example.
Nope, people who can’t figure out the answer to this question fairly quickly go out of business on about the same time scale :-)
How quickly actually depends on the trading strategy. One of the big advantages of high-frequency trading, for example, is that the trader will know there is something wrong with the strategy in a matter of hours. He may fine-tune it for months, but whether it works or not is clear very quickly. On the other end of the spectrum are long-term illiquid investments like private equity. In those cases it can, indeed, take years before you know whether your choices were good ones.
I would say that typically a few weeks to a couple of months of losses (or a few months of neither making nor losing money) is enough to subject a trading strategy to scrutiny.
Aha. I think our actual misunderstanding was about what counts as “rapid” feedback; I had in mind a shorter timescale than you did.
Keep in mind two additional points.
First, we’re talking about typically complex strategies that are designed to run for a long time. Usually they accept short-term variation, but think that they’ll come out ahead after the noise diversifies away. But trading ideas usually have much faster feedback cycles. For example, if you think that at the moment some asset XYZ is trading cheap because of a temporary imbalance in supply and demand and it will revert to its normal valuation in a few hours—you’ll find out if that idea was correct in a few hours.
Second, the strategies I mentioned have already passed a very high bar—they convinced a sufficient number of people that they are good enough to commit real money to. At the research stage, figuring out if a strategy looks decent takes a few minutes—you feed it to a trading simulator based on historical data and look at the results. The strategies in the grandparent post are the equivalent of clinical trials in pharma—there are already a lot of people convinced they would work.
It’s also a field in which one can get “positive feedback” (ie: make a profit) by taking completely randomized actions and then just waiting for the world to reward you anyway. Most available studies show that most professional money-managers don’t beat the market most of the time. On the whole, making money off macroeconomic growth doesn’t yield scientific bona-fides.
Heh. Why don’t you go try it? Sounds like free money X-/
Besides, aren’t you confusing finance and marketing?
Do you not have savings at all, or did you actually just say you’ve put no portion of your savings in market-tracking index funds?
Market-tracking index funds are decidedly non-random. Lumifer’s point is that you’re confusing the professional money-manager’s ability to make money off people who don’t want to manage their own money (i.e. marketing) with the professional money-manager’s ability to make money off picking individual stocks (i.e. finance), which on average does not exist. The money-management field does not represent ‘free money’ in that you need to actively market for clients, and there is not free money to be had in doing mediocre money management with your own money.
The point that you were trying to make, that there’s money to made simply by having capital and investing it in the entire economy as a whole, doesn’t seem like a knock-down objection to Lumifer’s point; by watching how my index fund holdings are doing, I am getting rapid and unambiguous feedback about how the overall economy is doing relative to various segments or other holdings.
Did I actually say? Quote me.
Let me also point out three different things and note that they are different.
Thing one is financial services. They are services—they provide you with something in exchange for some sort of a fee. For example, providing an index-tracking mutual fund is a service. If you want to use this service, you pay for it and if the service is sufficiently popular, the provider makes a profit. This is not different in principle from buying the services of, say, a gardener or a car mechanic. Of course, some providers make inflated claims about their services, but that’s hardly limited to finance.
Thing two is investment/trading where you are trying to, basically, extract (more) money out of a market.
Thing three is putting capital to work which, strictly speaking, doesn’t even require markets. If you have some value and you put that value to productive use, you can expect (subject to a large number of caveats) to get some profit. This is not even finance, but basic economics.
Note that in none of these cases anyone is “taking completely randomized actions” or is guaranteed a profit.
This is an example Goodhart’s law. Real sciences of course ultimately reach a consensus around the truth, but trying for consensus for the sake of consensus is likely to result in a consensus around a false belief being reached.
I think the aim is not consensus, but consistency. If two camps hold irreconcilable views, one of them is wrong and it’s highly useful to know which one. The fact that both views have some domains where they seem to work better than the other is not a good excuse.
Unless of course you’re operating in a situation best modelled by paraconsistent logic.
But finding that out would require looking.
Would you care to provide some examples along with arguments why paraconsistent logic is the best way to model them?
Consistency with what. If you mean consistency with the truth, I agree. However, the context talks about consistency among expert positions, which is what is normally called consensus. Downvoted for introducing terminology for seemingly no purpose other than to confuse the issue.
True, however, pushing for consensus or consistency will not tell you which one is wrong. Rather it will result in one of them being declared “wrong”, not necessarily the one that actually is.
Ok, did you mean to write this in reply to some other comment?
Internal consistency of the science as a body of knowledge. “Consistency with the truth” is better expressed by the simple adjective “true”.
It’s not consensus either. Consistency is a property of a theory (or a set of theories). Consensus is a property of a set of experts.
No, but it will provide impetus and motivation to find out which one is wrong.
Romer goes on to write:
He should reread Kuhn. Kuhn says that the cause of persistent disagreement is usually the lack of a relevant and workable scientific paradigm which can identify important problems, resolve disputes, and thereby mandate researchers to come to consensus. Romer’s use of the phrase “the norms of science” indicates that he believes in a singular, universal, monolithic set of principles which is valid for all types of scientific inquiry. But economists obviously cannot use the same principles as physicists, simply because they cannot run experiments. What Romer is really complaining about is that there is no good paradigm for economics, but that’s not anyone’s fault—the discovery and articulation of a paradigm is as difficult as doing the science that the paradigm supports. A more valid criticism of the field would be “We are trying to do science without a strong enough paradigm, and the weakness of the paradigm is preventing us from resolving our disagreements definitively. Instead of trying to do more research along the same old lines, we should go back to the philosophical foundations and re-examine what it means to do economics.”
This is a field in which the discoverer of the theorem that rational agents cannot disagree was given the highest possible honours...
I’ve been wondering lately whether it is possible for economics to get a more empirical foundation. Clearly, a serious difficulty in the field is our lack of having a way for doing controlled trials. Does anyone know if anyone has tried bribing people to live in small-towns/enclaves (one to serve as control) for a time to see if we can isolate some effects at small levels that may or may not scale up? Or is this just too ridiculously impractical? (Or just too expensive?)
That’s not true. Economics (in particular, microeconomics) can and actually does do a lot of controlled trials. I don’t think that’s the problem. Consider psychology—it does a LOT of controlled trials and generates a very impressive amount of garbage.
Do you happen to know anywhere I can read simplified (layman-readable) results of some of these?
Psychology has recently been implicated in the “can’t reproduce your results” scandal, suggesting that a lot of the garbage they generate is due, more or less, to pressure to publish, bias towards confirming expectations, and insufficient safeguards. Do microeconomics trials suffer the same problems?
A couple of links.
Found… Database for registering economic controlled trials and a (unpublished?) paper that suggests economic RCTs have more problems than medical trials.
Excellent, thank you!