Economic Theory: Rigor and Relevance

Physicist Mark Buchanan recently wrote an article in BloombergView entitled "Explaining How Economists Explain." In the article Buchanan makes some good points about how economists conduct theoretical work. He writes:

Several years ago, in the immediate wake of the financial crisis, economist Ricardo Caballero wrote about what he called the “pretense-of-knowledge syndrome” in academia. Economists, he argued, had become “so mesmerized” with the internal logic of their theories that much of the discipline -- even that part concerned directly with policy making -- had spiraled off into fantasy. Even when they studied issues close to the crisis, such as bubbles, panics and fire sales, they relegated them to the periphery of macroeconomics, which at its core valued mathematical elegance over usefulness. [n.b. Caballero has written a handful of papers over the past several years analyzing the financial crisis through the lens of a version of Austrian Business Cycle Theory]

Not much has changed since then. That, at least, is the conclusion of Itzhak Gilboa and a group of economists who recently tried to understand why their profession operates so differently from most sciences. Academic economists, they say, use the term "explanation" in a way that other scientists never would. Instead of developing realistic and testable theories like those in biology or physics, they often aim only to develop "theoretical cases" -- imaginary mathematical worlds with their own rules of cause and effect.

The main point here is that internal validity has trumped external validity as the focus of economic theory (at both the macro and micro levels). This has produced, broadly speaking, a body of literature that is more concerned with mathematical elegance (or rigor) than policy relevance (whether that means government policy or smart business strategy) or an improved understanding of economic reality. This doesn't imply that mathematical modeling can't be useful or that some focus on internal validity isn't necessary, but there's a tradeoff between mathematical rigor and real-world practicality. For every unrealistic assumption made we get more tractability and measurability, and less practicality and generalizability.

He also mentions cause and effect. This is a huge issue, I think, mostly because causality in economics has to be determined by reason, not only by a look at the data. A biologist in a laboratory can carefully construct experiments with treatment and control groups. The control group tells the biologist what would happen if the treatment weren't applied. Thus, causality can be established in the physical sciences by a look at experimental data.

However, in economics, we don't have access to counterfactuals (i.e. the control group). We can't sort out causality with data because the data can't tell us what would have happened if some policy hadn't been implemented or some economic condition hadn't changed. In microeconomics, this can be chalked up to subjectivity of costs and benefits and the possibility of preference changes over time. In macroeconomics, it's simple enough to state that no laboratory exists and the number of factors that potentially drive some observed change is very large. The implication is that hypothesis testing in economics should be done with even greater humility than it should be in the physical sciences. Based on data alone, we can only provisionally reject or accept a given hypothesis. The real battle over cause and effect has to be fought on purely logical grounds with the data as a supplement or illustration.

There's a lot more to be said specifically on the mathematical rigor point. This post is already fairly long, so I'll just make a quick comment on math. Richard A. Levins, Emeritus professor at the University of Minnesota, has a classic Choices article from 1989 that points out in very simple terms some of the biggest problems with focusing on mathematical rigor in economic theory. His analysis of the "as if" assumption reaches farther than he points out because the fundamental assumptions necessary for the use of calculus in economics only apply if the "as if" assumption is valid. Levins argues it isn't and I'm inclined to agree. Bryan Caplan of George Mason University has a pair of blog posts on the topic of math in economics (here and here) that are also very good.

The bottom line is that economic theory which is policy-relevant and improves our understanding of how real-world markets, firms, and governments work is likely to be far more useful to society than that which focuses primarily on mathematical rigor. On a positive note, I think applied economists, generally speaking, do a very good job of providing interesting illustrations of good theory using statistical methods. Of course, I'm biased on that point. Don Boudreaux has a very good (and short) post that incorporates a lot of what I've said here.

He concludes:

The good economist, in short, sees with his reason that which is often invisible to his eyes.

And often invisible also, I here add, to his or her econometric tools – tools which can never capture, much leas [sic] measure accurately, the full range of the important and relevant details that are always changing in a modern market economy.  

Well said.