On Regulatory Cost-Benefit Analysis

by Levi Russell

I recently ran across a fantastic article in Regulation magazine written by George Washington University regulation expert Susan Dudley. The article, entitled "OMB's Reported Benefits of Regulation: Too Good to Be True?" tackles an issue not often raised in policy discussions: What are the assumptions underlying cost-benefit analysis of regulation? Dudley explains in detail the way in which benefits are counted and how the scope of the analysis differs for benefits and costs. A single benefit category, reductions in fine particulate matter (PM 2.5), is responsible for the bulk of benefits calculated by OMB.

Given this focus on fine particulate matter, it would make sense that the science on the harm caused by PM 2.5 would inspire a lot of confidence. On the contrary, Dudley writes:

The OMB identifies six key assumptions that contribute to this uncertainty in PM2.5 benefits estimates. One assumption is that “inhalation of fine particles is causally associated with premature death at concentrations near those experienced by most Americans on a daily basis.” The EPA bases this assumption on epidemiological evidence of an association between particulate matter concentrations and mortality; however, as all students are taught, correlation does not imply causation (cum hoc non propter hoc), and the agency cannot identify a biological mechanism that explains  the  observed  correlation.  Risk  expert  Louis  Anthony  Cox raises questions as to whether the correlation the EPA claims is real. His statistical analysis (published in the journal Risk Analysis) concludes with a greater than 95 percent probability that no association exists and that, instead, the EPA’s results are a product of its choice of models and selected data rather than a real, measured correlation.

Another  key  assumption  on  which  the  EPA’s (and therefore the OMB’s) benefit estimates hinge is  that  “the  impact  function  for  fine  particles  is approximately  linear  within  the  range  of  ambient  concentrations  under  consideration,  which includes concentrations below the National Ambient Air Quality Standard” (NAAQS). Both theory and data suggest that thresholds exist below which further  reductions  in  exposure  to PM 2.5 do  not yield changes in mortality response and that one should expect diminishing returns as exposures are reduced to lower and lower levels. However, the EPA assumes  a  linear  concentration response  impact function that extends to concentration below background levels. The OMB observes, “indeed, a significant portion of the benefits associated with more  recent  rules  are  from  potential  health  benefits in regions that are in attainment with the fine particle standard.”

Based  on  its  assumptions  of  a  causal,  linear, no-threshold relationship between PM 2.5 exposure and premature mortality, the EPA quantifies a number  of  “statistical  lives”  that  will  be  “saved” when concentrations of PM 2.5 decline as a result of regulation. If any of those assumptions are false (in other words, if no association exists, if the relation-ship is not causal, or if the concentration-response relationship is not linear at low doses), the benefits of reducing PM 2.5 would be less than estimated and perhaps even zero.

Further, as the OMB notes, “the value of mortality risk reduction is taken largely from studies of the willingness to accept risk in the labor market[where the relevant population is healthy and has a  long  remaining  life  expectancy]  and  might  not necessarily apply to people in different stages of life or health status.” This caveat is particularly important in the case of PM2.5 because, as the EPA’s 2011 analysis reports, the median age of the beneficiaries of these regulations is around 80 years old, and the average extension in life expectancy attributable to lower PM 2.5 levels is less than six months.

 It's clear that there are some serious, objective problems with the way some benefits of regulation are calculated. Dudley concludes:

The OMB’s role is to serve as a check against agencies’ natural motivation to paint a rosy picture of their proposed actions. While it cannot ensure that agencies consider all the possible consequences of an action in their analyses, it should try to ensure that the boundaries of those analyses are set with some regard to objective science. When a few categories of benefits that have questionable legitimacy puff up benefits by a five-fold margin or more, that does not appear to be the case.

Beyond the objective, scientific questions concerning the benefits of regulation, analysis of the costs are important as well. In my recent piece in Perspective, a magazine published by the Oklahoma Council of Public Affairs, on the costs of environmental regulation of agriculture, I point to the fundamental uncertainty facing regulators. This uncertainty is not accounted for in the cost calculations of the regulations they enforce:

The uncertainty and compliance costs associated with these regulations represent serious concerns for producers. Recent surveys of row crop producers, cattle producers, and feedlot operators indicate that future environmental regulation is a top concern for their businesses over the long term.
...
This is not to say that regulators are ill-intentioned. They face a highly complex and difficult problem: implementing the will of Congress for the betterment of the American people. The knowledge and information required to regulate even one industry is immense. Not only is it costly to obtain the information necessary to pass effective regulations, regulators can’t be sure that unforeseen unintended consequences won’t diminish the effectiveness of their rules or cause more harm than good. Proposed measures to ensure effective regulation that is not overly burdensome, such as sunset provisions that would require regulations to lapse on a periodic basis, have been put forth but have not been implemented widely. Other propositions include less federal and more local and state control over environmental policy and greater use of common law courts to deal with environmental problems. Both of these proposals acknowledge the information problems inherent in the regulation of agriculture.

There are significant political hurdles to overcome if we are to inject more scientific and objective analysis into regulatory cost-benefit calculation. Knowing how that calculation is done is a crucial first step; Susan Dudley's article is a great way to inform the public so we can get the reform ball rolling!