"Metrics for Evaluating Policy Commitments in a Fragmented World: The Challenges of Equity and Integrity"
Discussion Paper 08-17, Harvard Project on Climate Agreements, Belfer Center
Belfer Center Programs or Projects: Harvard Project on Climate Agreements
Despite the uncertainties about the nature and stringency of national emission reduction commitments, some things are clear: the international negotiations not only will include national targets and timetables, but also will have to take account of diverse policies and measures undertaken by individual nations, including those inside the current Kyoto group, as well as among developing countries. The evaluation of these diverse policies poses a number of challenges. For example, how can one assess the fairness of the relative contributions of different nations? And even if fairness is agreed upon, how is it possible to determine the credibility of the commitments?
Ex post, determining whether particular policies have been implemented is a relatively simple matter, although assessing their effectiveness is not always straightforward. Ex-ante, however, the integrity of the international process requires at least some evaluation of the policies and measures proposed by individual nations to estimate their likely impacts. The absence of such evaluation may handicap the negotiators in reaching credible agreements.
The current system for reporting national actions to the international community is highly nonuniform and insufficient to understand differences among countries’ policies and their effectiveness. Thus, a first order of business should be the development of a much tighter, narrowly defined set of guidelines designed to reflect genuine differences in activities among nations.
Regarding the fairness of the commitments, certain metrics, like emissions as a share of GDP, population, or historical emissions, are straightforward to calculate, and generally informative, albeit imperfect indicators of burden. Other metrics, like emissions reductions or total costs of policies undertaken, are unlikely to be reported reliably. The metric of marginal abatement costs—at least among market-based policies—has the advantage of indicating the cost-effectiveness of the international distribution of effort. It is also an important indicator of the competitiveness impacts of climate policies vis-à-vis trading partners. We recommend greater focus on this measure, but caution the difficulty in attributing the marginal costs of non-market-based policies, especially inefficient measures. The key question is what carbon price would achieve the same reductions as the suite of policies selected, either by sector or for the whole economy. This would be analogous to the calculation of the level of effective protection applied in analyses of trade disputes.
Regarding the integrity of the commitments, we see related but distinct issues associated with the ex-post verification of performance—essentially compliance—and the ex-ante challenges faced by international negotiators in comparing often quite dissimilar policies and measures. For ex-post verification, the simplicity of an aggregate, economy-wide emissions target, or even one expressed as emissions intensity, is quite appealing. Existing data and reporting systems are certainly compatible with such approaches. When subnational or specific regulatory or voluntary policies are used, the commitment should be expressed as a transparent, verifiable goal, such as a fuel efficiency standard or level of technology deployment. However, while such goals may be clear, their effects on emissions are less transparent. Therefore, descriptive, institutionally-oriented information must be supplemented with micro data on the actual implementation and performance of these measures. Focusing on specific emission goals as opposed to regulatory standards can help avoid excessive reliance on model-based counterfactuals. R&D programs are by their long-term nature difficult to compare to near-term emissions targets, but these activities should at least be made more comparable across countries. We see no alternative to relying on actual expenditure and deployment data, although care should be taken to link such data to specific program activities, and to include transparent baseline information.
Assessment of the integrity of ex-ante commitments is, perhaps, the most important but also the most problematic area. The greatest challenges are associated with the unavoidable need to model counterfactuals, with all the attendant complexities. New guidelines should focus on the need for greater transparency in models and data, and greater standardization in methodologies in order to improve the consistency of analysis across sectors, policies and countries. Another priority is the strengthening of the mandate of the international group of experts that evaluates the submissions. The current practice of UNFCCC peer review is far too loose an arrangement for the reports to be credible inputs to climate negotiations. Other international processes may provide lessons for evaluating the quality, consistency and value of the estimates of ex-ante commitments. For example, the World Trade Organization has a Trade Policy Review Mechanism, by agreement in the Uruguay Round, which offers regular, comprehensive reviews of individual Members’ trade policies and practices and their impacts on the functioning of the multilateral trading system.
While the multilateral trading system offers some lessons in negotiating and supporting international agreements, the circumstances and party incentives are quite different for a climate framework. A greater role may need to be played by independent institutions, international organizations, academic researchers, and other third-party groups in supporting the evaluation efforts that, in turn, support the negotiations.
SUMMARY FOR POLICYMAKERS
Development of effective strategies to address climate change will require collective effort on the part of many countries over an extended period and across a range of activities. The challenge for the international community will be to judge the equity and integrity of the various national commitments.
Since diverse actions by different nations seem an unavoidable part of future climate policy, a useful input to international negotiations would be some means of talking in a coherent and widely accepted fashion about what individual nations are doing to help reduce climate risk. This paper explores various metrics for evaluating the equity and integrity of individual nations' policy commitments.
Equity is a major concern because international climate negotiations are fundamentally about sharing a burden. There is a strong desire to compare efforts and assess whether countries are contributing their fair shares. Yet, comparing efforts involves two kinds of exercises, neither of which lends itself to clear and fair metrics. The first exercise is to take a portfolio of disparate national policies and compare them according to some consistent measure that reflects effort, cost burden or emissions reductions. The second exercise is to place that measure of effort for each country into an appropriate context reflecting its socio-economic and other circumstances, in order to assess fairness. The basic problem is that clear metrics are not always fair, and fair metrics are not always clear. This paper considers a number of alternative approaches to measuring climate policy contributions, including measures of emissions performance, reductions and costs. Each can provide some
valuable information, but none is terribly satisfying as a reliable measure of effort or equity.
Evaluating the integrity of a collection of country commitments also requires two levels of analysis. First, the credibility of the commitments must be assessed; that is, will the countries actually undertake the measures, and can they be monitored and verified? This question requires the ability to conduct ex post analysis to support enforcing the agreement and engaging in commitments. The second level of analysis related to integrity involves assessing whether the proposed effects of the commitments are themselves credible; that is, do we reasonably expect the set of policies being undertaken to lead to the stated
KEY FINDINGS & RECOMMENDATIONS
- There is a clear need to improve the current reporting system in order to provide greater confidence to negotiators about thecredibility of countries' activities. Most importantly, reported activities need to be presented in a more uniform, consistent
fashion. The breadth of the different reporting practices currently used can mask genuine differences among countries. A first order of business should be the development of a much tighter, narrowly-defined set of reporting guidelines designed to
reflect genuine differences in activities among nations.
- Consensus on a particular metric for indicating equitable burden sharing is likely to be elusive. Each country has its ownincentive to choose measures of effort by which it is likely to perform relatively well. Some metrics are straightforward tocalculate, and they are somewhat informative, although imperfect indicators of burden. Other metrics are unlikely to be
reported reliably. One metric has the advantage of indicating the cost-effectiveness of the international distribution of effort,
and that is marginal abatement costs. It is also an important indicator of the controversial competitiveness impacts of climate
policies vis-à-vis trading partners.
- For ex post verification, the simplicity of an aggregate, economy-wide emissions target, or even one expressed as emissions intensity, is appealing. Existing data and reporting systems are certainly compatible with an aggregate approach. When subnational or specific regulatory or voluntary programs are used, descriptive, institutionally-oriented information must be
supplemented with detailed data on the actual implementation and performance of these measures. Expressing the commitment goal in a way that focuses on subsector aggregates (such as total auto sector emissions) rather than reductions attributed to policies will allow for more direct comparison and verification of effectiveness.
- Assessment of the integrity of ex ante commitments is the most important but also the most challenging area, because it requires modeling of counterfactuals. The main focus should be on greater transparency in models and data, and greater standardization in methodologies to improve the consistency of analysis across sectors, policies, and countries. Another priority is the strengthening of UNFCCC peer reviews, which currently are not sufficiently rigorous to provide credibility to the negotiations.
- While the multilateral trading system offers some lessons in negotiating and supporting international agreements, the circumstances are quite different for a climate framework. In trade, countries negotiate the removal of barriers to foreign goods in exchange for the benefits of greater access to foreign markets. In climate change there is no such exchange; the negotiations are to share a global burden, from which the benefits are far removed in time and not excluded from non-members. Perceptions of fairness and effort thus play a greater role. National governments may not provide the objective evaluation that is essential to the serious comparison of national mitigation proposals. A greater role may need to be played by independent institutions, international organizations, academic researchers, and other third-party groups in strengthening the evaluation efforts that support the negotiations, and in integrating the evaluations into the full negotiation process.
No single metric can adequately address the complex issues of equity and integrity central to international agreement on climate change mitigation. Development of a common, consistent, and credible set of indicators should be prioritized to build the foundation of trust and transparency needed to underpin multifaceted commitments.
For more information about this publication please contact the Harvard Project on Climate Agreements Coordinator at 617-496-8054.
For Academic Citation: