SDG indicators: Counting the trees, hiding the forest

After a two days meeting in Bangkok, at the end of October, the statistical experts of the UN agencies have come with a controversial list of 159 “generally agreed” indicators to measure the Sustainable Development Goals approved last September by the UN.

Traditionally, development agencies have tried to summarize in a single indicator or index complex development goals. Thus, UNICEF emphasized infant mortality as a proxy indicator of its mandate to protect children, the World Bank has traditionally equated development with per capita income, and UNDP enriched that idea with its Human Development Index based on a larger but still reduced set of indicators. Those single figures were considered to correlate so closely with other relevant indicators (for example infant mortality correlates with child underweight) that they were considered as useful to represent broad trends, progress or regression, in realities that everybody understood as being complex.

Now, instead of attempting to find summary indicators for each of the 17 Sustainable Development Goals, the experts are attempting to be exhaustive and find at least one (frequently more) indicator for each of the 169 targets that the governments agreed should describe the goals or lead to their achievement. Of those targets, 121 have now at least one indicator marked as “green” in the list (which sum up to 159 indicators because many targets have more than one), leaving 48 targets with indicators still to be determined because more disucssion or methodological development is still needed.

This approach seems to imply that each of the 17 goals is equal to the sum of the targets agreed under it, that they are all equally relevant and exhaustive. But counting the trees can hide the forest instead of understanding it, as the popular saying wisely warns us. The analysis of the “green” goals shows this to be the case for many of the goals.

Thus Goal 10 proposes quite clearly and understandably in its title to “reduce inequality within and among countries”. This is a very straightforward proposition. The governments did not agree on a benchmark on where to stop. At what point would inequalities be reduced so much that they acceptable? But if there was a worldwide political agreement that current inequalities are to be reduced, because they have reached intolerable level, the indicators should at least help establish if reduction is indeed happening.

Yet, the list proposed does not include anywhere the Gini index of inequality, which is the well established scientific standard, nor the “Palma ratio” (between the top 10 per cent and the bottom 50 percent) that is easy to understand and popular in academia. Both this indicators already exist, are available at global and country level for most countries and backed by abundant literature.

Instead, the key indicators proposed as “green” for Goal 10 such as “people living below 50% of median income” or “ income per capita among the bottom 40” seem designed to carefully avoid looking at the high income end of the scale (the upper 10 or 1%) and completely ignore inequalities among countries.

Many of the targets are very precise and include the indicator in their formulation. This is the case of Target 3.1: “By 2030, reduce the global maternal mortality ratio to less than 70 per 100,000 live births”. Targets like those do not generate any controversy.

Targets that require specific policies can be assessed by a binary indicator (yes or no). Target 5.1, for example demands to “End all forms of discrimination against all women and girls everywhere”. And the proposed “green” indicator is a binary one: “Whether or not legal frameworks are in place to promote equality and non-discrimination on the basis of sex.” The implementation of those policies and their impact will still have to be assessed but their very existence is a starting point that needs to be monitored.

But other targets are programmatic statements, that imply a multiplicity of policies and potential results that cannot be captured within the self-imposed restriction of one or two indicators per target. See, for example, target 1.4: “By 2030, ensure that all men and women, in particular the poor and the vulnerable, have equal rights to economic resources, as well as access to basic services, ownership and control over land and other forms of property, inheritance, natural resources, appropriate new technology and financial services, including microfinance”.

This is one of “grey” cases. The experts asked UNEP (the UN Environment Program) “to review the ontology about basic services.” But even if this review comes out with an acceptable assessment tool, basic services is only one of the many relevant issues spelled out of this target and key issues like access to property, technology and finances will be left out.

Under Goal 12, which addresses “sustainable production and consumption patterns” the indicators agreed upon refer to the existence of policies in place but the key indicator of “footprint” which measures actual impact was left as “grey”.

Finally, the indicators for Goal 17, on Means of Implementation (MoI) and for the targets around MoI in the other 16 SDGs are those that proved most controversial. Not only do they have the greatest number of greys and yellows, but when indicators are listed as “green” they frequently miss the point or completely distort the intention of the target.

As if the name of “means of implementation” was not enough, the very first MoI target (1.a.) requests explicitly to “ensure significant mobilization of resources from a variety of sources,

including through enhanced development cooperation, in order to provide adequate and predictable means for developing countries, in particular least developed countries, to implement programmes and policies to end poverty in all its dimensions”. The proposed indicators, labeled as green, measure the “percentage of resources allocated by the government directly to poverty reduction programmes” and the “spending on essential services (education, health and social protection) as % of total government spending”.

Thus, developing country governments will be judged negatively if they are unable to spend more on the poor, but the promise of “enhanced development cooperation” will be left unmonitored.

Target 17.1 Strengthen domestic resource mobilization, including through international support to developing countries, to improve domestic capacity for tax and other revenue collection

But the proposed indicator is “total tax revenue/GDP”, which again measures the ultimate expected result, but not the promised support or necessary global agreements and measure s to curb illicit financial flows.

17.3 is even more problematic. The target is to “mobilize additional financial resources for developing countries from multiple sources but the indicators proposed are “Foreign Direct Investments as % of total FDI + ODA” and “additional Volume of remittances (USD)/GDP.” On the one hand it is not admissible to account remittances from migrants to their families as “additional resources for development”. On the other hand not every FDI contributes to development and if capital inflow weights positively in the balance of payments, it is the net balance what counts and thus outflows (profits, royalties, etcetera) should not be ignored.

Target 17.4 to Assist developing countries in attaining long-term debt sustainability” has as indicator “Debt service as a percentage of exports of goods and services”, ignoring again to look at the promised assistance.

The main point of Target 17.6 is to “Enhance (…) cooperation on and access to science, technology and innovation and enhance knowledge sharing (…) including through improved coordination among existing mechanisms (…) and through a global technology facilitation mechanism.”

Instead of a simple yes or no to the question of whether such a mechanism or an assessment of cooperation was enhanced, the proposed indicator is the “access to the WIPO Patent

Database and use of the international IP system”. This is adding insult to injury, because the current global intellectual property system is precisely the obstacle that this goal wants to overcome.

Target 17.8 that wants to Fully operationalize the technology bank” is measured by the proportion of individuals using the Internet.

Key demands like enhancing global macroeconomic stability (which is important everywhere and not just for developing countries), or “enhance the global partnership (among countries) for sustainable development” still lack agreed indicators, while the promotion of “effective public, public-private and civil society partnerships” will be measured by the money “committed on public-private partnerships”, not assessing whether those funds were actually disbursed or their real contribution and leaving civil society out.

The commitment to “respect each country’s policy space and leadership to establish and implement policies for poverty eradication and sustainable development ” is left without an agreed indicator, even when developing countries suggested to use a simple count of the “numbers of constraints” (conditionalities) that are embodied in ODA or loan agreements as well as investment and trade agreements.

To acknowledge the difficulties in monitoring the Agenda 2030 because of the complexities of the issues, the lack of statistical capacity in many countries or even the ambiguities in the wording that made the agreement possible is sensible. To propose indicators that substantially rewrite key aspects of the consensus is simply unacceptable.

By Roberto Bissio, Social Watch.