chapter 3 Looking at Well-being Data in Context

The OECD as a case study of what lies behind objective well-being data

Measuring well-being and progress has been and will continue to be a key priority for the OECD, in line with its founding tradition to promote policies designed to achieve the highest living standards for all.

(OECD 2011a, 4)

The OECD have been key to the ‘second wave’ of framing well-being as important to measure¹. National well-being initiatives have tended to be in OECD or EU countries, and it is thought that the OECD had a hand in the process of the influential Sarkozy commission². The OECD Framework for Measuring Well-Being and Progress is said to be based on the recommendations from the commission³. We are going to peer under the bonnet of how the OECD devised its well-being indicators to reveal the decisions and care that go into such a programme.

The OECD claim that:

the ultimate objective of this work is not just measurement per se, but to strengthen the evidence-base for policy making. Better measures of well-being can improve our understanding of the factors driving societal progress. Better assessments of countries’ comparative performance in various fields can lead to better strategies to tackle deficiencies.

(OECD 2011a, 4)

The OECD wanted to understand well-being in a way that can both offer comparisons across nations and potentially inform policy evaluations. They decided the qualities that best represented well-being; made objective lists; researched appropriate proxy indicators using existing data that can answer the dimensions of well-being. They tested the indicators that they have used to meet the demands of their well-being framework and ensured that they meet additional quality criteria; they sought expert advice on these moving parts and then offered a caveat on the experimental and evolutionary nature of these metrics: they will change and they are not perfect. This level of detail is not always readily available when research is published. So, we are going to look at the decisions made in the devising of the index in order to understand what lies behind these well-being data.

The OECD devised a list of criteria of what would be good to measure. Crucially, they also undertook a review of the data available from member countries (who, of course, may not be measuring the same thing). Prior to finalising the index, a compendium was released, which contained the framework on which decisions were made regarding which well-being indicators they might use⁴. This was the criteria they published in the compendium:

• the well-being of people in each country, rather than on the macro-economic conditions of economies; hence, many standard indicators of macro-economic performance (e.g. GDP, productivity, innovation) are not included in this Compendium.
• the well-being of different groups of the population, in addition to average conditions. Measures of inequalities in people’s conditions will figure prominently in the “How’s Life?” report but are only discussed briefly in this Compendium.
• well-being achievements, measured by outcome indicators, as opposed to well-being drivers measured by input or output indicators.
• objective and subjective aspects of people’s well-being as both living conditions and their appreciation by individuals are important to understand people’s well-being.⁵

The OECD were also keen that their framework distinguished between current material living conditions and quality of life, on the one hand, and the conditions required to ensure their sustainability over time, on the other. Notably ‘material living conditions’ do not always mean economic, and often the term elsewhere incorporates quality of life dimensions, as discussed above.

• Material living conditions (or ‘economic well-being’) determine people’s consumption possibilities and their command over resources. While this is shaped by GDP, the latter also includes activities that do not contribute to people’s well-being (e.g. activities aimed at offsetting some of the regrettable consequences of economic development) while it excludes non-market activities that expand people’s consumption possibilities.
• Quality of life, defined as the set of non-monetary attributes of individuals, shapes their opportunities and life chances, and has intrinsic value under different cultures and contexts.
• The sustainability of the socio-economic and natural systems where people live and work is critical for well-being to last over time. Sustainability depends on how current human activities impact on the stocks of different types of capital (natural, economic, human and social). However, suitable indicators for describing the evolution of these stocks are still lacking in many fields. For this reason, indicators of sustainability are not included in this Compendium, although some of them will feature in ‘How’s Life?’⁵.

It is within the tension between conceptual soundness and the quality of data that the sustainability indicators sit: they would be what we would ideally be measuring if we want to capture well-being; remembering that the principle of an objective list is that the indicators included⁶ are all vital to well-being. It is interesting that the OECD consulted with individual statistics offices on which indicators to select. The UK’s ONS also state they consulted the OECD to decide their well-being metrics⁷.((The politics of who were involved in well-being measurement are discussed by Bache (2012) in greater detail.)) Therefore, despite apparently separate investigations, the same experts were informing different indices. Sharing expertise is undoubtedly a good thing, especially when it comes to methodological rigour, but it might arguably limit the possibility for independence or innovation in how countries measure the well-being of their citizens. Notably, despite the fact that Bhutan’s measures of Gross National Happiness are often cited as inspiring the OECD, Sarkozy commission, and so on, expertise from Bhutan is not very evident on these advisory groups. We return to this in Chap. 6, but who the experts are, are always important questions to ask.

Another important thing to note about the OECD’s contribution to well-being data are the caveats that were presented alongside these domains, namely that the indicators are:

• experimental, in that the proposed selection of indicators has not yet reached the stage of meeting all agreed standards;
• evolutionary, as the indicators proposed in this Compendium are, in many cases, only proxies of a broader underlying outcomes, for which ideal measures are currently lacking.⁸

The report also notes that the selection of indicators will change in the future as better measures are developed, and as member countries reach agreement on indicators that are more appropriate to summarising conditions in the various dimensions of people’s lives⁴. So, whilst these national indicators tend to be presented as absolute, or fixed, in some way, like other forms of science and social science, they are invented to respond to developments and improvements. This is rarely acknowledged when objective indicators are presented in official reports and briefings.

So, what might these indicators look like?

The description ‘bewildering array’⁹ may come to mind when looking at Fig. 3.1. As a result, Table 3.4 shows only the domains and indicators in 2010. There are 21 indicators across the 11 domains, with a row for each member country. This is how the indicators were presented in 2010. Some of these have now changed, perhaps imperceptibly to most. It can be difficult to establish exactly what is meant by or what has changed about an indicator, why, and when that change happened, because this information is not readily available.

To explain what I mean here, we are going to zone in on the ‘domain’ of ‘Personal Security’, in our case study. Personal Security has two indicators in our 2010 visualisation: intentional homicides and self-reported victimisation. So, one question might be, ‘why not just say crime, if you mean crime?’ If you look at all the domain names, they are all positive in their inflection: environmental quality might read as pollution, or litter, for example. What is also interesting about the idea of personal security is that it does not necessarily mean crime, really. It could possibly include financial security to most people: do you have a pension; do you own your own home, and so on?

Table 3.4 Summary of the OECD indicators in 2010

Source: Adapted from Compendium of OECD Well-Being Indicators 2011

Another question is why, then, has the domain changed in the current 2020 version of the index? The Personal Security domain name is now called ‘safety’. The OECD explain this domain as follows: ‘Personal security is a core element for the well-being of individuals, and includes the risks of people being physically assaulted or falling victim to other types of crime’ (OECD website/topics/safety). Therefore, the headline domain name has shifted from ‘personal security’ to ‘safety’, but has retained the credibility of the original measures.

Not only has the domain name changed. The indicators themselves have shifted: ‘homicide rate’ remains the same, but ‘self-reported victimisation’ has been replaced with ‘feeling safe walking home at night’. Thus, an objective indicator has been replaced with a subjective indicator, as the data were collected by surveying how someone feels, rather than the administrative data from reporting crimes.

There are methodological reasons why this is a sensible change. In some places people do not report crimes, as they happen, so as a chosen proxy measure of a domain well-being, this is not necessarily the best indicator of the relationship between crime and quality of life. Secondly, it could be argued that it is in the ‘feeling safe’, rather than the reporting of crime that we experience well-being. This is why more subjective measures—even on an objective list—can be a better way of capturing what it is about well-being that we need to know.

In the previous section we encountered what objective indicators are, and this section has presented a lot of detail on one well-being index, as it is not always clear where such official-looking data come from. We focussed on some of the decision-making aspects of devising an index. This also revealed their methodological complexity—even without the quantitative modelling involved in statistics. We have also questioned the nature of the data assembled in objective lists and what is implied by their naming as objective. We have learnt that they are, in fact, shifting rather than fixed sets of measures. They evolve and respond to reflections on their limitations and how they could be done better. As we continue to use these sorts of data as objective facts, we lose these qualities, which are not considered important. Yet, the contexts of these data practices are both valuable and credible. It is a disservice to statistics and people who wish to understand them, that they remain obscured.

1. Bache and Reardon 2013 [↩]
2. Bache 2012, 26, 30 [↩]
3. OECD n.d.b [↩]
4. OECD 2011a [↩] [↩]
5. OECD 2011a, 5 [↩] [↩]
6. BetterEvaluation 2012 [↩]
7. Oman 2017 [↩]
8. OECD 2011a, 7 [↩]
9. Scott 2012, 36 [↩]