chapter 3 Looking at Well-being Data in Context
Objective well-being data and measures
In terms of quantitative data, you might imagine that the key question is how should well-being be measured? Really, this is a much bigger question, or series of inter-related questions, which are how should well-being be conceptualised, operationalised and measured? Or before well-being can be measured, we need to decide what we mean by well-being (conceptualise) and find measurable dimensions of our concept (operationalise),((Box 7.1 explains operationalisation in research in greater detail. Notably, Chap. 6 talks about operationalising an idea in policy, which is different from operationalising a concept for measurement in quantitative research. These are different applications of the same word, which can be confusing.)) and then we can decide on a way of measuring it.
We have discussed some of the methods of collecting well-being data. Many decisions are involved that are not always made obvious, but are all important. The point here is that the conceptualisation of ‘what is it we’re actually trying to get at when we want to understand well-being’ is distinct from its operationalisation. It is also worth noting that to operationalise a concept in research has a slightly different meaning than it does in everyday life. We come back to this in Chaps. 6, 7 and 8. If someone operationalises something, it generally means they put it to use, or bring it into use. In research, it is more a process of establishing how we can measure. So, conceptualisation is different from operationalisation, but connected. The operationalisation of ‘here is the form of words we’re using to ask the question’ is different again from ‘here are the options for the answers people can be provide (and if applicable, how we’ll combine these answers from different questions to give people an overall well-being score)’.
As we have hopefully established in the introduction to this chapter, money is important in most contexts, but is far from everything. There are many more features that shape people’s lives and that need to be understood if we aim to understand well-being as quality of life1. You could ask a population any number of questions to understand aspects of their quality of life. For example, is your housing adequate? How sanitary is your local environment?((For additional detail, you may notice the first two questions will collect different kinds of data. Is your housing adequate? invites a yes/no answer (probably with a don’t know option for best practice). How sanitary is your local environment? invites a scale, so you will probably offer someone a scale to mark. Perhaps a Likert scale, as described in note 18.)) Do you have public institutions that respond to your needs? Would you say have an active social life? Are quality healthcare and education services easily available to you? You may note that all of these questions are phrased in such a way that they ask for people’s opinion on aspects that are thought to affect our quality of life. They are therefore going to produce data that are subjective.
All of these issues can also be measured using data that are objective indicators. For example, administrative data such as GP visits and hospital wait times could be used to generate a benchmark for ‘fair access to healthcare’, and then community-level data could be measured against this benchmark. These are proxy indicators because they do not directly answer the question ‘does this person have fair access to healthcare’, but are used to stand in for data that could.
Proxy indicators have a number of pros. They are not biased by people’s inaccurate memories of how long they waited in hospital, which, for obvious reasons, may be clouded with frustration. You do not have to worry about issues of sampling bias (see Box 3.4). Also, proxy data have often already been collected and cleaned by someone else, or a statistical organisation. So, while they can only partially answer the question of how many people have fair access to healthcare, the pros will have been thought to outweigh the cons. Similarly, being able to answer a research question on fair access to healthcare doesn’t tell us everything we need to know about well-being: it is one aspect of well-being. It only partially indicates someone’s quality of life, and so to understand quality of life more completely at population level, we need more indicators.
Objective measures of well-being are based on assumptions regarding human needs and rights, believed to impact on quality of life. Herein is the difference between quality of life and well-being. The academic literature tends to assume that quality of life involves material conditions, whereas well-being also involves life satisfaction, mood and meaning (although as we know from the previous chapter, this is not always clear-cut). It is the quality of life aspects of well-being that are measured with objective indicators using the objective list theory that most indexes are based on.
The existence of the list, of course, suggests that a person or people with expertise have decided what should go on the list: what is important and what standard measures should be used, or indeed to whose standard? There is even an ‘objective list theory of well-being’2 that is pluralistic. This means that instead of identifying a single feature common to all states of well-being (think of an overarching argument for ‘what is the meaning of the good life?’), it identifies a number of characteristics of what makes for a good life. This philosophical theory is applied to lists of objective indicators, of what would be all the qualities needed for a good life. The key is that the aim is to cover all the important domains in life, so unlike a simple index, like the HDI, these tend to have lots of indicators. In other words, the well-being data are about lots of aspects of life.
The previous chapter explained a brief history of the move away from a single measure of progress (GDP) towards multiple measures of well-being in the twentieth century. These tended to be an index of multiple objective indicators of quality of life, associated to different ‘domains’ of life. Some organisations and nations recognise the same six major objective and observable dimensions for the measurement of objective well-being. These include international organisations, such as the Organisation for Economic Co-operation and Development3 and the United Nations Development Programme4, as well as national statistics offices, such as the Italian Statistics Bureau5. Notably, within each dimension are multiple indicators (ordinarily two or three). Figure 3.1 shows just how many indicators there are within domains in the OECD’s index and per member country. As we shall discover in the following chapter, these bodies all heavily influence each other by way of advisory groups and drawing on perceived best practice.
Given that the theory behind the objective list approach means you need to analyse data from across all these dimensions, this can make it difficult to interpret these data, even at headline level (see Fig. 3.1), but also to compare them. Changing the unit of analysis from each indicator, to per country, or domain, makes them more readable. This is the same as with the Dow Jones, where the index is designed to have a single measure for readability. With the HDI,((It is important to note that something being easier or more readily available for measurement does not necessarily mean it is accurate. Remember that the advice from the important, game-changing Sarkozy commission (see Chap. 2) was that each nation should devise its own measures. This is because each country has its own culture and priorities that may not be reflected in existing large-scale indices.)) the three dimensions are combined into a single measure for easy comparison.
With more complex indexes than the HDI, such as the OECD’s (Fig. 3.1), decisions need to be made on balancing the importance of the different domains. As we know, each of the three domains contributes equally to a country’s overall HDI score6, this is not the case with all indexes. If domains are not equally weighted, then decisions have to be made about the relative importance of each to overall well-being decisions. As Table 3.2 demonstrates, establishing the importance of different domains of well-being is not a neutral process.
To this end, these weights involve subjective decisions by experts on what is more important about the objective indicators. That is not to say it is not a rigorous process, that it is not based on much evidence, and that experts do not debate and review these processes to ensure robustness. Yet, the term objective can obscure what is going on behind the scenes, or underneath the hood, if you like, of what are called ‘objective indicators’ of well-being, or imply that they arrive at some sort of universal truth about well-being. As criticisms over the HDI surface, people do not value these aspects of life equally, or, indeed, the same as each other. Remember that there is a difference between measuring what is valuable and what is valuable to measure—to whom and why.
An attempt to counter criticisms of weights applied by experts, The OECD states that its ‘Better Life Index is an interactive composite index that aggregates average measures of country’s well-being outcomes through weights defined by users’7. What does this mean, and why have the OECD attempted to do this? Let’s break this down.
The OECD’s Better Life Index website has an interactive dashboard, enabling people to use sliders to order and balance the importance of different aspects of well-being. When people use the sliders, they are effectively applying weights to the different aspects of well-being to construct an overall index that is personal to them.((See the OECD Better Life Index website (OECD n.d.).)) In this instance, the index aims to avoid representing the experts’ view of what is valuable, presenting those of the person interacting with the dashboard back at them.
This is all well and good, but how does this impact on change for social good? Are the OECD listening/watching/recording these interactions, and how might it change the way they value what is important? While some analysis has been done on people’s interactions and values8, and this dashboard implies democratic engagement or participatory decision-making to a degree, there is no commitment to this. People are also only able to interact with the pre-defined categories: were something of importance to you not there, there is no way to include this in the dashboard or tell anyone it should be included.
Box 3.4 Weights and Sampling Bias Weights
The term ‘weighting’ is used in several different ways in the analysis of quantitative data, and it’s important to be clear about which way we’re talking about.
In this section, we are concerned with how different bits of information about countries are combined to give an overall score for those countries. Or, how important money is, as opposed to education or health. The HDI applies an equal weight to these categories.
Weighting is also used to describe a technique when working with survey data to correct for sampling bias. As we have discussed, it is rare to achieve a whole population, and so most survey data are a sample. No matter how large that sample is, your sample is unlikely to look the same as the whole population, so you need to adjust for different proportions who answered the survey. For example, younger people are often less likely to respond to surveys, so estimates based on surveys often weight young people’s responses more heavily to adjust for this difference.
These two different meanings of the term ‘weighting’ are applied in very different ways—in one case, to the questions that are being asked, and in another, to the people who are being asked the questions—and shouldn’t be confused.
The terminology, processes and decisions behind what are used for objective well-being data, and how they are used together—as an objective list of indicators—are complex. I have tried to cover specific examples and drill down into the processes of why things happen in certain ways and to explain some of the terminology. We are going to look at one index in greater detail in the next section. This is to help those who wish to understand what goes on underneath the hood of a well-being index and to have a better understanding of what decisions are made about what good data practices might be for well-being data.
