Understanding Well-being Data

chapter 3 Looking at Well-being Data in Context

Well-being measurement (other data are available)

It measures neither our wit nor our courage, neither our wisdom nor our learning, neither our compassion nor our devotion to our country, it measures everything in short, except that which makes life worthwhile. And it can tell us everything about America except why we are proud that we are Americans.

(Robert F. Kennedy 1968)

These remarks from Robert F. Kennedy are often found in arguments for measuring well-being,((One example of this is that the UK’s national newspaper, The Guardian, offered him his own blogpost to put the UK’s Measuring National Wellbeing measures into context. See Rogers 2012)) as an alternative to gross national product (GNP, and what Kennedy calls ‘it’).((Gross domestic product (GDP) and gross national product (GNP) are measures of a country’s aggregate economic output. They are both widely used, differing in what exactly they measure: GDP is a measure of (national income = national output = national expenditure) produced in a particular country. GNP = GDP + net property income from abroad.)) As touched on in the previous chapter, GNP (and GDP) are ‘national accounts’ and are administrative data that capture the economic activity of a country. Data on economic activity are used to measure financial success, compare countries against each other, and track progress over time.

Robert F. Kennedy’s comments are from a speech at the University of Kansas on 18 March 1968, forming part of his campaign for nomination for the US presidency.((This speech was a few months before he was sadly assassinated)) Fondly called ‘Bobby’, he is remembered for his advocacy for the civil rights movement. In this speech, he also declares support for student protests as good for society, and against the Vietnam War happening at the time1. Interestingly, his questioning of the value of GDP to measure human flourishing did not make much of an impact at the time. It is only retrospectively, and with hindsight, that this quote has gained notoriety, thus implying that it resonates more now than it perhaps did to American citizens in 1968.

Why is this speech important? Kennedy advocates changing priorities of public policy-making in line with altering values (both how we value and what we value). It indicates that it was politically prudent for a politician like Robert F. Kennedy to argue for replacing GDP as the main indicator of human progress at that time; it also suggests that believing in measuring well-being, rather than GDP, was ideologically aligned with supporting student protests and problematising the Vietnam War. Likewise, it tells us that there is an alternative to GDP or GNP to measure at that time. With the previous chapter, we can historicise this speech as coinciding with the social indicators movement that characterised what Bache and Reardon2 called the ‘first wave of well-being’. This means we can contextualise this political speech as from a time when different measures were called for—by people with particular values—to understand human flourishing, or how a nation was progressing. We are acknowledging that these comments were little repeated at that moment in time, but were later revisited to justify another ‘second wave of well-being’3.

So why are these historical and political settings for measuring well-being valuable for this chapter? Because they help contextualise well-being data. Context is key to recognising the role of methods in generating well-being data, as this chapter will show. Exploring the stories that lie behind data, and looking under the bonnet of how they are generated, is important to understanding: what they measure; whether they measure what they say; and the reasons why they have been collected and analysed in particular ways.

This is all part of what I call ‘data contexts’, arguing it is important to know how data work in what contexts4. What do I mean by this? Well, understanding where data come from, and why they were generated, is important. Were they generated in a lab or in a real-world setting? Why do they exist? Were they collected for one purpose and are being used in another? Who has analysed them and how may that affect how we view the data? We also need to think about how different techniques of analysis are applied and how they are operationalised in different contexts. What do they achieve? Do they monitor people’s toilet breaks in a call centre or how many steps a day we take while working from home? Do people know these data are being collected and why? Do the data help to hold governments accountable for national poverty or are they used to decide welfare distribution?

Measuring well-being as a political and scientific project does not have a consistent historic arc. There are moments where various technical and intellectual disciplines, and people with differing political interests, gather around ‘the well-being agenda’ as a project. This results in different types of well-being data being foregrounded, even acting as the catalyst for political change, at different times. The UK’s national well-being measures are often called ‘Cameron’s happiness index’5 after the UK’s Prime Minister contributed to the launch of the Measuring National Well-being (MNW) project6. As we shall see, the next section of this chapter opens with evidence that the idea of well-being measures for the UK (to become the MNW project) developed under the previous New Labour administration. The history of these measures is, therefore, not always obvious.

Similarly, it is not always clear what might be well-being data, and what are not. Data about well-being have long been valuable because they could help to understand how well a population was doing. Sometimes the data collected were believed to capture a specific aspect of happiness; other times to understand a particular part of a population, or indeed, one person’s quality of life. Therefore, data about well-being do not all look the same, do not have the same unit of analysis (individual people, nations or communities), are not used the same way and do not all exist for the same reason. Again, this is why context is important.

This chapter considers how well-being data is collected: the diversity of methods and the range of data that can be called well-being data. This includes background and context to the well-being statistics you might read in newspapers, online, or have seen in COVID-19 briefings and press conferences. It also begins to look at claims about what can possibly be concluded from different kinds of well-being research. We will continue to break down technical terms to show well-being data and measurement are complex, and their uses in policy are not universal. It aims to show that this language and these ideas can be more accessible when you know where they come from.((These contexts of data can be notoriously difficult to find out about! It can be difficult to know where to begin looking. Even all the fact-checking, and then re-checking, to finalise this book (and I have been doing this for years, now) required hours wrestling with broken links and inconclusive information on websites and in reports. I even emailed international statistics bodies for clarification. Most people probably don’t even know that this is a thing you can do if you have questions. The ONS and the OECD have both replied extremely quickly to my general queries this last year, and they are mandated to answer queries. Hopefully this book offers a starting point to help answer some of your queries.))

Well-being data as a term most often describes well-being metrics or indicators. This chapter offers some examples of how many decisions are made when choosing an objective indicator of well-being. Despite the name ‘objective’, which implies they are not affected by feelings or opinions, they do not fall from the sky as facts. If truth be told, they are the product of a specific methodology, which means they must fulfil certain practical and theoretical criteria that satisfy often long-established opinions of what are the best methods to capture the most objective data, and then how to go about analysing them.

Objective well-being indicators predominantly originate from survey data (like the census) or administrative data (such as mortality rates). They also include some subjective data where people are asked about aspects of their lives, such as how satisfied they are with their health. We come to this later. These datasets will include enough of the population that it is sensible to analyse them numerically—as quantitative data. These quantitative analyses are not always conducted by the person or the organisation who collects these data. Similarly, secondary uses of data can make the data useful as well-being data, when it may not have been collected for such a purpose.

Whether objective or subjective, it is mostly agreed that:

  1. all well-being measures must be theoretically grounded7, meaning that there is a clear, agreed rationale as to what exactly is being measured, what for and how the data are collected and handled
  2. the limited impact of previous attempts to measure well-being lies in deficient theoretical grounding, and therefore failed understanding of what the measures are for and who they benefit8
  3. assessing one’s own well-being is a subjective and aesthetic((We tend to think of aesthetics as a sense of beauty, but more generally it means being actively engaged and conscious of the world’s effect on us, whilst at the same time appreciative how we might affect the world. According to philosopher John Dewey9, this enables us to appreciate how our experience is organised, making it coherent, and allowing us to appreciate the past, present and future—whether we are satisfied, or dissatisfied.)),((According to Rapley, ‘asking about the quality of life amounts to a request for an aesthetic judgement’, rather than a scientific one, from the person asked. You cannot take for granted that people have the same notion of quality of life, and therefore its assessment is a qualitative appraisal of how things stand. ‘Aesthetic judgement’, according to Kant10, is dependent on discriminatory abilities at a sensory, emotional and intellectual level all at once.)) experience11
  4. well-being survey questions should involve concepts which are readily understandable and easy to relate to, such as ‘satisfaction’ and ‘happiness’12
  5. well-being measures need to be subject to harmonisation (GSS), meaning that they should be able to work with other wellbeing measures

Not all well-being data are numbers, or the result of large-scale data collection, however. It can be easier than you may imagine to produce and use well-being data. To discover how accessible other methods are, we will explore other ways of collecting data, such as interviews and focus groups. We will also look at policy documents as data, like the speech above, finding that ideas of measurement and well-being are used together, and how that can reveal the all-important context to why data are used to make certain arguments. As with the quantitative data found in well-being indicators, it is also important to understand the limitations to what we can claim to know as a result of analysing qualitative data. Whether from a few policy documents or interviews with a community in a particular place (rather than a whole population), these data tell us a lot about a small number of people and may not describe how things work on a larger scale. These are matters of methodology.

Box 3.1 Methodology

Methodology is more than the methods used to collect data (e.g. a questionnaire or interview) or analyse data (i.e. statistical techniques or thematic analysis((Thematic analysis groups people’s responses into themes to help a researcher understand commonalities and differences across their sample.))). It is more than who is using methods, whether in academic research, in national-level surveys, or in evaluations of how much a policy decision or an individual project has impacted on well-being. It is the system behind methods: why people have decided to do these things in these ways. This is what makes data ‘theoretically grounded’ (see above).

As we go about our day-to-day activities, we don’t tend to consider the theory of what we are doing and why, but odd moments might make us stop and think about why we have done something in a certain way and whether that is the best possible, or the one most suited to our situation (how much time we have and where we are, for instance). Think about when we hear how other people do something, their tips or techniques might be different from ours and can be about something quite mundane.

Think about a cup of tea (English tea to non-native Brits, or depending on dialect: ‘a cuppa’ or ‘a brew’). It has different names, depending on where you come from, and there are often discussions about how to make tea the right way: milk first or second; let the bag stew or not; in a teapot, cup or mug, and for how long. There are also TikTok videos and Facebook posts on the issue, Reddit feeds exclaiming the crimes of others’ tea-making methods, and reports in the national press, saying certain methods ‘spark outrage’ (Morris 2020). What works best, and in which order, is therefore not a universal truth and there are opinions on how these all work together.

Methodology, similarly, involves the theory behind how stages of working with data work together. Working with theory doesn’t only mean reading philosophers, but more practically involves careful consideration of each process.

Some useful questions to ask about these stages include:

Was it appropriate to apply this particular approach to collecting and analysing data to the particular issue the researchers want or need to know more about?

Or would it have been more appropriate to analyse data already available or accessible in a different, perhaps easier, and less intrusive way?

Would people have been easily able to answer the questions?—

we’ve all answered plenty of surveys where we cannot answer the questions truthfully, because the questions are badly designed. Or, indeed, because we do not want to tell the truth, exactly.

Is it fair to ask people to answer this question about themselves in this context (on the street, in a room full of others, at work where their screens might be viewed by colleagues, etc.)?

Is this ethical?

Methodology is often described as bringing theory to method. It is not so different from debating how tea is made, and how that affects the result. Methodology discussions are also often tribal, with in-fighting and disciplinary arguments—even disagreements over namings and meanings. In the case of data, this more simply involves thinking through what we do with data and how we have thought about collecting them. What order certain processes go in and what are our approaches to each process, and why that is best suited to the situation at hand. It is the foundations of why research has been done in a particular way.

There is often a tendency in the social sciences to feel the need for academics to take a position on the value of quantitative data over qualitative data or vice versa. This is colloquially called the ‘Quants-Quals debate’, which I had never heard of until I became an academic, but it is rife.((There is much written on this so-called debate, but Gary Goertz and James Mahoney are interesting on how it is A Tale of Two Cultures (2012).)) Other academics have requested I make it clear where I stand in the past. So, I want to make it clear that in this chapter—and the whole book, in fact—I resist this assumption that any data is better than another because we read them as text or count them as numbers, or collect them differently. All well-being data might be valuable to understanding well-being. Whether they are qualitative or quantitative is not the issue at hand. Instead, context is the issue: where the data came from, are they used appropriately and how are they applied? Are their uses ethical and fair? What are the limitations to the data we have? What can we know as a result of the data? What happens next?

The chapter describes different sorts of data: a moment from my research, hypothetical examples, as well as case studies from international statistics agencies to reveal some of the contexts of data collection, interpretation and uses of well-being data. It does this to show that all data have origins of thought, process and practice and are therefore rarely completely neutral or objective. All methodologies have their limitations, which thereby limits the claims that can be made. These are not always fully recognised.

If limitations are acknowledged in one place, that place is often far removed from the headline findings((Headline findings are provided in separate documents and executive summaries and are written to underpin messages that are the intended ‘takeaway’ findings from research. They are presented accessibly for the interested public, policy-makers and media with the intention that people will know what they need to know from reading a few bullet points, rather than looking at detailed results.)) to make caveats clear when interpreting results. The de-contextualising of data removes how we understand their limits and appropriateness. It must, therefore, impact on how ‘good’ the data can be in understanding society and well-being. It also affects the capacity for data to do good and inform societal change in such a way as to improve social, personal or national well-being. We need to account for the data used and we need to heed different accounts of what well-being means, as well as how we might understand it better.

  1. Kennedy 1968 [↩]
  2. 2013 [↩]
  3. Bache and Reardon 2013 [↩]
  4. Oman n.d. [↩]
  5. Clinton 2011; Mirror 2011 [↩]
  6. Cameron 2010 [↩]
  7. Haybron 2008 [↩]
  8. Scott 2012 [↩]
  9. [1934] 1958 [↩]
  10. [1790] 1951 [↩]
  11. Rapley 2003 [↩]
  12. Fleche et al. 2012, 9 [↩]