chapter 3 Looking at Well-being Data in Context
Secondary qualitative data
Qualitative data are increasingly collected with a view to the data being used again. This means those collecting data must be mindful of this when designing the questions asked and ensuring interviewees give permissions for storage, secondary access (used by someone else) and re-use (in publications or otherwise). Secondary data usage involves analysing data collected by someone else, as opposed to analysing primary data that you collect yourself and is more common with quantitative data.
It is sometimes possible to ask permission to access qualitative data that were not collected with the same questions in mind. This would mean that the same data, collected for a different purpose, could possibly be reanalysed to answer the specific question: ‘what were the impacts of X social policy on the well-being of a specific community between X and X date, for example?’ However, much qualitative data are too specific, in that they contain too much data and information about issues that are too personal to the people involved. For instance, given the sensitive nature of Bogue’s data on the bedroom tax, it would be unlikely that these data could be reanalysed to answer a broader question on well-being for ethical reasons, even if it were a practical possibility. It is unlikely that permissions for re-use would have been sought at the time of collection, and were people told the data might be placed in a repository, they may have not been as honest. These kinds of data are extremely difficult to anonymise in a way that completely protects participants and were one to try, perhaps there would be very little left to analyse. The benefits of qualitative data in capturing the specificities of people’s experiences, therefore, mean there can be barriers to secondary qualitative research.
Data collected by international bodies, such as the International Monetary Fund (IMF) or the UN, and national statistics agencies, such as the ONS in the UK, make their data publicly available for secondary analysis. These are primarily quantitative data and in addition to the findings that these bodies publish themselves (often presented as tertiary data). The ONS have pages and pages of findings and data on their website under well-being now, and there are a lot of data available from the UN’s HDI on its website.((See ONS n.d. and UN n.d. for more information.))
Sometimes large surveys managed by national and international agencies, and available for secondary analysis, contain free text data (as shown above). If qualitative data has been collected at a large enough scale, then there is sometimes value in coding these and then adding up (aggregating) the answers which are similar, and turning this qualitative data into quantitative data. In 2013, I requested permission from the ONS to access free text fields from the Measuring National Well-being debate. I had developed a hypothesis from reading a report which contained quotes from the debate that I wanted to investigate.
My research question for these data, related to the issue we found outside the imaginary concert (described earlier in this chapter). If the evidence we have about the well-being impact of particular leisure and cultural activities can be argued as circumstantial, and from leading questions, the credibility of data is called into question—most specifically, its collection1. This is an issue that plagues arguments over the quality of the evidence on the relationship between aspects of culture and well-being that we return to in the second half of this book. If the data can be dismissed as resulting from leading questions and years of research projects that are therefore not able to offer generalisable results, then how might this issue be addressed?
I proposed we turn this question on its head. How does that help us overcome some of these issues with the context in which these data are collected? What if the question was more like: ‘When people describe well-being, how often do they talk about participating in different kinds of activities—and what might that tell us about aspects of social and cultural policy?’ I coded 6787 free text fields on well-being that were collected by the ONS and collated them into themes of all the things they talked about. I then quantified the themes2 and then ordered them in terms of prevalence of response.
Table 3.2 shows the difference in order according to what the ONS said it found in the overall debate (34,000 responses) and what I had found in the free text fields. Again, people did not refer directly to concerts (using the word) in the national debate, and only once in a subsequent consultation3, but people did refer to the importance of broader concepts of social and cultural participation4.
Table 3.2 ‘A re-ordering’ of priorities in the Measuring National Wellbeing Debate Questionnaires
| ONS’ ordering of tick box responses (most prevalent at the top) | A re-ordering of free text field responses (most prevalent at the top) | ||
|---|---|---|---|
| 1st | Health | Leisure and spare time | 1st |
| 2nd | Having good connections with friends and relatives | Quality of natural environment | 2nd |
| 3rd | Job satisfaction and economic security | Family | 3rd |
| 4th | Present and future conditions of the environment | Security | 4th |
| 5th | Education and training | Protect planet/nature | 5th |
| 6th | Personal and cultural activities, including caring and volunteering | Freedom/power | 6th |
| 7th | Income and wealth | Access to leisure possibilities | 7th |
| 8th | Availability to have a say on local and national issues | Healthcare | 8th |
| 9th | Crime | Equality | 9th |
| 10th | Other | Happiness/well-being of others | 10th |
| Government | 11th | ||
| Fairness/social justice | 12th | ||
| Access to services | 13th | ||
| Politics | 14th |
Adapted from (Oman 2019, 2020)
Well-being data include many sorts of data beyond those used in national indicators or the statistics we read in the media. They can all be extremely useful to inform work of many kinds from social work and policy, to arts administration, to the management of a particular company or understanding how to better care for students away from home at university. The data required, and how they are analysed, involve a balance of what needs or wants to be known (see Table 3.1). It is also a practical matter of preference of approach, skill and resource; all need to be balanced and there are various limits on different kinds of data to answer different questions. Table 3.3 offers an overview of how different data can help answer different questions for different reasons and/or audiences.
Table 3.3 Overview of data types and possibilities for answering well-being questions
What do we want to know? | About who/what? | For who/what? | How to find out? | What kind of data? | Does this answer our Q? |
|---|---|---|---|---|---|
| Did this concert improve people’s well-being? | People attending a local event | To report to local council | Ask people after the event with questionnaire | Ask them to comment = qualitative Ask them to rate it on a scale = quantitative | Yes, but with many limits, such as people not understanding the question, interviewer effect, etc. |
| Does government funding in the arts improve social inclusion? | Social impact of government funding | Think tank or government evaluation | Secondary analysis of longitudinal data | National survey data and administrative or monitoring data (such as financial accounts) | Yes, but with many limits. Relationship between money spent and proxy indicator of social inclusion (access to further education is an example) can only tell so much. |
| What is the social value of public parks in London borough? | Residents and visitors to London borough | Local council | Public consultation | Online questionnaire | Access only to some users, if online, but also recruiting in person will have limits. |
| Evidence review | Looking at findings and/or data from other reports and synthesising viii | Constrained by other people’s findings and methods. Evidence synthesises often don’t evaluate how well the methods of others answer the question. | |||
| Does money buy happiness? | Global population over time | Academic study | Secondary data analysis | Life satisfaction data and a proxy for wealth (GDP), tracked over time, per country | GDP is a limited understanding of the lived experience of wealth. |
| How does crime rate affect well-being? | Population of different countries | International well-being index, that is OECD Better Life Index | A combination of primary survey data collection, complemented by some secondary data | Online questionnaire (in this case the Gallup World Poll, but OECD mainly use data they collect itself) | Not exactly, it is a proxy indicator as risk of crime affects well-being, but it cannot tell you how one affects the other without modelling. |
| How did ‘the bedroom tax’ affect people’s well-being? | A specific community identified as particularly affected | Academic research to inform policy decisions | Ethnographic approaches | Qualitative data from observations and interviews | This research may not be aiming to answer this question explicitly, but would be expected to show a relationship between the policy and well-being. The research answers the question about the people studied in one community in great depth. It may not be generalisable to a wider population. |
| What is important to people about well-being? | All people— within a specific population? (i.e. the UK) | To inform the national measures of well-being | Various methods, including survey and live events | Qualitative and quantitative data | Yes, but different methods may find different answers, so would then have to be looked at together. |
Understanding whether data are ‘good data’, as in good at the job you need it to do, requires an appreciation of all the many aspects of the context, situation and/or population you want to understand. It requires an understanding of what you want to know about well-being, which we have discovered is contestable and varied in different contexts. Thus, for them to be data for good (and thus good for well-being) requires context and reflection. There is a tendency to view and to use certain kinds of data as if they are objective and unaffected by human decisions. The next sections look at objective data and their issues.
