chapter 4 Discovering ‘the new science of happiness’ and subjective well-being
Case study: Subjective well-being, by the Office for National Statistics’ design
The UK’s national well-being measures are categorised into ten domains. These are as follows: Our Relationships; Health; What we do; Where we live; Personal Finance; Economy; Education and Skills; Governance; Environment; Personal Well-being.((See ONS 2019, ‘Measures of National Well-being Dashboard’.)) Each of the ten domains is composed of multiple indicators, just like the OECD’s index that is described in detail in Chap. 3. The subjective well-being domain was named personal well-being, because it was thought to make this domain more understandable to a general audience, which was considered particularly important to the MNW programme.((The MNW debate was more than simply a data collection exercise; it was also a way of engaging the public in the new measures of well-being (Oman 2015a).)) This domain comprises ‘the ONS4’.((The personal well-being domain also includes a measure of ‘population mental well-being’, using data from Understanding Society: UK Household Longitudinal Study. I found it difficult to establish why his additional measure was in the domain, as it gets overshadowed by ‘the ONS4’, with numerous ONS pages on personal well-being, only showing ‘the ONS4’. However, the population mental well-being (SWEMWBs) question was developed to capture a broad concept of positive mental well-being, including psychological functioning and affective emotional aspects of well-being. Respondents to Understanding Society complete the seven question SWEMWBs survey questions. Each response is given a score of between 1 and 5, resulting in a total score of between 7 and 35.)) Table 4.2 presents the questions, together with their rationale.
‘The ONS4’ were designed to capture three types of subjective well-being: evaluative, eudaimonic and affective experience. The four individual subjective well-being questions ask people to give their answers on a scale of 0 to 10, where 0 is ‘not at all’ and 10 is ‘completely’. The ONS considered consolidating the figure of all four measures to provide a single measure of personal well-being. Just as with the HDI in Chap. 2’s discussion of objective lists, this single number is easier to communicate and is most often discussed in national media and by politicians. It was, however, not considered conceptually robust to do so. Here, again, we see a tension between robust and easy to understand.
Table 4.2 The ONS4 capture different aspects of well-being
| ONS’ questions on personal well-being | Specific perspectives on personal well-being, from which the questions are drawn |
|---|---|
| ‘Overall, how satisfied are you with your life nowadays?’ | This comes from the evaluative approach to measuring subjective well-being (i.e. a cognitive assessment of how life is going) |
| ‘Overall, to what extent do you feel the things you do in your life are worthwhile?’ | From the eudaimonic approach |
| ‘Overall, how happy did you feel yesterday?’ | This is about experience, specifically positive affect |
| ‘Overall, how anxious did you feel yesterday?’ | Experience, negative affect |
Source: Adapted from Allin and Hand (2017)
The first results from trialling the ONS4 were published in April 20111. The aim was to gather responses from survey participants which are an ‘assessment of their life overall, as well as providing an indication of their day-to-day emotions’2. ‘The ONS4’ gained National Statistics status in September 2014 and, since then, have continued to be introduced to surveys across government. They are, therefore, not necessarily intended to be used by themselves. Table 4.3 shows the variety of these surveys and the sorts of data they capture. The Government Statistical Service has more recently published advice on the harmonisation of the ONS43. This aims to ensure subjective well-being statistics and data are ‘comparable, consistent and coherent’ across government departments and beyond.
While we know that the ONS4 capture the different aspects of subjective well-being, and there were many reports and working papers from the time, it was quite difficult to find methodological or administrative detail readily available on how the questions themselves were decided on. In particular, the final wording chosen. In parallel to my PhD research, and after much searching, I found a detailed report to the Technical Advisory Group4 on the findings from 44 interviews.
Table 4.3 Surveys containing the ONS4
| Organisation | Survey | Topics covered | First asked | Frequency of update |
|---|---|---|---|---|
| Office for National Statistics (ONS) | Annual population survey | Labour market data including employment and unemployment, as well as housing, ethnicity, religion, health and education. | April 2011 to March 2012 | Annual |
| Wealth and assets survey | Level of assets, savings and debt; saving for retirement; how wealth is distributed among households or individuals; and factors that affect financial planning. | July 2011 to June 2012 (Wave 3) | Bi-annual | |
| Living costs and foods survey | Household spending patterns for the consumer prices index and for GDP figures and detailed information on food consumption and nutrition. | April 2011 to March 2012 | Annual | |
| Crime survey for England and Wales | Experience of crime and attitudes to crime-related issues such as the police, the criminal justice system, and perceptions of crime and anti-social behaviour. | April 2012 | Annual with quarterly updates | |
| Opinions and lifestyle survey | Collects information on a variety of topics that are too small to have surveys of their own. Topics that have been previously commissioned include smoking habits, cancer awareness, charitable giving, climate change and disability. | April 2011 | Monthly | |
University of Oxford and ONS | Time-use survey | Diary entry survey. The substantive domains are main activity (49 categories), secondary activity (10 categories), location and means of transport (11 categories) and with whom (8 categories). The temporal identifier holds information on the time when episodes start and end. | April 2014 to March 2015 | Annual |
| Cabinet Office | National Citizenship Services Evaluation | Social mixing; transition to adulthood; teamwork, communication and leadership; and community involvement. | 2014 | Not updated |
| Youth social action survey | Social action (only satisfaction and worthwhile included). | 2014 | Annual | |
| Department for Work and Pensions (DWP) | Life opportunities survey | Measures how disabled and non-disabled people participate in society in a number of areas which include: • work • education • social participation | 2013 to 2014 | Not updated |
| The National Study of work search and well-being findings | Psychological health and well-being of jobseekers allowance (JSA) claimants. | 2011 | Not updated | |
| English longitudinal study of ageing (ELSA) | Information on the health, social, well-being and economic circumstances of the English population aged 50 years and older. | April 2012 to March 2013 | Annual | |
| Department of Health | What about YOUth? Sur vey | Young people’s health, diet, what they do in their free time, bullying and whether they smoke, take drugs or drink alcohol. | 2014 | Not updated |
| Ministry of Defence (MoD) | Armed forces continuous attitude survey (AFCAS) | Information on the views and experiences of MoD personnel which helps shape policies for training, support, and the terms and conditions of service. | 2012 | Annual |
| Department for Business, Energy and Industrial Strategy (BEIS) | Families continuous attitude survey (FAMCAS) | Information on personals in the MoD spouses in a number of areas including accommodation, healthcare, education and childcare, and deployment. | 2012 | Annual |
| Impact of FE learning survey | Attitudes towards further education, including funding, readiness of information, guidance and decision-making process. | 2012 | Not updated | |
| Department for Communities and Local Government (DCLG) | English housing survey | Age, type, condition and energy efficiency of housing stock and the characteristics of households. | 2013 to 2014 | Annual |
| The Department for Digital, Culture, Media & Sport (DCMS) | Taking part survey | Participation in and engagement with cultural and sporting activities at the individual level, and pathways in and out of participation and engagement. | 2013 to 2014 | Annual |
| Community life survey | Volunteering, charitable giving, local action and networks, and well-being. | 2013 to 2014 | Annual | |
| Food Standards Agency | Food and you | Reported behaviours, attitudes and knowledge relating to food issues such as reported food purchasing, storage, preparation and consumption. It also looks at eating habits, influences on where respondents choose to eat out and experiences of food poisoning. | 2014 | Bi-annual |
| Welsh Government | The National Survey for Wales | Opinions on a wide range of issues affecting people living in Wales and their local area. | April 2012 to March 2013 | Annual |
| Central Statistics Office Ireland | Quarterly national households survey | Labour force estimates that include the official measure of employment and unemployment in the state (International Labour Organisation (ILO) basis). | 2013 | Well-being module not updated |
| Natural England | Monitor of engagement with the natural environment (MENE): The natural survey on people and the natural environment | How people use the natural environment, includes the: • type of destination • duration • mode of transport • distance travelled • expenditure • main activities • motivations • barriers to visiting | 2012 to 2013 | Annual |
| UK Civil Service Sainsbury’s, Oxford Economics and National Centre for Social Research | Civil service people survey Living well index | Civil service staff attitudes and experiences of work. What does it mean to live well? How well are we really living as a nation, and why? This study aims to provide the answers—by defining, measuring and tracking, over a number of years, what it means to live well in Britain. | 2012 | Annual |
| Higher Education Statistics Agency | Measuring graduate subjective well-being outcomes through destination of leavers from higher education (DLHE) | The survey which will gather insightful and comprehensive information about graduate outcomes. The four ONS personal well-being questions are optional. | 2017 | Annual |
| One Parent Families Scotland and Scottish Poverty and Inequality Research Unit at Glasgow Caledonian University | Single parents’ community connections survey | Aims to be the largest ever survey of single parents in Scotland. The results will feed into OPFS and GCU’s community connections project funded by the Scottish government innovation fund. The project aims to tackle isolation, loneliness and poor mental health among single parents. | 2018 | Not updated |
| The Land Trust | Perceptions survey and social value study | The Land Trust is dedicated to providing free public open space for the benefit of communities. Land Trust commissioned carney green to undertake a social value assessment of its sites. | 2015 | Not updated |
| Natural Resources Wales | People Survey 2015 | Our people survey was carried out in order to gauge honest opinions from staff on how they feel about working for Natural Resources Wales. | 2016 | Not updated |
| Active Lives survey | Sport England | Measuring the number of people aged 14 and over taking part in sport and physical activity. | 2015 | Annual |
| Centre for Regional Economic and Social Research (CRESR) and Institute for Employment Research (IER), University of Warwick | Active lives survey— children and young people survey | Includes 3 of the ONS4—does not include the anxiety question. | 2017 | Annual |
| Big lottery talent match survey | An evaluation survey of the initial entrants onto the talent match programme. The overall objectives of the programme are to support 25,000 individuals with the goal of 5400 entering employment. | 2014 | Not updated | |
| Isle of Man Government | Health and Lifestyle Survey 2017 | The areas of interest for this survey were: • general health • diet and physical activity • smoking • alcohol and drug consumption • well-being | 2016 | Annual |
| Higher Education Policy Institute | Student academic experience survey | The survey investigates the learning and teaching experiences of students, including satisfaction with courses, reasons for dissatisfaction, experience of different-sized classes, total time spent working, perceptions of value for money, institutional spending priorities and a focus on student well-being. | 2014 | Annual |
Source: ONS
This report is phase 2 of qualitative findings from testing the ONS4. Notably, not all the responses to the trials were positive in this report. Limitations were found in how able people were to answer the questions.
Interestingly, when it came to the life satisfaction question (thought to be the most robust, as you may remember), not everyone thought that being satisfied with life was positive; some believed it neutral and some thought it a negative commentary on their lives5. With the ‘worthwhile’ question, answers were affected by what was seen as social desirability, leading to inflated scores. This is known as response bias, and meant that certain people (arguably with lower subjective well-being) did not want to appear as if they did not have worthwhile lives to the interviewer6. A later phase in the cognitive testing also details how, when the questions are administered face to face, people felt uncomfortable giving negative scores in front of loved ones7.
When it comes to understanding the meaning of the questions, the qualitative report also states that:
Where the question was not understood this tended to be by those with lower educational attainment. This group simply did not understand the term ‘worthwhile’.
(Ralph et al. 2011, 5)
In some ways, what is more concerning is that:
For the most vulnerable respondents, answering this question was distressing and in some cases respondents became visibly upset. It is recommended that ONS investigate the possibility of creating a flier that interviewers can leave with respondents, which tells them where they can seek help if it is required.
(Ralph et al. 2011, 5)
Having a protocol at the end of research interview, should the interview have covered sensitive issues, is standard ethical practice in qualitative research, but less so in survey collection methods. It is not clear whether filers were trialled after asking participants these questions.
In summary, there were a number of issues that the qualitative research in 2011 uncovered with these four questions. These include: how accurately people were able to answer, based on their understanding of the questions; how honestly people felt capable of being when answering sensitive questions; and that arguably these questions could be detrimental for someone who was not experiencing good well-being. These issues revealed by the testing were brought to the attention of the programme’s advisory groups.
The minutes from the Technical Advisory Group in 2011 outline the importance placed on these four questions. Lord Layard refers to these questions as ‘the work of the ONS’ and outlines that it is the status of this work that is the aim of the wider MNW programme, which reiterates the importance of this new subjective well-being data to the broader agenda. Layard also outlined his concerns that the ‘UK is less likely to set international agenda if introducing unnecessary changes’8. These minutes might suggest that what was learnt from the trials were unlikely to be able to change the new measures, which we have discovered were built from a synthesis of disciplines and authority.
The Technical Advisory Group (TAG) had disappeared from the ONS publications archive when I was originally undertaking this research to try and ‘follow the data’, and understand the methodological origins of the questions. However, I was able to find a record of the group by way of a fellow researcher. The National Statistician, Jill Matheson, refers to a National Statistician’s Advisory Forum and a Technical Advisory Group. All traceable records of TAG meetings are headed by a list of those present. Only ONS, civil service and academic economists were present at the meetings in the minutes I was able to locate. However, another academic researcher confided to me during my ethnography fieldwork that there was a clear hierarchy in the programme and psychologists were rarely listened to, with the economics experts dominating proceedings. This appears to be substantiated by minutes regarding the development of the SWB measures1. It also corroborates claims that economists dominate how evidence is presented, acknowledged and applied in these forums. However, it is important to note that these are not impartial accounts, either.
Psychologists reflecting on phase 2 of the testing of the questions advised that they could cause psychological distress in some participants, but this concern is absent from other outputs. Notably the report on phase 39 mentions it found no issue of difference in legibility for different people, unlike phase 210. More importantly, however, it does not acknowledge that one phase of research found the ONS4 questions to be detrimental to well-being. As you can see, looking under the bonnet of the data presents questions about how the measures work in practice, how they are decided on and by who, and what evidence of success becomes part of record and what disappears. It also reveals issues with regards to how data collection on well-being can be detrimental to well-being that are rarely considered.
