Understanding Well-being Data

chapter 4 Discovering ‘the new science of happiness’ and subjective well-being

Case study: Subjective well-being, by the Office for National Statistics’ design

The UK’s national well-being measures are categorised into ten domains. These are as follows: Our Relationships; Health; What we do; Where we live; Personal Finance; Economy; Education and Skills; Governance; Environment; Personal Well-being.((See ONS 2019, ‘Measures of National Well-being Dashboard’.)) Each of the ten domains is composed of multiple indicators, just like the OECD’s index that is described in detail in Chap. 3. The subjective well-being domain was named personal well-being, because it was thought to make this domain more understandable to a general audience, which was considered particularly important to the MNW programme.((The MNW debate was more than simply a data collection exercise; it was also a way of engaging the public in the new measures of well-being (Oman 2015a).)) This domain comprises ‘the ONS4’.((The personal well-being domain also includes a measure of ‘population mental well-being’, using data from Understanding Society: UK Household Longitudinal Study. I found it difficult to establish why his additional measure was in the domain, as it gets overshadowed by ‘the ONS4’, with numerous ONS pages on personal well-being, only showing ‘the ONS4’. However, the population mental well-being (SWEMWBs) question was developed to capture a broad concept of positive mental well-being, including psychological functioning and affective emotional aspects of well-being. Respondents to Understanding Society complete the seven question SWEMWBs survey questions. Each response is given a score of between 1 and 5, resulting in a total score of between 7 and 35.)) Table 4.2 presents the questions, together with their rationale.

‘The ONS4’ were designed to capture three types of subjective well-being: evaluative, eudaimonic and affective experience. The four individual subjective well-being questions ask people to give their answers on a scale of 0 to 10, where 0 is ‘not at all’ and 10 is ‘completely’. The ONS considered consolidating the figure of all four measures to provide a single measure of personal well-being. Just as with the HDI in Chap. 2’s discussion of objective lists, this single number is easier to communicate and is most often discussed in national media and by politicians. It was, however, not considered conceptually robust to do so. Here, again, we see a tension between robust and easy to understand.

Table 4.2 The ONS4 capture different aspects of well-being

ONS’ questions on personal well-beingSpecific perspectives on personal well-being, from which the questions are drawn
‘Overall, how satisfied are you with your life nowadays?’This comes from the evaluative approach to measuring subjective well-being (i.e. a cognitive assessment of how life is going)
‘Overall, to what extent do you feel the things you do in your life are worthwhile?’From the eudaimonic approach
‘Overall, how happy did you feel yesterday?’This is about experience, specifically positive affect
‘Overall, how anxious did you feel yesterday?’Experience, negative affect

Source: Adapted from Allin and Hand (2017)

The first results from trialling the ONS4 were published in April 20111. The aim was to gather responses from survey participants which are an ‘assessment of their life overall, as well as providing an indication of their day-to-day emotions’2. ‘The ONS4’ gained National Statistics status in September 2014 and, since then, have continued to be introduced to surveys across government. They are, therefore, not necessarily intended to be used by themselves. Table 4.3 shows the variety of these surveys and the sorts of data they capture. The Government Statistical Service has more recently published advice on the harmonisation of the ONS43. This aims to ensure subjective well-being statistics and data are ‘comparable, consistent and coherent’ across government departments and beyond.

While we know that the ONS4 capture the different aspects of subjective well-being, and there were many reports and working papers from the time, it was quite difficult to find methodological or administrative detail readily available on how the questions themselves were decided on. In particular, the final wording chosen. In parallel to my PhD research, and after much searching, I found a detailed report to the Technical Advisory Group4 on the findings from 44 interviews.

Table 4.3 Surveys containing the ONS4

OrganisationSurveyTopics coveredFirst askedFrequency of update
Office for National Statistics (ONS)Annual population surveyLabour market data including
employment and unemployment,
as well as housing, ethnicity,
religion, health and education.
April 2011 to March 2012Annual
Wealth and assets surveyLevel of assets, savings and debt; saving for retirement; how wealth is distributed among households or individuals; and factors that affect financial planning.July 2011 to June 2012 (Wave 3)Bi-annual
Living costs and foods surveyHousehold spending patterns for the consumer prices index and for GDP figures and detailed information on food consumption and nutrition.April 2011 to March 2012Annual
Crime survey for England and WalesExperience of crime and attitudes to crime-related issues such as the police, the criminal
justice system, and perceptions
of crime and anti-social
April 2012
Annual with quarterly updates
Opinions and lifestyle surveyCollects information on a variety of topics that are too small to have surveys of their own. Topics
that have been previously commissioned include smoking habits, cancer awareness, charitable giving, climate change and disability.
April 2011Monthly

University of Oxford and ONS

Time-use survey
Diary entry survey. The substantive domains are main activity (49 categories), secondary activity (10 categories), location and means of transport (11 categories) and
with whom (8 categories). The temporal identifier holds information on the time when episodes start and end.
April 2014 to March 2015Annual
Cabinet OfficeNational Citizenship Services EvaluationSocial mixing; transition to adulthood; teamwork,
communication and leadership;
and community involvement.
Not updated
Youth social action survey
Social action (only satisfaction and worthwhile included).
Department for Work and Pensions (DWP)Life opportunities surveyMeasures how disabled and non-disabled people participate
in society in a number of areas which include:
• work
• education
• social participation
2013 to 2014Not updated
The National Study of work search and well-being findingsPsychological health and well-being of jobseekers
allowance (JSA) claimants.
2011Not updated
English longitudinal study of ageing (ELSA)Information on the health, social, well-being and economic circumstances of the English population aged 50 years and older.April 2012 to March 2013Annual
Department of HealthWhat about YOUth? Sur veyYoung people’s health, diet, what they do in their free time,
bullying and whether they smoke,
take drugs or drink alcohol.
2014Not updated
Ministry of Defence (MoD)Armed forces continuous attitude survey (AFCAS)Information on the views and experiences of MoD personnel
which helps shape policies for
training, support, and the terms
and conditions of service.
Department for Business, Energy and Industrial Strategy (BEIS)Families continuous attitude survey (FAMCAS)Information on personals in the MoD spouses in a number of
areas including accommodation, healthcare, education and
childcare, and deployment.
Impact of FE learning surveyAttitudes towards further education, including funding,
readiness of information, guidance and decision-making process.
2012Not updated
Department for Communities and Local Government (DCLG)English housing surveyAge, type, condition and energy efficiency of housing stock and the characteristics of households.2013 to 2014Annual
The Department for Digital, Culture, Media & Sport (DCMS)Taking part surveyParticipation in and engagement with cultural and sporting activities at the individual level, and pathways in and out of participation and engagement.2013 to 2014Annual
Community life surveyVolunteering, charitable giving, local action and networks, and well-being.2013 to 2014Annual
Food Standards AgencyFood and youReported behaviours, attitudes and knowledge relating to food
issues such as reported food
purchasing, storage, preparation
and consumption. It also looks at eating habits, influences on where respondents choose to eat out and experiences of food poisoning.
Welsh GovernmentThe National Survey for WalesOpinions on a wide range of issues affecting people living in Wales and their local area.April 2012 to
March 2013
Central Statistics Office IrelandQuarterly national households surveyLabour force estimates that include the official measure of employment and unemployment
in the state (International Labour Organisation (ILO) basis).
2013Well-being module not updated
Natural EnglandMonitor of engagement with the natural environment (MENE): The natural survey on people and the natural environmentHow people use the natural environment, includes the:
• type of destination
• duration
• mode of transport
• distance travelled
• expenditure
• main activities
• motivations
• barriers to visiting
2012 to 2013Annual
UK Civil Service Sainsbury’s, Oxford Economics and National Centre for Social ResearchCivil service people survey Living well indexCivil service staff attitudes and experiences of work.
What does it mean to live well? How well are we really living as a nation, and why? This study aims to provide the answers—by defining, measuring and tracking, over a number of years, what it means to live well in Britain.
Higher Education Statistics AgencyMeasuring graduate subjective well-being outcomes through destination of leavers from higher education (DLHE)The survey which will gather insightful and comprehensive information about graduate outcomes. The four ONS personal well-being questions are optional.2017Annual
One Parent Families Scotland and Scottish Poverty and Inequality Research Unit at Glasgow Caledonian UniversitySingle parents’ community connections surveyAims to be the largest ever survey of single parents in Scotland. The results will feed into OPFS and GCU’s community connections project funded by the Scottish government innovation fund. The project aims to tackle isolation, loneliness and poor mental health among single parents.2018Not updated
The Land TrustPerceptions survey and social value studyThe Land Trust is dedicated to providing free public open space for the benefit of communities. Land Trust commissioned carney green to undertake a social value assessment of its sites.2015Not updated
Natural Resources WalesPeople Survey 2015Our people survey was carried out in order to gauge honest opinions from staff on how they feel about working for Natural Resources Wales.2016Not updated
Active Lives surveySport EnglandMeasuring the number of people aged 14 and over taking part in sport and physical activity.2015Annual
Centre for Regional Economic and Social Research (CRESR) and Institute for Employment Research (IER), University of WarwickActive lives survey— children and young people surveyIncludes 3 of the ONS4—does not include the anxiety question.2017Annual
Big lottery talent match surveyAn evaluation survey of the initial entrants onto the talent match programme. The overall objectives of the programme are to support 25,000 individuals with the goal of 5400 entering employment.2014Not updated
Isle of Man GovernmentHealth and Lifestyle Survey 2017The areas of interest for this survey were:
• general health
• diet and physical activity
• smoking
• alcohol and drug consumption
• well-being
Higher Education Policy InstituteStudent academic experience surveyThe survey investigates the learning and teaching experiences of students, including satisfaction with courses, reasons for dissatisfaction, experience of different-sized classes, total time spent working, perceptions of value for money, institutional spending priorities and a focus on student well-being.2014Annual

Source: ONS

This report is phase 2 of qualitative findings from testing the ONS4. Notably, not all the responses to the trials were positive in this report. Limitations were found in how able people were to answer the questions.

Interestingly, when it came to the life satisfaction question (thought to be the most robust, as you may remember), not everyone thought that being satisfied with life was positive; some believed it neutral and some thought it a negative commentary on their lives5. With the ‘worthwhile’ question, answers were affected by what was seen as social desirability, leading to inflated scores. This is known as response bias, and meant that certain people (arguably with lower subjective well-being) did not want to appear as if they did not have worthwhile lives to the interviewer6. A later phase in the cognitive testing also details how, when the questions are administered face to face, people felt uncomfortable giving negative scores in front of loved ones7.

When it comes to understanding the meaning of the questions, the qualitative report also states that:

Where the question was not understood this tended to be by those with lower educational attainment. This group simply did not understand the term ‘worthwhile’.

(Ralph et al. 2011, 5)

In some ways, what is more concerning is that:

For the most vulnerable respondents, answering this question was distressing and in some cases respondents became visibly upset. It is recommended that ONS investigate the possibility of creating a flier that interviewers can leave with respondents, which tells them where they can seek help if it is required.

(Ralph et al. 2011, 5)

Having a protocol at the end of research interview, should the interview have covered sensitive issues, is standard ethical practice in qualitative research, but less so in survey collection methods. It is not clear whether filers were trialled after asking participants these questions.

In summary, there were a number of issues that the qualitative research in 2011 uncovered with these four questions. These include: how accurately people were able to answer, based on their understanding of the questions; how honestly people felt capable of being when answering sensitive questions; and that arguably these questions could be detrimental for someone who was not experiencing good well-being. These issues revealed by the testing were brought to the attention of the programme’s advisory groups.

The minutes from the Technical Advisory Group in 2011 outline the importance placed on these four questions. Lord Layard refers to these questions as ‘the work of the ONS’ and outlines that it is the status of this work that is the aim of the wider MNW programme, which reiterates the importance of this new subjective well-being data to the broader agenda. Layard also outlined his concerns that the ‘UK is less likely to set international agenda if introducing unnecessary changes’8. These minutes might suggest that what was learnt from the trials were unlikely to be able to change the new measures, which we have discovered were built from a synthesis of disciplines and authority.

The Technical Advisory Group (TAG) had disappeared from the ONS publications archive when I was originally undertaking this research to try and ‘follow the data’, and understand the methodological origins of the questions. However, I was able to find a record of the group by way of a fellow researcher. The National Statistician, Jill Matheson, refers to a National Statistician’s Advisory Forum and a Technical Advisory Group. All traceable records of TAG meetings are headed by a list of those present. Only ONS, civil service and academic economists were present at the meetings in the minutes I was able to locate. However, another academic researcher confided to me during my ethnography fieldwork that there was a clear hierarchy in the programme and psychologists were rarely listened to, with the economics experts dominating proceedings. This appears to be substantiated by minutes regarding the development of the SWB measures1. It also corroborates claims that economists dominate how evidence is presented, acknowledged and applied in these forums. However, it is important to note that these are not impartial accounts, either.

Psychologists reflecting on phase 2 of the testing of the questions advised that they could cause psychological distress in some participants, but this concern is absent from other outputs. Notably the report on phase 39 mentions it found no issue of difference in legibility for different people, unlike phase 210. More importantly, however, it does not acknowledge that one phase of research found the ONS4 questions to be detrimental to well-being. As you can see, looking under the bonnet of the data presents questions about how the measures work in practice, how they are decided on and by who, and what evidence of success becomes part of record and what disappears. It also reveals issues with regards to how data collection on well-being can be detrimental to well-being that are rarely considered.

  1. ONS 2011a [] []
  2. ONS 2015a, 5 []
  3. Nickson 2020 []
  4. Ralph et al. 2011 []
  5. Ralph et al. 2011, 5 []
  6. Ralph et al. 2011, 5 []
  7. ONS 2012, 7 []
  8. ONS 2011b []
  9. ONS 2012 []
  10. Ralph et al. 2011 []