chapter 7 Evidencing Culture for Policy
Policy decisions and investments using well-being data
Lord Richard Layard (the Happiness Tsar from Chap. 4) has previously stated that ‘policy is not going to be framed around [well-being] for decades, but unless you have the index you’ll never get to a point where you can influence things’1.(( Legatum Commission Chairman Lord O’Donnell said: ‘We now know much more about what drives the wellbeing of people and communities than we did 10 years ago, and our knowledge and understanding is set to increase significantly over the next few years. I look forward to working on this exciting project which could transform the way we develop policy’ (Legatum 2012).)) This is a far more measured take on well-being data and evidence being used for policy-making than suggested by the prime minister’s speech that opens this chapter. Lord Gus O’Donnell, another major advocate for well-being in policy-making, is also an economist and an extremely influential civil servant.((Lord O’Donnell served as the Cabinet Secretary between 2005 and 2011. Cabinet Secretary is the highest official in the British Civil Service and it is notable that he held this position under three prime ministers: Blair, Brown and Cameron.)) He explained that same year:
We now know much more about what drives the wellbeing of people and communities than we did 10 years ago, and our knowledge and understanding is set to increase significantly over the next few years.
(O’Donnell in Legatum Institute 2012)
As recognised by the OECD and the ONS early on in their programmes to measure well-being (see Chap. 3), there was a general acknowledgement at this time that well-being measures were evolving and exploratory. So, while a simple visualisation of how life satisfaction over time might interact with arts funding or suicide rates, not all well-being measurements are equally robust, and all have limits that are not often made clear when data are expressed. This is also the case when the concept of well-being is operationalised with another concept, such as culture.
Well-being valuations are far more complex than the way tertiary or headline data were ‘described’ in the previous section’s simple line graphs. As we discovered at the end of Chap. 6, demands from and on government departments to evaluate the impact of their decisions, evolved from the descriptive to more complicated modelling in the 2010s. These models can analyse primary or secondary data and enable a more sophisticated reading of the data. A model helps researchers understand far more complex relationships, including what might be interfering with our understanding (confounders). It can also express a relationship between two things, such as culture and well-being, in monetary terms. We will look at an example of well-being valuation modelling, and how complex this is, in greater detail in the next chapter.
Box 7.4 What Is a Model?
Earlier in this book, I stated that data don’t just fall from the sky as facts. Neither do the models that analyse them. A model will probably contain assumptions about how concepts like ‘well-being’ and ‘culture’ are associated.
There are two main kinds of models: exploratory and confirmatory.
Exploratory models
These allow you to try numerous variables that may be associated, and see what emerges as of possible interest. In other words, you are exploring the possibilities of the data. Developments in machine learning have sped up this kind of exploratory modelling with Big Data, as we discovered in Chap. 5.
Confirmatory models
Most of the chapters in this book refer to work that aims to confirm a hypothesis. Statisticians and others who model quantitative data in this way don’t just throw a bunch of variables into a model and hope for the best. Their models are designed with a theoretical foundation and that will most likely be arrived at from what we already know from previous studies about how one thing (say income) affects another, well-being, for example.
Before a good confirmatory model is designed, it is important to establish ‘what counts’ in the issues you are considering, and how things are expected to fit together.
In exploratory analysis, you won’t need to guess how concepts fit together (although you might have an inkling), and won’t need the same level of attention to the variables you pick in relation to the concepts.
An example of what a model does
A simple model might be based on the hypothesis of a positive correlation. Say, between the average wealth of a nation and its average happiness (as with Easterlin). Imperfect measures tend to be used that represent far more complex concepts like wealth and happiness. For example, variables for life satisfaction and income will not tell us all we need to know about wealth or well-being. Also, resources dictate that it is unlikely a researcher will examine the entire path between income and well-being; instead it will examine whether the two measured concepts (variables) have a statistical association.
It is likely that the relationships examined in any one study represent only small parts of a larger theory. This is not unusual, but is it always explicit when research is presented?
As Chap. 6 describes, government departments including DCMS were indeed looking at how to use well-being data in valuations((Some pivotal examples from the broader DCMS evidence programme include O’Brien (2010); Matrix Knowledge Group (2010); Miles and Sullivan (2010).)) before Cameron’s speech in November 2010 from the official launch. This is because DCMS and the areas it funds were addressing HMT’s preference for valuation techniques2. There are a couple of approaches that have been called well-being valuation. Fujiwara’s (2013) seems the most influential in the UK, but other examples3 called well-being valuation take a different approach. Following the increase in using subjective well-being data to value the impact of services, there has been a growing number of studies investigating the impact of the arts or specific cultural organisations in this way4. These studies use responses to subjective well-being questions in national-level surveys, together with data on, say, theatre attendance, and estimate the impact of that artform. Such valuations assess data which can tell you that people who go to the theatre are more or less likely to have answered subjective well-being questions in a particular way. The magic is in the modelling.
Important questions remain, however, when it comes to the limits of the data and the extent that valuations can advise policy; particularly when it comes to stating one thing is more valuable than another. The practice of ordering the value of one thing over another does not seem to be presenting us with findings that corroborate each other. In one study, one artform is more important than another. As we saw in Chap. 4, ‘excessive TV watching’ is pitted against an unspecified amount of gardening when reporting on data collected to understand how people are spending their time in lockdown and measuring their well-being5. Bias is brought to the data, which means they can be read in ways that confirm prior beliefs about what is an excessive amount in one area, but not necessary to measure about another. Consequently, this bias will feed into the presentation of findings and shape recommendations to decision-makers. In other words, ostensibly rational, neutral decisions which are supposedly made on the basis of well-being data are in danger of reproducing prior judgements and beliefs of the researchers—especially if they confirm those of the policy-maker reading the recommendations.
