chapter 8 Talking different languages of value
How was the value of the relationship between museums and happiness calculated?
The previous sections have walked you through some of the contexts of the research in ‘Museums and Happiness’: the data, the concepts and the relationships being modelled—as well as an aside about how appealing headline findings are when they are formulated in monetary terms that appear easy to grasp. We have looked at what this example of quantitative research was aiming to do with a hypothesis on various relationships, but fundamentally: that museums improve people’s happiness.
This one hypothesis emerges from a series of contexts: the naturalised relationship between culture and well-being and a hypothesis that this can be measured; a desire to isolate the qualities of museums in this relationship to argue their value; philosophical reasoning on how this is possible; prior research indicating other values that help understand the relationship in question—and prior research indicating methods and models that will be useful. We are now going to look at how the numbers were generated.
We discussed how the same model could be run over and over again, changing one variable each time. The calculations, when taken together, can model how when an individual goes to a museum, their happiness goes up because of the experience. This can account for some of the additional things that could be going on. One might be that their happiness could be going up directly because of the specific experience, and also indirectly, because their health could be getting better because their happiness has improved. So, again, it is not ‘museums’ that is valued, per se, but a series of variables which are different in each set of models, but some of these variables are about going to museums. I am reproducing Table 8.1, together with Table 8.2, so you can see the variables together.
As a reminder, the key findings are summarised as follows in ‘Museums and Happiness’1:
- People value visiting museums at about £3200 per year.
- The value of participating in the arts is about £1500 per year per person.
- The value of being audience to the arts is about £2000 per year per person.
- The value of participating in sports is about £1500 per year per person.
Table 8.2 Variables modelled in ‘Museums and Happiness’ that are not about participation
| Other variables |
|---|
Binary variables for each of: • marital status, • religiosity, • educational qualifications (having General Certificates of Secondary Education (GCSEs) and above vs not), • sex, • employment status, • frequency of meeting friends (at least once a month vs less than that), • being in London, • satisfaction with the local area (‘satisfied’ and above vs less than that), • smoking, • ethnicity (white vs other), • volunteering; Scales for: • numbers of children in the household • and how often participants drink (from ‘never’ to ‘every day’). • The self-rated health measure is also incorporated into the x vector. Bands of: • income in £5000 bands |
There are four regression tables in the report that estimate the relationships between
• museum participation and happiness
• museum participation and health
• ‘audience to arts’/arts attendance, arts participation and happiness • ‘audience to arts’/arts attendance, participation and health
Just to remind you, that all these variables in the regressions tables began their life around someone’s kitchen table, or on their sofa, answering the questions of an interviewer, using the TPS script. Let’s consider two questions again. We have already thought about the subjective well-being question. I have copied the explanations from the report as to why each variable was used in the table. It is not normal practice to display these two aspects of methodology together like this, but I find it helpful to see the what and the why (Table 8.3).
Table 8.3 Health and subjective well-being variables, questions and rationales in ‘Museums and Happiness’
| Variable | Question from TPS | Rationale for the question |
|---|---|---|
| Subjective well-being | ‘Taking all things together how happy would you say you are?’ on a scale from 1 to 10 where 10 is described as ‘extremely happy’ and 1 as ‘extremely unhappy’ | ‘Happiness taps in to people’s emotions, technically their affective state, and hence tries to gauge people’s moods at that moment’2 |
| Health | ‘How is your health in general?’ on a scale from 1 to 5 where 1 is ‘very good’ and 5 ‘very bad’ | ‘ …questions on general health will cover mental health and so we may be able to pick up some aspects of well-being or happiness that are not captured in the stand-alone happiness question’3 |
It is interesting that the rationale behind using health is stated as it may pick up on mental health, which may pick up on well-being and happiness. Of course, it does not necessarily follow that responses to a health question will ‘pick up’ on happiness and there is much work on these complex relations. For example, Clark et al.4 find that measures of mental health explain more variation in well-being than measures of physical health. Again, it is not that this is not going on, but it is hard to say that it definitely is.
As the reader, you can make your own decisions on whether this question ‘how is your health in general?’ may be likely to collect meaningful data regarding subjective well-being for respondents. You can do this by imagining how you might answer this question, and whether you feel you would respond about your general health in a way that incorporated your subjective well-being. You might also do this for others you know well, who, for example, might identify as having poor physical health, but are generally happy, and vice versa. Again, this is not to say that because people with poor physical health are susceptible to poor subjective well-being that the health question cannot pick this up.
The other thing to remember here is that this representative sample was asked these questions between the years 2005 and 2011. When this report was written in 2013, the general public would have made less association between health and happiness. Arguably, much advocacy and attention raising have happened in the last few years, which would possibly change
how people align health and subjective well-being. Back in 2005, culturally, it would have been different again. Therefore, when claims are made about how one thing picks up another, we can all think about the contexts in which the questions were asked, how we might answer them, and begin to think about the assumptions made on this basis.
Perhaps another reason that the study includes health is that it would have helped the process of comparison across the two models, in that it offers two measures of subjective well-being (according to the theory that health will pick up on subjective well-being). It will therefore be possible to check for robustness. This is because, and we should continue to bear this in mind, no measure is perfect. Having multiple measures that are shown in previous studies to be related to the relationship you want to understand will add confidence to your finding. That is, if they are all pointing in the same direction.
In summary, the research reported on in ‘Museums and Happiness’ compares the relationships between participation (various) and subjective well-being, and income and subjective well-being, by interpreting what the coefficients mean. In line with standard practice, assumptions about the measures of subjective well-being and everything else have been stated. So, there is a theory behind why particular variables are used, and what they can tell us (and the limits to what they cannot), and efforts have been made to communicate them. It gets confusing when the coefficient of the relationship between income and subjective well-being is then substituted with other estimates (multiplying by 8) that emerge from other reports which used different modelling techniques, different variables and concepts. They may also be based on other conceptualisations that may have been used in previous examples of the well-being valuation technique. The researcher also points out that this part of the process is also established, however, and has also been accepted by Treasury((HM Treasury is the government’s economic and finance ministry, maintaining control over public spending, setting the direction of the UK’s economic policy and working to achieve strong and sustainable economic growth. https://www.gov.uk/government/organisations/hm-treasury)) (p. 8).
Following the key findings on page 8, some caution is advised in the report, for a number of reasons. I summarise these below (the text in brackets aims to explain the reason behind caution being required):
- arts participation and museum attendance are not randomised (without a randomised sample, claims to causation are limited)
- there are likely to be hidden factors that affect both participation and the outcome variables
(it is not possible to isolate participation from all other possible variables to be sure that the effects measured are because of participation in museums, rather than any of a number of other things that could affect well-being) - it is possible that the described causal relationship could be backwards (it might be that happier people tend to go to museums, rather than museums make you happier)1.
These caveats, briefly explained, are: if you really wanted to understand the relationship between museums and happiness. In a theoretically perfect world, you would engineer a sample of people that you could then randomise, making sure that half had gone to museums and half of them not, and see whether one group’s happiness is higher on average at the end than the start. This is a randomised control trial (RCT) used as the gold standard in medicine to understand the effects of medication or other interventions and has become increasingly popular in policy-making5. Yet, such a test is not really practical or ethical in the social sciences—making it very imperfect for a well-being researcher. As I have said before, it is important to (1) use the best available data and be clear on their limitations, and (2) imagine the origins of data. For example, imagine a reality in which people were surveyed en masse in an RCT like this. It would be unethical for the cultural sector to force half the population into a museum and forbid the other half from going in for a year in order to model its value! Also, RCTs use placebos, so people who have not been dosed don’t know. It’s not as if there’s a placebo version of a museum you can send people to.
When it comes to the hidden (latent) factors, the explanation in the report is useful. There are always likely to be some influences that cannot be observed in the data available.
For example, extraverted people may be more likely to participate in the arts and also are more likely to report higher happiness and wellbeing, which means that any observed relationship between the arts and happiness may in part be driven by this personality trait rather than the act of participation itself.
(Fujiwara 2013, 8)
Latent traits are personal characteristics that affect what people do, but which cannot be measured directly. So, for example, some people are more curious about the world than others. This would mean ‘curiosity about the world’ is your latent variable of interest, and maybe those people are both more interested in going to museums and are happier, but you can’t just ask people ‘how curious are you about the world?’ to find out.
As a result of these typical limitations, it is hard to be sure whether it is the going to museums per se that means people are happier, or whether it’s some latent trait that means that people who are more likely to go to museums are more likely to be happier. As we know, one of the key issues with the evidence base on the value of culture is that most of the research struggles to argue that ‘doing culture makes you well-er’, rather than people who are more well participate in culture.
These caveats are all threats to causal inference. Yet, as the report points out, this ‘level of rigour… is anyway normally acceptable in public policy-making and policy evaluation in OECD governments’1. Therefore, the report implies that there are a number of limits to the claims that can be made, but that these limits are considered acceptable. In other words, there is a shared understanding that this is acceptable between experts who do valuations and experts in government who accept them as evidence.
