Home » related media »
Counting What Counts: What Big Data Can Do for the Cultural Sector
In 2013, I wrote an extended blogpost review of Counting What Counts: What Big Data Can Do for the Cultural Sector that was hosted on the Cultural Value Initiative site. Thanks to Ele Belfiore for agreeing to it being re-posted here.
Review of NESTA’s Counting What Counts
I am thrilled to introduce this post by Susan Oman, which marks a bit of a departure from the usual posts on this blog. This has been a very busy time for cultural policy: Maria Miller’s debut arts speech has kept us all busy commenting for the past few weeks, and the radically different one from her Scottish counterpart, Fiona Hyslop, has had social media enthralled for days now (and a comment post on Hyslop’s speech is indeed coming soon!). Giving a platform for debate on live cultural policy developments between academics, creative practitioners, policy makers and other interested observes is precisely what The #culturalvalue Initiative is all about. However, when a policy report or
a publication comes out that seems to have particularly important implications for the sector, it might be worth slowing down, and engaging with a more thoughtful and complex reflective piece of writing. Susan Oman’s mini-review essay on a report recently published by NESTA is an excellent way to introduce this new type of cultural value post, which I hope to be able to host more of in the future. Susan is overall left disappointed by the report, which makes the claim that the cultural sector is “in need” of big data, with little consideration for the fact that it is indeed the cultural sector that significantly contributes to big data generation, and that there are marked differences across the cultural sector in both organisational scale and potential to engage fruitfully with ‘big data’.Eleonora Belfiore editor of Cultural Value initiative, (2013)
The observation that “big data is us alienated from ourselves and sold back to us” feels disturbingly pertinent when reading Counting what Counts. The report positions the ‘cultural sector’ as in need of big data, when in fact the sector contributes so significantly to big data generation. Given that it is the relationality of information that forms much of the ‘big data’ imperative, this oversight is perhaps the report’s most disappointing omission.
Viktor Mayer-Schonberger and Kenneth Cukier’s recent bestseller, Big Data: A Revolution That Will Transform How We Live, Work and Think talks about the “treasure hunt for the dormant value of data.. [its] ..intrinsic value” (17). With this in mind, NESTA’s recent report, Counting What Counts: What Big Data Can Do for the Cultural Sector by Anthony Lilley and Professor Paul Moore promises a wealth of possibilities, assimilating the intrinsic and instrumental values of ‘culture’ with values of the data it produces and demands. The title implies progress in the quest for a holy grail of data collection, from which useable ‘evidence’ will flow for funders, policy-makers and makers of culture alike.
‘Big data’ has significant implications for evidence based policy-making, the repercussions of which remain largely under-theorised. Mayer-Schonberger and Cukier suggest “society will need to shed some of its need for causality in exchange for correlation” (p. 7). This is an attractive prospect for the ‘cultural sector’ which has struggled to ‘prove’ impact through causation. The existing ‘evidence base’ primarily represents the correlation between a specific project or product and its implied effect on an individual, audience or wider population. If big data evolve the nature of ‘evidence’, there is the potential to assuage the historical criticism of the sector’s inadequacies at evidence production.
Moore and Lilley do not look at this opportunity that ‘big data’ presents. The report focusses instead on the obligation to keep pace with the commercial sector, especially those “compet[ing] for a share of leisure time, from sports to restaurants” (p. 13). It argues that non-participation in big data will “no longer be an option” for the cultural sector, but this context is misleading, as it is doubtful that small businesses such as local restaurants and fitness classes are yielding improved growth opportunities from expensive and unwieldy big data technologies and methodologies.
The report’s universalising of sectors of ‘culture’ into a ‘Cultural Sector’ is a familiar issue and difficulties surrounding this simplified definition morphed into the Creative Industries mapping of 1998, a debate resurrected by a DCMS consultation this month. In generalising the sector without acknowledging this, the report obscures differences in requirements and capacity for data technologies, multiplied by variance in organisation size, type, purpose, mission and cultural offering across and within sectors.
Alternatively, it stratifies the sector by technological capabilities into what it calls Data 1.0; Data 2.0 and Data 3.0 organisations. The arcane coding system is no more germane to ‘big data’ than smaller data objects, and the lack of explanation hinders its utility. The report’s ‘analysis’ of the sector makes no reference to research, and labels that use terminology more familiar to most in descriptions of web technologies seem a little dated and unsophisticated. Furthermore, the categories do not reflect the suitability of technological expansion to develop understanding of how people might interact with an organisation or product in a way that generates data (beyond ticket sales), and how that might positively impact on an organisation’s understanding of productivity or survival. Most significantly, it fails to demonstrate what big data can do for the cultural sector.
Counting What Counts advocates the competitive necessity of big data, rather than adequately describing what it might be or offer. ‘Big data’ refers to a large dataset which improves opportunities to understand people according to Schonberger and Cukier. Moore and Lilley on the other hand, neglect definition in the report’s first quarter, finally describing it as “a loose term which is commonly used to mean the increasing volume, velocity and variety of data created by digital technology, and in particular, the amount of data which arises from the scaling effects of digital networks”(p. 10).
The unsatisfactory definition and lack of wider context compromises the report’s usefulness to the sector or its decision-makers. It also fails to elaborate that it is people’s use of technology that creates useful data about people and preferences, rather than the existence of data in itself. The authors claim “[e]ach sector of cultural activity can benefit if it is able to respond to the opportunities” and talk about “taking advantage of big data”, but this remains unhelpfully abstract. Whilst it is acknowledged (p. 26) that big data can be ‘debilitating’, this is not explored fully, which would have supported comprehension of the possibilities.
Much of the space in which data’s potential to benefit the sector might have been explained was waylaid by two arguments as to why the authors believed the sector needed big data. The first outlines the investment versus subsidy argument (specific to the parts of the sector reliant on funding, and thus not representative of the whole sector). Moore and Lilley suggest that investment obliges a return on investment (ROI), whether financial or social. They explain that as a consequence, reporting mechanisms demanding ROI evidence, compromise the position of the individual organisation (p. 21).
The second involves the “extensive literature around the definition and indeed the usefulness of the concept of cultural value much of which is referenced in the bibliography to this report” (p. 24). They choose McCarthy’s (2005) definition, which recognised the limits to public value as contributing to “broad social and economic goals.. recognis[ing] how both instrumental and intrinsic benefits of the arts contribute to the public realm”. It is postured that big data “holds out potential to measure this kind of value more effectively”, enabling individual organisations to present and appraise their own data productively to assess their value. That ‘public value’[1] and ‘cultural value’ are not synonymous is not adequately accounted for, despite the authors’ “extensive research” and bibliography.
Referencing public value and its relationship to the cultural value debate in order to instrumentalise big data is useful context, but frustratingly incomplete. The reader wonders why the cultural value issue is pertinent to data and measurement when “economics is more than capable of encompassing financial and public good outcomes” (p. 23), suggesting that to a degree for the authors, at least, the issue is resolved.
Furthermore, the ‘extensive’ cultural value literature is not extensive enough. Despite aiming the report at senior decision makers (p.3), (we can assume policy-makers, owing to the emphasis on investment and competition), it accounts for none of DCMS’ attempts to explore cultural value. This includes eight years of Taking Part data, the Culture and Sport Evidence Programme (CASE) and Dave O’Brien’s report ‘Measuring the Value of Culture’. Not to mention Miles and Sullivan’s concurrent DCMS report highlighting the value of cultural activity outside of ‘formal participation’ (and consequently unmeasured). It also neglects a plethora of work from other academics, practitioners and sector professionals, many of whom feature on the Cultural Value Initiative blog in which this review is published.
The reader struggles through the report’s unclear structure to discover on page 37/45 that “the hypothesis of this project was that leading-edge social network analytics might be a fertile breeding ground for new analytical techniques which would allow improved tracking of cultural value”. There are other sections in which social networking is discussed as sites of big data, but it is not explicit until this point that this forms the crux of the report’s recommendations.
The proposition seems to be that investment should be made to better appreciate the utility of harnessing social network communications, as these might offer additional methods of measurement of a cultural product’s merits. In part, this refers to how comments on social networking sites such as twitter might offer insight into audiences’ reception of TV programmes/plays/exhibitions/films, and how these might contribute to a definition of quality.
There is no appraisal of the constraints to analysis of social network comments. One of many examples might be that positive comments on a cultural event or product on twitter often result in a ‘retweet’ by that organisation to its ‘followers’. This arguably encourages praise in the hope of publicity. Anonymous feedback has traditionally been valued for its alleged honesty, but social media contradicts anonymity by design.
In the last few weeks, a DEMOS blog commented that our data are being “hoovered up” by the commercial sector, and one might wonder whether this aspect of the ‘big data’ phenomenon is especially distasteful to much of the cultural sector and its participants/customers. The authors claim that listening to big data should inform ‘product development’ and this is unlikely to curry favour in those believing in the artistic integrity of creation.
A new play reliant on, or resulting from mining big data rather than, say, investment in new writing, is incomprehensible to those in performing arts sectors. It would also have negative value ramifications for those who ‘invest’ in tickets. Whilst such methods might appeal to broadcast, where viewing figures steer production, they cannot be universalised across a united ‘cultural sector’. Most new cultural products do not emerge from data unless an artist/practitioner has chosen it as their subject/medium.
That the report refers to evidence-based decision making, familiar to regulators such as OfCOM, is one of many indicators of the report’s broadcast and media leaning. These sectors dominate the list of contributors and interviewees (Appendix 3), perhaps reflecting (the limits of) the knowledge, expertise and research of its authors.
Big data proffer an evolution beyond audience figures, subsidy and market failure as default case-making for cultural and social value. These debates can seem cyclical in their narrow focus and big data should and could be mobilised to broaden the horizon of debate. It should enable organisations and practitioners to cultivate conceptions of their value outside of economic and social impact scales (if they are as unhelpful as often claimed).
Reading the report, “based on extensive research and practical experience” (p. 6), one wonders how it might have benefitted from more analysis of sector data. Recommendations of gaining funding from the private sector are vague and do not acknowledge that this forms much of the publically funded sectors’ current focus. Furthermore, it fails to convey a nuanced understanding of diversity of ‘product’ or organisational structure across ‘the sector’; even more ironic when considering its main recommendation is network analysis.
It is fair to commend the report for aiming to address the necessary relationship between the cultural sector and big data. It is also fair to acknowledge that many of the big data discussions outside of the sector remain unsophisticated. That is not to say that projects are not underway, but these are largely within the academy, and detached from the cultural value debate thus far. Prompted by Savage and Burrows’ observation of the comparable improved efficacy of commercial datasets in understanding certain aspects of social life (2007), these discussions are predominantly happening in social science domains. Six years later, they have evolved into projects such as Evelyn Ruppert et al.’s The social lives of digital data, which “seek[s] to develop a ‘social literacy’ about big data rather than re-iterating the need to respond to ‘the data deluge”. Whilst Counting What Counts calls for investment in the sector to research big data potential, it fails to convey an understanding or advocacy of either ‘culture’ or ‘big data’, and as such omits the intrinsically important relationship between the two. This lack of advocacy within NESTA itself is evidenced by this report’s absence from its big data pages.
Most data is audio-visual and thus cultural products. Much of the rest is habitually created in the socio-cultural spheres of social media, or as a by-product of human interactions, including shopping at Tesco, for example. “Big data is us alienated from ourselves and sold back to us”, was a comment from the floor at a recent CRASSH, Cambridge event. This feels disturbingly pertinent when reading Counting what Counts. The report positions the ‘cultural sector’ as in need of big data, when in fact the sector contributes so significantly to big data generation. Given that it is the relationality of information that forms the ‘big data’ imperative, this oversight is perhaps the report’s most disappointing omission.
As Livetheatre recently reported, theatre ticket purchases trebled following the BBC’s Great British Class calculator[2]. This is an interesting example of how big data might demonstrate an unexpected relationship of one area of the sector to another, but more importantly an insight into how human behaviour is manifest in cultural sector interactions, which become uniquely apparent through big data analysis. The BBC online tool implied that theatre attendance improved your class ‘rating’. That this provided an imperative to adjust practices and behaviours through consumption provides research impetus across many facets of the sector looking at participation, investment and cultural capital as well as the competitive and commercial aspects of such knowledge.
Big data and the expansion of relations between what and how we measure will produce other correlations, enlightening associations and ancillary benefits of one cultural phenomenon impacting upon others. There is much potential in the sector upping its engagement with ‘big data’ principles and technologies but this report’s centring on the competitive edge, rather than on the potential for insight was another missed opportunity.
Susan Oman is a doctoral student at the ESRC Centre for Research on Socio-Cultural Change (CRESC), University of Manchester. Her inter-disciplinary research is linked to the AHRC-funded project ‘Understanding Everyday Participation – Articulating Cultural Values’ and investigates the politics of cultural practices, participation and well-being.
Susan was involved in a big data project to implement a CRM (customer relational database) in a public sector creative organisation, and as such recognises the potential of and institutional obstacles to big data in the cultural sector. One of the facets of her PhD research addresses tensions in large datasets and how ‘evidence’ gathering methodologies tendentially omit the ‘experiential’, which are vital to the conception and articulations of ‘cultural value’ and she argues part of the promise of big data.
[1] The cited Building Public Value (BBC 2004) report discusses ‘public value’ as an administrative mechanism to understand success of public sector management. The much debated term, ‘cultural value’ is more varied and nuanced.
[2] The BBC Class Calculator was based on academic research that can be found here, in the journal Sociology: A New Model of Social Class: Findings from the BBC’s Great British Class Survey Experiment
