What can Big Data do for the cultural sector?

«Big Data» is a buzz phrase of the moment, of that there can be no doubt. In the many discussions I had in travelling the UK and Europe discussing the potential of data-driven decision-making in the cultural sector, there is almost unanimous agreement that data can be a powerful asset and that, in many parts of the cultural sphere, it is currently underused.
Whilst the trendiness of the phrase itself will certainly decline, the characteristics of big data are here to stay. It is trite to say that the world we live in includes an unprecedented volume of data, capturing, recording and measuring a greater variety of activity than ever before and coming at us at increasing velocity. Whilst the precise definition of «Big Data» varies depending on the author and the audience, the behaviours and insights to which volume, variety and velocity can give rise are profound. «Before Big Data, our analysis was usually limited to testing a small number of hypotheses that we defined well before even collecting the data. When we let the data speak, we can make connections that we never thought existed.»¹

Aggregating data about the behaviour of cultural consumers could provide powerful new arguments both for the provision and allocation of public funding and for the measurement of its impact. But, ideally, this will need to rest on a culture which genuinely sees public money as an investment (rather than subsidy) and which values a two-way, regular and honest exchange of data and information between funder and funded which goes beyond the necessary elements of accountability and governance. Such an approach could have a genuinely revolutionary effect on cultural provision.

How can Big Data help the average cultural sector player?

Firstly, it helps to go where the Big Data already is. What could something like Google Analytics tell you about demand for a new production? How might analysis of the behaviour of a Facebook Group be compared to large-scale segmentation using Facebook’s own ad tools and then compared to the behaviour of the part of «your» audience, which has similar demographic characteristics? These examples uses some element of big data-type approaches to rethink at least one component of traditional decision-making.
And it is the decision-making which is the key. Whether the data is technically defined as «big» is of comparatively little importance in some ways. It is the use of data-driven approaches to drive insight and change behaviour which matters. 
Put simply, almost all cultural organisations already live in a world of greater volume and variety of data - even if they don’t yet harness it. Many are exploring the opportunities and challenges of velocity (for instance through the liveness of Twitter). But very few have an integrated strategic approach and the skills and tools to make the most (or even much sense) of the potential that they are faced with.
One way to think about data-driven decision-making at the current time is to move away from thinking about simply finding definitive relationships of cause and effect between data points (e.g. single transactions in a ticketing system). Even the smallest of «bigger data» sets benefits more from a process of separating the signal from the noise - of letting the data «speak». Many of us have been trained or conditioned to use data to prove our point - and this still has a place - but the confident use of larger data sets can go beyond that.

Even if many cultural organisations have «bigger» data rather than Big Data, the challenge of increasing data-driven decision-making is made up of many layers.
Firstly, many organisations believe their data to be poor quality. This may indeed be the case - but there is no such thing as perfect data. So long as a process of continuous improvement and cleaning is in place and that any obvious problems are ironed out before relying too much on data, it’s undoubtedly better to make a start using existing data rather than waiting for it to be «perfect». If you think your data really is terrible, there’s so much of the stuff about nowadays you might even think about starting again - then using your new dataset to check back against the old stuff to find out what, if anything, might be useful after all.

Think about what you need to measure and then start to measure it

From then on in, the tasks get technical for a little while. Systems such as ticketing software must be developed and improved in many organisations. Integration between websites, databases and ticketing needs to be a priority of the user-centred organisation - which is what everyone should be looking to be able to be. Non-ticketed organisations should look for ways in which they can create opportunities to connect with and capture the data they need that is meaningful. Given the velocity of data, reporting can be more about dashboards and less about print outs, more about visualisations than spreadsheets.
But even where (or when) the technical capability exists, real challenges remain. As Nate Silver explains in his book The Signal and the Noise, there is still lots of room for human judgment, pattern spotting and the wisdom of good decision-making. It’s all very well if we can get the data to speak, but someone needs to be listening and taking action for there to be any positive effect.

This is where analysis comes in. A characteristic of Big Data led decision-making is the freedom with which analysts explore data and make connections, test them, tell stories about them through visualisation. There are analytic tools in many box office systems available for interrogation of web and social media traffic (e.g. via Google Analytics and for social media) and yet, from discussions with suppliers, many remain un- or under-used. It could be that this results from a lack of skills. Bringing data scientists from other, non-cultural fields into the sector could be an important way to explore the needs of cultural organisations and to build capacity. Another culprit could be a lack of demand for data in the director’s office or board room.

Few people in the sector would now argue that data and culture don’t, can’t or even shouldn't mix. In an increasingly quantified world, just to stay on top of the zeitgeist, almost by necessity, involves being aware of data in a way which would have seemed alien less than a decade ago when no one had «followers» and «like» was a verb.
But, more than that, the use of data as an asset inside cultural organisations would accelerate if the stories it was used to tell were better told, more beautiful, playful and startling. Around the world, artists themselves are working with data in amazing and thought-provoking ways and, more mundanely, people are using dashboards to control their social media feeds or weight-loss. Assuming that the foundation of the raw materials is strong enough, the analytics robust and the people in the room willing to listen; data-driven decision-making could and perhaps should be a key element of increasing artistic impact and commercial resilience both for individual organisations and for the sector as a whole. And it could be based on beautifully evidenced stories.

Audiences expect personalitation

How do we use the data - big, bigger or just bog-standard, to generate new ways of looking at old (or novel) questions? That’s where experience, creativity and necessity are a powerful combination. Counting what counts is important and there are new ways of counting things which had previously been beyond the realms of numbers. Looking to new ways, such as sentiment or semantic analysis to measure aspects of artistic impact could also be an important new tool for the cultural sector.
But there’s plenty that can be done today. The measurability of everyday life is growing at an amazing rate, as personal devices such as smartphones and tablets proliferate and as social data, loyalty cards and the rest combine to record aspects of life that were outside the scope of measurement in the quite recent past. Couple this with the developing expectations of audiences for personalisation and the levels of service provided by digital companies such as Amazon, and data is already a key tool in the armoury if used effectively.
This is definitely a marathon not a sprint and the current buzzy popularity of the phrase «Big Data» is helping to get people thinking. That can only be a good thing. After all data-driven decision-making has been around for years and is certainly no fad, it’s just becoming more useful all the time.