Thursday, 7 November 2013

Big Data and GI similar evolution, similar challenge and similar benefits?

Over the weekend the Sunday Telegraph published an article called ‘Big Data – Not just for geeks’ discussing how big data is moving from being the preserve of companies IT departments to something of interest to the wider organisation. I was struck with the similarities seen with the expansion of GI from the core practitioners to the rapidly widening sphere of users, especially since the introduction of BIM.

Of course this is not entirely surprising as GIS and BIM systems are based on Big Data and Big Data being collected by other industries will often have a spatial component.

There are many different claims for what represents the birth of ‘big data’ with the term first being coined in 1997. Originally it was discussed as a challenge to systems and networks, having too much data to store and transfer in an economical fashion. However with improvements in technology big data became increasingly seen as an opportunity. In the UK one of the first (and now famous) mainstream applications of big data was the inception of supermarket club cards in the mid 1990’s. Under the auspices of rewarding customer loyalty Tesco with Clubcard and Sainsbury with Nectar (I can’t remember what it used to be called, they kept changing it) suddenly had unparalleled access to customer insight.

Initially (in fact until very recently) for most companies Big Data was there preserve of their IT or software engineering departments overseen by a CIO or equivalent. The ownership belonged to this technical ‘elite’ (for want of a less pejorative word) with a focus on the technical challenges of collecting and analysing this data. This is comparable to the GI industry which consisted largely of products produced by GI professionals for GI professionals, a paradigm that has shifted rapidly with the proliferation of web based applications. This is not to say that these traditional roles do not exist and are not still crucial, just that the technology and usage has spread much further.

In the Sunday Times article AGI Chair Anne Kemp (among others) suggest that Big Data is undergoing a similar shift which will present a similar skills and knowledge challenge. Coincidentally one of my old clients was a company that specialised in some very clever real time analytics and I remember their Vice President (US company) saying something very similar about the proliferation of big data to new users and taking a more holistic approach.

At the end of the day Big Data and GI are both just tools for enabling better informed decision making. It stands to reason therefore that the greater number of people who are able to access relevant information the greater the positive impact on decision making. In both Big Data and GI this presents a twofold challenge for producers and users of the data.

The first challenge is at the suppliers’ end whether this be a companies IT department or a GI software and data supplier. The challenge here is to provide the data in a way that is accessible to a wide range of users with differing skill sets that may not be particularly technical. The second challenge is for the organisations themselves in ensuring they have the right skills amongst their staff to interpret the data. This doesn’t mean everybody needs to be able to code just that people have an appreciation as to how to handle data and how it impacts and supports their role.

Meeting these challenges and a general cultural appreciation of data (and GI) as a tool that can enable organisations to make better decisions is a key change that will allow these organisation to save time and money and deploy limited resources in a more effective fashion.

In short Big Data is going through the same maturation process as GI forcing a cultural shift in attitudes to how data is used and who it is used by. Both sectors (if they can truly be called that given their interconnectivity with all other sectors) can learn from each other in how to espouse their benefits and encourage an ever greater uptake.

No comments:

Post a Comment