V3 Big Data Summit: Making sense of vast amounts of disparate readings, statistics, facts and figures that make up big data is no easy task.
Many technology companies have found ways to visualise enterprise data to make it more easily digested and analysed, but the sheer volume of big data makes that process challenging.
However, Imperial College London, a university renowned for its academic prowess in the fields of computing and engineering, appears to have found a way to put big data in the big picture.
The KPMG Data Observatory, part of the university's Data Science Institute (DSI) built in partnership with consultancy firm KPMG, has created a physical space in which to view masses of data from real-time and historic sources.
The largest of its kind in Europe, the Data Observatory consists of a circular wall of 64 monitors powered by 32 desktop PC-based graphics processing units (pictured below) connected through a mesh network of web browsers.
The 313-degree facility can render a wide range of data visualisations to effectively give a 130MP view of big data.
The big data picture
However, the Data Observatory is not merely a tool to create large-scale visuals of big data. It offers in-depth interaction and analysis on a wide variety of data sets and can build predictive models to discover how changes in parameters affect the outcome of analytics.
Dr Mark Kennedy, associate professor and director of the KPMG Centre for Business Analytics at Imperial College Business School, compared the Observatory with a military situation room, where a lot of data needs to be taken into account to assess the long-term implications of an action.
He explained in an interview with V3 that this approach is just as applicable to global businesses making decisions concerning the release a product or service to market, as it provides an insight into the ramifications of such a move without needing to commit to an actual launch.
"The visualisation discipline brings you a way to get to the point of being able to fully experience something before you take the risk of fully implementing it," said Kennedy.
"You're able to look at patterns, see interactions, build simulations, look at the dynamics of systems," he added, noting that the ability to spot patterns can lead to better insight and inform potential actions.
Filling a virtual canvas
The Data Observatory is suitable for multiple data visualisations and analytics, and V3 was shown during a tour of the facility several distinct ways it could be used.
Most striking was the visualisation of bitcoin, the cryptocurrency that threatens to challenge traditional payment providers in the e-commerce sector. Using publically available data the Observatory created a real-time visual representation of transactions occurring on bitcoin nodes across the world (as seen in the video below).
The visualisation can separate individual payments from mergers of the digital currency and other bitcoin activity. When used with historical data, the Observatory can identify and track incidences of money laundering and sustained hacking attacks made against the bitcoin network (pictured above).
Such data-based visual analytics could be put to use by financial companies looking to better understand the impact and proliferation of bitcoin, and assess whether it is something they wish to get involved in.
Another visualisation that stood out was a heatmap of the activity in the Shanghai metro system (pictured below). The map of the city's transit network showed which areas of the metro were busiest at which times, such as a station by a stadium getting increasingly busy at the end of an event.
Other supporting visuals were displayed on nearby screens to show the peaks and troughs of activity along the network, offering what was effectively a ‘surround' view of the data collected from recording when people entered and exited metro stations.
As the data can be manipulated, transport companies can see the effect on the rest of the network of closing a line at certain times, based on historical data. Unsurprisingly, the effect of closing one line caused others to swell with activity.
But this system could be useful for civil engineers and planners looking to better understand how line closures and disruptive activity, such as strike action, can affect the metro, and thus be better prepared.
It is not hard to see such visual analytics being put to use at Transport for London, the body responsible for transit around the UK's capital and coincidentally a client of KPMG's.
The Data Observatory was brought about by working with industry partners and funding from KPMG, which is part of the company's £20m investment into Imperial's DSI.
Mazhar Hussain, director at the KPMG Centre for Advanced Business Analytics, told V3 that working with an academic institute was the only way to make the best use of the diverse big data sources to which its clients have access.
"We realised that collaboration is really critical to be able to innovate and push new boundaries. And that's really where this relationship was born. Just over 12 months ago we signed an eight-year alliance with Imperial College all around big data and analytics," he said.
Kennedy explained that the Data Observatory and the wider DSI allow the university to work with industry players and foster collaboration between students and researchers in different subject areas who might not normally cross paths during their academic work.
"This is a place where we can bring researchers together with data and people that might have a stake in what we can show in the data and experience things that they wouldn't have been able to see otherwise," he said.
"What you find without something like the DSI is you have people in computing doing their bit, people in engineering doing their bit, people in social sciences doing their bit, but no place does it quite come together.
"With the DSI we have a meeting ground for people that have different pieces of the puzzles or solutions or both. It's a way to come together."
This concept of cross-discipline and industry collaboration could in many ways point to the future of big data handling and analysis.
As some tools become more specialist to help enterprises mine large datasets for nuggets of information specific to their business and operations, the larger side of data analytics and its impact will be realised by institutions using their collective knowledge and expertise to realise the effects of big data.
A fast, gorgeous but expensive display
Intel wants to get inside your car, despite missing out on mobile
'We'll keep fighting to fight to keep the web free and open,' claim EFF
Breached in March by the same attackers, claim 'insiders'