Big data analytics faces challenges before mainstream adoption

Industry analysts have pinpointed big data analytics as the next wave of innovation for the IT industry, while user case studies are starting to emerge that demonstrate the value big data can bring to society.

However, not all organisations are ready for the latest analytics technology, and professionals with the skills to make sense of big data are in short supply. This may mean that only the most forward-thinking organisations invest in big data analytics in the near future, and that it may be being years before the average business jumps on the bandwagon.

Big data analytics technology differs from traditional data warehousing and reporting technology because it can cope with extremely large data sets that often include unstructured data coming from a range of internet applications, devices and sensors. Big data environments are able to answer specific questions on historic as well as projected data.

According to the Gartner hype cycle, big data is showing the first signs of early adopter investigation this year, and the analyst firm believes that it will be one of the top enterprise technologies in 2012.

Meanwhile, Ovum has released bullish figures on how it expects big data analytics take-up to increase. A recent survey by the analyst firm showed that 44 per cent of businesses storing more than 1TB of data have plans to invest in the technology in the next two to five years.

Ovum analyst Tony Baer told V3 that the field of data science had seen a significant growth spurt since a number of large IT vendors planted stakes in the industry.

EMC bought Greenplum in 2010, and has since launched the Greenplum Data Computing Appliance (DCA), which combines storage and recovery options with the Greenplum massively parallel processing (MPP) database. The DCA also contains support for the Apache Hadoop database, which is the leading NoSQL analytics platform for processing unstructured data.

"Greenplum's strength is its MPP architecture, which means that the processing nodes can deal with multiple queries at the same time. For example, financial institutions using the database could query the nodes to find out which customers are over their credit card limits, at the same time as carrying out real-time credit checks," said Mark Sears, Greenplum solutions architect, in an interview with V3.

Because the DCA is also integrated with Hadoop technology, the sentiment of customers can be further processed and analysed at the same time, said Sears.

IBM, HP and Teradata are some of the other large IT vendors that have acquired pure-play big data analytics SQL firms in the last year. These being Netezza, Vertica and Aster Data respectively.