Big data analytics faces challenges before mainstream adoption

But even given these multiple challenges, Rozairo urged businesses to start thinking about big data analytics. He pointed out that the financial crash occurred because mortgage brokers and banks had failed to recognise big data.

"We have one customer called CoreLogic who did analyse big data before the crash using Sybase IQ, and studied all detailed mortgage payment data for every residential mortgage applied for in the US. They warned people of what was going to happen but no one wanted to hear," said Rozairo.

Steve Jones, global head of master data management at Capgemini, warned organisations not to embark on any big data analytics projects until they are sure of having good quality data.

"It's easy to find patterns in massive amounts of data but this is no good if you don't have a clean set of data to start off with," he said.

Jones added that many organisations can analyse large amounts of data by integrating the data warehouses they already own. It's only when they need real-time processing of data such as CCTV footage, phone calls or comments left on social networking sites that they should consider updating their analytics suite to something geared towards processing structured and unstructured data.

"Unstructured data analytics is a whole different category people are taking on. It counts as big data even though the data may not be in the exabytes," said Jones.

Jones suggested that the big data analytics market is still relatively immature and that there is lots of room for the vendors in the space to flesh out their offerings.

"There is no key player leading the market, and no big data analytics product boxed up that everyone can use," said Jones.

"Vendors need to focus on integrating data quality and security into big data environments. They also need to produce environments that can be queried by both Hadoop and SQL technologies."

Jones said that EMC Greenplum DCA is one such environment, but pointed out that it still lacks interoperability with other types of processing technology businesses may use, such as those supplied by Oracle or Teradata.

A number of issues have to be addressed before the full potential of big data can be exploited by organisations, such as the price, maturity and interoperability of big data analytics technology. The shortage of professionals with the skills needed to manage big data is another challenge facing the industry.

Promoting use cases of big data analytics as they arise will alert organisations to the potential that can come with big data. However, before an organisation does embark on the next field of analytics, the IT department needs to put strong processes in place to ensure that business data is of a high standard, and can all be analysed in the same place.