Making the most out of big data can be a daunting task for even the largest of businesses, and the sheer volume of raw data alone is enough to give IT teams a headache.
Collecting, storing and processing data has become more sophisticated in recent years, but turning raw data into insightful information still requires an expert touch.
Analysis lies at the heart of the big data trend, and the effort and expense of collecting the data is wasted without a considered approach to using and understanding the information.
Data analysis has been at the heart of business intelligence for some time, but analysing the vast volume of disparate information that underpins big data renders traditional approaches all but useless.
To this end companies wishing to exploit big data need a team with the appropriate skills.
Mat Keep, product manager at database firm MongoDB, believes that data analysis requires a team that understands the problems big data can solve for a company.
"Big data analysis requires three core skills - computer science, analytics and statistics - as well as domain knowledge, blended with strong communication skills. Often these skills exist across different staff members, so you need to bring them together as a team," he said.
With this in mind a successful big data team will need to start with a data scientist.
Regardless of their field of expertise the base level of analytics expertise inherent in data scientists gives them the scope to spot trends in data that others might miss.
However, analyst house Gartner warned that these data-focused analytical skills need to be paired with business analysis ability, which is less common among data scientists.
"These skills are guided by an understanding of the business context that drives the analysis. What are the goals, the questions that need to be answered, the business decisions that may be affected, and the constraints around executing?" Gartner analysts Douglas Laney and Lisa Kart explained in a report looking at the emerging role of the data scientist.
"Business analysis also includes the ability to distinguish ‘cool facts/analysis' from insights that will matter to the business, and to communicate those insights to business executives."
A team is therefore likely to need a data scientist and a business analyst to extract the desired insights from big data effectively.
Once the right analytical skills are in place, a team will need the right technical skills to maximise the potential of software and systems available for processing and disseminating big data.
A big data analysis team will need coding and programming skills, along with familiarity with platforms such as Hadoop, HBase and Cassandra.
With these skills, such tools can be tailored to extract the right datasets from a mass of raw information. This allows analysis that links with business objectives, rather than leaving a data scientist to sift through a deluge of disparate information.
Martin Gollogly, a regional director at SAP, believes that, without the proper human input, a company could find itself analysing vast amounts of data with no clear process or objective.
"Technology by itself is not the silver bullet, and there is no benefit to collecting lots of data just because you can. Big data needs robust analysis that is relevant to the business. Technology is a critical enabler only after you have figured out the first part of the equation," he said.
A company can extract the most valuable information from big data only by combining an understanding of traditional and business analysis with the appropriate technical skills.
However, finding such skills is becoming increasingly difficult, as demand for people with relevant IT and digital skills is outpacing supply.
This is a concern echoed by Mike Merritt-Holmes at the Big Data Partnership. "As the big data market expands and evolves, we have seen an increasing challenge for our clients, and across the industry, in finding the right skills and the right technologies," he said.
"Whenever new technologies such as Spark enter the market, they present an enormous opportunity but create another skills deficit to fill."
Finding skilled data analysis specialists has become challenging, particularly as market demand makes them costly to hire. But alternatives do exist.
Merritt-Holmes believes that many companies embracing big data may already have an IT team with the necessary skills.
"It's possible that they may already employ people with existing traditional data analytics skills that can be skilled up. If the big data project is simply a challenge of scale, it is possible that existing team members could manage the workload with help," he said.
Equally, companies that lack the time or resources to upskill existing employees or hire a data scientist may wish to bring in a third-party consultancy to help them understand the big data challenges and needs.
Ensuring that the right skills are available, as part of an existing IT team or an external consultancy, is paramount to the successful dissemination and analysis of big data.
Hardware, software and infrastructure are essential considerations, but companies will also need the people in place before they can truly explore the benefits big data can deliver.
Security firm claims that 117,638 sites out of 135,035 analysed contain serious security flaws
AimBrain adds lip-sync capabilities to its biometrics system
Canadian scientists claim to have found a way to mass produce plastic semiconductors
RAND claims AI could enhance strategic stability by improving accuracy in intelligence collection and analysis