A quarter of global corporations will be using big data technologies to secure their organisations in the next two years, according to Gartner, with the value of such tools becoming "too big to ignore".
Today, just eight percent of these companies employ big data-backed security analytics, but as technology becomes more readily available, advanced internal and external threats will become easier to spot.
In a blog post, Gartner's vice president Avivah Litan explained that real-time data with a very short shelf life will be the most important element for firms looking to protect themselves from attackers.
"Information needed to uncover security events loses value over time, and timely intelligent data analysis is critical as criminals and bad actors move much more quickly to commit their crimes," she explained. She added that hackers were skipping the "reconnaissance phase" and instead going directly to attack, thus not giving firms the opportunity to detect their initial attempts.
With big data analytics, companies are now able to combine behavioural information with contextual information such as the content of emails, attachments or social media activity.
In an interview with V3, HP Autonomy CTO Fernando Lucini gave examples of how HP's ArcSight technology combined with its Haven big data tools could detect impending attacks.
He said integrating language analysis tools into security software would reduce the amount of false alarms security officers would have to face.
"If some of my [corporate account] emails are sent to an account at Yahoo, and you reported every email that went to Yahoo your security guy's life would be a misery.
"So now we're reading the email as well, and if one happens to have an email attachment that is a document which is marked as secure – or has content in it that's clearly the company's information capital – we want to make this detection much faster so we can catch bad guys."
He added that attaching a similar form of analytics to social media activity would also provide security teams with better warnings of potential areas of attack.
"If you watch all the social information going around and allow these guys to attach sentiment value – scoring the information as positive, negative and neutral, and finding out what that intent is – the machine finds these things out and provides the security officer with this intelligence. It's pretty spectacular," he said.
Gartner's Litan recommended that firms start small and gradually scale up their deployments of such software. "Enterprise are recommended to start small, but think big, and develop a road map that encompasses multiple use cases and applications across the organisation," she said. "The return on investment on big data analytics is typically too big to ignore."
Electronics and computer chain the latest high street retailer to fall into difficulties
Incisive Media and Investec Asset Management supported fundraiser crosses Atlantic in 40 days
Alphabet's health sciences division Verily have been messing with AI algorithms
North Korea's cyber attack capabilities are expanding fast - and turning their fire on a wider range of targets