The US Defense Advanced Research Projects Agency (DARPA) has begun working on advanced big data solutions.
DARPA director Arati Prabhakar revealed the agency's plans during a briefing on emerging threats with the House Armed Services Committee's Subcommittee on Intelligence.
"We can see the information revolution unfolding across every aspect of military operations. In response, DARPA is creating a new set of cyber security capabilities that will ensure that networked information is trustworthy," she said.
"We're also inventing the new tools that let us get a handle on this explosion that's happening with data so that, instead of drowning in the data, we can actually get deep insights out of all of that information out there."
The DARPA director said the development of big data tools is essential as the country's existing solutions are not robust enough.
"The classic approach to these complex military systems leads us to a place where these systems are so costly and inflexible that they're really not going to serve our needs for the next generation," she explained.
While DARPA's research is focused on military applications the news could have both positive or negative outcomes for the wider market. DARPA has a track record for creating complex and sophisticated technologies for military departments, which are eventually rolled out for use to the general public.
In the past DARPA has played a role in the creation of NavSat, a predecessor to the Global Positioning System (GPS) technology used today, and background technology used in most modern virtual-reality systems.
As such, any big data tools developed by DARPA could reach the market in the future and provide big benefits to businesses.
However, the data tools DARPA is developing could have more nefarious uses as it could be used by intelligence agencies, such as the NSA, to improve the efficiency of mass-surveillance operations such as PRISM when sifting through data.
Big data is a growing issue for businesses of all sizes. Representatives from companies from various industries listed security and governance as key concerns around big data at a roundtable hosted by Rackspace earlier in March.