Big data platforms are found in a variety of industries, from manufacturing firms using analytics to identify maintenance needs in a production line, to healthcare companies crunching large databases to aid medical research.
Tennis, a game that has gradually evolved over the past 150 years or so, is not the place you would expect a big data platform to reside.
But the Association of Tennis Professionals (ATP) thinks otherwise, which is why the organisation has worked with Infosys to collect and analyse all the data generated during the ATP World Tour competition, from systems such as Hawkeye and statistical information gleaned from each game.
To do this Infosys provided the ATP with a customised version of its Information Platform, a cloud-based data ingestion and analytics service designed to mix real-time data with historical information to enable the public, media and players to see a range of information based on games and tournaments, all displayed through a web portal or app.
For example, you can compare the serves of different players during an all-important break point, or see how different players apply topspin.
Infosys and the ATP wanted the platform to serve information that can get tennis fans more engaged in the games, provide the media with extra insights beyond the action on the court, and give players more information on their performance.
Serving such data is one thing, but the clever part of the Infosys Information Platform is the way it can tap into historical data from ATP tennis matches and five years' worth of Hawkeye replay data and apply machine learning to effectively predict the outcomes of games.
This required Infosys to train the machine learning algorithms to understand the nuances of tennis rather than the operations of enterprises, but the platform itself is created from a host of open source components put together by Infosys and controlled through a user interface that hides the complexity below the surface.
You could argue that adding predictive analytics to tennis erodes some of the fun of watching a tightly contested match, but what it does showcase is the flexible applications of the Infosys platform and the ways in which big data analytics can be applied to all manner of things.
Infosys has capitalised on this flexibility by basing the Information Platform on open-source engines and frameworks such as Hadoop and Apache Spark, commonly used in the IT and enterprise world, thereby enabling the platform to be deployed across numerous sectors without requiring masses of retooling and integration.
Further adding to this flexibility is the ability to deploy the Information Platform in the cloud or on-premise, with support for global cloud platforms such as Amazon Web Services and Microsoft Azure.
Looking for extra-terrestrial life is not usually associated with big data analytics and IBM technology, and is usually left to enthusiastic stargazers and people who may have been in the Mojave desert for too long.
But the SETI (Search for Extra-Terrestrial Intelligence) Institute begs to differ. SETI was showcased as an IBM customer at the company's Insight 2015 conference in Las Vegas, and makes use of IBM's Cloud Data Services and Apache Spark to analyse huge amounts of data harvested from the Allen Telescope Array in California.
SETI's goal is to find obvious narrow-band aspects of radio signals that differ from background astrophysical and human signals.
Dr Jill Tarter, holder of the Bernard Oliver Chair at SETI, said that four years of listening to signals has resulted in a collection of 100 million signals and a vast amount of raw data on the frequencies to which they relate.
This has given SETI a large database of signals that it has identified as interference from humans and non-alien sources.
SETI uses a combination of analytical resources in IBM's Cloud Data Services portfolio and Apache Spark to query this data and determine whether SETI may have missed something in the recorded interference.
The institute also uses this combination of cloud-based analytics and in-memory framework to find faster ways to diagnose signals.
"Capabilities like Apache Spark are opening up these previously unexplored data sets. We want to do what we've been doing faster and we want to do things that we didn't know we could or should do. Ultimately, we want to be able to analyse that overwhelming fire hose of data flowing from antennas. We want to listen better. We want to really find a signal," said Tarter.
So while such data processing and analytics tools have been championed as a way for enterprises to derive business-boosting information from data, they could also help discover whether humanity is alone in the universe. After all, the truth is out there.
IT conferences are normally an excess of major brands excitedly enthusing over their latest products and touting aspirational but theoretical use cases for their technology.
But the 2015 iteration of Spunk's .conf conference held in Las Vegas is a different beast all together.
Rather than announce a new product that shakes up its portfolio but lacks a real-world use, Splunk preferred to release a series of small but solid updates and products that evolve its core operational analytics platform.
One could argue that Splunk has done nothing revolutionary to its software offerings, but having attended the main conference hall full of whooping attendees and spoken to several customers at the conference, V3 can attest that what the company is doing is garnering strong praise from its followers.
This is probably because Splunk seems to have simply given its customers what they want with updates set to benefits them, and then sitting back to see what the customers do with the extra functionality and enhancements.
With the likes of Gatwick airport, shoe brand Kurt Geiger, BMW, and luxury smartphone maker Vertu showcasing different uses of Splunk at .conf, it is not surprising the company has almost taken a back seat in its own conference, allowing customers to showcase its technology rather than dictate uses for their products.
Snehal Antani, chief technology officer at Splunk, said this fandom Splunk has from its customers, means the company allows its direction be dictated by its customers.
"People come up with really cool new ideas that I've never thought of," he told V3. "Our customers are inventing these uses cases and letting us know what they're doing and we're trying to internalise it. [Splunk] is customer driven and so customers are teaching us, it's not the other way round."
It is this approach that paints Splunk as the Lego of the IT world, whereby Splunk provides the pieces for users to build with, rather than providing an overly ridged and closed platform with uses cases limited to specific industries.
And Splunk is being very smart here, as the world of machine data analytics is still a relatively immature facet of the technology world with a myriad of use cases, and if the company tried to prescribe the path the tech trend should be on then it may end up stifling innovation rather than enjoying the fruits of its open platform.
Speak to any motorist who's spent time navigating Britain's rural B-roads and you'll probably set them off on a tirade about journey-ruining, tyre-shredding, pothole-riddled roads.
But Jaguar Land Rover (JLR) might just have the answer to these tarmac-based woes after revealing research into using cloud and connected car technology to enable vehicles to identify the location of potholes and broken manhole covers and share that knowledge with other motorists.
Pothole Alert has the potential to save motorists billions of pounds a year on punctures and vehicle repairs, according to JLR.
The system is an evolution of the MagneRide technology found in the Range Rover Evoque and Discovery Sport, which uses sensors to profile the road surface and monitor vehicle motion and changes in suspension height.
The system then adjusts the suspension to give passengers a comfortable ride when they are travelling on rough or damaged roads.
Dr Mike Bell, global connected car director at JLR, said the Pothole Alert research stemmed from the potential the company saw for wider use of the information harvested by the MagneRide system.
"We think there is a huge opportunity to turn the information from these vehicle sensors into big data and share it for the benefit of other road users," he said.
Bell explained that the most accurate data comes from vehicles that have already driven over a pothole, but that JLR is researching ways to scan the road ahead to provide data on such obstacles so that action can be taken before a vehicle reaches them.
JLR said that such alert systems could be used to deliver pothole and road damage data to local councils via the cloud to inform them of road sections in need of repair, something the carmaker is working on with Coventry City Council.
The Pothole Alert research is an example of connected car and cloud technology being explored in a granular and very practical way, rather than from a high-concept and large-scale perspective, in turn helping to inject a ‘real-world' element into modern and often nebulous technology.
That being said, Bell noted that JLR's research is a stepping stone towards developing autonomous vehicle systems and driverless cars, and will help make autonomous driving "a safe and enjoyable reality".
Driverless cars might seem to some like a far-fetched concept lifted from the pages of science fiction novels.
But the fact that Google's driverless cars have been involved in only 11 minor accidents in six years, none of which was the car's fault, and having clocked up thousands of miles of autonomous driving, suggests that driverless cars will be on UK roads sooner than many would have predicted.
The Internet of Things (IoT) is going to be big. According to Microsoft chief executive, Satya Nadella, IoT will generate 44 zettabytes of data in the future. That's a lot of 1s and 0s.
Nadella made the claim during his keynote speech at Convergence 2015 in Atlanta, as part of Microsoft's push around IoT on its Azure cloud platform. This is a collection of cloud-powered tools designed for companies looking to build software that exploits data collected from a myriad of sources.
It is data that is underpinning Microsoft's approach to IoT, with Nadella stating it will drive the direction of the company. "Devices will come and go," he said. "The most interesting thing is the data that's being collected."
Nadella said Microsoft has a prevailing objective to find ways that seek out the value in data harvested from IoT devices.
The majority of the products announced during Convergence 2015 all had data use at their core. Nadella explained how Microsoft wants to empower people and businesses to use the insightful information that can be gleaned from diverse datasets.
Nadella even went so far as to showcase his own use of data, revealing to thousands of onlookers his fitness and wellness data recorded by the Microsoft Band he wears.
V3 was interested to see that the chief executive of one of the largest companies in the world still manages to find time to exercise and get over seven hours sleep.
Nadella also introduced Seattle Seahawks star quarterback Russell Wilson on stage to discuss how data can be used to monitor players' performances and vital signs.
Given Microsoft's investment into what it calls a ‘hyper scale' multi-purpose cloud, it likely has the capability to handle 44 zettabytes of IoT data. But the Redmond company will be entering an area that is rapidly filling up with other technology giants, such as Intel, ARM and IBM.
10 May 2013
President Obama recently signed an executive order that will require government organisations to release their data to the public in an easily digestible form. The White House says that the move will give entrepreneurs and innovators the information they need to create engaging new products.
To see what sort of potential this government data holds for the private sector, look at the past. Prior to the 1980s GPS data was mostly relegated to military and government organisations. It wasn't until 1983 and the release of GPS data to the public that consumer mapping technology really began to take shape.
At the time, then-President Ronald Reagan ordered GPS data become freely available to the public. Reagan's decree came following the crash of Korean Air flight 007, which was shot down after getting lost and flying to near Soviet airspace.
By 1989, US company Magellan released the first commercially available portable GPS system. The Magellan NAV 1000 used GPS data from government satellites to put GPS right next to a citizen's Walkman.
The release of the device, and the government data it used, is why we have GPS navigation today. From Google Maps to Apple Maps, all of the world's most basic turn-by-turn navigation wouldn't be possible without government data.
The example goes to show that the government has types of data that would be almost impossible to get without an open initiative to release it. The US government has the resources to do things that the private sector cannot.
Government agencies have the abilities and options to collect massive amounts of data on things that private firms would never spend money on. If it wasn't for the military's work with GPS, the private sector could be years behind what today's mapping apps are capable of.
A startup would never be able to map the globe or launch a satellite for the sake of a navigation app. By giving out government data, the Obama administration has opened the door for clever entrepreneurs to use data without doing the legwork.
Oracle chief executive Larry Ellison once called cloud computing a fad. Well, that fad is now making Oracle money.
According to an IDC study, Oracle software revenues grew over three percent on the back of big data and cloud software. According to the study, Oracle owned a 21 percent share of the application development and deployment (AD&D) software market.
Ellison unveiled Oracle's first IaaS offering last October at OpenWorld. Now, the firm is seeing growth in the software market. Oracle's slow march to cloud software looks to be paying off this year.
However, Oracle still has a while to go before it can truly compete in the evolving world of enterprise software. The firm still lags behind IBM and Microsoft in overall enterprise sales.
Microsoft has been the top dog in the enterprise software world for a long time. Its grasp on the market doesn't seem to be slowing anytime soon either with a 17 percent share of the sector.
However, Oracle can still look to get into the number two spot by leaping over IBM. Big Blue didn't see much growth in enterprise software revenues for 2012. The firm only registered a little under a one percent growth mark year-over-year.
That in comparison to Oracle growth could mean big things for the firms future. If Oracle continues its trend towards growth, and IBM continues to stay consistent, it can take the number two spot.
To do that Oracle would need to continue to bring out cloud software offerings going forward. The world of enterprise is increasingly becoming cloud-centric. IDC reported that the cloud would be a major growth sector moving towards 2015 and that has proven to be coming true.
For Oracle to capitalise on that growth it would need to continue its push towards the ether. Oracle's executives have been hesitant to embrace the cloud on the level of some of its competitors, but it's getting there.
It will be interesting to see if Oracle will attempt to bring out ground-breaking cloud software offerings in the future or if it will stand content to just play catch up with its competitors.
The firm's Q3 earnings were less than impressive. Commentators mentioned that a probably cause of the poor earnings was the firm's failure to offer compelling cloud services.
The products the firm released over the second half of last year were a good start. However, to truly capitalise on the cloud market it will need to do more by stop playing catch up and start being an innovator.
11 Apr 2013
Google just announced its plans to enlist big data in the fight against human trafficking. The search giant will work with three advocacy groups to collect and analyse data from human trafficking hotlines.
The work is aimed to stifle human trafficking by bringing about a shared data platform for anti-trafficking groups. By using big data, advocacy groups can identify trafficking hotspots and create stronger strategies to put an end to traffickers.
Google's work in the field is an illuminating reminder of the types of projects big data can take on. Big data doesn't have to be used just to create the perfect targeted ad or discover the biggest IT bottleneck.
Big data can also be used to solve a variety of the world's ills. The potential big data holds for the greater good can't be underestimated. From being able to project future crime sprees to solving big city traffic jams, big data holds the key to fighting a variety of societal troubles.
That is one of the reasons why the lack of qualified big data analysts is so troubling. We can have all the data in the world but if we don't have qualified analysts it won't mean anything.
Knowledgeable and creative big data scientist will be crucial if the industry ever hopes to create some sort of major social change. The world will need scientists who not only know what they are doing technically, but also have the creativity needed to use data in unique ways.
Last year, Oracle president Mark Hurd made the comment that most big data is "worthless". According to Hurd, 99.9 percent of big data is unusable.
His assessment may hold weight in the sense that most data will not help a business improve its infrastructure. However, the idea that most big data is useless in the greater context of society is off base.
To truly use data to uncover societal truths we need imaginative analysts, who can take seemingly benign data and transform it into real-world solutions.
By now it's become a cliché to say the world needs more Steve Jobs, but it's the truth. Steve Jobs (and the many pioneers of the computing age) took the technology of their time and brought a sense of creative thinking to it.
We need a generation of Steve Jobs. The technology exists to such a point that creative thinking can change the world. Tech like big data can be used to revolutionise how we think about the world's problems.
With creativity and know-how a data analyst can do amazing things. Not just in business, but also for society as a whole.
Now, it's up to clever people to take up an interest in the field. To do that people will need equal parts ingenuity and opportunity. They'll need the opportunity to learn and discover the power of the trade. They'll also need to understand big data is more than just statistics.