Intel makes big data push with launch of Apache Hadoop distribution

Intel has revealed a major push into the big data analytics market, releasing its own version of the Hadoop database and partnering up with a number of analytics firms, including Pentaho and Teradata.

Open source Apache Hadoop is a collection of database software, including a distributed file system that can handle large amounts of unstructured data storage, unlike traditional relational databases, and MapReduce that processes the data. Different versions of such Hadoop databases are referred to as distributions.

The Intel Distribution for Apache Hadoop will offer organisations performance boosts and improved security features on other Hadoop distributions, according to Intel.

Intel AES Instructions in the Xeon processor will provide the Intel Distribution with complete encryption. Intel said this would allow organisations to securely analyse their datasets using the Hadoop Distributed File System without compromising performance.

Meanwhile, Intel claimed the Xeon processor platform would lead to new levels of analytic performance in the Hadoop database.

The addition of the Intel Manager for Apache Hadoop software will also simplify the deployment, configuration and monitoring of new applications for system administrators.

Intel has also developed Active Tuner for Apache Hadoop software that will automatically configure the performance of each application.

Boyd Davis, vice president of the Intel Architecture Group, said Intel would not be trying to lock down customers with the release, keeping with the open source ethos of the Apache Software project.

In a webcast, Davis highlighted the work Intel has been doing to improve the performance of MapReduce and the security of the distributed file system.

"We have the most intimate knowledge of computation performance with our leadership in the Xeon processors, and we are the number one vendor in solid state discs. In combining these technologies we will produce the best results for the Hadoop cluster," he said.

"The solution will be sold with Intel support, services and technology collaborations."

Davis said support would either be offered to customers for 24 hours, seven days a week, or for eight hours, five days a week.

One ‘technology collaboration' Intel announced today was with analytics firm Pentaho.

Through an OEM licensing agreement, Intel will integrate the entire range of software applications in the Pentaho Business Analytics platform into the Intel Hadoop Distribution, including data mining, interactive reporting, analysis, data discovery, dashboards and predictive analytics.

Davy Nys, Pentaho European vice president, said the Intel Hadoop distribution with the Pentaho analytics capabilities will allow IT departments to recognise potential infrastructure problems before they occur.

"Machines generate so much data and that data is captured on a range of hardware and devices. The log files are there but people only generally look at when things go wrong. Many organisations want the data before something happens," said Nys, speaking to V3.

Ben Woo, Neuralytix analyst, said the Intel Distribution for Apache Hadoop announcement shows the chip maker is keen on integrating Hadoop directly into the silicon level.

"Intel know the chip so well so they can truly integrate the Hadoop framework into the depth of the chip. Because there are so many variations of the Intel chip, and alternative chips as well, developers always tend to work with the lowest common denominator. This is Intel's way of ensuring their CPUs are being used," he said.

"Intel has always built microprocessors for high-end processing of data. But what is new for Intel is the extent to which it's moving to touch individual customers. Intel is no longer looking to partner in OEM agreements with computer companies to reach customers. Intel is gently walking the line to get closer to the users' technology."

Meanwhile Helena Schwenk, an analyst for MWD Advisors, said she was not surprised by Intel's big data push.

"In the last 18 months, Intel has been investing in a number of software start-ups that have some foothold in the data warehousing market, so this seems like a logical move in some respects."

"It's interesting from my perspective to see a different kid on the [big data] block. It looks like they are ramping up their efforts particularly on the encryption and administration side."

Mike Davis, principal analyst for MSMD Advisors, said the announcement is further signs of Intel moving into the software market.

"First with McAfee, now with this. Intel doesn't want to be known as just a chipmaker ever again."