Microsoft has announced the pubic preview of Apache Spark for Azure HDInsight (HDI), expanding its cloud-based offerings for big data crunching with Hadoop.
The addition of Spark in Azure HDI enables the open source framework to run large-scale big data analysis with in-memory databases.
This means that the high-speed queries that Spark facilitates during big data analysis can be delivered as software-as-a-service through Microsoft's Azure platform.
Companies looking to use Spark for crunching data have the flexibility to scale the number of compute nodes up or down running in a Spark cluster, rather than needing to manage resources on-premise or in a private data centre.
Businesses using Azure HDI also benefit from storing large amounts of data in the Azure cloud rather than in Spark clusters.
This reduces the cost of running such systems as there is no need to keep compute power in a cluster continuously up and running.
Microsoft is also offering 99 percent uptime guaranteed through service level agreements.
Oliver Chiu, product marketing manager of big data and data warehousing at Microsoft, said in a Microsoft blog post that adding Spark capabilities into Azure HDI expands the company's big data services offered through Azure.
"HDI eliminates much of the heavy lifting associated with deploying, managing and executing tasks on Spark, thus raising the bar on what it means to process big data in the cloud," he said.
By tapping into the in-memory capabilities of Spark, Microsoft is touting Spark for Azure HDI as a service to carry out streaming analytics for data harvested from Internet of Things networks, as well as working with data in real time.
Spark for Azure HDI can also be used with the data analytics visualisation tools supported by Microsoft, which make digging through data a much more visual process for analysts looking to glean insightful information from huge datasets.
Power BI is Microsoft's own data visualisation tool, which will move out of preview status to general availability on 24 July. Users can connect Power BI with Spark to get big data reports in a visual rather than numerical form.
Spark for Azure HDI also supports third-party visualisation and business intelligence tools from vendors such as Tableau Software, SAP and Qlik.
Big data is becoming a major focus point for Microsoft, as seen with the firm's expansion of big data choices for customers with the addition of more Hadoop offerings in Azure from MapR and Hortonworks.
Russian Taiga smartphone promises snoop-proof communications - coming soon to employees of Russian state-owned firms
Eugene Kaspersky's ex outs smartphone that claims to prevent apps from spying on users
Deloitte accused of leaving its internal Active Directory server exposed to the internet with RDP open
Deloitte accused of lax systems administration and security practices over email hack
Lax systems administration practices blamed for exposing millions of sensitive client emails
The new processors support Intel's Optane memory acceleration technology