LAS VEGAS: Luxury smartphone brand Vertu has used Splunk's operational data analysis services to overhaul Android software testing for its premium handsets.
Known for producing high-end smartphones that come complete with a tailored interface and a concierge service, Vertu needs to ensure the apps on its Android-based handsets are as robust and bug-free as possible before they are shipped to customers with deep-wallets and high expectations.
In an interview with V3 at Splunk's .conf 2015 conference in Las Vegas, Rob Charlton (pictured), Cloud DevOps Architect at Vertu, said the company's developer and testing team opted to use a cloud-hosted version of Splunk Enterprise to carry out operation analytics on its software testing processes.
"The impetus was to try and work out a better way of deciding when is the right time to launch the phone software and when it is of high enough quality," said Charlton.
"In previous products we relied on a combination of gut feeling, instinct and experience, based on the number of areas reported, how far they've come down and the results of some automated testing, to decide now is the right point to launch.
"And it was the experience with our first Android phone that made us realise that we needed to do this in a smarter way as the software is getting ever more complex and the consequence of errors in the layers of software for our customers could be quite serious.
"So we wanted to find a way of being data led to get an actual metric in the software quality to be able to say: ‘when it's got to this level, that's when we launch'."
Charlton explained that Vertu uses Splunk Enterprise to dig out errors in the deeper layers of the smartphone's software, going from the third-party app analysis, through to the underpinning Android software layer and then the Linux-based foundations Android is built upon, down to the Qualcomm driver software layer, which links the phone's components together.
He explained that errors in the deeper levels of an Android-based phone can wreak havoc for its users, so Vertu uses Splunk technology to gain a deeper insight into what is going on within their phone's software during testing processes.
"If those layers go wrong the consequences for the customer are a lot more serious, so the phone could crash, or it could reboot silently, the modem could reboot, which means it could drop a call, or you might not be able to get a signal or the GPS might not work," he said.
"We needed to be able to track who is testing the phone, how many people are testing it, how long have they been testing it for and how many errors or crashes were found.
"Previously we relied on manual methods for trying to assess that, but it doesn't always work properly. For instance, if I give you a phone to test for a month it's not really guaranteed that you're actually going to test it for a month, as people get bored and forget about it.
"So we build a little agent that runs on the phone during the [product development] phase, and that agent will collect various product health metrics, like the amount of time it's been on for, the battery health, disk space, as well as tracking crashes and errors.
"And that agent will upload those metrics to a web service we've got running in our data centre on a regular basis, and we have a Splunk forwarder running on that same instance and that will feed that data into a Splunk cluster that we've got."
Charlton explained that with this process in place Vertu's product managers can use Splunk's dashboard to assess the health of each version of a smartphone's software to decide exactly when it meets the level needed for wide distribution.
Vertu also uses Splunk Alerts, which send an email to the crash analysis team every time an error or crash is detected, meaning software problems are tackled immediately rather than waiting to be reported within a finite window of time.
"This has completely turned around our ability to find and fix crashes, as previously we relied on somebody to come to our teams and say, ‘my phone has just crashed, can you take a look?'," he said.
Charlton said the result of using a more automated and operational data driven analysis process has given Vertu the scope to be more confident in its software releases, which is useful given it sells smartphones costing several to tens of thousands of pounds.
"It gives us a lot more confidence when we're launching and I'm convinced we've found and fixed a lot more crashes in more recent phone releases," he said.
The insight into the testing process also enables Vertu to pick up on the most disruptive flaws so that it can better prioritise software tweaks and fixes.
"We're a small company, we can't afford to have millions of people testing our phones to track everything down so it's very important that we work smart and use tools to help us get the quality level up," he added.
Charlton also told V3 that Vertu has opted to use more cloud services, in order to remove the need for legacy IT infrastructure.
"Over the last five years we've gone through a bit of a digital transformation in Vertu. We started from probably what's quite a common position of having a heterogeneous mix of different data centres, managed services, stuff on premise - we even had some stuff in a nuclear bunker guarded by dogs - and although that was exciting, managing that was quite complex and expensive," he said.
"So we've undergone a few changes since then, migrating and consolidating all of those different services into a single VMware private cloud in 2013."
Following a smooth and almost seamless cloud migration back in January, Charlton noted how the company had moved away from its own IT estates to a cloud-based alternative.
"Over the course of last year, finishing in January this year, we migrated everything to Amazon Web Services. So now all our infrastructure is running in Amazon, which gives us great flexibility, and the ability to scale up or scale down," he noted.
Vertu is yet another company that has pushed its IT into the cloud, joining the likes of Eurostar, British Gas, and other companies featured in V3's CIO Insight section.
Users are told that their non-existent 'iPhoneID' is expiring soon
Expansion of SDK intended to expand Amazon Alexa ecosystem
Locky returns from a prolonged rest with two new variants
AMD lambasted over Radeon RX Vega pricing that will add an extra £100 to RX Vega 56 and 64 graphics cards
Company accused of failing to tell anyone that the launch prices were only introductory offers