Realising opportunities from data is all about thinking like a small business, according to Thomas Lee-Warren who set up the Technology Data Group at Royal Mail.
The group comprises data scientists, developers, software engineers and visualisation experts, and aims to take business problems, model them in code and analyse them for possible solutions.
"I'm interested in bringing the best of small company culture to large corporations and vice versa," Lee-Warren said.
"The ability to understand and solve problems quickly is very important. The guys and girls can take a problem, cut code, make models, knock up some visualisations and return and say: 'This is what we've done.'
"They've managed to squash that process down to days or a couple of weeks, and that's revolutionary for a business that is used to tasking IT with something and when they come back they've got grey hair."
Royal Mail uses Hortonworks Hadoop HDP as a central data lake, a flat repository that takes all sorts of data from multiple sources. The firm deploys the Apache Nifi real-time data flow solution, now rebadged as Hortonworks DataFlow, to ingest data rapidly from the many silos across the business.
"The combined data from two sources has much more value than two sources on their own," Lee-Warren said.
Having the data located centrally enables the data scientists to come up with fresh insights, and simplifies compliance with the many regulations that govern data flows and access in large international businesses.
"You can better understand the data. You've got it all together and you've got the lineage. You know where the data came from and how many copies you have, whereas before it was siloed. You can introduce much more disciplined processes around it when it's centralised," he explained.
Hortonworks has recently sought to unify security and governance issues by integrating Apache Atlas and Apache Ranger in its Hadoop distribution. This was in response to demands from large customers, including Royal Mail.
Lee-Warren said that the board has been very enthusiastic about the data innovations introduced by the Technology Data Group, which provides a list of possible interventions that the directors then prioritise.
Those chosen so far have been mostly about adding value by improving the customer experience rather than optimising business processes.
Projects included building customer churn models and predicting the volumes of letters and parcels expected to arrive at particular sorting offices. This is an example of predictive analytics that will become increasingly important.
"It's about how we balance our workforce with our volumes in an increasingly competitive environment," said Lee-Warren.
IBM and Technical University of Munich team demonstrate how Shor's algorithm, which can't be cracked by conventional computers, can be solved quickly with quantum computing
Hubble Space Telescope finds superflares from young red dwarfs could strip away planetary atmosphere
Younger stars are 100 to 1,000 times more energetic than when they're older
Two of the big four supermarkets will use the system to control sales of restricted products
PUBG news and updates: November's Update #23 to bring new Skorpion pistol and changes to blue zone visibility
Genuinely useful side-arm coming to PUBG in Update #23