Few companies in the world have quite the same combination of challenges as Ticketmaster. The company has to meet sharp spikes in demand while shutting out touts, coordinate that across 21 countries, and consolidate a sprawling IT estate established over a period of 40 years.
This is partly why the company embarked on a shift to DevOps methodologies 18 months ago, according to Simon Tarry, engineering lead at Ticketmaster.
The company maintains a wide array of systems largely geared to ticketing and meeting spikes in demand for popular events. This IT estate has been fuelled by a combination of organic growth and overseas expansion.
"Ticketmaster was one of the original tech leaders in its field, and grew into a global giant simply by having some of the best ticketing technology. We still have mainframe platforms. You might not believe it, but we do," said Tarry at Computing's DevOps Summit 2016 this week.
"We've expanded across the globe, so we now have a ton of different technology platforms selling tickets on a whole range of different stacks."
And these days it's not just about selling tickets, but about connecting clients into the company's systems. Hence, if Madonna is on a world tour, her management needs to tap into Ticketmaster's systems wherever they are. These systems (and what's required to log-in to them) are currently different from one country to the next.
It was against this backdrop that the company started a shift towards DevOps as part of a bid to better manage its IT estate and an army of 200 or so developers across five locations worldwide.
"It's really high-traffic, high-intensity e-commerce. People expect to be able to buy their tickets from anywhere, and then there's the whole social aspect that we have to cater for," said Tarry.
In addition to dealing efficiently with fans keen to buy tickets for tours as soon as they are put on sale, the company has to prevent tickets being scooped up by high-tech touts.
"When you're buying tickets, it's not just other people that you are competing with, but automated bots hoovering up tickets and trying to resell them," said Tarry.
"So one of the big challenges is trying to figure out whether you are a real person or not. We're putting a lot of R&D into that side of things."
Ticketmaster embarked on its journey towards DevOps in early 2015, but one of the early lessons Tarry picked up was the importance of communication, not just from the top down but between team members. An important prerequisite, he believes, is a soft skills programme.
"We started about a year and a half ago. We had all these teams across different locations and we wanted to be able to release our products far more quickly than we were," explained Tarry.
"We have about 15 different platforms, most of which are selling tickets, which is a ridiculous waste, and we are trying to consolidate them. But what we really wanted to do was try to get stuff out the door more quickly."
Goals included delivering business value, and improving the efficiency of the IT team and the overall quality of development.
"That was around automated testing and getting the continuous integration pipelines working. The general reliability is all about our monitoring pipelines, health checks and alerting," said Tarry.
Perhaps most important, however, was delivering the service. "That was the culture ‘piece' about developers owning their code in production," said Tarry.
"So we came up with a number of maturity matrices with five different levels of maturity for all of those areas. That enabled each team to figure out where they were on this matrix and where they wanted to get to."
In other words, the matrices provided a mapping tool so that each team could see how far along the journey to full DevOps they had travelled and what else they needed to do.
Kicking Palantir off of AWS is among their demands, too
Rafaela Vasquez was watching The Voice at the time of the crash, new evidence shows
PUBG price slashed on Steam after selling more than 50 million copies - as daily player numbers plunge
Use the same password for every website? It might be time to change them all