The maker and broadcaster of TV hits like "Coronation Street, Downton Abbey" and "I’m A Celebrity" is ITV. Advertising generates £1.7 billion in annual revenue. Andrew Duncan, principal engineer of ITV, works with the Airtime Sales (ATS) team using technology to improve the management of this vital source of revenue.
"Technology underpins what we do, and it must be able to work at scale," he explains. "It must allow us to respond very quickly, especially as our business becomes more and more digital. And that means we need to use more technology and automation to support the operational side of things and augment our traditional human skills and capabilities."
In alignment with how ITV relies on an end-to-end supply chain that is increasingly digital, the ATS organization recently completed the transition to a new software platform for managing advertising sales. Key to the success of this complex and challenging migration was full observability with New Relic, which is now a key element of how ATS is growing as a data-driven digital business.
Starting a modernization program
ITV had a five-year modernization program for the ATS department, which has been challenging because of the broadcaster’s long history. The company is the amalgamation of 12 regional broadcasters, each of which had its own advertising sales teams, systems, and packages.
"There’s a lot of legacy in the systems we use, and a great deal of complexity around how ads are still sold regionally," says Duncan. "We’ve spent a great amount of time and effort in modernizing, streamlining, and tackling all the technical debt. The end goal is ensuring we are set up for the future with more integrated advertising across broadcast and online, and being more reactive to what happens in the market." Duncan and his team decided on a digital transformation plan to migrate from Java-based applications running in on-premises servers to a software management platform that utilizes decomposed microservices running in the Amazon Web Services (AWS) public cloud.
Yet, even as the software was fully ready to launch, Duncan’s team faced end-user resistance to the change-over. To resolve this, New Relic was chosen to help complete the most important stage of the digital transformation journey.
"With some irony for a TV drama maker, there was the potential for the drama of the migration to overtake us," he says. "Many people in ATS had visions of that £1.7 billion revenue being put at risk by technical problems as we moved from the old to the new. There was a real need to build trust, given that the original proof of concept was five years old, and the migration had been postponed before. This is where New Relic and its ability to allow us to instrument everything proved so crucial."
"By being able to trace across all of those systems and then very quickly tie down where our performance bottlenecks are with New Relic, we can get to the root of what is actually involved in serving a user request and fix it," says Duncan.
Smooth migration to the cloud
The first critical test for using New Relic was one very tense weekend when all 300 users and air time sales activity would be moved over to the new platform. There was a real concern among the ad sales teams, who need those systems to be uninterrupted because they often sell ad spots only hours before transmission.
To reassure the end users, the team instrumented New Relic for application and infrastructure monitoring, as well as for the user interface in the browser so they could access end-user experience in real time. This gave full end-to-end observability of how the new applications were performing and allowed the team to rapidly pinpoint problems. New Relic was especially useful for observing how the multiple systems that made up the new platform were constantly interacting with each other.
"Having instrumented everything with New Relic, we built dashboards that would focus purely on the go-live weekend," says Duncan. "The dashboards were designed to answer the most important things our business owners needed to know. We shared those New Relic dashboards with everyone in the ATS department and ran them on giant TVs all around the office."
Everyone could see how the migration was performing with no hidden secrets or problems. As Duncan recalls, there was even a countdown when a legacy piece of software used to manage ad inventory was turned off, and the new applications came online. The ability to communicate so clearly using New Relic was a major factor in how the whole process proved relatively trouble-free. New Relic enabled the team to ensure certainty of performance and to show the business that the migration to the new platform and the move to the public cloud wasn’t damaging performance.
As Duncan says, "New Relic was capturing what we said was happening and there were no big dramas. The dashboards showed it was going very well and that was what got us across the line. Without being able to build that trust through that visibility and observability of the software, I don’t think we would have got through the launch. It also meant my team could finish earlier than planned."
Thanks to New Relic, we’re now communicating much more and working towards a shared goal. There’s a much better understanding of what we're trying to do in the organization and how technology can be adapted to meet the challenges of a rapidly changing marketplace.
Managing a billion-plus ad business
The ATS team had spent five years building toward this single moment. So, what came next? The same end-to-end observability that proved critical during migration continued to be integral in running a digitally transformed operation. With New Relic’s support and guidance, the team embraced the disciplines of site reliability engineering to create scalable and highly reliable software systems for ATS. After the go-live, New Relic was used to build up a baseline of application and infrastructure performance to help proactively observe what was happening.
Dashboards for business owners are also driving better internal conversations and collaboration about business application development and management. Says Duncan, "In the program’s last months there had been a lot of tension, and it was very much a culture of 'us versus them.' The dashboards created a common language that broke down barriers, and enabled us to discuss together what the business wants from software."
As New Relic has promoted familiarity with data about software performance, there have been interesting changes within the organization. "We had given the business owners New Relic accounts over the go-live weekend so they could look at dashboards and run basic queries," Duncan says. "We didn’t disable the accounts, so they kept using them and even started learning more through New Relic University training. This is leading the business owners to experiment with using New Relic data to better understand and improve business processes."
Both ATS business managers and the technology team are now using their New Relic dashboards to answer queries about how their teams are selling ad spots, and how efficiently ITV is selling its airtime. The level of observability allows the technology team to spot patterns in how advertising management processes can be improved.
"What excites me about New Relic is I can see what my users are doing, and I can understand why they‘re trying to do it. Understanding what to create is the hardest part of software development. Now we’ve got this insight that lets us see what people are doing in real-time," says Duncan.
As New Relic increasingly helps ATS manage its advertising business in addition to its software operations, the team is looking at how New Relic can log more business objects about campaigns, airtime, and ad spots directly.
"Software modernization is never a finished process," explains Duncan. "What we’ve learned is that communication, visibility, and observability are absolutely key to its success. All the fear and distrust that had built up stemmed from a lack of communication between the business and the technology side of the department. Thanks to New Relic, we’re now communicating much more and working towards a shared goal. There’s a much better understanding of what we're trying to do in the organization and how technology can be adapted to meet the challenges of a rapidly changing marketplace."
