How 10x Banking gains confidence to perform zero-impact deployments

Banking is one of the most fundamental services we use, but it has lagged behind other consumer-facing industries in adapting to new technologies and placing the customer at the center of operations. 

At 10x Banking, our cloud-native core banking platform enables banks to understand their customers in real-time, release products in minutes, unlock data, and deliver connected experiences. Our core banking infrastructure is built on Amazon Web Services (AWS), powering some of the world’s biggest banks and supporting millions of customers. Our goal is to continue scaling the platform across a growing client base while consistently improving features and minimizing disruption. Continuous delivery—the ability to deliver software in short, successive cycles—ensures that we’re always delivering the best and most reliable experience to our customers and their end users, which is an absolute necessity in the banking industry.

In my role as release manager, I’m responsible for the success and reliability of our deployments; it’s my job to reduce error rates and mean time to resolution (MTTR) in order to improve the consistency and reliability of our software. After joining Banking 10x, I recognized the importance of observability and monitoring not only to understand how our technology is performing on a broader scale but also to zoom in on the immediate impact of any deployment or update. This is why we began using New Relic for deployment markers and change tracking.

Here’s how our release processes evolved over our first year with deployment markers:

Consolidating tools and data sources

Before New Relic, we were using multiple tools to monitor and collect data from different areas of our tech stack. This tool sprawl meant that we weren’t able to resolve issues quickly. Not only did we have a hard time identifying the root cause of an error, but we also lacked the detailed information that would help us solve the problem. 

Our mission as a company is to drive the banking industry forward through technology innovation. But the absolute priority for any bank is uptime. An outage represents a significant breach of trust for the end user because they won’t have access to their money when they need it. Achieving a high-level view of our architecture with observability meant that we would be able to address anomalies more quickly and resolve issues before they become outages. This in turn helps us move towards continuous delivery: when major emergencies instead become minor issues, the team can focus on consistency rather than disruption. 

Once our tools and telemetry data were consolidated on New Relic, we started using change tracking to see the impact of deployments and configuration changes. The impact is huge. If we see a spike in error rates, we can drill down on the root cause and fix the problem before it ever reaches the customer. New Relic is particularly helpful for us because we can tailor which areas of the architecture our clients can see and which they can’t—our clients are responsible for the front end, while we just need to make sure the back end is running smoothly. 

Shifting toward continuous delivery

We have to identify anomalies and reduce MTTR, but our mission as a company is to build innovative software. New Relic dashboards give us a complete picture of our software lifecycle, which enables us to make an informed "go" vs. "no-go" decision on each deployment. These dashboards display deployment data across multiple environments at the same time. This function was essential for getting stakeholders across the company on the same page. Everyone in the company has a unified view of how each deployment impacts various elements of the environment. 

We’re making several deployments per day; deployment markers give us confidence in each release and allow us to quickly identify the source of an error. The New Relic integration with Jenkins makes our lives easier too. The OpenTelemetry plug-in displays jobs and pipeline executions as distributed traces, which allows us to easily monitor our deployment data and make improvements to our pipelines as a result. This visibility—and our shift in general towards continuous delivery—has made my life easier as a release manager. 

Release management can be defined by chaos or it can be defined by confidence. The data we collect from New Relic allow us to make sense of the chaos and understand performance in real time. In a fast-paced environment focused on continuous delivery, these real-time metrics provide us with a lens through which we can understand the system as a whole—and identify the services and APIs that need improvement. At any moment in time, we can see the state of a release and proactively resolve any issue that may impact the customer experience.