What the ANZ App does to monitor its users in real time

At ANZ Bank, we’ve leaned into our use of New Relic to create the best possible user experience for our customers. Our ANZ App is used by four million personal and business account holders across Australia.

We’ve seen many iterations of this app. We were among the first to release a native iOS app back in 2008 for our New Zealand market, and then we had the release of the ANZ goMoney App in 2011 for Australia and New Zealand. The current iteration — the ANZ App — was introduced in February 2018, and we are continually innovating and evolving to improve our customer experience. In this digital age, services are always just a tap away, and we have a responsibility to live up to that customer expectation.

Observability across the software lifecycle

Observability is a crucial part of effective technology rollouts, which is why we have New Relic embedded into all aspects of our software lifecycle. From integrated testing environments to quality assurance and PNV (Performance & Volume Testing), observability plays a big part in our production releases. This end-to-end involvement ensures that our pipeline functions like a well-oiled machine, offering optimal performance and excellent service to our banking customers.

To ensure that our technology is highly performant, we hold ourselves accountable to strict performance and service standards:

1. Our systems are responsive and always meet or exceed defined response times. 
2. Our app and its underlying infrastructure stack are consistently available. 
3. Error rates are always maintained within baseline thresholds.

New Relic tracks these metrics against our established benchmarks, meaning that our services align with established standards. We have also started down the path of looking at New Relic service level capabilities, and turning Service Level Agreements into objectives and indicators. The optimal performance of the ANZ App is about being responsive, reliable, and providing an error-free experience, as outlined in our non-functional requirements which are monitored through New Relic.

When we develop microservices, New Relic helps us to identify, troubleshoot, and fix any bugs early in the lifecycle, while also supporting changes in app deployment, and vetting performance issues or regression. In production, we rely on New Relic day in and day out to keep everyone informed of how things are performing and functioning.

Monitoring mobile in real time

Real user monitoring on mobile is critical to all of our processes; enabling us to gain actionable insights into user engagement on the ANZ App. It also supports us to quickly address any potential pain points that may fly under the radar. With real user monitoring on our dashboards, we can spotlight customer pain points that often evade detection, and be empowered to offer seamless digital banking experiences that avoid performance degradations.

Real user monitoring utilises the New Relic mobile agent to collect telemetry data from each mobile device running the app. This enables us to gain visibility into the app's performance and detect issues or errors early in the user experience, even for issues that happen outside of our infrastructure stack.

This allows us to deploy new mobile builds with confidence, and understand how each version is performing while being able to roll back to previous iterations if necessary. We have seen the benefit of this for an issue that happened on our Web Application Firewall (WAF) layer, where users were unable to login to the ANZ App.

Simplifying alerts and dashboards

New Relic dashboards and alerts play a pivotal role in our operations by enabling us to swiftly detect and resolve problems. Through dashboards and alerts that show the user journey in real time, we can start drilling down to focus on specific areas and pinpoint issues based on how they affect our customers rather than just the state of the technology. When new features are released, we can see how these changes impact our customers immediately, such as detecting unexpected errors or behaviours, which helps us retain service stability and user satisfaction.

Dashboards provide an easy way to look at baselines and do comparisons that we rely on for changes in our application and infrastructure. Our operations and mobile development team have also benefited from this, by being able to compare the same sets of data across teams. 

One key example was a .NET framework upgrade for both our infrastructure and application stack. Infrastructure agents and APM agent stats displayed through these dashboards showed a deviation in CPU utilisation, memory usage, response time, latency and other key metrics we use to monitor the health of our application. As a result, we were able to view the data we receive from real user monitoring as well as data collected from microservices to accurately see where the root cause was, which created much greater visibility.

Creating a continuous feedback loop with engineers and customers

The success of the ANZ App largely depends upon listening and responding to user feedback. We have a Slack channel that is continuously updated thanks to New Relic alerts and sits at the core of our operations. We read and discuss every piece of feedback that’s submitted by users.

New Relic real user monitoring and APM logs feed into these channels when there has been a deviation to APM or real user monitoring and provide a valuable connection between feedback and potential technical issues. This creates a continuous feedback-loop mechanism that helps us prioritise improvements while keeping customer satisfaction at the heart of everything we do.

The integration of New Relic into our development, testing, and deployment processes has substantially streamlined the work behind the ANZ App. It has not only improved our service stability and efficiency, but has equipped us to promptly address issues before they hurt our business. The unified view of our service that we get from New Relic gives us full stack observability in a single platform, making it easier to deliver new innovations with the level of service our customers expect.