How Statista cut latency by 95% with New Relic’s free tier

When I joined Statista in January 2023 as director of infrastructure & technology, our technology was the opposite of cutting edge: Many of our services were built on legacy systems, including websites built on WordPress, most of which were improvised or poorly maintained. Instances were configured by hand without automation and backups, so when an instance failed, we had to go through the painstaking process of fixing it manually. 

I had a mandate: urgently address our immediate challenges and formulate long-term plans to modernize our technology stack, ultimately enhancing customer experiences. Recognizing the need for substantial changes, I approached the situation with a positive hypothesis. To validate and measure our infrastructure and application performance, I initiated a proof of concept project using New Relic’s free tier. This exploration aims to determine if adopting New Relic as our long-term monitoring solution is not only feasible but also beneficial.

First steps with New Relic

Upon joining Statista, we were using Amazon Web Services (AWS) on a very limited scale, while much of our architecture still lived on physical servers outside of AWS. I introduced New Relic to our environment by instrumenting our AWS stack, and I was surprised by how quick and straightforward the process was.

My goal was to use the free tier to demonstrate value to our company’s leadership and make the case for an expanded observability deployment. The justification was easy to understand: our monthly bill with New Relic is equivalent to four developer hours while delivering massive savings in money and resources. We can instrument our entire tech stack—AWS, physical, and virtual servers—simplifying the process of identifying bugs and anomalies.

Demonstrating value and expanding observability deployment

Before implementing New Relic, our WordPress instances experienced sluggish performance due to accumulated plugins, unrefactored websites, and underlying bugs. New Relic played a crucial role in identifying and addressing these issues, enabling us to optimize hosting configurations. The enhancements facilitated the adoption of auto-scaling on our previously challenging legacy tech stack without unnecessary expenses on oversized instances.

While some companies focus on cost savings in tech stack optimization, Statista prioritized enhancing the customer experience. We aimed to improve performance without increasing spending, with New Relic efficiently enhancing instance performance and a migration plan to replace physical servers with AWS instances for significant long-term cost reductions.

Massive latency reductions with limited spending

When it came to improving our customer experience, I knew we had to reduce latency. We were able to introduce new caching mechanisms and identify which sites and pages are being used, and we used New Relic to identify the root causes of most of our latency. We discovered some N+1 problems that were resulting in a dramatic increase in database calls. This unnecessary volume of database calls meant that our services were much slower than they needed to be. We identified malfunctioning plugins and replaced them with modern, efficient solutions. Additionally, we removed many plugins that were not contributing significantly.

Once we solved the N+1 problems and replaced the problematic plugins, we saw latency drop dramatically. Pages that previously took 2 seconds are now being served in just 100 milliseconds—that’s an improvement of 95%. The difference in user experience is night and day: when customers browse the website, they experience a positive look and feel.

We didn’t have to invest tremendous amounts of time and money to achieve these improvements. It was simply a matter of using a modern observability tool—New Relic—to identify how to optimize our infrastructure and applications. Now that we’ve cleaned up some of the persistent problems, we can begin focusing on how to build a better tech stack for the future.

Shifting left with observability

We are on an exciting journey to modernize our tech stack and enhance our customer experience. Currently, we are strategically planning our transition from physical servers to enable scalable instances on demand, thereby reducing operational expenses. Our focus is on making all applications cloud-ready before migrating to AWS. Upon achieving these milestones, scaling our infrastructure will become significantly more streamlined.

New Relic is going to play a key role in our migration. We’ll use New Relic to baseline performance and make a before and after comparison, and we want to integrate observability from the first moment in our development cycle. Over time, we want to take advantage of custom events and synthetic monitoring so we understand what’s going on in the system and get notified as soon as things are out of the ordinary. I expect that this transition will cut down the time we spend on development and QA significantly.

We are currently in the initial phases of our journey. By actively hiring developers and project managers, adopting DevOps practices, and modernizing our toolset, we are making significant progress. The introduction of the New Relic free tier has been a pivotal moment for Statista, enabling us to identify and address some of our most persistent challenges. As our observability practice continues to evolve, it will guide us towards enhanced performance and an even better user experience.

Easy to get started: Check out our launch template

If you want to see how easy it was for us to get started, check out the example of our launch template below.

#!/bin/bash

# copy the correct configs to /etc

# setup New Relic Infrastructure Agent
echo "license_key: eu01xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxNRAL" | sudo tee -a /etc/newrelic-infra.yml

# setup New Relic PHP Agent
sudo NR_INSTALL_SILENT=1 newrelic-install install

# setup New Relic Logs Integration
curl -Ls https://download.newrelic.com/install/newrelic-cli/scripts/install.sh | bash && sudo NEW_RELIC_API_KEY=NRAK-XXXXXXXXXXXXXXXXXXXXXXXXXXX NEW_RELIC_ACCOUNT_ID=1234567 NEW_RELIC_REGION=EU /usr/local/bin/newrelic install -n logs-integration -y

# setup New Relic NginX Integration
curl -Ls https://download.newrelic.com/install/newrelic-cli/scripts/install.sh | bash && sudo NEW_RELIC_API_KEY=NRAK-YYYYYYYYYYYYYYYYYYYYYYYYYYY NEW_RELIC_ACCOUNT_ID=1234567 NEW_RELIC_REGION=EU /usr/local/bin/newrelic install -n nginx-open-source-integration -y

# setup New Relic Memcached Integration
curl -Ls https://download.newrelic.com/install/newrelic-cli/scripts/install.sh | bash && sudo NEW_RELIC_API_KEY=NRAK-ZZZZZZZZZZZZZZZZZZZZZZZZZZZ NEW_RELIC_ACCOUNT_ID=1234567 NEW_RELIC_REGION=EU /usr/local/bin/newrelic install -n memcached-open-source-integration -y

# restart services
sudo systemctl restart memcached nginx php-fpm newrelic-infra