Why New Relic
Real User Monitoring and single unified interface
- Problems discovered and fixed before customer is aware of them
- Incremental performance improvements become a reality
- End-user experience and customer satisfaction improved
BlackLine has completed the Statement on Standards for Attestation Engagements No. 16 (SSAE 16) report – an internationally recognized auditing standard developed by the American Institute of Certified Public Accountants (AICPA) which sets guidelines for auditors to use in order to objectively assess the internal controls of service organizations. SSAE 16 effectively replaces the prior Statement on Auditing Standards No. 70 (SAS 70) as the authoritative guidance on service organizations. (BlackLine completed its original SAS 70 report in 2009.)
While undergoing the SSAE 16 audit, BlackLine also successfully completed an International Standard on Assurance Engagements No. 3402 (ISAE 3402) evaluation, showing that the company has the proper controls and processes in place to meet the stringent international audit and accounting reporting standards for service organizations, as set forth by the International Auditing and Assurance Standards Board (IAASB).
BlackLine offers a secure, state-of-the-art, hosted solution and provides all backups, disaster recovery, maintenance, support, etc. Both the BlackLine application and Rackspace data center have gone through successful SSAE 16/ISAE 3402 audits, giving clients even greater peace of mind that their data is safe and secure.
Headquartered in Los Angeles with offices in Atlanta, Chicago, London, New York, Portland, Ore. and Sydney, BlackLine delivers its services to companies around the globe.
Rackspace Hosting, New Relic
“We would not want to troubleshoot anything on the application layer without New Relic. It kick starts the process. We can quickly see if the issue is on the database side or web services side. That simple piece of information gives you a head start and saves you time.”
With a growing business and an expanding global customer base, BlackLine Systems needs reliable high-speed performance to ensure delivery of its SaaS-based month-end close and reconciliation application to clients. Serving companies of all sizes that must adhere to compliancerelated financial deadlines, BlackLine Systems has self-imposed performance and uptime requirements designed to meet customers’ varying needs. However, ensuring superior performance has become more challenging as the BlackLine’s customer base continues to grow rapidly. For the BlackLine app team, understanding application performance and having the ability to quickly find and diagnose performance issues before they affect customers is increasingly important.
Month-end close is critical to all companies and, as the provider of those services, BlackLine must have the uptime and high performance required to ensure its customers meet internal financial reporting requirements every month. A critical part of meeting that requirement is the hosting environment. BlackLine chose Rackspace as its hosting provider, in large part because of the company’s reputation for high quality performance and reliability. Since BlackLine needed a very high level of reliability, the company chose the Rackspace Critical Sites Service, which commits to 100% production platform uptime.
As a Rackspace customer, BlackLIne was eligible for free New Relic, and when New Relic announced Real User Monitoring (RUM) as part of its base offering, the BlackLine team was immediately interested. They knew that the ability to get a complete view of application performance that included the browser response times and end user and application performance was vital to making improvements in their application going forward. They quickly adopted New Relic’s easy-to-use interface and complete end-to-end view of the application to enhance their service offering to their customers.
BlackLine started using New Relic in October of 2011. In no time, they were getting information that simply had not been available to them before. After three months of using New Relic, the team had baseline information on their application’s performance characteristics. The ability to look at twelve weeks of performance history provided insight into the application’s progression over time. For example, the average response time hovered around three seconds, regardless of the load and the app server performance ran at about 250 milliseconds, regardless of the number of transactions. Server load was not overloaded in the present environment. Knowing how their application behaved in the norm gave them a starting point to tackle small incremental performance improvements that could translate directly into improved end-user experience and overall customer satisfaction.
“The BlackLine application performance baseline tells us where to start. Knowing the average response time, that we are not stretching our servers is important to us and helps us measure the impact of our changes,” said Nils Rurack, Senior Web Engineer at BlackLine Systems. “With New Relic, we can verify and prove theories, pick the low hanging fruit and make incremental performance improvements.”
“Our Operations Department utilizes and leverages New Relic to communicate more effectively with other departments making it a lot faster and easier to troubleshoot, isolate and remedy issues for enhanced productivity. We spend significantly less time on performance issues. On balance we save 10-20% time on problems depending on their complexity.”
BlackLine Systems has achieved significant results using New Relic. Since adding New Relic, BlackLine has seen solid improvement with a major positive impact to the tens of thousands of end-users interacting with the BlackLine SaaS application each month..
“BlackLine’s goal is to discover any problems and issues before the customer is aware of them,” added Nils. “We want to eliminate customer frustration and to stay ahead of the curve. We want to know what’s going on in our application before our customers experience any impact. New Relic helps us do that. It heads us in the right direction, helps us prioritize and saves us time. New Relic helps us discover the things to work on to improve our performance, one step at a time.”
“New Relic is a fabulous tool to quickly identify what’s going on. It is so much easier for us to identify and remediate system issues in a timely manner,” added Ed Ramon, Director of Infrastructure Operations at BlackLine. New Relic’s alerts are a major factor in achieving this goal. The team has found that their Apdex score and New Relic alerts operate like an early warning system as they provide key information in just a few hours so the team can see if there are any unexpected performance changes. If their Apdex score drops below the acceptable threshold, the team quickly sees and troubleshoots the issue. This is critical when the team makes changes to the application. “It’s vital to look at everything before and after the change,” commented Nils. “Having seven days of data prior to the change is terrific as it helps us pinpoint exactly what and where the issue is. It’s much easier to correlate a change to any positive or negative performance impact.”
Using New Relic, BlackLine has made even more incremental performance improvements over time. They’ve been able to identify and remedy long running SQL queries, add some new indexes, etc. “We’ve definitely improved the average page load time,” said Nils.
“Our Operations Department utilizes and leverages New Relic to communicate more effectively with other departments making it a lot faster and easier to troubleshoot, isolate and remedy issues for enhanced productivity,” Ed added. “We spend significantly less time on performance issues. On balance we save 10-20% time on problems depending on their complexity.”
As they move to a more agile deployment model with many more deployments, the BlackLine team sees New Relic as essential to make these more rapid product evolutions work successfully. As Nils said, “We would not want to troubleshoot anything on the application layer without New Relic. It kick starts the process. We can quickly see if the issue is on the database side or web service side. That simple piece of information gives you a head start and saves you time.”