The Twelve-Factor App: What It Is and How to Monitor It

I started my IT career as a Java developer, but I haven’t done full-time coding for almost eight years. Truthfully, I’ve done a poor job keeping up with new trends in development and the Software Development Life Cycle (SDLC). Lately, though, I’ve been thinking a lot about the Twelve-Factor App, which is a set of simple, powerful guidelines for building Software-as-a-Service (SaaS) apps with the modern tools available today. As the introduction states, this methodology is for creating apps that:

• Use declarative formats for setup automation, to minimize time and cost for new developers joining the project;
• Have a clean contract with the underlying operating system, offering maximum portability between execution environments;
• Are suitable for deployment on modern cloud platforms, obviating the need for servers and systems administration;
• Minimize divergence between development and production, enabling continuous deployment for maximum agility;
• And can scale up without significant changes to tooling, architecture, or development practices.

If you haven’t already read through all 12 factors, I encourage you to do so before reading further. In this post, I walk through each of the 12 factors and explain how to use New Relic to achieve the goals of each one.

1. An app is built from one version-controlled codebase

In my early days as a software developer, we didn’t use source code versioning at all—we manually synced code between developers, dropped code in a common folder, and hoped for the best. Setting up IDEs was a challenge, application packaging/deployment was a wild “artform,” and a lot of things could (and did) go wrong.

In a Twelve-Factor App, you work from a single codebase, which is tracked in a version control system (such as GitHub or Subversion) and use multiple deployment environments (staging, dev, production, etc.). Look at it this way: If you have multiple codebases, you’ve got a distributed system, not an application.

How New Relic can help with this factor:

• Use deployment markers or record deployments to capture an application’s deployments and versions.
• Use custom variables to inject version numbers into the agent’s configuration , so you can track specific versions of your app in New Relic Insights.
• Check out this Git to Insights prototype I created that allows you to track some GitHub data in Insights.

2. An app explicitly declares its dependencies

This is another amazing benefit of modern software languages and frameworks. Gone are the days of hunting jars/dependencies to drop into a project to try to compile it. And gone are the days of dependency hell in large projects when components needed different versions of the same package. In a Twelve-Factor App, you guarantee a deterministic build process, so the app can always be built from scratch and dependencies will never be missed or collide internally.

How New Relic can help with this factor:

• You can track jars and dependencies loaded by JVMs from New Relic APM’s Environment snapshot page.
• Be sure to include New Relic APM agents in your package and dependency managers to ease management of monitoring configurations.

3. Store an app’s configuration in environment variables

An application’s configuration is likely to vary from environment to environment. For example, various versions of an app’s code base could be deployed to development, testing, staging, or production environments.

The Twelve-Factor App calls for “strict separation of config from code,” and suggests that we store application configuration in environment variables.

How New Relic can help with this factor:

• Capture environment variables as custom attributes and send them to Insights.
• Use environment variables to define specific environment-dependent agent configuration. Refer to the relevant agent documentation for details on configuration options (for example, see environment variables for the Java agent).
• Create labels using environment variables for application grouping/alerting.

4. Backing services are resources

Most apps interact via API with any number of services as part of their standard operation; backing services include local or third-party services such as databases, message queues, and cache services. “A backing service,” according to the Twelve-Factor App, “is any service the app consumes over the network as part of its normal operation.”

“The code for a twelve-factor app makes no distinction between local and third-party services,” says the Twelve-Factor App. “To the app, both are attached resources, accessed via a URL or other locator/credentials stored in the config.”

How New Relic can help with this factor:

• You can monitor your app’s backing services right out of the box with our agents, particularly with New Relic Infrastructure.
• You can also use service maps to visualize your app’s topology.

5. Separate the stages of your app

In the Twelve-Factor App, there are three main stages: build, release, run.

In the build stage, you compile all your code from your version control repo and package that version of the app. The release stage is where you combine the build and the app’s configuration in preparation to deploy it live. Finally, in the run stage, you’ve fully executed the app and have created a new release.

The Twelve-Factor App states that “every release should always have a unique release ID, such as a timestamp of the release (for example, 2011-04-06-20:32:17) or an incrementing number (such as v100).” Any new changes to the build (for example, bug fixes) require a new release.

How New Relic can help with this factor:

• As we noted for factor #1, use deployment markers or record deployments to capture an application’s deployments and versions.
• Similarly, use custom attributes in New Relic agent configurations to inject version numbers, so you can track specific versions of your app in Insights. Custom attributes can also help identify and track performance in Blue/Green Deployments.

6. Stateless processes are essential

This is a key factor for creating flexible, scalable cloud apps. You deploy the app into the execution environment as one or more processes, and the processes are stateless and share nothing. “Any data that needs to persist,” according to the Twelve-Factor App, “must be stored in a stateful backing service, typically a database.”

Even session state data (think “sticky sessions”) should be stored in a tool that offers time-based expiration, the methodology says, typically in a datastore like Memcached or Redis.

I remember dealing with sticky sessions and load balancing … it now seems like so much wasted time. Stateless-ness is the backbone of scaling, resiliency, and redundancy. If your app components are stateless, you can multiply your resources to respond to load “automagically.”

How New Relic can help with this factor:

• Monitor your backend and caching platforms that make sharing state data possible across application instances with New Relic Infrastructure.
• You can create dashboards that compare end-user performance vs. load vs. number of instances to evaluate how effectively your application scales when necessary.
• If your application auto-scales, keep watch over how often it is doing so, and how much it’s costing you. New Relic’s AWS Billing integrations can help you track the real costs of your scalable application.

7. Use port bindings instead of packaging a web server

A Twelve-Factor web app does not include a web server, and instead ships its services over HTTP. The app does this by binding to a port to export and receive requests. This ensures that the app is completely self-contained and doesn’t rely on a specific web server.

How New Relic can help with this factor:

• You can monitor an application’s host:port usage (as in this example from a Java app).
• With Insights and NRQL query alerts, you can visualize load and distribution across instances:ports.

8. Run processes concurrently

In a Twelve-Factor App, different types of work are handled by different processes: “For example, HTTP requests may be handled by a web process, and long-running background tasks handled by a worker process … [and] this does not exclude individual processes from handling their own internal multiplexing.” In this model, scale represents your number of concurrently running processes, and workload is a count of the number of running process types.

Essentially, a Twelve-Factor App must be able to scale horizontally, and if you’re using stateless processes, such scale should be achieved seamlessly.

How New Relic can help with this factor:

• If you use modern, elastic apps that scale horizontally, consider New Relic’s compute unit (CU) pricing model.
• Use New Relic Infrastructure integrations to better track resource usage. Additionally, you can use New Relic APM and Insights to visualize the load and number of process and instances your app is using at any given time.
• You can measure application, resource, and code efficiency with New Relic’s capacity analysis report. This can also help you make critical decisions about fixes and code improvement initiatives.

9. Processes are disposable

Long before we started developing apps to run in the cloud, I had application servers that took 10 minutes (or more!) to be available in production. If a production server fell out of rotation, our system faced considerable risk. To aggravate the situation, major components of the applications were stateful and kept (in memory) user information and state, so any server or process shutdown—graceful or not—caused disruptions for our users.

A Twelve-Factor App’s processes are disposable: “they can be started or stopped at a moment's notice.” This facilitates fast and elastic scaling, rapid deployment of code or config changes, and robust production deployments.

From the time they start, processes should be able to run jobs as quickly as possible. Similarly, in the case they receive something like a SIGTERM signal, they should be able to gracefully shut down (i.e., finish any active requests before ending).

This disposability of processes is essential for applications that use auto-scaling (such as containerized apps orchestrated by Kubernetes). Today’s containerized world shows disposability in action. By looking at how our customers are running applications on Docker, New Relic learned that most containers have an average lifespan of less than an hour.

How New Relic can help with this factor:

• As with factor #8, you can use New Relic Insights to track instances and processes as they’re created and killed. Tracking this along with load can help you gauge application efficiency.
• You can also use this data to assess the auto-scale settings: Is the app provisioning resources at the right time? Is it causing you to spend more money than you need to?

10. Wherever possible, keep dev and prod deploys in sync

Back in the day, how many times did I make live edits to a local deploy of the app without keeping those changes in sync with production? Let’s put it this way: If you wanted to call this a software crime, I’d have to plead “guilty as charged.”

According to the Twelve-Factor App, the lack of parity can manifest itself as three types of gaps: time, personal, and tools. To prevent these gaps, developers should have their code deployed as soon as possible; they should be present when ops—or whoever—deploys the code; and they should use the same tool stack that’s used in production.

How New Relic can help with this factor:

• You can use New Relic Infrastructure to compare packages and versions (via the Inventory page) across different environments.
• Use New Relic in all environments (development, staging, and production) to guarantee developers have access to all the information they need to create and improve their apps across the entire development lifecycle.

11. Logs are event streams

According to the Twelve-Factor App, a log is “the stream of aggregated, time-ordered events collected from the output streams of all running processes and backing services.... [They] have no fixed beginning or end, but flow continuously as long as the app is operating.”

Today, it’s virtually impossible to think about running production-grade apps without it. But before Application Performance Management became mainstream, logs were usually the only monitoring tools we had.

I remember writing extensive logging “frameworks” designed to standardize log formatting and output in my applications. Logs would then be parsed by yet another process to alert us when things weren’t working as expected. Sometimes I wonder how much time I wasted maintaining all of that code.

Unfortunately, even today, there are still applications and technologies that can’t be instrumented and instead rely heavily on logs for monitoring.

How New Relic can help with this factor:

• If you rely on logs, try New Relic’s Splunk or SumoLogic integrations.
• Use the New Relic Insights APIs to import data from log scrapers.

12. Run admin/management processes in the same environment as regular processes

“The process formation,” according to the Twelve-Factor App, “is the array of processes that are used to do the app’s regular business (such as handling web requests) as it runs.” In some cases, however, such as database migrations, developers need to run management processes for an app, so the methodology urges that “one-off admin processes” run in the same release environment as do regular processes, using the same code and configuration as any other process running in that release.

How New Relic can help with this factor:

• Use New Relic Infrastructure to monitor the execution of “one-off” processes in your hosts, and track logins and updates to the hosts.

Conclusion

We may never see a true “one-size-fits-all” application framework, but methodologies such as the Twelve-Factor App provide a fantastic set of best practices for architecting, building, and maintaining modern applications.

Similarly, New Relic won’t magically solve all issues in your applications, but the critical, actionable data we gather for you helps you ensure your applications—and your customer experience—are as powerful as they can be.