Distributed Monolith vs. Microservices Architecture: 4 Ways New Relic Can Tell You Which Is Which

As more and more software organizations adopt DevOps processes and culture, a major initiative they commonly undertake is breaking their monoliths into microservices. However, even after organizations have put in a considerable amount of work, it sometimes turns out that these monoliths weren’t actually turned into collections of microservices so much as they became “distributed monoliths.” In You’re Not Actually Building Microservices (a post that inspired me to write this one), SimpleThread’s Justin Etheredge writes that a distributed monolith is the “worst of all worlds. You’ve taken the relative simplicity of a single monolithic codebase, and you’ve traded it in for a deeply intertwined distributed system."

So, how do you tell if you actually have a true microservices architecture or just a distributed monolith? Let’s walk through four questions to see how New Relic can help you find out. Keep in mind, though, that while these questions will give you insight into the true nature of your systems, the final answer may not always be clear-cut exceptions—after all, modern software is nothing if not complicated.

Question 1: Does a change to one microservice require changes to or deployments of other microservices?

A major benefit of microservices is the decoupling of development and deployment. In a true microservices architecture, a team can make changes (within reason) to the service they own without having to communicate those changes to other upstream or downstream teams responsible for other services. If your team does synchronized deployments or creates pull requests across multiple services to make a change to your service, you’re not really running decoupled microservices.

A examination of your deployment processes or version control strategy can provide an immediate answer to this question. You can also get a good idea of service interdependencies by looking at deployment markers in New Relic service maps.

If you see deployment markers lining up across the multiple services in your map, you haven’t effectively decoupled your services.

To monitor deployment dependencies in New Relic Insights, record your deployments as custom events using the New Relic rest API (v2) and insert those custom events via the Insights API. Then build an NRQL query to create a widget for your Insights dashboard.

If your widget shows spikes of many deploys occurring simultaneously, that’s a sign of poor decoupling. Ideally, in a microservice architecture, you deploy apps using continuous integration and continuous delivery (CI/CD), in which case the graphs in your widget will appear chaotic, rather than organized.

Question 2: Are your microservices overly chatty?

Each microservice should have a clearly defined purpose that it can execute with minimal communication with other services. If you see a service that has to send many back-and-forth requests to the same downstream services, that’s a red flag.

Use New Relic APM to look at the central service in question and examine its interaction with external services.

A distributed monolith can actually perform pretty well, and may never experience significant issues with response times. With that in mind, we typically want to examine services by highest throughput. If the number of calls per minute (cpm) for a given external service is greater than the throughput of the application itself by a large factor, you most likely have not decoupled your services.

You can also investigate this on a per-transaction level in APM. The same logic applies here—if the average calls per transaction is greater than 2 or 3 calls per external service, that’s a good sign of “chattiness” and poor decoupling.

For larger and more complex environments, use New Relic Insights to monitor external call counts across all your applications.

• To create a widget to track external calls made by each of your applications over time, use this NRQL query:
  

  SELECT average(externalCallCount) FROM Transaction FACET appName SINCE 1 week ago TIMESERIES

• To create a widget to track all external calls made by each of your applications, use this NRQL query:
  

  SELECT max(externalCallCount) FROM Transaction FACET appName SINCE 1 week ago

It’s important to remember that some microservices (such as payment gateways) can record a lot of external calls because of the function they perform. If you identify such services in your environment, just filter them out in the NRQL queries for these widgets:

SELECT max(externalCallCount) FROM Transaction WHERE appName != “name-of-payment-gateway” FACET appName SINCE 1 week ago

Question 3: Do several microservices share a datastore?

Even a set of microservices that appears to be well separated can fall prey to this problem. It’s typical that most of a monolith’s services will use the same datastore. But this should not be the case in a distributed microservices architecture. In a microservices architecture, shared datastores create deployment problems and database contention issues; and schema changes can create problems for the services communicating with the database. Therefore, each service should have its own datastore.

As with tracking interdependent deployments for services, you can use service maps to discover the dependencies between services and datastores.

In this case, we see that each service in our WebPortal app has its own datastore. This greatly simplifies deployment and data management for each service.

With New Relic, you can monitor your databases using New Relic Infrastructure on-host integrations.

If, for example, your application is running in Amazon Web Services (AWS) and uses the Amazon Relational Database Service (RDS), you can use New Relic’s AWS RDS integration to monitor this information and create a widget using this NRQL query:

SELECT uniqueCount(appName)/uniqueCount(entityName) as 'Applications per Datastore' FROM DatastoreSample,Transaction SINCE 1 week ago

(Note that the DataStoreSample datatype in this example is unique to AWS RDS. If you’re hosting your services in Microsoft Azure, for example, you could use an Azure monitoring integration and the AzureSqlDatabaseSample datatype. Or see the Infrastructure integrations documentation for other databases we integrate with.)

Question 4: Do my services scale dynamically?

One of the main reasons to use microservices is to isolate bottlenecks to small units that may be scaled independently as needed, rather than having to scale your entire architecture. A healthy microservices deployment should be very dynamic, with throughput swings resulting in non-uniform scaling across the services. For example, for a spike in global throughput, I’d expect to see certain applications (the bottlenecks) scaling up more rapidly than, say, a payment gateway service that serves as a request router and includes little logic.

You can use Insights to determine if your services do scale dynamically. A couple of widgets that monitor throughput per application and hosts per application are a good way to start.

• To create a widget to track throughput per application, use this NRQL query:
  

  SELECT count(*) FROM Transaction FACET appName TIMESERIES since 1 month ago

• To create a widget to track hosts per application, use this NRQL query:
  

  SELECT count(*) FROM Transaction FACET host TIMESERIES since 1 month ago

In a full microservices architecture, you should see spikes in services populating on hosts correspond to spikes in throughput on individual services, which would indicate dynamic scaling of services. If instead you see corresponding spikes across all services and hosts, this is good indication that your services aren’t truly decoupled.

Be honest in your answers

If your system is able to cleanly pass all these checks, it may very well be a fully functioning microservices architecture. If one or more of these questions raise red flags for you, however, it may be time to re-evaluate or redesign your approach to properly tame the complexity of microservices. This programatic shift requires that you not only build this architecture piece by piece but that you also never stop developing it.