This blog post is part of the Understand OpenTelemetry series.
Part 1 gave an overview to OpenTelemetry and discussed why it is the future of instrumentation.
Part 2 explored the core components of the OpenTelemetry open source project.
Part 3 focused on the primary OpenTelemetry data sources: traces, metrics, and logs.
Part 4 explored instrumenting a couple of simple Java applications with OpenTelemetry.
Now Part 5 continues that journey, with videos that show how to run the OpenTelemetry Collector locally with Docker, how a Java app instrumented with the OpenTelemetry Java agent uses OpenTelemetry semantic conventions, and what you can do with the data exported to your backend observability platform.
In this blog post, we go over the OpenTelemetry Collector, a powerful new tool for building telemetry data pipelines that you’ll want to consider bringing to your environment. Then we show a Java application collecting instrumentation data aligned with the semantic conventions, and exporting it to the Collector. Finally, the data flows from the Collector into our observability platform, and we visualize it in New Relic One.
The OpenTelemetry Collector
Modern software environments often consist of a mixture of open source and proprietary components, instrumented with different technologies with varying levels of configurability. You can gain control in this kind of environment with the OpenTelemetry Collector, a highly configurable and extensible means of collecting, processing, and exporting data. Do your components expose telemetry data in a variety of formats, like Prometheus, Jaeger, Kafka, and Zipkin? You can configure the Collector as a Gateway cluster to accept all of these formats (and more) and apply common processing before exporting to the backend observability platform of your choice. Want to send an extra copy of the data to a data warehouse? Add an exporter to your configuration, and if one doesn’t exist that fits your needs, you can write your own.
Another valuable way to use the Collector is as an Agent. The Collector is deployed as an Agent on every host, where it can scrape metrics about the system. Processes on the host can send telemetry data to the Agent, which can enrich the data with host metadata before forwarding it on. Our demos here do not show this option, but it’s something to consider.
The following video demonstrates how to run the Collector locally with Docker (an example from newrelic-opentelemetry-examples). The Collector is configured to receive data through OpenTelemetry Protocol (OTLP) and export it to New Relic One. It also shows a Java application exporting data to the Collector. You can adapt this type of setup and deploy it to any container-based environment.
As discussed in Part 2, the OpenTelemetry semantic conventions define common conventions for collecting instrumentation data for similar operations. For example, if two web servers written in Java and Python are each instrumented and follow the semantic conventions, they produce similar looking trace and metric data. This makes it possible for people using different platforms to derive more meaning from data and tailor improved experiences.
There are a variety of semantic conventions currently defined for resources, traces, and metrics, and more to come as OpenTelemetry continues to mature. Note that the conventions are still experimental and subject to change.
The following video shows a Java application instrumented with the OpenTelemetry Java agent JAR file. The agent does a good job of following the semantic conventions. When we run the application, you see how it sends data to the Collector aligned with the conventions.
Visualizing data in New Relic One
Next, let's look at the data in New Relic One and see where the semantic conventions come into play.
After OpenTelemetry data is flowing into New Relic, it’s accessible to all the tools of New Relic One, including querying, alerts, custom dashboards, and more. Trace data is ingested as Span objects. Metrics data is ingested as dimensional Metric objects. An entity is created to identify the component reporting the data. If you navigate to that entity in the Explorer, you see a dashboard that displays a variety of interesting signals. The dashboard relies on the semantic conventions, and the quality of the experience is better for applications that closely follow them.
The following video shows how to verify data is flowing using simple NRQL queries and how to use the entity explorer to navigate to the dashboard for the application. Finally, we look at the UI for a more complex microservices setup.
In this blog series, we’ve gone from core concepts to working examples. We showed how to wire together Java applications with the OpenTelemetry Collector, export telemetry data to New Relic, and visualize it in New Relic One. Now that you have completed this blog series on understanding OpenTelemetry, you can put what you’ve learned to work in your own environment. Hopefully, we have demystified OpenTelemetry—so you can leverage this powerful new set of tools to reach your observability goals!
Sign up for New Relic’s free tier and start sending your OpenTelemetry data today.
The views expressed on this blog are those of the author and do not necessarily reflect the views of New Relic. Any solutions offered by the author are environment-specific and not part of the commercial solutions or support offered by New Relic. Please join us exclusively at the Explorers Hub (discuss.newrelic.com) for questions and support related to this blog post. This blog may contain links to content on third-party sites. By providing such links, New Relic does not adopt, guarantee, approve or endorse the information, views or products available on such sites.