Pour le moment, cette page n'est disponible qu'en anglais.

Elasticsearch is a search and analytics engine that has become one of the most popular distributed document stores since its release in 2010. It's an open source tool that delivers lightning-fast and highly relevant search results to users. If you’ve searched for a show on a streaming service or ordered groceries from an app for delivery, chances are Elasticsearch was part of that journey.

Monitoring Elasticsearch in context with the rest of your Ruby application can help you identify and solve performance issues and errors related to requests to your Elasticsearch clusters. 

Version 8.12.0 of the New Relic Ruby agent introduces automatic instrumentation of versions 7.x and 8.x of the Elasticsearch gem, giving you a more complete picture of how your Elasticsearch implementation is working over time. Because we’ve automatically instrumented Elasticsearch, you just need to upgrade to Ruby agent version 8.12.0 or later to monitor requests from your Elasticsearch client. 

In this blog post, you’ll learn how our team instrumented Elasticsearch with Ruby metaprogramming, how to update the Ruby agent, and how you can configure the Ruby agent to capture or obfuscate queries as needed.

Instrumenting Elasticsearch with metaprogramming

The New Relic Ruby agent uses metaprogramming techniques to insert New Relic instrumentation code into commonly used public methods of the libraries we want to monitor. Many libraries, including the Elasticsearch gem, aren't designed to use callbacks or hooks. To work around that, we use metaprogramming to add custom instrumentation methods to these libraries.

We automatically default to using a technique called Module#prepend to insert instrumentation code in the ancestor chain of an existing method but lean on alias_method chaining when a conflict occurs between these two techniques. Here's an example from the source code that uses chaining.

      to_instrument = if ::Gem::Version.create(::Elasticsearch::VERSION) <
          ::Gem::Version.create("8.0.0")
        ::Elasticsearch::Transport::Client
      else
        ::Elastic::Transport::Client
      end

      to_instrument.class_eval do
        include NewRelic::Agent::Instrumentation::Elasticsearch

        alias_method(:perform_request_without_tracing, :perform_request)
        alias_method(:perform_request, :perform_request_with_tracing)

        def perform_request(*args)
          perform_request_with_tracing(*args) do
            perform_request_without_tracing(*args)
          end
        end

The Ruby agent offers both instrumentation options to give you greater flexibility. New Relic uses Module#prepend by default, but if you have another gem included in your application that uses method chaining on the same methods as the New Relic agent, errors can occur. That’s because method chaining (alias_method) and Module#prepend work together only in particular situations. When not used together properly, they can cause non-terminating recursions. Our troubleshooting guide for the SystemStackError provides details on these two options if you have a conflict with one of the instrumentation methods.

The next image shows the default option for auto-instrumentation of the Elasticsearch library in New Relic. See the Ruby agent documentation for more information.

The New Relic Ruby agent inserts the instrumentation code into the perform_request method at runtime because the Elasticsearch ruby gem routes all API calls through this method. 

The New Relic Ruby agent supports versions 7.x and 8.x for the Elasticsearch gem.

The perform_request method doesn’t include information about the specific operation type, such as update, delete, and so on. To solve this problem, we found that we could check the caller_locations in order to see which method was responsible for invoking perform_request.

Here's the code where caller_locations is used:

    def nr_operation
      operation_index = caller_locations.index do |line|
        string = line.to_s
        string.include?('lib/elasticsearch/api') && !string.include?(OPERATION)
      end
      return nil unless operation_index

      caller_locations[operation_index].to_s.split('`')[-1].gsub(/\W/, "")
    end

This provides us with the specific operation being called on Elasticsearch, so we can attach the correct operation type to spans.

Here's a screenshot of a distributed trace of Elasticsearch spans in New Relic.

Elasticsearch spans have an attribute, db.instance, that displays the name of the cluster the Elasticsearch request is sent from. This can help identify if there are any specific clusters encountering errors or performance issues. Spans will also record the query parameters passed to perform_request, which are obfuscated by default to protect sensitive information.

And this screenshot shows the attributes recorded for an Elasticsearch update span, including attributes db.instance for the cluster name and db.statement for the query parameters.

Elasticsearch configuration options

We included two configuration options, both of which are enabled by default: capture_queries and obfuscate_queries.

When elasticsearch.capture_queries is enabled, the agent will attempt to record the query being passed to the request. We also added obfuscate_queries to keep sensitive information hidden. These options can be configured through your newrelic.yml file or environment variables.

This screenshot shows the configuration for elasticsearch.capture_queries in New Relic.

The next screenshot shows a New Relic Ruby agent configuration for elasticsearch.obfuscate_queries.

Upgrade your Ruby agent

You can start monitoring Elasticsearch by upgrading your Ruby agent. Version 8.x of the New Relic Ruby agent includes new instrumentation for threads Tilt (8.2.0), threads (8.7.0), gRPC (8.10.0), and Ruby’s Logger class with logs in context (8.6.0), along with some bug fixes and enhancements like code-level metrics (8.10.0).

Connect with us

The Ruby agent is open source software. We welcome contributions to our source code and improvement suggestions, and we always look forward to connecting with the community. We’d love to hear about what you like and what you want to see in the future. Connect with us on GitHub.