Optimizing XPath Selectors For New Relic Synthetics Monitors

A key feature of New Relic Synthetics are scripted browser monitors. These are scripts you create with a JavaScript-like scripting language that drive Selenium-powered Google Chrome browsers. A scripted browser monitor lets you track multi-step processes on a website with a real browser. Among several object locator options available when creating scripted monitors, XPath selectors are perhaps the most powerful.

XPath selectors point to object elements within the tree structure of an XML document. By optimizing the XPath selectors in your Synthetics monitors, you can:

• Reduce the likelihood that changes in your web application will break XPath selectors. This helps avoid unnecessary monitor failures and alerts, as application changes could cause your script to find no matches, or cause it to interact with incorrect elements.
• Improve the readability of your monitors, making troubleshooting and future revisions easier.

The examples tips included in this post can help you make the best XPath optimizations and avoid unnecessary toil in your monitor scripts.

Why create your own XPath selectors?

XPath selectors created by browser exports or recorders like the Katalon Automation Recorder work, but often provide overlong XPath selector statements that impact readability and could lead to stability issues if you make changes in your application. The Katalon Recorder does well with ID, linkText, and other simple object identifiers, but it can create XPath selectors that are hard to read and may not correctly identify the necessary objects.

Identifying and revising XPath selectors

Developer tools in Google Chrome greatly simplify the process of writing monitor scripts. To get started, identify the element you want to inspect and then right click on it.

This opens Chrome’s developer tools. Then, within the elements view, right click on the object you want to interact with in your monitor and select Copy > Copy XPath, which creates an XPath statement; in this example:

//*[@id="post-47124"]/div[1]/h3[1]/strong

The XPath the browser produced in this example is far from ideal. First, the XPath will no longer match the correct object if you make changes to the page structure under the post-47124 object. Second, this XPath makes the script more difficult to read and troubleshoot. We could find this same object with a simpler XPath selector, for example:

//*[text()="What it means to be entity-centric"]

We can even validate this selector without a monitor. In the elements view,  activate Find with command + F. To the Find search field, add the XPath selector

//*[text()="What it means to be entity-centric"]

Chrome will highlight the matching object and indicate how many matches were found, as shown here:

Learning from examples

Let’s look at a couple of examples of poorly constructed “bad” XPaths and show suggestions for how to improve them.

1. Bad XPath: /html/body/div[2]/form/div/div[1]/div[1]

Suggestion: Avoid using an absolute XPath, which includes the entire path from the object you need all the way up to the HTML element. With an absolute path, if you make any application changes that impact the path down to the div object your script interacts with, the monitor will either fail or cause actions (clicks, sendKeys, etc.) to interact with an incorrect object on the page. 

Use a relative XPath instead, and use the least number of levels in the XPath as possible. Check the div element to determine if there are any identifying attributes that you could use in your XPath. For example, if the element had the text of "Hello World", you could use //div[text()="Hello World"]as your selector.

2. Bad XPath: (.//*[normalize-space(text()) and normalize-space(.)='Next Slide'])[1]/following::span[2]

Suggestion: This example is an XPath created by the Katalon Recorder that will click the "Introducing New Relic One" link from the New Relic homepage. While this XPath technically works, it’s complicated, difficult to read, and fragile should the application change.

The XPath targets the second span element after the first element with the text of "Next Slide," which isn’t ideal for a few reasons. First, it would work better to target the <a> element instead of the span. Second, the example unnecessarily locates an object based on the location of a separate element. Changes to this object could break the selector. Finally, the XPath itself doesn’t make it clear what it’s looking for, which could complicate troubleshooting efforts. Better to use the title attribute of the <a> element in the selector:

//a[@title="Introducing New Relic One"]

Five tips for XPath selector optimization

Finally, these tips can help optimize your XPath syntax:

1. Use the contains function to match a selector to an object based on partial text contents or portions of an attribute:

Text: //strong[contains(text(), "Absolute")]

Attribute: //img[contains(@src, "attachment-menu-button.png")]

2. If there are additional tags, such as <strong>, for example, as part of the object you’re matching, use the dot (“.”) instead of text()://div[., "Hello World"]

3. Instead of looking for a specific div or span, use conditional logic such as and/or expressions and search for any objects that match using the “*” wildcard.  This XPath looks for an object with a class attribute that contains panel-title and also has the text Reports://*[contains(@class, "panel-title") and text() = "Reports"]

4. For case-insensitive text (switching upper case letters to lower case letters), use the translate function. This converts the object text to lower case so that you no longer need to worry about case sensitivity://a[contains(translate(text(), "ABCDEFGHIJKLMNOPQRSTUVWXYZ", "abcdefghijklmnopqrstuvwxyz"), "newsroom")]

5. Use the normalize-space function to handle new lines, trailing spaces, and so on. This removes leading and trailing whitespace and replaces multiple spaces with a single space://span[normalize-space(text()) = "Introducing New Relic One >"]

Obviously this isn’t an exhaustive list of XPath best practices, but hopefully these tips will give you some ideas on how to best use XPath syntax in your scripted monitors. For more detail, check out this Guru99 tutorial on XPath in Selenium Webdrivers, and don’t miss our blog post on Tips for Creating Resilient Selenium Scripts with CSS Selectors.