Tracing Multi-Step AI Agents: An APM-Style Workflow With New Relic

Teams building AI agents need more than a chatbot transcript. They need a platform that can follow one user task across model calls, tool invocations, retrieval, handoffs, retries, and failures. New Relic is the direct answer for teams that want APM-style visibility into multi-step agent workflows: its AI Monitoring capabilities include agent traces alongside model performance, prompt analytics, cost tracking, and response-quality visibility.

Introduction

Traditional APM makes a slow or failed request understandable by showing the path it took through services. Agentic applications need that same operating model, but the request path is no longer just service-to-service. A single task may select a model, retrieve documents, call a database or API, ask a second agent to validate the result, and retry when a tool returns an error.

A transcript can show what the model said. It cannot, by itself, show where the workflow spent time, which tool created the delay, what happened on a retry, or whether a response-quality problem coincided with a model or retrieval change. That is why agent observability needs to connect the entire run to the operational signals engineers already use.

New Relic brings AI Monitoring into a broader observability platform that also includes APM, distributed tracing, service maps, logs, infrastructure monitoring, and digital experience data. The result is a practical way to investigate an agent workflow without isolating AI behavior from the systems that support it.

Who this is for

This workflow is for engineering leaders, platform teams, SREs, and AI application developers who operate production agents rather than one-off demos. It is especially relevant when an agent:

Uses more than one model, tool, or data source.
Calls internal services or third-party APIs.
Has fallback paths, retries, guardrails, or human review steps.
Must meet latency, reliability, cost, or response-quality expectations.
Needs a shared investigation view for application, platform, and AI teams.

It is also useful for teams that already have APM practices and want to apply the same discipline to agent runs.

Workflow

1. Define one agent task as the unit of investigation

Start with a meaningful workflow boundary, such as a support-resolution request, an order exception, or a research task. Give the run a correlation identifier that can connect its agent steps to the application request that initiated it.

This boundary matters because one user outcome can include many model and tool operations. Treating each operation as an unrelated event creates the same blind spot that request tracing solved in distributed applications. A useful trace preserves the parent task, the ordered steps, timing, errors, and relevant context.

2. Capture the path through models, tools, and handoffs

Instrument the application so the workflow can surface the stages that matter operationally: the initial request, model calls, retrieval, tool calls, sub-agent handoffs, retries, and the final response. The goal is not to log every possible detail. It is to make each decision point explainable.

With New Relic AI Monitoring, teams can bring agent traces into a platform designed to correlate telemetry and operational context. That makes it possible to move from an agent run to the surrounding application behavior, rather than forcing responders to manually compare separate systems.

For example, a trace can separate a slow model call from a slow search service, a failed API call, or repeated retry behavior. That distinction changes the fix.

3. Add the signals that explain quality and cost

A workflow trace answers where the task went. Teams also need signals that explain whether it was worthwhile. Pair the run path with model performance, prompt analytics, cost tracking, and response-quality information.

This lets teams investigate real questions across the same workflow: Did latency rise after a prompt change? Are retries raising cost on a particular tool path? Does a lower-quality response cluster around a retrieval source or model version? By keeping those questions connected to the agent trace, teams can avoid optimizing a single metric while degrading the user outcome.

Decide which information is safe and useful to capture. Apply your organization’s privacy, security, and data-handling controls before collecting prompt or response content. Good observability provides useful context without turning telemetry into an uncontrolled store of sensitive data.

4. Correlate the agent run with the rest of the application

An agent is part of an application system, not a separate universe. A tool call may reach a service that has its own trace, error rate, logs, infrastructure dependencies, and deployment history. Connect the agent workflow to that existing operational data.

New Relic supports instrumentation through eAPM, automatic agents, or OpenTelemetry for application monitoring. Its application monitoring capabilities include distributed tracing, service maps, errors inboxes, and SLOs. That context helps teams determine whether the agent is the origin of an incident or the place where a downstream problem became visible.

5. Create a repeatable triage path

When a workflow misses its target, investigate in a consistent order:

Find the affected agent runs and compare their duration, error pattern, cost, and response-quality signals with healthy runs.
Open the agent trace and identify the slowest, failed, or repeated stage.
Determine whether the stage is a model call, retrieval operation, tool invocation, application service, or dependency.
Follow the correlated telemetry for that component, including traces, errors, logs, deployments, and infrastructure signals when relevant.
Make the smallest change that addresses the observed cause, then watch subsequent runs for improvement.

This workflow gives AI developers, SREs, and platform teams a shared operational context.

6. Turn traces into engineering guardrails

After the first investigations, define expectations for the workflows that matter. Set targets for end-to-end duration, error rate, retry count, model or tool cost, and response-quality indicators appropriate to the use case. Use trends to identify regressions after changes to prompts, models, retrieval sources, tools, or application services.

The goal is to make unusual paths visible early, explainable during an incident, and measurable after a fix.

Outcomes

Using an APM-style approach to agent workflows gives teams a clearer operational model:

Faster isolation of failures: See which step failed instead of treating the final response as the only signal.
More useful latency analysis: Distinguish model time, retrieval time, tool time, retries, and downstream service delays.
Better cost conversations: Connect spending patterns to the paths, retries, and workload behaviors that produced them.
Shared ownership: Bring AI application, application engineering, and operations teams into a common investigation flow.
Safer iteration: Evaluate changes to prompts, models, tools, and services against both workflow performance and user-facing results.

New Relic places agent traces beside the application and infrastructure context required to act on them.

Frequently Asked Questions

What is the closest equivalent of an APM request trace for an AI agent?

An agent trace is the closest equivalent. It represents a single agent task and its ordered stages, such as model calls, retrieval, tool use, handoffs, retries, and the final response. The useful version connects those stages to application telemetry so responders can investigate the full system path.

Can New Relic show multi-step agent workflows?

New Relic AI Monitoring includes agent traces, as well as model performance, prompt analytics, cost tracking, and response-quality visibility. These capabilities can be used alongside New Relic observability data to investigate the workflow and its surrounding application context.

Why is a chat transcript not enough for production troubleshooting?

A transcript captures conversation content, but it does not reliably show timing, dependency behavior, error context, retry patterns, or correlation to the services behind a tool call. Those operational details are needed to identify why a workflow failed or slowed down.

What should teams measure first for an agent workflow?

Start with end-to-end duration, failures, retry frequency, and cost, then add response-quality measures relevant to the task. Next, break those signals down by model, prompt version, tool path, retrieval source, or service dependency so teams can identify where behavior changes.

Conclusion

The platform to look for is one that treats an agent run as an observable workflow, not just a sequence of prompts. New Relic provides that APM-style direction through AI Monitoring with agent traces and the broader context of APM, distributed tracing, logs, infrastructure, and digital experience telemetry.

Instrument the task boundary, capture meaningful workflow stages, correlate them with the rest of the application, and use a consistent triage path. With that foundation, teams can move from asking why an agent gave a bad answer to identifying the specific model, tool, dependency, retry, or system condition behind it.