If you’re evaluating Datadog LLM Observability, you’ll want to know how it fits alongside your existing application performance monitoring (APM) tools and what it costs as your LLM applications scale. Datadog now calls the product Agent Observability, though you’ll still see Datadog LLM Observability in its URLs and documentation. Both names refer to the same product.

This article looks at Datadog’s tracing, evaluations, and pricing, along with the alternatives worth considering.

What is Datadog LLM Observability?

Datadog Agent Observability is a platform for monitoring, troubleshooting, and improving LLM applications and AI agents. It provides:

  • End-to-end LLM and agent tracing: Tracks model calls, retrieval steps, tool calls, and agent decisions.
  • Quality and security evaluations: Flags potential hallucinations, prompt injection attempts, and sensitive data exposure.
  • Experiments and prompt testing: Tests changes to prompts, models, and configurations before deployment.
  • Cost and token tracking: Measures latency, error rates, token usage, and costs across LLM workflows.

Datadog LLM Observability became generally available in June 2024. At DASH 2025, Datadog expanded the product with AI Agent Monitoring, which maps agent decisions in an interactive graph, and introduced LLM Experiments and AI Agents Console in preview.

Key features of Datadog LLM Observability

Datadog LLM Observability provides four core capabilities for monitoring LLM applications and AI agents, covering request tracing, output quality, testing, and costs.

End-to-end LLM and agent tracing

Datadog traces requests across prompts, retrieval steps, tool calls, and agent decisions, with inputs, outputs, and metadata captured in individual workflow spans. For AI agents, Datadog Agent Observability displays decision paths as interactive graphs, helping developers investigate incorrect tool calls, repeated loops, and failed retrievals.

The platform also provides Patterns and Insights to identify recurring topics and issues across production traces.

Quality and safety evaluations

Datadog runs built-in and custom evaluations against production traces to flag potential hallucinations, model drift, and unsafe outputs. Security evaluations also check for prompt injection attempts and sensitive data exposure.

The Sensitive Data Scanner identifies and redacts sensitive information in LLM prompts and responses.

Experiments and prompt testing

LLM Experiments lets developers test prompts, models, and agent configurations against datasets before deployment. Evaluators score the results so teams can compare output quality across different configurations.

Experiments are included in the product rather than priced separately.

Cost and token tracking

The platform tracks latency, token usage, retries, errors, and cost across LLM workflows. Developers can examine individual model calls to identify expensive requests, excessive token consumption, and performance issues.

How Datadog LLM Observability pricing works

Datadog prices Agent Observability as a standalone product, billed by the number of LLM spans processed. Charges are separate from APM, infrastructure monitoring, and logs.

As of November 2026, the plans on Datadog’s pricing page are:

 

Free

Pro

Monthly price

$0

From $160

Included LLM spans

40,000/month

100,000/month

Trace retention

15 days

15 days

Evaluations

Unlimited

Unlimited

Additional spans

Not available (upgrade to Pro)

From $3.50 per 10,000

Pro pricing depends on the billing commitment: $160/month annually, $200 month-to-month, or $240 on demand. Additional spans cost $3.50, $4.20, or $5 per 10,000, respectively.

What counts towards billing?

An LLM span is one model call to a provider like OpenAI or Anthropic, and that’s what Datadog bills for. Tool, workflow, agent, embedding, and retrieval spans don’t count, meaning an agent that makes five model calls to answer one request bills five spans.

What's included?

Both plans provide:

  • Tracing, evaluations, and dashboards
  • Datasets, experiments, and annotations
  • Playground, CLI, and Model Context Protocol (MCP) access
  • 15-day trace retention, with paid add-ons extending traces to 30, 60, or 90 days and experiments to 6, 9, or 12 months

With annual billing, the per-span rate increases to $5, $6.50, or $7.50 per 10,000 LLM spans for 30-, 60-, or 90-day retention, respectively. Monthly and on-demand rates are higher.

You can purchase Agent Observability without having any other Datadog subscription. Teams already using Datadog for APM, infrastructure, or RUM get correlation across those signals through a shared trace ID without extra configuration.

What to consider about Datadog LLM Observability before you commit

Before choosing Datadog Agent Observability, consider how its pricing, instrumentation, and integration fit your existing stack:

  • Separate pricing: Agent Observability is a standalone product, so you can use it without another Datadog subscription. If you already pay for APM, infrastructure, or logs, it adds another cost to forecast as LLM usage grows.
  • Span-based billing: Datadog charges for model calls, not tool or retrieval spans. Compare this with alternatives that bill for traces, observations, or evaluation scores.
  • Instrumentation: Datadog supports Python, Node.js, and Java SDKs, plus OpenTelemetry and HTTP API integration. Check which providers and frameworks support auto-instrumentation. Using OpenTelemetry can also make it easier to export telemetry to other platforms.

Before making a choice, decide whether you need a dedicated AI observability solution or want LLM monitoring integrated with your existing APM.

Datadog LLM Observability alternatives

Datadog isn't the only option for monitoring LLM applications. These four alternatives offer different approaches to tracing and evaluations, and some integrate directly with your existing APM. For a wider view of observability tools, see our roundup of Datadog alternatives.

New Relic AI Monitoring

New Relic AI Monitoring extends application performance monitoring (APM) to generative AI applications. Developers can enable AI monitoring through existing New Relic APM agents, bringing LLM traces, model performance, and application telemetry into the same platform.

AI agents and tools also appear in the entity map and distributed trace waterfall, so developers can follow requests from an LLM call through the services and databases involved.

Key features:

  • Integrated APM: Investigate LLM latency, error rates, and application dependencies in the same distributed traces.
  • AI response monitoring: Review model outputs in a consolidated view to investigate unusual responses.
  • Model comparison: Compare model performance, costs, and output quality to guide model selection.
  • Agent observability: Track AI agents and tool calls alongside application services in the entity map and trace waterfall.
  • Zero-code monitoring: eBPF-based AI monitoring detects LLM calls to OpenAI, Anthropic Claude, Google Gemini, and Amazon Bedrock without SDKs or code changes.

Considerations:

  • Configuration: eBPF AI monitoring must be enabled explicitly. Prompt and response capture is optional and disabled by default. When a language APM agent is already reporting AI telemetry, eBPF monitoring automatically backs off to avoid duplicates.
  • Usage-based pricing: AI Monitoring uses New Relic's existing ingest and user pricing, so telemetry volume affects costs.

Best for: Teams monitoring AI applications in production that want LLM performance connected to APM, distributed traces, and infrastructure telemetry.

Langfuse

Langfuse provides tracing, evaluations, and prompt management for LLM-powered applications. Its MIT-licensed core can be self-hosted, with managed cloud plans available. Developers can also link prompt versions to production traces to see how changes affect model performance over time.

Key features:

  • Self-hosting: Run the core platform on your own infrastructure and retain control of trace data.
  • Evaluations: Score model outputs and track quality across prompts and versions.
  • Trace analysis: Inspect LLM calls, agent workflows, and token consumption.

Considerations:

  • Usage-based billing: Cloud usage is metered across traces, observations, and scores, so workflows generating many observations and evaluation scores can consume more billable units.
  • Free-tier limits: The Hobby plan only has two seats and 30-day data access.

Best for: Teams wanting self-hosted LLM monitoring with a managed cloud option.

LangSmith

LangSmith, developed by the team behind LangChain, helps developers trace, test, and evaluate AI agents. It supports applications built with LangChain and other frameworks. Developers can turn production traces into test datasets, making it easier to reproduce failures and check whether changes improve agent behavior.

Key features:

  • Agent tracing: Inspect individual steps, tool calls, and model responses.
  • Online evaluations: Measure output quality against defined criteria.
  • LangChain integration: Connect tracing and testing to existing LangChain workflows.

Considerations:

  • Retention: Base traces are retained for 14 days. Extended traces can be retained for 180 days at an additional cost.
  • Deployment and pricing: Self-hosting requires Enterprise, while cloud plans combine per-seat fees with usage-based charges.

Best for: Teams building AI agents with LangChain or needing dedicated evaluation workflows.

Arize Phoenix

Arize Phoenix provides local-first tracing, evaluations, and prompt testing for LLM applications. It supports OpenTelemetry instrumentation and can run locally or on self-managed infrastructure. Phoenix uses OpenInference semantic conventions built on OpenTelemetry, helping developers capture consistent trace data across supported models and frameworks.

Key features:

  • OpenTelemetry integration: Capture traces across supported models and frameworks.
  • Evaluations: Test output quality using human feedback and LLM-as-judge scoring.
  • Local deployment: Keep application traces and prompts within your own environment.

Considerations:

  • Licensing: Phoenix uses Elastic License 2.0, which is source-available but not OSI-approved open source.
  • Managed hosting: Arize AX has separate SaaS plans, with self-hosted enterprise deployment options.

Best for: Teams prioritizing local development, evaluation workflows, and control over AI telemetry.

LLM observability works best as part of the whole stack

When an AI application slows down or returns an error, the model isn't always responsible. The problem could be:

  • A slow retrieval query to a vector database
  • An API gateway timing out before the model responds
  • Excessive token consumption from unnecessary context

To find the cause, developers need to trace the LLM call alongside the application services and infrastructure supporting it.

New Relic AI Observability brings this visibility into your existing APM, so you can:

  • Trace AI agents, model calls, and application dependencies in the same request.
  • Investigate latency, errors, and token usage alongside application performance.
  • Monitor AI workloads on the same ingest and user pricing as the rest of the platform, without a separate AI product SKU.

When you compare New Relic with Datadog, you’ll see that New Relic includes AI monitoring within its existing APM capabilities, while Datadog prices Agent Observability as a separate product. New Relic also supports tool consolidation by keeping AI telemetry alongside application and infrastructure monitoring.

Request a New Relic demo to see AI agents and application services in the same distributed trace, and discuss how New Relic would handle your LLM monitoring requirements.

Frequently asked questions about Datadog LLM Observability

Is Datadog LLM Observability included with Datadog APM?

No, Datadog LLM Observability is not included with Datadog APM. Agent Observability, formerly LLM Observability, is a standalone product billed separately from APM, infrastructure monitoring, and logs.

Pricing is based on LLM spans, with a free tier covering 40,000 spans per month and Pro starting at $160 per month with annual billing. Teams using Datadog APM can correlate AI agent activity with application services and infrastructure through shared trace IDs.

What is the difference between LLM observability and APM?

The difference is that LLM observability focuses on AI-specific behavior, while APM monitors application performance. APM tracks throughput, latency, error rates, and dependencies. LLM observability adds visibility into prompts, model responses, token usage, costs, agent workflows, and output quality.

An LLM call can complete successfully but still produce an inaccurate or unsafe response. Combining LLM monitoring with APM helps developers investigate both model behavior and application performance.

What are the best alternatives to Datadog LLM Observability?

The best alternatives to Datadog LLM Observability include New Relic AI Monitoring, Langfuse, LangSmith, and Arize Phoenix.

New Relic AI Monitoring integrates with existing APM agents, providing end-to-end tracing across AI and application services, plus optional zero-code eBPF monitoring. Langfuse offers self-hosted LLM observability, LangSmith supports agent tracing and evaluations, and Arize Phoenix provides OpenTelemetry-based tracing and local deployment.

New Relic is the strongest fit for teams that want to investigate LLM behavior within their existing APM workflows, with application traces and infrastructure data available in the same platform.

현재 이 페이지는 영어로만 제공됩니다.