Introducing AI Observability for OpenTelemetry

As engineering teams standardize on OpenTelemetry (OTel) for generative AI architectures, a structural data disconnect is creating visibility gaps for enterprise leaders. This fragmentation can spawn unmanaged applications and unmonitored LLM token usage that silently drains cloud budgets, while leaving disconnected OTel trace streams completely cut off from core safety pipelines. Without unified tracking, production open-source AI services operate exposed to potential issues such as undetected prompt injections, PII leakage, and toxicity threats, introducing severe financial and compliance risks.

New Relic AI Observability for OTel eliminates this friction with a powerful "Normalize on Read" architecture that unifies open-source telemetry and native APM agent data into a single, cohesive view. Instead of forcing costly data duplication or requiring developers to rewrite code, the platform dynamically aligns disparate schemas on the fly to instantly restore complete observability. By automatically discovering AI footprints and establishing a stateless bridge to enterprise security pipelines, platform leaders can regain absolute financial governance and operational control over their AI initiatives.

Financial Governance & Cross-Instrumentation Model Benchmarking

Scaling generative AI across enterprise environments frequently triggers severe cloud billing spikes when engineering teams lack visibility into aggregate token spend across fragmented microservices. AI Observability for OTel eliminates these financial blind spots by delivering unified, account-wide Model Inventories that combine open-source and native APM agent telemetry into cohesive macro dashboards. Platform leaders gain instant, comprehensive visibility into global AI expenditures, including metrics such as total token consumption, request latency, and error rates, to govern cloud budgets proactively.

To prevent overspending on token usage, AI Program Managers and Platform Architects must continuously optimize their model selections based on objective ROI data. The platform’s Model Comparison interface features a flexible per-column query resolution engine that enables true cross-instrumentation benchmarking. Engineers can run direct, side-by-side performance and cost evaluations, comparing a legacy service instrumented with a proprietary APM agent directly against a modernized microservice using open-source OpenTelemetry. By rendering clear cost, latency, and quality data on a single screen, organizations can confidently identify and deploy the most cost-effective LLM configuration, maximizing their generative AI investments without compromising the end-user experience.

Architectural Agility: "Normalize on Read" for Lower TCO

Standard observability platforms handle OpenTelemetry by normalizing data on write, intercepting OTel spans at the ingestion gateway and mutating or duplicating them into proprietary event formats to fit legacy dashboards. This legacy approach drives up storage fees, increases processing overhead, and inflates cloud infrastructure costs. New Relic eliminates this data bloat through a strict Normalize on Read strategy. OTel telemetry is written natively to standard Span tables with zero ingestion-phase translation or background duplication. At runtime, the UI query engine dynamically detects the telemetry source and adapts its NRQL templates on the fly to query the Span table directly, delivering identical visibility without duplicating data.

Beyond frontend query agility, the platform leverages the structural design of OpenTelemetry GenAI semantic conventions to streamline backend pipeline efficiency. Traditional vendor agents often emit prompt inputs and model completions as separate telemetry events, requiring stateful tracking and external caching layers to stitch them back together. In contrast, a single OTel span inherently packages user prompts (gen_ai.input.messages) and LLM responses (gen_ai.output.messages) into a single, self-contained payload. A dedicated OtlpEvaluationHandler intercepts these trace spans and executes processing statelessly, completely removing Redis caching or session reassembly dependencies. Combining write-free data normalization with stateless processing significantly lowers the total cost of ownership for your observability stack. Enterprise engineering teams preserve complete fidelity to open-source standards while avoiding data duplication fees, infrastructure sprawl, and background processing overhead.

Eliminating "Shadow AI" Across Microservices

In complex, cloud-native microservice architectures, development teams frequently adopt open-source LLM libraries to build autonomous capabilities without notifying platform or security leads. This decentralized adoption creates "Shadow AI" across the enterprise, leaving untracked, unmapped microservices running in production without operational oversight, centralized budget allocation, or security governance. Relying on developers to manually register AI services or maintain static service catalogs inevitably leads to inventory blind spots, leaving platform teams unaware of where generative AI is deployed across their global infrastructure.

To eliminate this operational friction, New Relic embeds zero-touch footprint mapping directly into the core distributed tracing stream. Powered by a specialized TraceProcessor extension within the ingestion pipeline, the platform inspects incoming OpenTelemetry trace streams in real time, scanning payloads for gen_ai. semantic attributes. The moment an open-source LLM interaction is detected, the pipeline extracts the entity GUID and automatically publishes an aiEnabledApp:true tag to the global Entity Platform, requiring zero human intervention or manual code tagging.

This background discovery mechanism transforms passive telemetry into proactive governance. The split second an engineer drops an OTel-instrumented AI library into code, the microservice automatically catalogs itself within the central AI Monitoring UI. Platform teams and SREs gain an updated, audit-ready inventory of all open-source AI deployments, unlocking instant UI filtering, macro-level cost segmentations, and automated baseline alerts to stop runaway token spend and operational anomalies before they impact end users.

Extending Real-Time Protection to Open Stacks

Operating open-source AI applications in production without continuous evaluation exposes your organization to severe security vulnerabilities and regulatory liabilities. When OTel trace streams are disconnected from safety pipelines, malicious prompt injection attacks, accidental PII leaks, and toxic outputs bypass security oversight undetected. This gap forces risk and compliance officers to either stall open-source AI deployments or risk severe brand damage and regulatory non-compliance.

New Relic AI Observability for OTel closes this security loop by routing your open-source trace data directly into the core evaluation stream. This direct pipeline integration delivers enterprise-grade safety checks straight to your open-source AI stacks, instantly mitigating business risk. By establishing automated PII masking, real-time prompt injection defense, and toxicity screening on live traffic, security leaders can confidently clear generative AI applications for production. The organization remains continuously shielded from data breaches and reputational harm without requiring complex security middleware or slowing down deployment velocity.

Strategic Value Across the Enterprise

AI Observability for OpenTelemetry aligns workflows across development, platform operations, and security teams, enabling enterprises to scale generative AI rapidly without compromising governance or security. By removing data structure friction and extending native feature parity to open-source stacks, the platform delivers targeted business value across three key roles:

Accelerate Your Open-Source AI Strategy Today

Deploying AI Observability for OpenTelemetry requires zero custom code rewriting or vendor-specific SDK lock-in. Because the platform natively ingests standard OpenTelemetry GenAI semantic conventions, your engineering teams can continue using their preferred open-source libraries and instrumentation workflows. Simply route your existing OTel trace streams to New Relic, and the background processor will automatically map your AI footprint, populate model inventories, and deliver unified performance and financial metrics across your enterprise.

Don't let schema disconnects, hidden token expenses, or unmonitored compliance risks stall your open-source GenAI initiatives. Explore the central AI Entities page to instantly map your OTel footprint, configure automated evaluation guardrails to secure live trace streams, or request a live demonstration today to see how New Relic unifies open-source AI telemetry under a single pane of glass.

Sign up for a free account or request a demo to explore these capabilities, and join us at New Relic Now 2026 to learn more.