Inside Adobe's OpenTelemetry Pipeline: Simplicity at Scale

Why It Matters

The design gives large, acquisition‑heavy enterprises a low‑touch, scalable observability platform while keeping error visibility and backend control centralized. It demonstrates how OpenTelemetry can be turned into a production‑grade service offering without demanding deep expertise from individual teams.

Key Takeaways

• Adobe runs thousands of OpenTelemetry collectors per signal type
• Sidecar collector immutable config prevents app restarts
• Managed namespace isolates metrics, logs, traces for fault tolerance
• Custom extension returns 401 on backend auth failures
• Auto‑instrumentation requires only two Kubernetes annotations

Pulse Analysis

OpenTelemetry has become the de‑facto standard for cloud‑native observability, but many enterprises struggle to scale it beyond pilot projects. Adobe’s three‑tier collector architecture shows how a central team can provide a self‑service platform that supports diverse workloads while preserving operational simplicity. By deploying a sidecar collector inside each application pod and a configurable deployment collector, Adobe isolates configuration changes to the deployment collector, eliminating unnecessary pod restarts and reducing the operational burden on service owners.

The real breakthrough for Adobe lies in its auto‑instrumentation strategy. Leveraging the OpenTelemetry Operator, developers add just two annotations to their Kubernetes manifests and gain automatic telemetry collection across supported languages. This minimal‑effort approach accelerates adoption and ensures consistent data quality. Adobe also builds a custom collector distribution that strips out unused Contrib components, shrinking the binary footprint and improving security. A bespoke extension adds a circuit‑breaker that returns a 401 error when backend authentication fails, surfacing issues upstream instead of hiding them in downstream logs—a critical enhancement for large‑scale observability.

From an operational perspective, Adobe’s quarterly upgrade cadence and clear separation of signal types in the managed namespace provide resilience against rate‑limiting and backend outages. The team’s experience with version compatibility challenges underscores the need for coordinated rollouts between the Operator and collector versions. Their lessons—treat OpenTelemetry as a platform, prioritize simplicity for end users, and design for error visibility—offer a roadmap for other organizations aiming to turn observability into a scalable, enterprise‑grade service.