Ingress-Nginx to Envoy Gateway Migration on CNCF Internal Services Cluster

Why It Matters

Consolidating to a single Envoy Gateway cuts cloud load‑balancer costs and streamlines certificate management, setting a practical blueprint for other organizations facing ingress‑nginx deprecation.

Key Takeaways

• CNCF moved three internal services to Envoy Gateway after ingress-nginx retirement
• Shared Gateway reduces cloud load balancer count, cutting costs
• Setting externalTrafficPolicy to Cluster avoids health‑check failures
• ReferenceGrants enable cross‑namespace TLS secret access for Envoy
• BackendTLSPolicy replicates ingress-nginx HTTPS backend configuration

Pulse Analysis

The retirement of ingress‑nginx forces many cloud‑native teams to reevaluate their edge routing strategy. Envoy Gateway, built on the CNCF‑standard Gateway API, offers a modular alternative that separates traffic handling, TLS termination, and policy enforcement into distinct resources. By leveraging a single shared Gateway, CNCF eliminated the need for multiple cloud load balancers, translating directly into lower infrastructure spend and a stable, single IP address for all internal services. This architectural shift also aligns with the broader industry move toward declarative, API‑driven networking.

Operationally, the migration highlighted several nuanced challenges. The default externalTrafficPolicy of Local caused health‑check failures on Oracle Cloud’s load balancer, prompting a switch to Cluster to ensure every node could respond to probes. Cross‑namespace certificate access, a common hurdle when moving from Ingress to Gateway, was solved with ReferenceGrant objects, while BackendTLSPolicy resources replicated the legacy ingress‑nginx backend‑protocol annotations for services that require HTTPS upstream connections. These adjustments underscore the importance of understanding the multi‑layer design of the Gateway API before a production cutover.

Beyond CNCF’s own cluster, the experience serves as a template for enterprises seeking to modernize their ingress stack. The combination of cost efficiency, simplified TLS lifecycle (especially when paired with cert‑manager’s Gateway API support), and clearer separation of concerns makes Envoy Gateway a compelling choice. As more organizations adopt the Gateway API, we can expect a richer ecosystem of tools and best‑practice guides, accelerating the transition away from legacy ingress controllers while preserving reliability and security.