DevOps Videos

Fast & Asynchronous: Drift Your AI, Not Your GPU Bill // Artem Yushkovskiy

The talk introduced ASEA, an open‑source asynchronous‑actor framework designed to replace traditional batch pipelines for generative AI workloads. By decoupling each processing step into self‑hosted GPU actors that communicate via message queues, the team at a global food‑delivery platform eliminated rate‑limiting, reduced engineering overhead, and gained fine‑grained control over scaling. Initially, a Kubeflow pipeline calling external AI APIs hit random errors and consumed 60‑80% of engineering effort just to stay operational, while cloud API costs ballooned. The solution migrated the models in‑house, wrapped them in actors that auto‑scale on demand, and introduced a root‑step message format that carries payload enrichment through a cascade of actors. Built‑in error handling routes failed messages to retry or dead‑letter queues, and a lightweight synchronous gateway lets developers invoke complex flows via a simple HTTP call. Key examples highlighted the system’s performance: throughput reached the limits of the GPU cluster without hitting rate limits, and the architecture scaled from zero to 100 GPUs handling diffusion models. The framework supports near‑real‑time latencies measured in minutes rather than milliseconds, and developers can dynamically re‑route messages using an LLM‑powered router, enabling fan‑out/fan‑in patterns for future enhancements. For enterprises, ASEA offers a path to dramatically lower AI operating costs, avoid vendor lock‑in, and accelerate the deployment of sophisticated AI pipelines. Its open‑source release invites broader adoption and community contributions, positioning it as a foundational tool for AI‑ops teams seeking scalable, cost‑effective, and resilient workflows.