What It Takes to Run AI in the Real World: Lessons From Akamai Digital Leadership Summit

Why It Matters

Scaling AI efficiently in India determines which models become viable products, directly affecting market competitiveness and consumer access. The insights also signal broader industry pressures to embed security and cost discipline from day one.

Key Takeaways

• India needs AI at population scale, near-zero cost.
• Hybrid edge‑cloud models cut latency to ~1 second.
• Reusing queries reduces inference cost to a few cents.
• Security risks rise 1,000% with AI‑driven APIs.
• Open‑sourced Bharat ML stack handles trillions of inferences.

Pulse Analysis

The conversation at the summit reflected a global pivot: after years of racing to enlarge foundation models, enterprises are now wrestling with the practicalities of deploying those models at massive scale. In India, the challenge is amplified by a user base that runs into the hundreds of millions and a price sensitivity that demands sub‑cent per‑inference economics. Companies like Reliance Jio and Postman are adopting hybrid edge‑cloud strategies, pushing inference closer to the user to meet sub‑second latency expectations while offloading heavier workloads to centralized GPUs. This architectural shift not only improves user experience but also curtails bandwidth costs, a critical factor for a market where data transfer remains expensive.

Cost engineering emerged as a recurring theme, with GlanceAI illustrating how query deduplication and batch processing can slash image‑generation expenses from $30 to a few cents. By recognizing that up to 60% of requests are repetitive, firms can cache LLM outputs or route simple queries to smaller, specialized models. Meesho’s Bharat ML stack, now open‑sourced, exemplifies how building a home‑grown inference engine can handle 3‑4 trillion inferences and a million queries per second, delivering personalized experiences within a 500‑millisecond window even during peak sales. These tactics enable businesses to price AI‑enhanced services competitively, unlocking broader adoption across price‑sensitive segments.

Security considerations are equally paramount. The rapid expansion of AI‑enabled APIs—growing over 1,000% year‑over‑year—has enlarged the attack surface, prompting leaders like Akamai’s Vijay Kolli to stress "security‑by‑design." Deterministic, explainable models are gaining traction to meet regulatory demands, especially in finance and government services where data poisoning could have severe consequences. Meanwhile, sovereign AI initiatives, such as Gnani.ai’s end‑to‑end voice stack, aim to retain data ownership and reduce reliance on foreign providers. Together, these developments suggest that the next wave of AI success in India will hinge less on model size and more on disciplined engineering, cost‑effective infrastructure, and robust security frameworks.