Packt Deep Engineering

Small Language Models and the Future of Production AI with Karun Thankachan

Karun Thankachan, a senior scientist at Walmart, discussed the growing role of small language models (SLMs) for cost‑effective, task‑specific AI in retail. He introduced ReasonLite, an open‑source library that consolidates chain‑of‑thought distillation, program‑aided reasoning, self‑consistency, and token‑budget controls into a single hyperparameter‑tuning interface. He also unveiled SLM‑Fusion, a framework that routes, merges, and serves multiple specialized SLMs through an OpenAI‑compatible FastAPI gateway. The conversation highlighted why RAG and context engineering are currently outpacing fine‑tuning, and how diffusion models may reshape the landscape soon.