Software Architecture Monday (Mark Richards)

Lesson 217 - Supervisor Consumer Pattern

The video introduces the Supervisor Consumer pattern, a scalability technique that lets a single service handle fluctuating loads by automatically spawning and retiring consumer threads. Mark Richards explains that a dedicated supervisor component continuously polls an event queue, assesses the queue depth, and determines the optimal number of consumer instances based on current service instances and predefined thresholds. The core algorithm divides pending messages by the number of service instances, caps the result at a maximum thread limit, and then either launches additional consumer threads or shuts excess ones down. Key parameters include idle consumer count, polling frequency (e.g., one second), and a hard ceiling on thread or memory usage. Pseudo‑code illustrates how the supervisor maintains a list of active consumers, computes required capacity, and iterates this logic until the service shuts down. Richards demonstrates the pattern with a customer‑service example where multiple downstream services request a customer name. By wrapping the message‑API calls in dynamically managed consumer threads, the service can process hundreds of requests in the same 300 ms latency as a single request. He notes that this approach avoids the overhead of launching new service instances, delivering near‑instant elasticity. While the pattern offers programmatic scalability, consistent latency, and efficient resource utilization, it also introduces complexity, risk of thread saturation, and potential out‑of‑memory scenarios. Practitioners must enforce strict thresholds and monitor system health to reap the benefits without compromising stability.