Commiserating About AI with Ned and Kyler

Why It Matters

Enterprises that adopt AI‑driven DevOps must blend rapid, non‑deterministic experimentation with strict, deterministic controls to safeguard security, cost, and production stability.

Key Takeaways

• Built internal secure AI assistant "Vera" handling 32k internal queries.
• AI agents now manage full SDLC, from ticket to deployment.
• Weekly AI coffee chats keep team skills current amid rapid changes.
• Markdown‑based "skills" automate GitHub workflows, reducing manual scripting.
• Balance non‑deterministic AI creativity with deterministic scripts for production safety.

Summary

The episode centers on Kyler Middleton and Ned Bellavance’s hands‑on experiments with generative AI in a corporate DevOps setting. They describe building a secure, internal ChatGPT‑style assistant called Vera, which has already fielded roughly 32,000 employee queries and integrates with Slack, Teams, and email.

Key insights include the evolution from single‑turn chat models to autonomous AI agents that can ingest a Jira ticket, run code reviews, enforce security policies, and push deployments without human intervention. To keep pace with rapid model releases, the team runs weekly AI coffee‑chat sessions, sharing practical tips and avoiding stale training courses. They also leverage “skills” – markdown‑styled scripts – to automate repetitive GitHub actions such as committing, opening pull requests, and invoking Copilot reviewers.

Notable examples feature a push‑to‑GitHub skill that orchestrates the entire PR lifecycle, and a comparison of AI‑generated bash versus deterministic scripts for production pipelines. Kyler likens the shift to moving from hand‑drawing circles to instructing a robot to draw perfect circles of any size, emphasizing the need for precise language to avoid “terrible” outcomes.

The discussion underscores a balancing act: developers can exploit AI’s speed and creativity in sandbox environments, but must enforce deterministic, auditable scripts for staging and production. This dual‑track approach highlights emerging governance challenges as enterprises embed LLM‑driven automation deeper into their software delivery lifecycles.

Original Description

Hosts Ned and Kyler compare notes on everything they’ve been doing with AI, including the successes they've enjoyed and headaches they’ve suffered building and implementing AI agents. They talk about how AI has sped up their workflows, how managing multiple AI agents is akin to raising toddlers, the necessity of using deterministic scripts for increased reliability, and more.

Links:

PP100: Building and Securing AI Agents – A Case Study - https://packetpushers.net/podcasts/packet-protector/pp100-building-and-securing-ai-agents-a-case-study/

Day Two DevOps is part of the Packet Pushers network. Visit our website to find more great networking and technology podcasts, along with tutorial videos, the Human Infrastructure newsletter, and loads more resources for building your IT career. https://packetpushers.net