How to Build AI that Actually Ships in Production - Aleksandr Kim

Why It Matters

Aligning AI metrics with real business objectives turns costly experiments into measurable ROI, ensuring that AI initiatives truly ship and scale.

Key Takeaways

• Align ML metrics with business outcomes to drive real impact.
• Prioritize automation over fancy models for measurable productivity gains.
• Use conditional recall thresholds to balance precision and business utility.
• Pivot quickly when POCs reveal low user value, focus on pain points.
• Track AI success via actual usage and time‑saved, not just feedback.

Summary

The conversation centers on AI engineering—specifically, how to move sophisticated models from prototype to production at scale. Alexander Kim, a senior data scientist at Intuit, shares his journey from fine‑tuning a BERT model for cybersecurity to building AI‑driven automation tools for QuickBooks, illustrating the evolving toolkit but consistent mindset required for real‑world impact.

Kim emphasizes that aligning machine‑learning metrics with business outcomes is essential. In a support‑ticket classification project, he reframed a 200‑category problem to focus on the 20 actionable categories, then introduced a conditional‑recall metric tied to automation rate, ultimately cutting support costs by 20%. He repeats that “great ML metrics, zero business impact” is a common pitfall, and that the model itself is rarely the win.

A memorable anecdote involves a planned chatbot POC that proved to be a nice‑to‑have feature. By listening to analysts, Kim pivoted to an automation pipeline that aggregates data, generates insights, and pushes them to Slack, saving executives roughly 30 hours per week. He notes that success is measured through concrete usage and time‑saved rather than superficial user feedback.

The takeaway for enterprises is clear: AI projects must start with a business problem, choose metrics that reflect operational goals, and remain flexible enough to pivot when the data reveals a more valuable use case. This disciplined approach turns experimental AI into reliable, revenue‑generating products.

Original Description

Connect with DataTalks.Club:

- Join the community - https://datatalks.club/slack.html

- Subscribe to our Google calendar to have all our events in your calendar - https://calendar.google.com/calendar/r?cid=ZjhxaWRqbnEwamhzY3A4ODA5azFlZ2hzNjBAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ

- Check other upcoming events - https://lu.ma/dtc-events

- GitHub: https://github.com/DataTalksClub

- LinkedIn - https://www.linkedin.com/company/datatalks-club/

- Twitter - https://twitter.com/DataTalksClub

- Website - https://datatalks.club/

Connect with Alexey

- Twitter - https://twitter.com/Al_Grigor

- Linkedin - https://www.linkedin.com/in/agrigorev/

Check our free online courses:

- ML Engineering course - http://mlzoomcamp.com

- Data Engineering course - https://github.com/DataTalksClub/data-engineering-zoomcamp

- MLOps course - https://github.com/DataTalksClub/mlops-zoomcamp

- LLM course - https://github.com/DataTalksClub/llm-zoomcamp

- Open-source LLM course: https://github.com/DataTalksClub/open-source-llm-zoomcamp

- AI Dev Tools course: https://github.com/DataTalksClub/ai-dev-tools-zoomcamp

👉🏼 Read about all our courses in one place - https://datatalks.club/blog/guide-to-free-online-courses-at-datatalks-club.html

👋🏼 Support/inquiries

If you want to support our community, use this link - https://github.com/sponsors/alexeygrigorev

If you’re a company, reach us at [e-mail]