#350 How to Make Hard Choices in AI with Atay Kozlovski, Researcher at the University of Zurich

At the start of the episode, Atay Kozlovski outlines why AI failures are surfacing faster than safeguards. He cites automation bias—people accepting system recommendations without question—as a recurring pitfall, illustrated by a passport‑control scanner that misidentified a traveler and a chatbot hallucinating a nonexistent restaurant. Algorithmic bias, hidden in massive training data, compounds the problem, leading to discriminatory outcomes that are hard to detect. These examples show that even seemingly low‑stakes errors can erode trust, while high‑stakes mistakes threaten safety and reputation, underscoring the urgency of robust AI ethics frameworks.

To prevent such harms, Kozlovski introduces the concept of meaningful human control, which shifts analysis from pure algorithms to a broader socio‑technical perspective. He describes the LAVENDER system used by the Israeli Defense Forces, which generated risk scores for millions of civilians and produced a 10 % false‑positive rate, leading to potentially lethal misidentifications. The rapid, 20‑second approval workflow exemplifies an automation bias that bypasses human judgment. This case highlights a responsibility gap: when AI makes recommendations, it is unclear who is accountable, legally or morally, creating ethical blindspots that demand new governance standards.

Finally, the conversation turns to practical governance. Kozlovski argues that AI systems must be embedded within the cultural norms of their deployment environment—whether a hierarchical military unit or a collaborative hospital ward. By mapping organizational practices, designers can decide which decisions remain human‑led and where tracing mechanisms record accountability. Distinguishing legal liability from moral answerability helps close responsibility gaps, ensuring that humans, not machines, bear ultimate blame. For businesses, this translates into clear policies, audit trails, and continuous human‑in‑the‑loop checks, turning AI from a risky black box into a controlled, value‑aligned tool.