When AI Says Yes: Social Engineering the Bots in Our Systems | 2 Minute Drill with Drex DeFord

Why It Matters

As enterprises and health systems increasingly delegate sensitive tasks to AI agents, adversaries will shift to manipulating the agents themselves, creating risks to accounts, data privacy, and operational integrity unless organizations actively red-team and harden those systems.

Summary

Drex DeFord summarizes a recent wave of attacks that exploited AI customer-support agents to take over dormant Instagram accounts, including one tied to the Obama White House. Attackers used simple social-engineering—matching presumed location via VPN and politely requesting email changes—to get the AI to reassign account control, enabling propaganda posts and targeted grabs of valuable handles. Experts warn this reflects a broader vulnerability as organizations automate account recovery and other workflows: AI assistants are designed to be helpful and thus can be manipulated into completing harmful actions. Meta has patched the flaw, but similar weaknesses have appeared across other vendors' models and deployments.

Original Description

A dormant Instagram account tied to the Obama White House started posting pro-Iranian content. No hacked password. No malware. No phishing email. Just a polite conversation with Meta's AI support agent.

Researchers are calling it out plainly: AI agents are built to be helpful, and that eagerness is exactly what attackers are starting to exploit. The attack surface isn't the password anymore -- it's the agent.

Remember, Stay a Little Paranoid

Thanks to Cyderes for sponsoring this episode: https://thisweekhealth.com/partners/cyderes/

Linkedin: https://www.linkedin.com/company/ThisWeekHealth

Donate: Alex’s Lemonade Stand: Foundation for Childhood Cancer - https://www.alexslemonade.org/mypage/3173454