Black Hat Europe 2025 | Weaponizing Image Scaling Against Production AI Systems

Why It Matters

These aliasing attacks turn routine media uploads into hidden command vectors, threatening data security and trust in AI services. Implementing stronger preprocessing safeguards is essential to prevent covert exploitation.

Key Takeaways

• Image downscaling can hide malicious instructions invisible to users.
• Attack exploits aliasing from Nyquist‑Shannon sampling in common resizers.
• Researchers fingerprint preprocessing pipelines to craft targeted adversarial images.
• Alias attacks also succeed on audio resampling and neural codecs.
• Mitigations need anti‑aliasing filters and sample‑rate validation for inputs.

Summary

The presentation at Black Hat Europe 2025 revealed a new class of attacks that embed hidden commands in images uploaded to production AI systems. By exploiting the mathematical properties of downscaling algorithms—particularly the Nyquist‑Shannon sampling theorem—adversaries can craft high‑frequency perturbations that disappear to human eyes but reappear as readable text after the image is resized, enabling prompt‑injection attacks on models like Google Gemini, Vertex AI, and agentic browsers.

The researchers demonstrated a practical workflow: first fingerprint the target’s preprocessing pipeline (bicubic, bilinear, Lanczos, or nearest‑neighbor) using artifact analysis, then solve a least‑squares optimization to modify only the most influential pixels, typically in dark regions and the red channel, to maximize machine visibility while remaining invisible to users. The attack extends beyond images to audio, where downsampling without proper anti‑aliasing can fold ultrasonic content into the audible band, allowing hidden speech to be recognized by transcription services.

Key examples included exfiltrating calendar data via a disguised image sent to Gemini CLI and generating audible “Hello London” from an unintelligible high‑frequency audio clip after aggressive downsampling. The talk also highlighted that many production systems lack robust validation of sample rates or anti‑aliasing safeguards, especially in neural audio codecs like Meta’s EnCodec, making them vulnerable to both linear and non‑linear aliasing attacks.

The findings underscore a pressing need for developers to audit image and audio preprocessing stages, enforce anti‑aliasing filters, and verify input metadata. Without these defenses, seemingly innocuous media uploads can become covert command channels, compromising data privacy and system integrity.

Original Description

AI vision systems see differently than humans do. When platforms downscale uploads to save compute, the mathematical properties of interpolation algorithms create exploitable artifacts. In this presentation, we'll show how to craft images which use invisible pixel perturbations to reveal malicious prompts after downscaling, triggering unauthorized tool execution across Google Gemini, Vertex AI, Google Assistant, and Genspark. Beyond image downscaling, we'll explore the broader attack surface, including audio transformations, dithering algorithms, and other preprocessing steps that become prompt injection vectors. You'll learn to fingerprint vulnerable systems using test patterns that reveal specific downscaling implementations across AI libraries. We'll demo Anamorpher, our open-source tool for automated attack generation, with both Python APIs and visual interfaces, as well as examine practical mitigations from displaying actual processed images to implementing design patterns resistant to prompt injection, such as the action selector pattern.

By:

Suha Hussain | AI Research Engineer, Product Security, Harvey

Kikimora Morozova | Security Researcher, Trail of Bits

https://blackhat.com/eu-25/briefings/schedule/?#weaponizing-image-scaling-against-production-ai-systems-49911