Seed Hijacking of LLM Sampling and Quantum Random Number Defense

Why It Matters

A stealthy PRNG‑based backdoor can compromise downstream AI applications without detection, threatening data integrity and brand trust, while a low‑cost QRNG shield offers a practical mitigation path for AI providers.

Key Takeaways

• SeedHijack manipulates PRNG outputs to force specific token injection.
• Attack achieves 99.6% success on GPT‑2 and 100% on larger models.
• Bypasses alignment defenses like RLHF, SFT, and reasoning distillation.
• QRNG defense adds only 0.6% latency and 7.7 MB memory.
• Highlights sampling layer as critical, overlooked supply‑chain vulnerability.

Pulse Analysis

The sampling stage of large language models has long been treated as a peripheral concern, with most security research focusing on model weights, training data, or inference logits. Autoregressive generation relies on deterministic pseudorandom number generators to select the next token, creating a predictable pipeline that can be hijacked if the seed is compromised. This overlooked vector expands the AI supply‑chain attack surface, demanding new defensive thinking beyond traditional alignment and watermarking techniques.

SeedHijack demonstrates how an adversary can subtly manipulate the PRNG output to force exact token injection, achieving near‑perfect success rates across a spectrum of models—from the 124 M‑parameter GPT‑2 to 7 B‑parameter aligned variants. By operating entirely within the sampling routine, the attack leaves model logits untouched, allowing it to slip past RLHF, supervised fine‑tuning, and reasoning‑distillation safeguards. The researchers’ extensive benchmark—540 trials across nine sampling configurations—shows that the vulnerability is both robust and scalable, raising alarm for any organization deploying LLMs in customer‑facing or high‑stakes environments.

To counter this threat, the authors propose a hardware quantum random number generator that replaces the deterministic PRNG with true entropy. In their threat model, the QRNG‑based defense eliminates the backdoor with negligible performance penalties—just a 0.6% increase in latency and an additional 7.7 MB of memory. This lightweight, deployable solution positions quantum‑enhanced randomness as a pragmatic security layer for AI infrastructure. As enterprises accelerate AI integration, the findings urge product teams to audit sampling pipelines and consider QRNG or equivalent entropy sources, reshaping best‑practice roadmaps for trustworthy LLM deployment.