Know What's Happening in AI

Zero-Shot Alignment: Harm Detection via Incongruent Attention Mechanisms

A lightweight 4.7 million‑parameter adapter sits atop a frozen Phi‑2 model and routes hidden states through two opposing attention heads—standard softmax and non‑normalizing sigmoid. The positive head amplifies likely continuations while the negative head highlights discarded signals, and a gate combines them into a dissonance vector that spikes on harmful prompts. Remarkably, even a randomly initialized adapter produced consistent suppression signals on dangerous queries across both Phi‑2 and Qwen 2.5b, suggesting the architecture surfaces latent safety cues without training. The author notes a measurement problem: intervening at each token creates a feedback loop that obscures the model’s natural behavior.