Know What's Happening in AI

Pruning LLMs for Retrieval: Why Attention Matters and MLPs Don't

The paper introduces EffiR, a pruning framework that flips conventional LLM pruning wisdom for dense retrieval tasks. By aggressively removing MLP layers while preserving attention heads, the authors cut Mistral‑7B’s parameters by roughly 50% and doubled inference speed with minimal loss on BEIR benchmarks. Experiments show the pruned 3.4B‑parameter model outperforms native small models such as LLaMA‑1B and Gemma‑2B on retrieval tasks. The approach also integrates smoothly with 4‑bit quantization, delivering further efficiency gains.