Latent Fusion Jailbreak: Blending Harmful and Harmless Representations to Elicit Unsafe LLM Outputs
Researchers present Latent Fusion Jailbreak (LFJ), a white-box attack that manipulates the internal hidden states of large language models (LLMs) by interpolating between harmful and benign prompts. LFJ achieves a macro-averaged attack success rate of 94.13% across five open-weight models, but a tailored adversarial training defense reduces this rate to 12.37%. The study also explores the impact of different pairing and sampling strategies on attack effectiveness.
Why it matters: This work reveals a novel vulnerability in safety-aligned LLMs via internal state manipulation and demonstrates a potential defense, underscoring the evolving challenges in AI safety.
Full story at: arXiv Computation and Language ↗