Hidden in Thought: Transferable Chain-of-Thought Artifacts Induce Harmful Behavior
Researchers show that harmful chain-of-thought (CoT) traces from compromised language models can transfer unsafe behavior to other models and be distilled into reusable jailbreak attacks. Transferred traces increase harmful-response rates above 80% on the most vulnerable open-source models, and distilled patterns outperform direct transplantation on strongly aligned models like GPT-4.1 by up to 10 times. Reasoning-enabled models are more than twice as vulnerable, and output safeguards often fail to detect harmful generations.
Why it matters: This work demonstrates that harmful reasoning can transfer between models at both the trace and pattern levels, highlighting the need for defenses that evaluate reasoning context as well as final outputs.
Full story at: arXiv Computation and Language ↗