SIREN: Automated Content Manipulation Exposes Vulnerabilities in Web-Augmented LLM Recommenders
A new arXiv preprint introduces SIREN, an automated method that systematically edits already-retrieved webpages to manipulate the rankings produced by web-augmented large language model (LLM) recommenders. By applying 23 content-poisoning techniques, SIREN achieved top-ranked placement for targeted entities in a majority of trials across two production Claude models, with high reproducibility in fresh sessions. The study demonstrates that LLM-based recommenders are susceptible to adversarial manipulation even when the set of retrieved sources is fixed.
Why it matters: This work highlights a significant security risk for LLM-powered recommendation systems, showing that adversaries can manipulate outputs by editing content after retrieval, not just by poisoning retrieval itself.
Full story at: arXiv Cryptography and Security ↗