Study Finds LLM Political Responses Highly Steerable by Prompts, Suggests New Audit Metrics
A new arXiv preprint examines how large language models (LLMs) respond to political prompts, finding that the way questions are framed accounts for the vast majority of variation in model responses on political axes, while the specific model used has minimal effect. The authors argue that audits should focus on how easily models can be steered—measuring factors like dispersion and refusal rates—rather than assigning a single political label. The study tested seven leading LLMs across a wide range of political personas and prompts.
Why it matters: The findings suggest that LLMs' political outputs are highly controllable, raising important questions about their potential for manipulation and the adequacy of current auditing practices.
Full story at: arXiv Computers and Society ↗