TextGrad Fails to Learn Effective Agent Policies from Experience Despite Success of Human-Written Policies
A new arXiv preprint reports that while human-written natural-language policies can significantly improve the performance of frozen large language model (LLM) agents in the TextWorldExpress environment, the TextGrad method—which attempts to learn such policies from agent experience using natural-language feedback—does not yield better results than fixed prompting. The study finds that the main obstacle is not in executing policy updates, but in reliably generating and selecting effective policies from agent trajectories, even when using richer feedback or advanced search techniques.
Why it matters: This highlights a key limitation in current approaches to learning agent policies from experience using natural-language feedback, suggesting that simply leveraging agent trajectories is insufficient for policy optimization in LLM agents.
Full story at: arXiv Computation and Language ↗