LLMs Encode Relevance as a Layer-Wise Cross-Lingual Signal
A new study finds that query-document relevance is linearly decodable from the residual-stream activations of instruction-tuned large language models (LLMs), with the strongest signals emerging in the middle-to-late layers. Linear probes trained on these activations can match or even outperform the models' generated relevance judgments in preserving system rankings. The relevance signal shows partial portability across languages, though within-language decoding remains more effective.
Why it matters: This work offers a novel, representation-level perspective on how LLMs internally encode relevance, enabling new ways to diagnose and improve LLM-based information retrieval systems.
Full story at: arXiv Information Retrieval ↗