Expenditure Horizon: Measuring Optimization Ability, with an Application to NanoGPT
METR introduces 'expenditure horizon' as a new metric to measure an AI system's optimization ability, demonstrated using NanoGPT. The metric quantifies how long a system can sustain goal-directed behavior, aiming to provide a more nuanced evaluation than traditional benchmarks.
Why it matters: Expenditure horizon offers a novel approach to assessing AI optimization ability, which could inform understanding and management of advanced AI systems.
Full story at: METR ↗