KITE: Preventing Model Collapse in Iterative Instruction Tuning with Synthetic Data
A new preprint identifies that model collapse during iterative instruction tuning with synthetic data often appears as a polarization of competence, where strong skills are reinforced but weak skills degrade. The authors introduce KITE, a two-stage framework that uses failure-guided data generation and boundary-aware uncertainty curation to address this issue. Experiments across several datasets and open-source LLMs show that KITE achieves more stable improvements compared to strong synthetic-data baselines.
Why it matters: This work offers a practical approach to mitigating performance degradation in LLMs trained with synthetic data, addressing a growing challenge as model-generated data becomes more common.
Full story at: arXiv Computation and Language ↗