TopoTuner: Topological Finetuning of Large Language Models
TopoTuner is a topology-guided fine-tuning framework for large language models that selectively freezes attention projection matrices based on topological changes measured by Wasserstein distances between persistence diagrams. The method achieves competitive performance with full fine-tuning while updating only 1-2% of parameters, and outperforms LoRA in 7 out of 9 model-dataset settings. TopoTuner also reduces training time by 20.4% compared to full fine-tuning and 5.5% compared to LoRA, and enables the transfer of reusable freezing profiles across tasks.
Why it matters: TopoTuner offers a novel approach to efficient LLM adaptation by enabling reusable fine-tuning profiles, potentially reducing computational costs and improving transferability across tasks.
Full story at: arXiv AI/ML ↗