FLITE: Federated Low-Rank Iterative Training Engine Cuts Communication by 8718x
Researchers introduce FLITE, a federated fine-tuning method that leverages a low-rank, seed-regenerable factorization to reduce per-client communication to just 1,280 floats (~5 KB) per round—an 8,718-fold reduction compared to full-weight FedAvg. FLITE achieves 74.67% accuracy on CIFAR-100 with ResNet-18, within approximately 0.5 percentage points of FedAvg, and matches or exceeds FedAvg performance under strong non-IID data skew. The method can further compress communication to 648 bytes per round using int4 latents without loss of accuracy.
Why it matters: FLITE significantly lowers the communication requirements for federated learning, enabling practical fine-tuning of large models on devices with limited bandwidth.
Full story at: arXiv Machine Learning ↗