PsiLogic Optimizer Introduces Chaos-Aware Active Cancellation, Outperforms Adam on Key NLP and Vision Benchmarks
A new optimizer, PsiLogic, augments Adam with a dynamic Active Cancellation Term controlled by a dual exponential moving average of gradient norms, acting as an implicit warmup mechanism. In FairBench evaluations, PsiLogic achieved the best validation metrics in three out of four tested arenas, including NLP perplexity and top-1 accuracy for ViT and ResNet, with statistical significance. The optimizer incurs a 1.2–1.8x wall-clock overhead on transformer-heavy tasks but maintains comparable peak GPU memory usage. The authors provide open-source code and benchmarking tools for reproducibility.
Why it matters: PsiLogic offers a principled, automated alternative to manual warmup schedules, potentially improving training outcomes and reproducibility across diverse deep learning architectures.
Full story at: arXiv Machine Learning ↗