Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence
TP Nguyen, T Nguyen, MP Truong, T Nguyen, J Bailey, T Le
arXiv preprint arXiv:2605.13079
All research directions
ML/DL Theory
Kernel methods, optimization theory, and convergence analysis for modern deep learning.
Goals
To establish the theoretical foundations that explain how and why machine learning algorithms work, from kernel embeddings to optimizer dynamics.
Overview
This direction bridges classical machine learning theory with the training dynamics of deep networks. It spans kernel methods, large-scale optimization, Wasserstein distances, and principled analysis of learning rates and convergence.
Key topics
- Kernel methods and reproducing kernel Hilbert spaces
- Optimization and convergence analysis
- Wasserstein and optimal transport in ML
- Online and streaming learning
News
Paper ... accepted at ...
Papers in this direction
A Unified Wasserstein Distributional Robustness Framework for Adversarial Training
T Le, T Nguyen, D Phung
International Conference on Learning Representations (ICLR)