FOLDS seminar: Extragradient Methods for Modern Machine Learning Settings: New Convergence Guarantees, Step-Size Rules, and Stochastic Variants
October 8 at 12:00 PM - 1:00 PM
Details
Venue
Zoom link: https://upenn.zoom.us/j/98220304722
Extragradient methods are a fundamental class of algorithms for solving min-max optimization problems and variational inequalities. While the classical theory is largely developed under smoothness and other relatively restrictive assumptions, many problems arising in modern machine learning call for analysis in weaker regularity regimes and in stochastic large-scale settings. In this talk, we present new convergence results for deterministic and stochastic extragradient methods beyond the classical framework. In particular, we establish convergence guarantees under the (L0, L1)-Lipschitz condition and derive new step-size rules that expand the range of provably convergent regimes. We also introduce Polyak-type step sizes for deterministic and stochastic extragradient methods, leading to adaptive variants with favorable theoretical properties and practical performance. Our results focus primarily on monotone problems, with extensions to selected structured non-monotone settings. We conclude with numerical experiments that illustrate the theory and the empirical behavior of the proposed methods.
Speaker
Nicolas Loizou
Assistant Professor, Johns Hopkins University
Nicolas Loizou is an Assistant Professor in the Department of Applied Mathematics and Statistics and the Mathematical Institute for Data Science (MINDS) at Johns Hopkins University, where he leads the Optimization and Machine Learning Lab. He holds secondary appointments in the Departments of Computer Science and Electrical and Computer Engineering and is a member of the Johns Hopkins Data Science Institute and the Ralph O’Connor Sustainable Energy Institute (ROSEI).
Prior to this, he was a Postdoctoral Research Fellow at Mila – Quebec Artificial Intelligence Institute and the University of Montreal. He received his Ph.D. in Optimization and Operational Research from the University of Edinburgh, School of Mathematics.
His research interests include large-scale optimization, machine learning, distributed and decentralized algorithms, algorithmic game theory, deep learning theory, and federated learning. He currently serves as an action editor for Information and Inference: A Journal of the IMA, Optimization Methods and Software, and Transactions on Machine Learning Research. He has received several awards, including an NSF CAREER Award, an AFOSR Young Investigator Program Award, COAP 2020 Best Paper Award, and a CISCO Research Award.