Self-concordant analysis of Frank-Wolfe algorithms

Pavel Dvurechensky, Petr Ostroukhov, Kamil Safin, Shimrit Shtern, Mathias Staudigl*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review


Projection-free optimization via different variants of the Frank-Wolfe (FW), a.k.a. Conditional Gradient method has become one of the cornerstones in optimization for machine learning since in many cases the linear minimization oracle is much cheaper to implement than projections and some sparsity needs to be preserved. In a number of applications, e.g. Poisson inverse problems or quantum state tomography, the loss is given by a self-concordant (SC) function having unbounded curvature, implying absence of theoretical guarantees for the existing FW methods. We use the theory of SC functions to provide a new adaptive step size for FW methods and prove global convergence rate O(1/k) after k iterations. If the problem admits a stronger local linear minimization oracle, we construct a novel FW method with linear convergence rate for SC functions.
Original languageEnglish
Title of host publicationProceedings of the 37th International Conference on Machine Learning
PublisherProceedings of Machine Learning Research
Publication statusPublished - 2020
Event37th International Conference on Machine Learning - Virtual Conference Only, Unknown
Duration: 12 Jul 202018 Jul 2020
Conference number: 37


Conference37th International Conference on Machine Learning
Abbreviated titleICML 2020
Internet address

Cite this