Self-Concordant Analysis of Frank-Wolfe Algorithms

P. Dvurechensky; P. Ostroukhov; K. Safin; S. Shtern; M. Staudigl

Self-Concordant Analysis of Frank-Wolfe Algorithms

P. Dvurechensky, P. Ostroukhov, K. Safin, S. Shtern, M. Staudigl^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceeding › Academic › peer-review

Abstract

Projection-free optimization via different variants of the Frank-Wolfe (FW), a.k.a. Conditional Gradient method has become one of the cornerstones in optimization for machine learning since in many cases the linear minimization oracle is much cheaper to implement than projections and some sparsity needs to be preserved. In a number of applications, e.g. Poisson inverse problems or quantum state tomography, the loss is given by a self-concordant (SC) function having unbounded curvature, implying absence of theoretical guarantees for the existing FW methods. We use the theory of SC functions to provide a new adaptive step size for FW methods and prove global convergence rate O(1/k) after k iterations. If the problem admits a stronger local linear minimization oracle, we construct a novel FW method with linear convergence rate for SC functions.

Original language	English
Title of host publication	Proceedings of the 37th International Conference on Machine Learning
Subtitle of host publication	International Conference on Machine Learning, 13-18 July 2020, Virtual
Editors	Hal Daumé III, Aarti Singh
Publisher	Proceedings of Machine Learning Research
Pages	2814-2824
Number of pages	11
Volume	119
Publication status	Published - 2019
Event	25th Americas Conference on Information Systems of the Association-for-Information-Systems( AMCIS) - Cancun, Cancun, Mexico Duration: 15 Aug 2019 → 17 Aug 2019

Conference

Conference	25th Americas Conference on Information Systems of the Association-for-Information-Systems( AMCIS)
Country/Territory	Mexico
City	Cancun
Period	15/08/19 → 17/08/19

Keywords

OPTIMIZATION
CONVERGENCE
COMPLEXITY

Access to Document

https://proceedings.mlr.press/v119/

Cite this

Dvurechensky, P., Ostroukhov, P., Safin, K., Shtern, S., & Staudigl, M. (2019). Self-Concordant Analysis of Frank-Wolfe Algorithms. In H. Daumé III, & A. Singh (Eds.), Proceedings of the 37th International Conference on Machine Learning: International Conference on Machine Learning, 13-18 July 2020, Virtual (Vol. 119, pp. 2814-2824). Proceedings of Machine Learning Research. https://proceedings.mlr.press/v119/

@inproceedings{5f3f81af7d074d76ab359c4bd4b2b8e2,

title = "Self-Concordant Analysis of Frank-Wolfe Algorithms",

abstract = "Projection-free optimization via different variants of the Frank-Wolfe (FW), a.k.a. Conditional Gradient method has become one of the cornerstones in optimization for machine learning since in many cases the linear minimization oracle is much cheaper to implement than projections and some sparsity needs to be preserved. In a number of applications, e.g. Poisson inverse problems or quantum state tomography, the loss is given by a self-concordant (SC) function having unbounded curvature, implying absence of theoretical guarantees for the existing FW methods. We use the theory of SC functions to provide a new adaptive step size for FW methods and prove global convergence rate O(1/k) after k iterations. If the problem admits a stronger local linear minimization oracle, we construct a novel FW method with linear convergence rate for SC functions.",

keywords = "OPTIMIZATION, CONVERGENCE, COMPLEXITY",

author = "P. Dvurechensky and P. Ostroukhov and K. Safin and S. Shtern and M. Staudigl",

year = "2019",

language = "English",

volume = "119",

pages = "2814--2824",

editor = "{Daum{\'e} III}, Hal and Aarti Singh",

booktitle = "Proceedings of the 37th International Conference on Machine Learning",

publisher = "Proceedings of Machine Learning Research",

note = "25th Americas Conference on Information Systems of the Association-for-Information-Systems( AMCIS) ; Conference date: 15-08-2019 Through 17-08-2019",

}

Dvurechensky, P, Ostroukhov, P, Safin, K, Shtern, S & Staudigl, M 2019, Self-Concordant Analysis of Frank-Wolfe Algorithms. in H Daumé III & A Singh (eds), Proceedings of the 37th International Conference on Machine Learning: International Conference on Machine Learning, 13-18 July 2020, Virtual. vol. 119, Proceedings of Machine Learning Research, pp. 2814-2824, 25th Americas Conference on Information Systems of the Association-for-Information-Systems( AMCIS), Cancun, Mexico, 15/08/19. <https://proceedings.mlr.press/v119/>

Self-Concordant Analysis of Frank-Wolfe Algorithms. / Dvurechensky, P.; Ostroukhov, P.; Safin, K. et al.
Proceedings of the 37th International Conference on Machine Learning: International Conference on Machine Learning, 13-18 July 2020, Virtual. ed. / Hal Daumé III; Aarti Singh. Vol. 119 Proceedings of Machine Learning Research, 2019. p. 2814-2824.

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceeding › Academic › peer-review

TY - GEN

T1 - Self-Concordant Analysis of Frank-Wolfe Algorithms

AU - Dvurechensky, P.

AU - Ostroukhov, P.

AU - Safin, K.

AU - Shtern, S.

AU - Staudigl, M.

PY - 2019

Y1 - 2019

N2 - Projection-free optimization via different variants of the Frank-Wolfe (FW), a.k.a. Conditional Gradient method has become one of the cornerstones in optimization for machine learning since in many cases the linear minimization oracle is much cheaper to implement than projections and some sparsity needs to be preserved. In a number of applications, e.g. Poisson inverse problems or quantum state tomography, the loss is given by a self-concordant (SC) function having unbounded curvature, implying absence of theoretical guarantees for the existing FW methods. We use the theory of SC functions to provide a new adaptive step size for FW methods and prove global convergence rate O(1/k) after k iterations. If the problem admits a stronger local linear minimization oracle, we construct a novel FW method with linear convergence rate for SC functions.

AB - Projection-free optimization via different variants of the Frank-Wolfe (FW), a.k.a. Conditional Gradient method has become one of the cornerstones in optimization for machine learning since in many cases the linear minimization oracle is much cheaper to implement than projections and some sparsity needs to be preserved. In a number of applications, e.g. Poisson inverse problems or quantum state tomography, the loss is given by a self-concordant (SC) function having unbounded curvature, implying absence of theoretical guarantees for the existing FW methods. We use the theory of SC functions to provide a new adaptive step size for FW methods and prove global convergence rate O(1/k) after k iterations. If the problem admits a stronger local linear minimization oracle, we construct a novel FW method with linear convergence rate for SC functions.

KW - OPTIMIZATION

KW - CONVERGENCE

KW - COMPLEXITY

M3 - Conference article in proceeding

VL - 119

SP - 2814

EP - 2824

BT - Proceedings of the 37th International Conference on Machine Learning

A2 - Daumé III, Hal

A2 - Singh, Aarti

PB - Proceedings of Machine Learning Research

T2 - 25th Americas Conference on Information Systems of the Association-for-Information-Systems( AMCIS)

Y2 - 15 August 2019 through 17 August 2019

ER -