First-Order Methods for Convex Optimization

P. Dvurechensky, S. Shtern, M. Staudigl*

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

First-order methods for solving convex optimization problems have been at the forefront of mathematical optimization in the last 20 years. The rapid development of this important class of algorithms is motivated by the success stories reported in various applications, including most importantly machine learning, signal processing, imaging and control theory. First-order methods have the potential to provide low accuracy solutions at low computational complexity which makes them an attractive set of tools in large-scale optimization problems. In this survey, we cover a number of key developments in gradient-based optimization methods. This includes non-Euclidean extensions of the classical proximal gradient method, and its accelerated versions. Additionally we survey recent developments within the class of projection-free methods, and proximal versions of primal dual schemes. We give complete proofs for various key results, and highlight the unifying aspects of several optimization algorithms.
Original languageEnglish
Article number100015
Number of pages27
JournalEURO Journal on Computational Optimization
Volume9
DOIs
Publication statusPublished - 2021

Keywords

  • Convex Optimization
  • Composite Optimization
  • First-Order Methods
  • Numerical Algorithms
  • Convergence Rate
  • Proximal Mapping
  • Proximity Operator
  • Bregman Divergence
  • STOCHASTIC COMPOSITE OPTIMIZATION
  • PROJECTED SUBGRADIENT METHODS
  • INTERMEDIATE GRADIENT-METHOD
  • COORDINATE DESCENT METHODS
  • VARIATIONAL-INEQUALITIES
  • MIRROR DESCENT
  • FRANK-WOLFE
  • APPROXIMATION ALGORITHMS
  • THRESHOLDING ALGORITHM
  • MINIMIZATION ALGORITHM

Fingerprint

Dive into the research topics of 'First-Order Methods for Convex Optimization'. Together they form a unique fingerprint.

Cite this