A limited memory steepest descent method

Roger Fletcher

doi:10.1007/s10107-011-0479-6

A limited memory steepest descent method

Roger Fletcher

Research output: Contribution to journal › Article › peer-review

57 Citations (Scopus)

Abstract

The possibilities inherent in steepest descent methods have been considerably amplified by the introduction of the Barzilai-Borwein choice of step-size, and other related ideas. These methods have proved to be competitive with conjugate gradient methods for the minimization of large dimension unconstrained minimization problems. This paper suggests a method which is able to take advantage of the availability of a few additional 'long' vectors of storage to achieve a significant improvement in performance, both for quadratic and non-quadratic objective functions. It makes use of certain Ritz values related to the Lanczos process (Lanczos in J Res Nat Bur Stand 45:255-282, 1950). Some underlying theory is provided, and numerical evidence is set out showing that the new method provides a competitive and more simple alternative to the state of the art l-BFGS limited memory method.

Original language	English
Pages (from-to)	413-436
Number of pages	24
Journal	Mathematical Programming
Volume	135
Issue number	1-2
DOIs	https://doi.org/10.1007/s10107-011-0479-6
Publication status	Published - Oct 2012

Access to Document

10.1007/s10107-011-0479-6

Cite this

@article{e90a24eeed7040b3b5a96b36d029086f,

title = "A limited memory steepest descent method",

abstract = "The possibilities inherent in steepest descent methods have been considerably amplified by the introduction of the Barzilai-Borwein choice of step-size, and other related ideas. These methods have proved to be competitive with conjugate gradient methods for the minimization of large dimension unconstrained minimization problems. This paper suggests a method which is able to take advantage of the availability of a few additional 'long' vectors of storage to achieve a significant improvement in performance, both for quadratic and non-quadratic objective functions. It makes use of certain Ritz values related to the Lanczos process (Lanczos in J Res Nat Bur Stand 45:255-282, 1950). Some underlying theory is provided, and numerical evidence is set out showing that the new method provides a competitive and more simple alternative to the state of the art l-BFGS limited memory method.",

author = "Roger Fletcher",

year = "2012",

month = oct,

doi = "10.1007/s10107-011-0479-6",

language = "English",

volume = "135",

pages = "413--436",

journal = "Mathematical Programming",

issn = "0025-5610",

publisher = "Springer-Verlag GmbH and Co. KG",

number = "1-2",

}

TY - JOUR

T1 - A limited memory steepest descent method

AU - Fletcher, Roger

PY - 2012/10

Y1 - 2012/10

N2 - The possibilities inherent in steepest descent methods have been considerably amplified by the introduction of the Barzilai-Borwein choice of step-size, and other related ideas. These methods have proved to be competitive with conjugate gradient methods for the minimization of large dimension unconstrained minimization problems. This paper suggests a method which is able to take advantage of the availability of a few additional 'long' vectors of storage to achieve a significant improvement in performance, both for quadratic and non-quadratic objective functions. It makes use of certain Ritz values related to the Lanczos process (Lanczos in J Res Nat Bur Stand 45:255-282, 1950). Some underlying theory is provided, and numerical evidence is set out showing that the new method provides a competitive and more simple alternative to the state of the art l-BFGS limited memory method.

AB - The possibilities inherent in steepest descent methods have been considerably amplified by the introduction of the Barzilai-Borwein choice of step-size, and other related ideas. These methods have proved to be competitive with conjugate gradient methods for the minimization of large dimension unconstrained minimization problems. This paper suggests a method which is able to take advantage of the availability of a few additional 'long' vectors of storage to achieve a significant improvement in performance, both for quadratic and non-quadratic objective functions. It makes use of certain Ritz values related to the Lanczos process (Lanczos in J Res Nat Bur Stand 45:255-282, 1950). Some underlying theory is provided, and numerical evidence is set out showing that the new method provides a competitive and more simple alternative to the state of the art l-BFGS limited memory method.

UR - https://www.scopus.com/pages/publications/84866235041

U2 - 10.1007/s10107-011-0479-6

DO - 10.1007/s10107-011-0479-6

M3 - Article

SN - 0025-5610

VL - 135

SP - 413

EP - 436

JO - Mathematical Programming

JF - Mathematical Programming

IS - 1-2

ER -

A limited memory steepest descent method

Abstract

Access to Document

Other files and links

Fingerprint

Cite this