Performance of various BFGS and DFP implementations with limited precision 2nd order info
D. Byatt, I.D. Coope and C.J. Price
Abstract
The BFGS formula is arguably the most well known and widely used update method for quasi–Newton algorithms. Some authors have claimed that updating approximate Hessian information via the BFGS formula with a Cholesky factorisation offers greater numerical stability than the more straightforward approach of performing the update directly. Other authors have claimed that no such advantage exists and that any such improvement is probably due to early implementations of the DFP formula in conjunction with low accuracy line searches.
This paper supports the claim that there is no discernible advantage in choosing factorised implementations (over non–factorised implementations) of BFGS methods when approximate Hessian information is available to full machine precision. However the results presented in this paper show that a factorisation strategy has clear advantages when approximate Hessian information is available only to limited precision. These results show that a conjugate directions factorisation outperforms the other methods considered in this paper (including Cholesky factorisation).
Back to Research Reports
