We have seen these before in our study of linear systems. On the one hand,
minimization of 
is almost easier than finding a zero of
. We saw before that simple descent sends us in the locally
steepest direction, but that search direction may not be the best from a global
perspective.
To use minimization methods, consider the current values 
, where 
is the
stepsize in the direction of minus the gradient. The question we faced before is:
How do we choose this stepsize? If we could, the best procedure would be to
minimize 
as a function of
. Often this is not possible. So what then?
The recommended procedure is called backtracking. Evaluate 
and see
if this is less than the current value 
. If yes, accept that value and
continue. If not, consider a locally quadratic model 
. To evaluate these coefficients, use
the values we determined above - 
-
amd minimize this quadratic model at 
. Providing this
leads to a value of x sufficiently different than the current value, accept it.