\part{Numerical optimization}

\section{Bracketing method}
Before attempting to optimize a function, a \emph{bracketing method} should be run on the interval of interest. This helps narrow down the bracket considerably before even attempting an specialized method.

The idea is to split an interval into fine partitions, and for each mesh see if the sign of the gradient from beggining to midpoint differs from that of midpoint to end. 
- bracketing method 
- numerical derivative test

\section{Golden section search}
Given a bracket containing a minimum, one aims to find a way to shrink this bracket.
\begin{itemize}
\item the two regions should have equal length
\item the ratio of the nonoverlapping part of the region to the bracket the should be invariant for each iteration 
\end{itemize}

These conditions imply that a region should take up $\frac{1}{\phi}$ of the bracket length.

For the minimum problem, if the function is seen to increase from the edge of region 2 to the edge or region 1, bracket the minima in region 1. Else bracket it in region 2.

\section{Jarratt's method}
Since quadratic functions are easy to minimize, one idea is to interpolate the function on its bracket as a quadratic, take the analytical mininum, and use that result to shrink the bracket. \emph{Jarrat's method} embodies this concept.
Quadratic interpolation 
- golden section search
- Brent's method (optimization)
- Direction set method
- Powell's method
- Nedler-meadle method