\chapter{Differentiation}




\section{Differentiable function}


Imagine we have a function and we want to find its tangent at that point; there are numerous ways we can approach this, all leading to the same solution.

Tangents are lines, hence we can find a best linear approximation of our function.

If we 'magnify' our function

If we consider our function's slope

\begin{definition}[Derivative]
Let $f$ be a real function, the \emph{derivative of $f$ at $x_0$} is the following limit.
\[ f'(x_0) = \lim_{x \to x_0} \frac{f(x)-f(x_0)}{x-x_0}\]
\end{definition}



\begin{definition}[Real differentiable function]
A real function is \emph{differentiable at $x_0$} iff the derivative of $f$ at $x_0$ converges to a finite value.
\[ f'(x_0) = \lim_{x \to x_0} \frac{f(x)-f(x_0)}{x-x_0}\]
A real function is said to be differentiable on $U$ iff it is differentiable for every $u \in U$.
\end{definition}


\emph{Leibniz notation} expresses derivatives as follows. It heavily reflects the origins of the derivative as the ratio of image and domain infinitesimals, and offers much intuition as to the nature of differentiation by likening it to a 'fraction of infinitesimals'. Note that though this notation gives good intuition, we shouldn't cancel out 'infinitesimals' ad hoc; we must prove our results on differentiation by means of the Newton quotient. 
\[ \frac{d f}{dx}\]
\[ \frac{d^2 f}{dx^2}\]
\[ \frac{d^n f}{dx^n}\]


\emph{Lagrange's notation} expresses derivatives as follows. It's neat, compact, and generalizes well; definitely my personal favourite.
\[ f' \]
\[ f''\]
\[ f^{(n)}\]

\emph{Euler-Arbogast notation} expresses derivatives as follows. It is commonly employed in the fields of differential algebra and differential equations.
\[ Df \]
\[ D_{2}f \]
\[ D_{n}f \]

\emph{Newton's notation} expresses derivatives as follows. It is generally used more by physicists than mathematicians.
\[ \dot{f} \]
\[ \ddot{f} \]

\subsection{Linearity of the derivative}


\begin{proposition}
If $f,g$ are real functions that are differentiable at $x$, then $f+g$ is differentiable at $x$ and the following holds.
\[(f+bg)'(x) = f'(x) + g'(x)\]
\[(f-g)'(x) = f'(x) - g'(x)\]
Moreover for any $a\in \mathbb{R}$ the following holds.
\[(af)'(x) = af'(x)\]
\end{proposition}

\subsection{The product rule}

\begin{theorem}[The product rule]
If $f,g$ are real functions that are differentiable at $x$, then $fg$ is differentiable at $x$ and the following holds.
\[(fg)'(x) = f'(x)g(x)+f(x)g'(x)\]
\end{theorem}


\subsection{The chain rule}


\begin{theorem}[The chain rule]
If $f,g$ are real functions that are differentiable at $x$, then $f \circ g$ is differentiable at $x$ and the following holds.
\[ (f \circ g)'(x) = (f' \circ g)(x)g'(x) \]
\end{theorem}




\subsection{The quotient rule}

A simple corollary of the product rule and the chain rule is the quotient rule.
\begin{theorem}[The quotient rule]
If $f,g$ are real functions that are differentiable at $x$, then $fg$ is differentiable at $x$ when $g(x)\neq0 $ and the following holds.
\[(\frac{f}{g})(x) = \frac{f'(x)g(x) - f(x)g'(x)}{g^2(x)}\]
\end{theorem}


\section{Rolle's theorem}

\begin{theorem}[Rolle's theorem]
	If $f$ is a real function continuous on the interval $[a,b]$ and differentiable on the interval $(a,b)$ and $f(a)=f(b)=0$, then there exists an $\xi \in (a,b)$ such that $f'(\xi)=0$
\end{theorem}

Rolle's theorem (in a way) states the extreme value theorem in terms of a derivative; though it finds use in applications such as mathematical optimization, it can be used as a tool to prove more powerful propositions on differentiable functions.



\subsection{Mean value theorem}

We'll now use Rolle's theorem to obtain an even more powerful tool; the mean value theorem.

Many theorems in analysis justify intuitions of using differentials like fractions (such as the chain rule, and reverse chain rule as we will see with integration); the mean value theorem is the primary method for setting up such proofs.

- mean value theorem (MVT)
\begin{theorem}[Mean value theorem (MVT)]
	If $f$ is a real function continuous on the interval $[a,b]$ and differentiable on the interval $(a,b)$, then there exists an $\xi \in (a,b)$ such that $f'(\xi)=\frac{f(b)-f(a)}{b-a}$
\end{theorem}

This result can be generalized further.

\begin{theorem}[Generalized mean value theorem (GMVT)]
	If $f,g$ are real functions continuous on the interval $[a,b]$ and differentiable on the interval $(a,b)$, then there exists an $\xi \in (a,b)$ such that $\frac{f'(\xi)}{g'(\xi)}=\frac{f(b)-f(a)}{g(b)-g(a)}$
\end{theorem}


MVT 

Recall that sometimes the techniques of substitution and algebraic manipulation alone are insufficient in calculating what a limit results to. There exists an incredibly useful theorem that greatly simplifies the calculation of difficult limits.

\subsection{L'Hôpital's rule}

Calculating limits of functions can be difficult at times, however there exists an extremely powerful result that facilitates many situations.

\begin{theorem}[L'Hôpital's rule]
If $f,g$ are real functions that are differentiable on $U$ and their limits at $c\in U$ are both equal to $0$ or $\pm \infty$ and the limit of $\frac{f}{g}$ exists at $c$, then the following holds.
\[ \lim x \to c \frac{f(x)}{g(x)} = \lim x \to c \frac{f'(x)}{g'(x)}\]
\end{theorem}

This sort of makes sense; informally the rule compares the 'speed' that the 2 functions approach $0$ or $\infty$.

Some argue that it takes the fun out of finding elegant ways to calculate interesting limits, however if you just need to get a limit evaluated, L'Hôpital's rule is the way to go.

\subsection{Inverse function theorem}

\begin{theorem}[Inverse function theorem]
Let $f : D \subseteq \mathbb{R} \to \mathbb{R}$ be a continuously differentiable real function and $x_0 \in D$. If $f'(x_0)\neq 0$, then there exists open intervals $I$ of $x_0$ and $J$ of $f(x_0)$ such that $f$ is invertible on $I$ and $f^{-1} : J \to I$ is continuously differentiable.
Moreover, the following holds for all $y \in J$.
\[ (f^{-1})'(y) = \frac{1}{f'(f^{-1}(y))} \]
\end{theorem}

On a first pass, this theorem looks like a neat rule for diferentiating inverse functions without calculating the inverse itself. Though this is the case, the theorem actually tells us much more; it tells us that for such funtions the inverse function always exists (at least locally) and is always continuously differentiable!

The formula for calculating the derivative of an inverse function is a very sweet bonus indeed, but the crux of this theorem is the confidence that we can invent such functions locally and take derivatives without qualms.


\section{Smooth functions}

- smooth function
- continuously differentiable function


\begin{definition}[Continuously differentiable real function]
A \emph{continuously differentiable real function} is a differentiable real function whose derivative is continuous.
\end{definition}

\begin{definition}[Smooth real function]
A \emph{smooth function} is a function that can be differentiated an infinite amount of times.
\end{definition}
- power series
- Taylor series


\section{Taylor series}

\begin{definition}[Taylor series of a real function]
	\[ T(f,x_0;x) = \sum^{\infty}_{n=0} \frac{f^{(n)}(x_0)}{n!} (x-x_0)^n \]
Taylor coefficients
\end{definition}
 The idea is that we find a polynomial $T$ where the derivatives at $x_0$ are the same as those of $f$ at $x_0$.

 We're essentially matching derivatives, however is there any assurance that this converges to $f$ (assuming it even converges at all)? When our space of interest is an open interval, we often employ \emph{Taylor's theorem} as a way to check that a Taylor series converges to $f$ on an open interval.

\begin{theorem}[Taylor's theorem]
	Let $f$ be a real, $N$-times differentiable function on open interval $I$ with $x_0 \in I$, then for any domain element $x \in I \setminus \{x_0\}$ there exists some $\xi \in [x_0,x]$ where the following holds.
	\[f(x) = \sum^{N-1}_{n=0} \frac{f^{(n)}(x_0)}{n!}(x-x_0)^n) + \frac{f^{(N)}(\xi)}{N!}(x-x_0)^N\]
\end{definition}




\begin{definition}[Real analytic function]
A real function is \emph{real analytic on $E$} iff for any $x_0 \in E$, the Taylor series of $f$ converges to $f$ in a neighborhood of $x_0$.
A real function that is real analytic on its entire domain is an \emph{analytic function}.
\end{definition}


Taylor's theorem allows us to prove the following corollaries regarding what functions are real analytic.

Real analytic functions are real smooth functions.



\section{Optimization on differentiable functions}
- convex function
- concave funtion
- concave funtion
- global maximum
- global minimum
- local maximum
- local minimum 

\begin{proposition}[Derivative test]
If a real function $f$ with continuous second derivatives has $f'(x_0)=0$ and $f''(x_0) > 0$, then $x_0$ is a local minimum of $f$.
\end{proposition}