\chapter{Quadratic forms}

Though we are familiar with linear forms, we will now study \emph{quadratic forms}; functions of the following form.


\begin{definition}[Quadratic form]

\[ Q(c\mathbf{x}) = c^2 q(\mathbf{x}) \]
\[ Q(\mathbf{x} +\mathbf{y}) -q(\mathbf{x}) - q(\mathbf{y}) \text{ is a bilinear form}\]
\end{definition}

These functions have wide ranging applications in other areas of mathematics such as number theory, differential geometry, mathematical optimization, convex analysis, and statistics.

We know that linear maps can be represented as $T(\mathbf{x})=\mathbf{A}\mathbf{x}$, and linear forms as $f(\mathbf{x}=\mathbf{v}^{\intercal}\mathbf{x}=\mathbf{v} \cdot \mathbf{x}$. Naturally we would like some concrete 'linear algebraic' representation that will assist us in manipulating quadratic forms. As our luck would have it, there is such a representatio and it is not too hard to derive.

\begin{proposition}
$\mathbf{A}$ is a square matrix
\[ Q(\mathbf{x}) =\mathbf{x}^{\intercal}  \mathbf{A} \mathbf{x} \]
\[ Q(\mathbf{x}) = \sum_{i=1}^n \sum_{j=1}^n \mathbf{A}_{ij} x_i x_j \]
\end{proposition}


This matrix representation of a quadratic form is not unique, however they are unique if we choose its symmetric matrix representation.


\begin{proposition}
For any quadratic form $q$, there exists a unique symmetric matrix $\mathbf{A}$ such that $q$ is representable as such.
\[ q(\mathbf{x}) =\mathbf{x}^{\intercal}  \mathbf{A} \mathbf{x} \]
\end{proposition}

From now we will always assume we are using such a symmetric matrix representation (and transform any quadratic forms not in this representation). The following proposition gives us a way to transform any quadratic form into this symmetric matrix representation.

\begin{proposition}
Let $ q(\mathbf{x}) =\mathbf{x}^{\intercal}  \mathbf{A} \mathbf{x}$ be a quadratic form, then  $\frac{\mathbf{A} + \mathbf{A}^{\intercal}}{2}$ is a symmetric matrix that represents the same quadratic form
\end{proposition}

A keen suggestion is that one could also represent quadratic forms using a unique upper (or even lower) triangular matrix, however this is seldom mentioned! The reason symetric matrixes are more desirable will be seen when we study the \emph{principle axis theorem}.

\section{Definiteness}


\begin{definition}[Definite quadratic form]
A \emph{positive-definite quadratic form}
	\[  Q \text{ is positive-definite } \iff \forall \mathbf{x} \in V \setminus \{\mathbf{0}\} [ Q(\mathbf{x}) > 0 ] \]
A \emph{negative-definite matrix}
\[  \mathbf{A} \text{ is negative-definite } \iff \forall \mathbf{x} \in V \setminus \{\mathbf{0}\} [ Q(\mathbf{x}) < 0 ] \]
\end{definition}

\begin{definition}
A \emph{positive semi-definite matrix}
A \emph{negative semi-definite matrix}
\end{definition}

We could also talk about definiteness when refering specifically to the matrix in the quadratic form


\begin{definition}[Definite matrix]
A \emph{positive-definite matrix}
	\[  \mathbf{A} \text{ is positive-definite } \iff \forall \mathbf{x} \in V \setminus \{\mathbf{0}\} [ \mathbf{x}^{\intercal}\mathbf{A}\mathbf{x} > 0 ] \]
A \emph{negative-definite matrix}
\[  \mathbf{A} \text{ is negative-definite } \iff \forall \mathbf{x} \in V \setminus \{\mathbf{0}\} [ \mathbf{x}^{\intercal}\mathbf{A}\mathbf{x} < 0 ] \]
\end{definition}

\begin{definition}
A \emph{positive semi-definite matrix}
A \emph{negative semi-definite matrix}
\end{definition}


Definiteness is often important in determining concavity.

\subsection{Definiteness and eigenvalues}



multiplicity of symmetric eigenequation
eigenvalues of positive-definite eigenequation
eigenvalues of orthogonal eigenequation
Eigenvalues and trace
Eigenvalues and determinant
The theory of eigenvalues is closely related to the propeorties of matrix 'definiteness'.
\begin{proposition}
All eigenvalues of a square positive definite matrix are positive.
\end{proposition}

\begin{proposition}
All eigenvalues of a square positive definite matrix are positive.
\end{proposition}




\section{Principle axis theorem}


Because of our symmetric matrix representation for quadratic forms, and the fact that symmetric matrixes are orthogonally diagonalizable

\begin{theorem}[Principle axis theorem]
For any quadratic form $q$, there exists some linear map $T$ such that we have the representation $q(T(\mathbf{x})=(T(\mathbf{x})^{\intercal}\mathbf{D}T(\mathbf{x})$, where $\mathbf{D}$ is a diagonal matrix.
\end{theorem}

Although this is an existence theorem, the proof is constructive (i.e applying orthogonal diagonalization to the quadratic form's symmetric matrix). By orthogonally diagonalizing the quadratic form's matrix one can find this linear map (in matrix form) and diagonal matrix.


One application of this is to the sum of squares.