\chapter{Linear spaces}


Much of the time, students have previous experience with vectors in the form of \emph{Euclidean vectors} and even representing them as \emph{coordinate vectors} as defined in the previous chapter.



Linear spaces are an algebraic structure that allows a general framework to define and handle vectors.

We would like a system that defines vectors based on their algebraic behaviour than being confined to a single, concrete coordinate implementation; this could help us translate problems stated in different coordinate implementations with more ease.

This would also allow mathematicians to abstract the algebraic behaviour of vectors and study mathematical objects with the same properties. For example, functional analysis makes use of linear algebra by studying functions as vectors (rather studying function spaces as linear spaces)!

This is because a linear space is an algebraic structure; it doesn't require a literal interpretation of 'direction' to be; it just needs a set of our synthetically defined objects ( that we will eventually call vectors) to have certain algebraic properties.





\section{Linear space}



Behold, the ultimate algebraic structure at our disposal to categorize the behaviour of vectors.


\begin{definition}[Linear space]
A \emph{linear space} is an ordered pair $(V,F,+,\cdot)$ of a field and a set of objects called \emph{vectors}. We say that $V$ is a linear space over $F$.

\begin{itemize}
\item $\mathbf{u} + (\mathbf{v} + \mathbf{w}) = \mathbf{u} + (\mathbf{v} + \mathbf{w})$
\item $\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}$
\item $\exists \mathbf{0}  \in V[ \forall \mathbf{v} \in V [\mathbf{v} +\mathbf{0} = \mathbf{v}]] $
\item $\forall \mathbf{v} \in V [\exists (-\mathbf{v})  \in V[ \mathbf{v} + (-\mathbf{v}) = \mathbf{0}]] $
\item $c(d\mathbf{v}) = (cd)\mathbf{v}$
\item $c(d\mathbf{v}) = (cd)\mathbf{v}$
\item $1_F\mathbf{v} = \mathbf{v}$
\item $c(\mathbf{u} + \mathbf{v}) = c\mathbf{u} + c\mathbf{v}$
\item $(c+d)\mathbf{v} = c\mathbf{v}+d\mathbf{v}$
\end{itemize}
\end{definition}


The mention of a field is a notion that readers may not be familiar with.

A field is a type of algebraic structure of a set where some notion of 'addition' and 'multiplication' are defined and follow some other special properties (in many cases, these \emph{are} the addition and multiplication that we are familiar with).

For most situations we will come across, $F$ is the set of real or complex numbers.

Linear spaces are actually a special clas of algebraic structures called \emph{modules}, however often one learns linear algebra before ring theory, so we accept the completely equivalent and more accessible definition above.


\subsection{Properties of linear spaces}
\[0_{F}\mathbf{v} = \mathbf{0}\]
\[ \exists! \mathbf{0} \in V [ \forall \mathbf{v} \in V [\mathbf{v} +\mathbf{0} = \mathbf{v}]] \]
\[ \forall \mathbf{v} \in V [\exists! (-\mathbf{v}) \in V [ \mathbf{v} + (-\mathbf{v}) = \mathbf{0}]] \]
\[ c\mathbf{0}=\mathbf{0} \]
\[ (-c\mathbf{v})= c(-\mathbf{v}) \]
\[ c\mathbf{v}=\mathbf{0} \implies c=0_F \lor \mathbf{v}=\mathbf{0}\]


We will define subtraction of vectors in the following manner.
\[ \mathbf{v}-\mathbf{u} = \mathbf{v} + (-\mathbf{u}) \]

\subsection{Examples of linear spaces}

\begin{example}[Real coordinate spaces]
$(\mathbb{R}^n,\mathbb{R},+,\cdot)$
\end{example}

This is by far the most common type of linear space that one encounters, and you have probably used it implicity numerous times! It is the cornerstone for mathematical analysis, cartesian geometry, it is an object of particular interest in topology, and outside pure mathematics it is fundamental in physics; it's influence resounds throughout mathematics as a whole.

\begin{example}[Linear space of polynomials up to degree 5]
\end{example}

Notice that due to the requirements of a linear space, this space includes the zero polynomial, has dimension 6. Such a linear space may be useful in the area of field theory.

\begin{example}[Linear spaces of solutions of linear ODEs]
\end{example}

Since solutions of linear ODEs form a linear space, linear algebra becomes an extremely powerful tool in the analysis of solutions, for instance by the use of the Wronskian.


\section{Linear subspace}

Perhaps only a handful of the vectors within $V$ are enough to satisfy the properties of a linear space. When we consider subsets of linear spaces that form their own linear space, we call this a \emph{linear subspace}.

\begin{definition}[Linear subpsace]
A \emph{linear supspace} of $(V,F,+, \cdot)$ is a linear space $(W,F,+,\cdot)$ such that $W$ is a subset of $V$. $W \leq V$ denotes that $W$ is a subgroup of $V$.
\[ W \leq V \iff (W \subseteq V) \land (W,F,+,\cdot ) \text{ is a linear space} \]
\end{definition}

\subsection{Examples of linear subspaces}
\begin{proposition}[Intersection subspace]
\[(V,F,+,\cdot)\]
\[(W,F,+,\cdot)\]
\[V \cap W \leq V\]
\[V \cap W \leq W\]
\end{proposition}



\begin{proposition}[Additive superspace]
\[(V,F,+,\cdot)\]
\[(W,F,+,\cdot)\]
\[V \leq V+W \]
\[W \leq V+W \]
\end{proposition}


When we eventually study linear maps, we will see that they add substantial contributions to the theory of subspaces.




\section{Linear combinations}

Studying linear combinations is fundamental for analyzing linear spaces since it is a precursor to the notion of a \emph{basis}, which is perhaps the ultimate tool for studying a linear space.




\begin{definition}[Finite linear combination]
For a set of vectors and set of coefficients, a \emph{finite linear combination} is a finite sum of vectors scaled by some scalar.
\[\sum^{n}_{i=1} c_i \mathbf{v} \]
\end{definition}

\subsection{Linear span}

To represent the set of all finite linear combinations possible with a certain set of vectors, we use a \emph{linear span}

\begin{definition}[Linear span]
$(F,V)$
\[S = \{\mathbf{v}_1 , \hdots , \mathbf{v}_n\}\]
\[\mathrm{span}(S) = \{ \sum^{n}_{i=1} c_i \mathbf{v}_i : c_i \in F \} \]
\[A \subseteq V \land |A| < \aleph_0\]
\[\mathrm{span}(A) = \{ \sum_{\mathbf{v} \in B} c_{\mathbf{v}} \mathbf{v} : c_{\mathbf{v}} \in F \land B \subseteq A \land |B| < \aleph_0\} \]
\end{definition}

\begin{proposition}
Linear spans form a subspace.
\end{proposition}


\subsection{Linear independence}


Representing vectors by means of linear combinations is nice, however what's even better would be if vectors have a \emph{unique} linear combination, given that we're using some fixed set of spanning vectors.


\begin{definition}[Linearly independent set]
A finite set of vectors $A$ is \emph{linearly independent set} iff no element $\mathbf{v} \in A$ can be represented as a linear combination of element sin $A \setminus \{\mathbf{v} \}$
\[A \text{ is linearly independent } \iff [ \nexists \mathbf{u} \in A [ \exists B \subseteq A [ |B| < \aleph_0 \land \sum_{\mathbf{v} \in B} c_{\mathbf{v}} \mathbf{v} = \mathbf{u} \land c_{\mathbf{v}} \in F ] ]  \]
\end{definition}

There are alternative equivalent ways that a linearly independent set may be defined; different definitions give more intuition for the meaning of the concept, plus they may facilitate constructing a proof.

\begin{proposition}[Equivalent definitions for linearly independent set]
\begin{itemize}
	\item no element $\mathbf{v} \in A$ can be represented as a linear combination of element sin $A \setminus \{\mathbf{v} \}$
	\item There is no nontrivial linear combination of vectors in $A$ for $\mathbf{0}$
	\item The determinant of the matrix whose columns are the vectors in $A$ is $0$
\end{itemize}
	\[A \text{ is linearly independent } \iff  \nexists B \subseteq A [ |B| < \aleph_0 \land \sum_{\mathbf{v} \in B} c_{\mathbf{v}} \mathbf{v} = \mathbf{0} \land c_{\mathbf{v}} \in F ]   \]
\end{proposition}

\section{Basis (Linear space)}




\begin{definition}[Basis of a linear space]
A \emph{basis of $V$} is a set of linearly independent vectors $B$ such that every element in $V$ is a linear combination of the vectors in $B$. $|B|$ is known as the \emph{rank of $B$}.
	\[ \mathcal{B} \text{ is a basis of } V \iff \mathcal{B} \text{ is linearly independent } land \mathrm{span}(\mathcal{B})=V\]
\begin{definition}


\begin{definition}[(Hamel) Basis of a linear space]
Let $V$ be a linear space, a \emph{(Hamel) basis of $V$} is a set of vectors $B$ such that every element in $V$ is a linear combination of the vectors in $B$, and every finite subset of $B$ is linearly independent. $|B|$ is known as the \emph{rank of $B$}.
	\[ \mathcal{B} \text{ is a basis of } V \iff \mathcal{B} \text{ is linearly independent } land \mathrm{span}(\mathcal{B})=V\]
\begin{definition}

The term 'Hamel basis' is more common in the stuy of functional analysis where (spoiler alert), there are different types of basis' needed to generate more complicated spaces; we will not consider such things in this book.

\begin{proposition}[Equivalent definitions of a basis]
\begin{itemize}
	\item $\mathrm{span}(\mathcal{B})=V$ and the vectors of $\mathcal{B}$ are linearly independent to eachother
	\item There is no nontrivial linear combination for $\mathbf{0}$
\end{itemize}

\end{proposition}



\section{Dimension}


The notion of a basis is extremely useful, however can we \emph{always} analyze a linear space by means of a basis? This is an interesting question because it relies on the foundation of mathematics that we use.

We'll start by considering this over \emph{finite linear spaces}.

\begin{definition}[Finite linear space]
A \emph{finite linear space} is a linear space that is spanned by a finite set of vectors.
\[ V \text{ is a finite linear space } \iff \exists A \subseteq V [|A| < \aleph_0 \land \mathrm{span}(A)=V ] \]
\end{definition}

Assuming we have a finite linear space, we can prove this with no qualms.

\begin{proposition}
Let $(V,F,+,\cdot)$ be a finite linear space, there exists some $\mathcal{B} \subseteq V$ such that $\mathcal{B}$ is a basis of $V$.
\end{proposition}

We require the use of an axiom that was once controversial among mathematicians; the axiom of choice. As it turns out, infinite linear spaces always having a basis is actually logically equivalent to the axiom of choice, making the following result not just of interest in linear algebra but also in mathematical logic.

\begin{proposition}
Let $(V,F,+,\cdot)$ be a infinite linear space, there exists some $\mathcal{B} \subseteq V$ such that $\mathcal{B}$ is a basis of $V$ iff we accept the axiom of choice.
\end{proposition}

In this book we accept the axiom of choice, so we accept the previous proposition. That said, most work (especially in the 'fundamentals' part of the book) focuses on results pertaining to finite linear spaces.

Now we know that all linear spaces have a basis, but what is even nicer is the following fact.

\begin{proposition}
Let $(V,F,+,\cdot)$ be a linear space, all its basis' have the same cardinality.
\end{proposition}

This can be proven by showing that linear spans larger than the basis are associated with linear equations with free variables, and any set smaller than the basis can only span a linear subspace.

Since all basis' for the same linear space must have the same cardinality, we can define \emph{dimension} as an intrinsic property of a linear space pertaining to the cardinality of any basis it can produce.

\begin{definition}[Dimension of a linear space]
Let $(V,F,+,\cdot)$ be a linear space, the \emph{dimension of $V$} is the cardinality of any basis of $V$. We write $\mathrm{dim}(V)$ to represent this cardinality.
\end{definition}

\begin{corollary}
For $(V,F,+,\cdot)$ Any set of more than $\mathrm{dim}(V)$ vectors is linearly independent.
\end{corollary}

Our proof of the existence of a basis for any lienar space relies on a method of 'cherry picking' linearly independent vectors until a basis is constructed; this proof is nice because it is constructive! The proof essentially corresponds to an algorithm on how to generate a basis. For the finite linear spaces, we only need to pick linearly independent vectors until we reach the dimension.

\begin{theorem}[Basis theorem]
Let $(V,F,+,\cdot)$ be linear space with $\mathrm{dim}(V)=n$, any set of $n$ linearly independent vectors of $V$ is a basis for $(F,V)$
\end{theorem}


\begin{proposition}
All finite dimensional linear spaces over $F$ with the same dimension are isomorphic.
\[ (V,F,+_V,\cdot_V) \]
\[ (W,F,+_W,\cdot_W) \]
\[\mathrm{dim}(V) < \infty \implies [ \mathrm{dim}(V) =\mathrm{dim}(W) \iff V \cong W] \]
\end{proposition}

Though we can construct such an isomorphism for finite dimensional linear spaces, this result doesn't hold for linear spaces with infinite dimension.

This theorem is enlightening because it implies the behaviour of linear spaces is independent of any basis, with the isomorphism itsem being the 'change of basis' function!

A change of basis is often thought of a change of coordinate systems, which means a linear isomorphism onto the same linear space. This is exactly the definition of a \emph{linear endomorphism}.


\section{Spanning set theorem}


\begin{theorem}[Spanning set theorem]
For any $\mathbf{v} \in A$ that is linearly dependent to $A \setminus \{\mathbf{v}\}$, we have $\mathrm{span}(A)=\mathrm{span}(A \setminus \mathbf{v})$
\end{theorem}




\section{Linear space constructions}

%\subsection{Intersection of linear spaces}
%
%A simple yet interesting way of constructing a linear space is by merely taking the intersection of two linear subspaces!
%
%\begin{proposition}
%Let $U,W \subseteq V$ and $(F,W),(F,U)$ be linear spaces over $F$, then $(F,U \cap W)$ is a linear space.
%\begin{proposition}

\subsection{Direct sum of linear spaces}

\begin{definition}[Direct sum of linear spaces]
Let $U,W$ be linear subspaces of $V$, then the direct sum $U\oplus W$ contains all vectors that have a unique representation as the sum of a vector in $U$ and vector in $W$
\[ U \leq V \]
\[ W \leq V \]
\[U \oplus W \iff \exists ! \forall \mathbf{v} \in U \oplus W ( \mathbf{u} \in U, \mathbf{w}\in W [  \mathbf{v} = \mathbf{u} + \mathbf{w} ] ) \]
\end{definition}

One way to think of the direct sum is to think of the disjoint union of the subspaces as well as the zero element $[ (U \cup W) \setminus (U\cap W) ] \cup \{\mathbf{0} \}$ and then closing it up with vector addition.

\subsection{Direct product of linear spaces}

NOTE; THIS SECTION REQUIRES MORE EXPLANATION

Let $F$ be a field, then $F^n$ is a linear space over $F$.


	\begin{proposition}
direct product and direct sum have a linear isomorphism under finite applications.
	\end{proposition}
\subsection{Quotient linear space}

It is highly reccomended that readers be familiar with group theory before understanding quotient spaces.

\begin{definition}{Left coset (Linear space)}
Let $W \leq V$, then $\mathbf{v} + W = \{\mathbf{v} + \mathbf{w} : \mathbf{w} \in W \}$
\end{definition}


\begin{proposition}{Left coset (Linear space)}
Left cosets are either equal or disjoint. They form an equivalence relation.
\end{proposition}

%Those proficient in set theory recognize $\mathbf{v}+U$ as equivalence relations, and furthermore, those familiar with group theory know them as cosets.
As it turns out, left cosets form their own linear space!


\begin{definition}[Quotient linear space]
A \emph{quotient linear space} $(F,V / U)$ where $U \leq V$ is a linear space over $F$ with the distinct sets $\mathbf{v}+U=\{ \mathbf{v}+\mathbf{u} : \mathbf{u} \in U \}$ as its vectors.
\end{definition}






















\chapter{Linear maps}


Though the notion of a basis is arguably the most important tool to analyze linear spaces, linear maps are definitely a contender.

These types of functions a powerful tools used to study properties of linear spaces since they 'preserve' the algebraic structure of the domain elements in the codomain space (homomorphism).

Additionally they are also worthy of study in and of themselves, displaying interesting properties as well as finding various practical applications.


\begin{definition}
Let $V,W$ be vector spaces over $K$, a \emph{linear map} or \emph{linear transform} is a function between linear spaces $T : V \to W$ with the following properties.
\begin{itemize}
	\item $T(\mathbf{u}+_{V} \mathbf{v}) = T(\mathbf{u})+_{W} T(\mathbf{v})$ ($T$ is additive)
	\item $T(c\mathbf{v}) =	 cT(\mathbf{v})$ ($T$ is homogeneous of degree 1)
\end{itemize}
The set $\mathcal{L}(V,U)$ represents the set of all linear maps $T : V \to U$
\end{definition}


The following theorem explains why matrixes are all the rage in linear algebra, and it is because of their intimate relationship to linear maps.

\begin{theorem}
	For any linear map $T : \mathbb{F}^n \to \mathbb{F}^m$, there exists a matrix over $F$ ($\mathbf{A}$) such that $T(\mathbf{x})=\mathbf{A}\mathbf{x}$. Furthermore, any matrix over $F$ defines a linear map.
\end{theorem}

This theorem gets to the heart of matrixes in the context of linear algebra and why matrix addition and multiplication are defined the way they are. In these contexts, matrixes are not just arrays of scalars, but are used to characterize and specify linear maps of the form $T : F^n \to F^m$!

Matrix multiplication was defined to reflect the composition of linear maps, matrix addition to reflect the addition of linear maps; solving a simultaneous linear equations is essentially finding what domain elements of these linear maps are mapped to some image element. This allows us to view familiar problems from a more functional perspective; a milestone in our mathematical maturity.

%In Euclidean spaces, all linear maps have a matrix representation, and any matrix. This is what I meant before that matrixes \emph{are} linear maps; matrix multiplication and matrix addition was defined so to preserve composition of linear maps (and applying linear map to vector) and addition of linear maps respectively in a Euclidean space.

By considering the fact that any finite linear space is isomorphic to $F^n$ , we have the following as a corollary.

\begin{proposition}
If $\mathrm{dim}(\mathrm{dom}(T))$ is finite, then there exists some matrix $\mathbf{A}$ such that $T(\mathbf{x})=\mathbf{A}\mathbf{x}$.
\end{proposition}

This reiterates the point that although matrixes can be used as a means of indexing elements in a grid-like fashion, their real power lies in the fact that under matrix multiplication they represent linear maps of finite dimension


Linear transforms on scalars are quite mundane; they merely scale the number by some fixed factor. For larger dimensions however, linear transforms can be interpreted with more geometrical flavour; linear transforms rotate, shear, and scale vectors!



\subsection{Kernel (linear maps)}
\begin{definition}[Kernel (Linear map)]
Let $V,W$ be linear spaces over $K$ and  $T : V \to W$ a linear map.
\[ \mathrm{ker}(T)= \{ \mathbf{x} : T(\mathbf{x})=\mathbf{0} \}\]
\end{definition}

\subsection{Image (linear maps)}
\begin{definition}[Image (Linear map)]
Let $V,W$ be linear spaces over $K$ and  $T : V \to W$ a linear map.
\[ \mathrm{Im}(T)= \{ T(\mathbf{x}) : \mathbf{x} \in \mathrm{dom}(T) \}\]
\end{definition}




\subsection{Exaples of linear maps}

Identity map
Shearing maps
Rotation maps
Dilation maps
\[ T(\mathbf{x}) = (c\mathbf{I}) \mathbf{x}\]
Reflection map


\subsection{Linear isomorphisms}


\begin{definition}[Linear isomorphism]
Let $(V,F,+,\cdot)$ and $(U,F,+,\cdot)$ be two linear spaces over $F$. A \emph{linear isomorphism} $f : V \to U $ is a bijective linear map.
If there exists an isomorphism between $V$ and $U$, then the linear spaces are \emph{isomorphic} to eachother, also written as $V \cong U$.
\end{definition}




\subsection{Linear endomorphisms}

\begin{definition}[Linear endomorphism]
linear isomorphism between linear spaces over the same set of vectors; When two linear spaces are equal
\end{definition}

\begin{proposition}
Let $(V,F,+,\cdot)$ be a linear space, then the set of linear endomorphims where $(f+g)(\mathbf{v}) = f(\mathbf{v}) + g(\mathbf{v}),(cf)(\mathbf{v}) = cf(\mathbf{v}) $ is a linear space over $F$ with dimension $\mathrm{dim}^2(V)$
\end{proposition}





\subsection{Linear forms}
\begin{definition}[Linear form]
Let $(V,F,+,\cdot)$ be a linear space, then a \emph{linear form } is a linear map $f : V \to F$ Mapping the linear space to the field it is over.
\end{definition}










\begin{proposition}
\[ T: V \to W \text{ is a linear map } \implies \mathrm{ker}(T) \leq V\]
\end{proposition}

- Column space
- Row space
- Rank
\[\mathrm{rank}(\mathbf{A}) =  \mathrm{dim}(\mathrm{col}(\mathbf{A})) = \mathrm{dim}(\mathrm{row}(\mathbf{A})) \]

\[ \mathrm{nullity}(\mathbf{A}) = \mathrm{dim}(\mathrm{ker}(\mathbf{A})) \]
\section{Rank-nullity theorem}

\begin{theorem}[Rank-Nullity theorem (Linear maps)]
Let $\mathbf{T}: V \to W$ be a linear map, then the following holds.
\[ \mathrm{dim}(V) = \mathrm{nullity}(T) + \mathrm{rank}(T)\]
\end{theorem}


\begin{theorem}[Rank-Nullity theorem (Matrix)]
Let $\mathbf{A}$ be an $m \times n$ matrix of elements of the field $F$, then the following holds.
\[ n = \mathrm{nullity}(\mathbf{A}) + \mathrm{rank}(\mathbf{A})\]
\end{theorem}









































\chapter{Normed linear spaces}


NOTE; MOVE THIS CHAPTER AFTER ORTHOGONALITY

We have studied linear spaces as algebraic structures, however we can equip them with certain functions that link linear spaces to applications in topology and analysis.

We will not apply the theory here to other branches of mathematics, however we will discuss some elementary consequences of introducing these functions to a linear space.

One of the basic functions we can consider is the \emph{norm}.

\section{Norm}

The idea of the norm is to define a notion of distance of vectors from the origin. 

\begin{definition}[Norm]
Given a vector space $(V,F,+,\cdot)$ where $F$ is the field $\mathbb{R},\mathbb{C}$, a \emph{norm over $V$} is a function $\|\cdot\| : V \to \mathbb{R}$ with the following properties.
\begin{itemize}
	\item $\|\mathbf{v} + \mathbf{u}\| \leq \|\mathbf{v}\| + \|\mathbf{u}\|$
	\item $\|\lambda\mathbf{v} \| = |\lambda| \|\mathbf{v}\|$
	\item $\|\lambda\mathbf{v} \| = 0 \implies \|\mathbf{v} = \mathbf{0}$
\end{itemize}
\end{definition}

\begin{definition}[Normed linear space]
A \emph{normed linear space} is an ordered pair $(V,\|  \cdot \|)$ of a linear space $V$ over $\mathbb{R},\mathbb{C}$ and a norm over $V$.
\end{definition}

One can prove a few immediate properties from this definition.

\[| \|\mathbf{v}\| - \|\mathbf{u}\| | \leq \|\mathbf{v} - \mathbf{u}\|\]
\[ \| \sum^{n}_{k=1} \mathbf{v}_{k}\| \leq  \sum^{n}_{k=1} \|\mathbf{v}_{k}\| \]
\[ \|\lambda\mathbf{v} \| = 0 \iff \|\mathbf{v} = \mathbf{0} \]


Perhaps one of the main motivations for a norm is that it allows the creation of a \emph{distance function}; a function that defines the distance between any 2 vectors. We always want distance functions to obey these properties to match our intuition, and indeed our distance function obeys them all.
-triangle ineq
- symmetry
- |x|=0 iff x=0


In fact, this distance function will actually have some even nicer properties that not all distance functions may have.
-trasnlation invariance
-scaling invariance

A set with a distance function define upon it is an important object in topology called a \emph{metric space}.



\begin{proposition}
Normed linear spaces have the distance function $d(\mathbf{x},\mathbf{y})=\|\mathbf{x}-\mathbf{y}\|$.
Moreover, this distance function is translation and scaling invariant.
\end{proposition}

\subsection{Examples of normed linear spaces}


\subsubsection{Finite $p$-norm}
Expression that generalizes the Euclidean norm.

\begin{definition}[$p$-norm of $\mathbb{R}^n$]
When $p \geq 1$
\[ \|\mathbf{x}\|_{p} = \sum^{n}_{k=1} |[\mathbf{x}]_k|^{p}\]
\[ \|\mathbf{x}\|_{\infty} = \max_{k \in \mathbb{N}\cap [1,n]} (|[\mathbf{x}]_k|)\]
\end{definition}

It is customary to check that this is actually a norm for $p \geq 1$.



\section{Inner product spaces}

Linear spaces endowned with a special function called an inner product.


\begin{definition}[Inner product]
Given a vector space $(V,F,+,\cdot)$ where $F$ is the field $\mathbb{R},\mathbb{C}$, an \emph{inner product over $V$} is a function $\langle \cdot  , \cdot \rangle : V \times V \to F$ with the following properties.
\begin{itemize}
	\item $\langle \mathbf{u} , \mathbf{v} \rangle = \langle \mathbf{v} , \mathbf{u} \rangle^{*}$
	\item $\langle \lambda\mathbf{u}+ \mu\mathbf{w} , \mathbf{v} \rangle = \lambda \langle \mathbf{u} , \mathbf{v} \rangle + \mu \langle \mathbf{w} , \mathbf{v} \rangle$
	\item $\mathbf{v} \neq \mathbf{0} \implies \langle \mathbf{v} , \mathbf{v} \rangle > 0$
\end{itemize}
\end{definition}

Note that conjugate symmetry implies $\langle \mathbf{v} , \mathbf{v} \rangle$ is a real number, so we can discuss it with respect to the standard partial order over the real numbers ($<$).


\begin{definition}[Inner product space]
An \emph{inner product space} is an ordered pair $(V,\langle  \cdot , \cdot \rangle)$ of a linear space $V$ over $\mathbb{R},\mathbb{C}$ and an inner product over $V$.
\end{definition}

One can prove a few immediate properties from this definition.

\[\langle \mathbf{v} , \mathbf{0} \rangle = \langle \mathbf{0} , \mathbf{v} \rangle = 0\]
\[\langle \mathbf{v} , \mathbf{v} \rangle \in [0,\infty)\]
\[\langle \mathbf{v} , \mathbf{v} \rangle =0 \iff \mathbf{v}=\mathbf{0}\]
\[ \langle \mathbf{v} , \lambda\mathbf{u}+ \mu\mathbf{w} \rangle = \lambda^{*} \langle \mathbf{v} , \mathbf{u} \rangle + \mu^{*} \langle \mathbf{v} , \mathbf{w} \rangle \]



\subsection{Canonically induced norm}


If we have an inner product space, we can actually use our inner product to create a norm function.

\begin{proposition}
Let $V$ be an inner product space, then there exists some norm $\|\cdot\|$ defined on $V$ in the following manner.
\[\|\mathbf{v}\| = \sqrt{\langle \mathbf{v} , \mathbf{v} \rangle}\]
\end{proposition}

This means that any inner product space is also a normed linear space.



\subsection{Cauchy-Schwarz inequality}

From inner product spaces arises perhaps the most renown inequalities in all of mathematics.
\begin{theorem}[Cauchy-Schwarz inequality]
\[  \langle \mathbf{u},\mathbf{v} \rangle^{2}  \leq \langle \mathbf{u},\mathbf{u} \rangle \cdot \langle \mathbf{v},\mathbf{v} \rangle \]
\end{theorem}
It is proven by analyzing the nonnegative function $f(\lambda) = \langle \mathbf{u} -\lambda \mathbf{v} , \mathbf{u} -\lambda \mathbf{v}\rangle$, expanding it into a quadratic, noticing it has negative discriminant, and then manipulating the inequality on the discriminant.
Alternatively,

For specific spaces, Lagrange multipliers or induction can prove the inequality on $\mathbb{R}^n$, and a result called Hölder's inequality is used to prove it on $L^p$ spaces (which are out of the scope of this book).


\subsection{Parallelogram law}
\begin{theorem}[Parallelogram law (Linear algebra)]
Let $V$ be an inner product space whose norm is the canonical norm, then the following statement holds.
\[ 2\|\mathbf{u}\|^2 + 2\|\mathbf{v}\|^2 = \|\mathbf{u}+\mathbf{v}\|^2 + \|x-y\|^2 \]
\end{theorem}



\subsection{Pythagorean theorem}
\begin{theorem}[Pythagorean theorem (Linear algebra)]
Let $V$ be an inner product space, then the following statement holds.
\[ \|\mathbf{u}\|^2 + \|\mathbf{v}\|^2 = \|\mathbf{u}+\mathbf{v}\|^2 \iff \mathbf{u} \perp \mathbf{v}\]
\end{theorem}




\subsection{Unitary isomorphims}

Of great interest are the linear isomorphism that not only respect the algebraic structure of the operations, but also the inner product.

\begin{definition}[Unitary isomorphism]
A \emph{unitary isomorphism} Is a linear isomorphism $T$ with the following.
\[\langle T(\mathbf{v}) , T(\mathbf{u}) \rangle = \langle \mathbf{v} , \mathbf{u} \rangle\]
\end{theorem}









\section{Euclidean space}

The most notorious example of an inner product space is undisputedbly Euclidean space.

\begin{definition}
	A \emph{Euclidean space} is an inner product space $(\mathbf{R}^n,\cdot)$.
\begin{itemize}
	\item $\mathbb{R}^n$ is the set of $n$-valued real vectors
	\item $\cdot$ is the inner product on the Euclidean space, called a \emph{dot product}
\end{itemize}
\end{definition}


\subsection{Dot product}

We have an understanding of perpendicularity; vectors that have directions that are completely
\begin{definition}[Dot product]
Consider the Euclidean space $\mathbb{R}^n$, the \emph{dot product} is a map $\cdot: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ that follows the following algebraic properties.
\begin{itemize}
\item $\mathbf{u} \cdot \mathbf{v} = \mathbf{v} \cdot \mathbf{u}$
\item $(c\mathbf{u}+ d\mathbf{w}) \cdot \mathbf{v} = c ( \mathbf{u} \cdot \mathbf{v})+d(\mathbf{w}\cdot \mathbf{v})$
\item $\mathbf{u} \cdot \mathbf{u} \geq 0$
\end{itemize}
\end{definition}

One may recognize that this is essentially the definition of an inner product of $\mathbb{R}^n$ (conjugate symmetry implies symmetry when dealing only with real numbers); It turns out that there is only 1 such function that is an inner product on $\mathbb{R}^n$ so our definition talks about 'the' dot product rather than 'a' dot product.

By writing vectors as linear combinations of their standard cartesian basis, we can see this alternate definition for the dot product.

\begin{proposition}[Alternative definition for dot product]
$\mathbf{u} \cdot \mathbf{v} = \sum^{n}_{i=1} \mathbf{u}_{i} \mathbf{v}_{i}$
\end{proposition}

Historically, the dot product was a formalization of a coefficient in the law of cosines.
\[ c^2 = a^2 + b^2 -2ab\cos(C) \]

By representing vectors with polar (or spherical etc.) coordinates yet with the standard cartesian basis, one can use our previous form of the dot product to define the following.

\begin{proposition}[Geometric interpretation for dot product]
	$\mathbf{u} \cdot\mathbf{v} = \| \mathbf{u} \| \| \mathbf{v} \| \cos (\theta )$
\end{proposition}

This is exactly the $ab\cos (C)$ coefficient in the law of cosines, generalized as a dot product.

Notice that cosine of $\frac{\pi}{2}$ is 0, so the dot product ; this gives us the edge we need to formally define what perpendicularity means in Euclidean space!



\begin{definition}[perpendicular pair of vectors]
In $\mathbb{R}^n$, for any $\mathbf{u}$ and $ \mathbf{v}$ in the sert are a \emph{perpendicular pair of vectors} iff $\mathbf{u} \cdot \mathbf{v} = 0$
\[\mathbf{v} \perp \mathbf{u} \iff  \mathbf{v} \cdot \mathbf{u} = 0\]
\end{definition}

\begin{definition}[Perpendicular set]
A \emph{perpendicular set} is a set of vectors $S$ such that any distinct such that any distinct vectors $\mathbf{v},\mathbf{u} \in S$ are perpendicular to eachother.
\end{definition}




\subsection{Results of Euclidean geometry}
Euclidean geometry is characterized in these spaces, so all the familiar theorems of Euclidean geometry may be expressed in terms of this inner product space. For instance, here is a familiar friend of ours who has had a linear algebra 'glow-up'.

\begin{definition}[Pythagorean theorem (Linear algebra)]
\[\mathbf{u} \propto \mathbf{v} \implies \| \mathbf{u} + \mathbf{v} \|^2 = \|\mathbf{u}\|^2 + \|\mathbf{v}\|^2\]
\end{definition}


Indeed one of the powers (and original motivations) of linear algebra is providing a strong foundations for analytic geometry (geometry using coordinate system).





\section{Cross product}

Given any vector in $\mathbb{R}^2$, we can find exactly 2 perpendicular vector directions by making use of our knowledge of linear maps to rotate our vector by $\frac{\pi}{2}$; multiplying the vector by the matrix $\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}$ does the trick. The other direction is given by the negative of this vector.

We will take a rather modern way of defining and thinking about such an operator; if we consider $\mathbb{R}^3$, a given vector has an infinite amount of vector directions perpendicular to it. However given 2 vectors, there are exactly 2 vector directions perpendicular to both vectors. This begs the question; can we define an operator that calculates 1 of the 2 vectors perpendicular to a pair of vectors? The answer is yes, if we imagine some algebraic conditions that such an operator would follow, we end up with the cross product.

\begin{definition}[Cross product]
Consider the Euclidean space $\mathbb{R}^3$, the \emph{cross product} is a map $\times : \mathbb{R}^3 \times \mathbb{R}^3 \to \mathbb{R}^3$ that is linear in both arguments (bilinear map) and follows the following algebraic properties.
\begin{itemize}
\item $\mathbf{v}  \times \mathbf{v}=\mathbf{0}$
\item $\mathbf{e}_1  \times \mathbf{e}_2=\mathbf{e}_3$
\item $\mathbf{e}_2  \times \mathbf{e}_3=\mathbf{e}_1$
\item $\mathbf{e}_3  \times \mathbf{e}_1=\mathbf{e}_2$
\end{itemize}
\end{definition}

One can easily verify that the definition of the cross product defines a unique operator, that is any two operators following these properties are actually the same operator).

From a mathematical point of view, this example is interesting as it takes two arguments and is linear in both; the term for this is a bilinear map. In fact, this particular bilinear map is generalized by the notion of a Grassman algebra; this is covered in the advanced part of this book.

\begin{proposition}[Alternative definition for cross product]
	$\mathbf{u} \times \mathbf{v} = \begin{vmatrix} \mathbf{e}_1 & \mathbf{e}_2 & \mathbf{e}_3 \\ \mathbf{u}_1 & \mathbf{u}_2 & \mathbf{u}_3 \\ \mathbf{v}_1 & \mathbf{v}_2 & \mathbf{v}_3 \end{vmatrix}$
\end{proposition}

\begin{proposition}[Geometric interpretation for cross product]
	$\mathbf{u} \times \mathbf{v} = \|\mathbf{u}\|\|\mathbf{v}\|\sin ( \theta ) \hat{\mathbf{n}}$
\end{proposition}

We now derive some elementary properties of the cross product.
\begin{proposition}
\[\mathbf{u} \times \mathbf{v} = -(\mathbf{v}\times \mathbf{u}) \]
\[\mathbf{w} \cdot ( \mathbf{u} \times \mathbf{v} ) = \mathrm{det}(\mathbf{w},\mathbf{u},\mathbf{v}) \]
\[\mathbf{w} \times (\mathbf{u} \times \mathbf{v}) = (\mathbf{w}\cdot \mathbf{v})\mathbf{u} - (\mathbf{w} \cdot \mathbf{u})\mathbf{v}\]
\end{proposition}