\part{Advanced} \chapter{Advanced matrix algebra} For a reader with a background in the fields of abstract algebra, further insight using group and ring theory can bring some concepts to light. This chapter is designed for those who are familiar with basic matrix theory and abstract algebra, and want to enrich their understanding of matrixes by combining the fields. \section{General linear group} - $\mathrm{GL}(n,F)$ group \section{Determinant as a homomorphism} - Determinant as a group homomorphism from $\mathrm{GL}(n,F)$ to $(F,\cdot)$ \section{Special linear group} - $\mathrm{SL}(n,F)$ group \section{Orthogonal group} - $a\mathrm{O}(n,F)$ group - $\mathrm{SO}(n,F)$ group - $\mathrm{GL}(n,F) / \mathrm{SL}(n,F)$ isomorphic to $F^\times$ \section{Conjugate matrix} Since the inner product is conjugate-symmetric, the matrix conjugate plays a role in the theory of inner product spaces. We now introduce the conjugate go a matrix. \begin{definition}[Conjugate-transpose matrix] \[ [\mathbf{A}^{*}]_{ij} = [\mathbf{A}]_{ji}^{*}\] \end{definition} \subsection{Hermitian matrix} \begin{definition}[Hermitian matrix] A \emph{Hermitian matrix} is a complex square matrix that equals its own conjugate transpose \[ \mathbf{A} = \mathbf{A}^{*}\] \end{definition} Since the conjugate of a real number is the same real number, the following result holds trivially. \begin{proposition} Real symmetric matrixes are Hermitian matrixes. \end{proposition} For maps between complex Euclidean spaces, the complex Hermitian matrix plays the role of real symmetric matrixes regarding spectral decomposition; the spectral theorem can be generalized to Hermitian matrixes. \begin{theorem}[Spectral theorem (Hermitian matrix)] $\mathbf{A}$ is an $n \times n$ Hermitian matrix iff any of the following hold. \begin{itemize} \item $\mathrm{spec}(\mathrm{A})$ has cardinality $n$ and is a subset of $\mathbb{R}$ \item Eigenvectors associated with different eigenvalues of $\mathbf{A}$ are orthogonal \item $\mathrm{A}$ is orthogonally diagonalizable \end{itemize} \end{theorem} \subsection{Unitary matrix} Analogue of orthonormal matrix for complex matrixes. \begin{definition}[Unitary matrix] A \emph{unitary matrix} is a complex square matrix whose inverse is its conjugate-transpose \[ \mathbf{A}^{*}= \mathbf{A}^{-1}\] \end{definition} Indeed, unitary matrixes get their name from the fact that they represent unitary automorphisms on $\mathbb{C}^n$ \begin{proposition} A matrix is unitary iff the linear map associated to $\mathbf{U}$ represents a linear automorphism onto $\mathbb{C}^n$ \end{proposition} This can be seen simply by considering a unitary automorphism the dot product as defined on $\mathbb{C}^n$. \[\mathbf{v} \cdot \mathbf{u} = \mathbf{U} \mathbf{v} \cdot \mathbf{U} \mathbf{u} \] \[ = \mathbf{v}^{*}\mathbf{U}^{*} \mathbf{U} \mathbf{u} \] For this statement to hold we require that $\mathbf{I} = \mathbf{U}^{*} \mathbf{U}$, but since $\mathbf{U}$ invertible since it is an automorphism, we can multiply both sides of the equation by $\mathbf{U}^{-1}$ to arrive at the definition of a unitary matrix. \subsection{Normal matrix} When considering complex matrixes, the SVD can only be done for normal matrixes. \section{Matrix exponential} It id defined in an analogous fashion to the exponential function of real and complex analysis. \begin{definition}[Matrix exponential] \[e^{\mathbf{X}} = \sum^{\infty}_{n=0} \frac{\mathbf{X}^n}{n!} \] \end{definition} \begin{proposition} \[e^{\mathbf{X}} = \lim_{n \to \infty} (\mathbf{I}+\frac{\mathbf{X}}{n})^n \] \end{proposition} It arises in the study of simultaneous differential equations. \section{Cayley-Hamilton theorem} \begin{definition}[Characteristic polynomial (scalar, linear algebra)] \[ p_{\mathbf{A}}(\lambda) = \det (\mathbf{A}-\lambda\mathbf{I}) \] \end{definition} \begin{definition}[Characteristic polynomial (matrix, linear algebra)] \[ p_{\mathbf{A}}(\mathbf{B}) = \det (\mathbf{B}-\lambda\mathbf{I}) \] \end{definition} \chapter{Dual spaces} Not only are these spaces interesting to study and compare to their regular linear space, they will be necessary in our study of multilinear algebra to develop tensors. \begin{definition}[Dual space] Let $(V,F,+,\cdot)$ be a linear space over $F$, its \emph{dual space} $(V^{*},F,+,\cdot)$ is the linear space over $F$ formed by $V^{*}=\mathcal{L}(V,F)$ (all the linear forms AKA covectors on $V$) \end{definition} As usual, we verify that the dual space is indeed a linear space over $F$. \begin{proposition} Let $(V,F,+,\cdot)$ be a linear space spanned by n linearly independent vectors $\{\mathbf{b}_1 , \hdots, \mathbf{b}_n \}$. Then a basis for $V^*$ are the linear forms $\mathbf{b}^1 , \hdots , \mathbf{b}^n$ that obey the following. \[ \mathbf{b}^{i}(\mathbf{b}_i) = \delta_{ij} \] \end{proposition} Note that $\mathbf{b}^{i}$ are linear forms that are basis elements of our dual space; we typically represent covectrs with the same bold font as vectors, however we index covectors of a basis by superscripts rather than subscripts; This convention will prove very convenient in the study of tensors, since tensors will ultimately be a way of combining vectors and covectors. Since we can use the linear space to construct a basis for its dual space with the same amount of basis elements, our dual space has the same dimension as the original space. \begin{proposition} Let $(V,F,+,\cdot)$ be a linear space, then its dual space has the same dimension as $V$. \[\mathrm{dim}(V^*)=\mathrm{dim}(V) \] \end{proposition} An interesting physical interpretation due to our proposition is that dual linear spaces represent vectors by their projections onto basis vectors rather than linear combinations. \subsection{Covariant transform} \[ \mathbf{c}_i = \sum_{j} [\mathbf{P}]_{ji} \mathbf{b}_{j}\] \[ [\mathbf{v}]_{B} = \mathbf{P} [\mathbf{v}]_{C}\] \[ \mathbf{c}^i = \sum_{j} [\mathbf{P}]_{ij} \mathbf{b}^{j}\] \[ [\mathbf{v}]^{C} = \mathbf{P} [\mathbf{v}]^{B}\] Covectors transform covariantly with respect to the vector transition matrix. Vectors transform contravariantly with respect to the vector transition matrix. \subsection{Annihilators} \subsection{Transpose of linear map} \begin{definition}[Transpose linear map] Let $(V,F,+,\cdot),(W,F,+,\cdot)$ be a linear spaces over $F$ and $f : V \to W$ be a linear map, then the \emph{transpose of $f$} is the linear map $f^{\intercal} : W^* \to V^*$ defined as the following. $f^{\intercal} (l) = l \circ f$. \end{definition} \chapter{Multilinear algebra} \section{Multilinear maps} We have studied linear maps, and we have implicitly looked as some bilinear map; bivariate functions on two linear spaces that are linear (matrix multiplication and the inner product). Indeed there are many noteworthy multilinear maps aside from these, even if these two are the most renown. \begin{definition}[Multilinear map] A \emph{mulilinear map} or \emph{multilinear transform} is a function $f : V^n \to W$ \begin{itemize} \item $f$ is additive in every argument \item $f$ is homogeneous of degree $1$ in every argument \end{itemize} \end{definition} This can be though of as a 'multivariate linear map'! Inner products, the cross product, and quadratic forms are examples of multilinear maps that we are already familiar with. We will now commence a deeper study on these operations generally. \subsection{Outer product} \subsection{Hadamard product} \subsection{Kronecker product} \section{Direct sum} \section{Tensor product} - Constructing as a quotient space - Constructing from the basis of the underlying linear spaces - Constructing by a univeral property Universal properties are a concept from category theory that may be inaccessible to us, and although we are familiar with the notion of a basis the cleanest way is by quotient spaces. \begin{definition}[Tensor product] Let $Z$ be a linear space with $V\times W$ as its (hamel) basis. \[\mathbf{v} \in V \] \[\mathbf{w} \in W \] \[c \in F\] \[ c(\mathbf{v} * \mathbf{w}) = c\mathbf{v} * \mathbf{w} = \mathbf{v} * c\mathbf{w} \] \[ (\mathbf{v}_1+\mathbf{v}_2 ) * \mathbf{w} = \mathbf{v}_1 * \mathbf{w} + \mathbf{v}_2 * \mathbf{w} \] \[ \mathbf{v} * ( \mathbf{w}_1 + \mathbf{w}_2 ) = \mathbf{v} * \mathbf{w}_1 + \mathbf{v} * \mathbf{w}_2 \] \[V \otimes W = Z / S\] \[\mathbf{v} \otimes \mathbf{w} = \mathbf{v} * \mathbf{w} + S\] \end{definition} This is a rather formal algebraic construction, however to \[\mathbf{v} \otimes \mathbf{w} \in V \otimes W\] \[\mathbf{v} \in V \] \[\mathbf{w} \in W \] \[ c(\mathbf{v} \otimes \mathbf{w}) = c\mathbf{v} \otimes \mathbf{w} = \mathbf{v} \otimes c\mathbf{w} \] \[ (\mathbf{v}_1+\mathbf{v}_2 ) \otimes \mathbf{w} = \mathbf{v}_1 \otimes \mathbf{w} + \mathbf{v}_2 \otimes \mathbf{w} \] \[ \mathbf{v} \otimes ( \mathbf{w}_1 + \mathbf{w}_2 ) = \mathbf{v} \otimes \mathbf{w}_1 + \mathbf{v} \otimes \mathbf{w}_2 \] As it turns out, a basis of this product space is all the tensors between basis vectors of the two lienar spaces. As it turns out $\mathbb{R}^n \otimes V \cong V^n$ \section{Grassmann algebra} Tensor theory will become fruitful to the study of the Grassman algebra \begin{definition}[Grassman algebra] A \emph{Grassman algebra over $(V,K,+,\cdot)$} or \emph{exterior algebra over $(V,K,+,\cdot)$} is the quotient ring $\bigwedge (V) = T(V) / I$ \begin{itemize} \item $T(V)$ is the tensor algebra on $(V,K,+,\cdot)$ \item $I$ is the ideal formed by $x \otimes x$ where $x \in V$ \end{itemize} \end{definition} This definition relies heavily on ring theory, but what does it characterize? It is all about introducing the wedge product; an operation that generalizes the cross product to arbitrary linear spaces. The central rule encoded into the algebra is that $\mathbf{v} \times \mathbf{v} = \mathbf{0}$. It is not a commutative algebraic structure, however we can prove that it is anticommutative. Wedge product k-blade bivector multivector Hodge star operator \chapter{Tensors} The demands of physics have required more complicated classes of linear functions. The mathematical attempt to classify them has been by the invent of \emph{tensors}. We are at least familiar with scalars, vectors, linear maps, multilinear maps, linear forms, bilinear forms, and multilinear form. These are indeed all examples of different types of tensors. \section{Vectors and covectors (linear forms)} $[\mathbf{x}]_k$ means the scalar at row $k$ of the column vector $\mathbf{x}$ $[\mathbf{A}]_{ij}$ means the scalar at row $i$, column $j$ of the matrix $\mathbf{A}$ $T(\mathbf{x}) = \mathbf{A}\mathbf{x}$ $\mathbf{f}_{k} = \mathbf{e}_{i}$ Consider a linear space $V$ with an old basis $\mathbf{e}_i$ and new basis $\mathbf{f}_i$. We use some rule of the following form to translate each $\mathbf{e}_i$ to the new basis (allowing the transformation of any vector to the new basis). \[\mathbf{f}_i = \sum_{j=1} ^{n} [\mathbf{P}]_{ij} \mathbf{e}_j \] \[\mathbf{e}_i = \sum_{j=1} ^{n} [\mathbf{P}^{-1}]_{ij} \mathbf{f}_j \] contravariant \[\mathbf{P}^{-1}[\mathbf{v}]_{\mathcal{E}} = [\mathbf{v}]_{\mathcal{F}}\] Vectors have the contravariant property covariant \[\mathbf{P}[\phi]_{\mathcal{E}} = [\phi]_{\mathcal{F}}\] Linear forms (covectors) have the covariant property \section{Ricci calculus} Tensors are quite complex objects, so we introuce some notation that compactifys expressions while retaining all necessary information of the tensor. \begin{definition}[Tensor (indexed array definition)] A \emph{$(p,q)$ tensor} is a \end{definition} \subsection{Indexing laws} Superscript is used for contravariant coefficients Subscript is used for covariant coefficients \subsection{Einstein notation} Einsteun notation is essentially the omission of the Sigma summation symbol when the context is clear. \[\sum_{i} \mathbf{e}_i x^{i}\] The main idea is that \emph{sums are implicitly implied over an index when you have a subscripted symbol times a superscripted symbol}. These expressions are so common that it becomes well worth introducing this notation. the summation index may not appear more than twice to invoke Einstein notation. \section{Tensor} We have investigated scalars and vectors, and we've had the epiphany that matrixes represent linear maps on vectors; however just as we create an algebra upon matrixes (linear maps), can we form an algebra on 'tensors' (multilinear maps)? \begin{definition}[Tensor (multilinear map definition)] Let $(V,F,+,\cdot)$ be a linear space. A \emph{$(p,q)$ tensor} is a multilinear map that takes $p$ covectors from $V^{*}$ and $q$ vectors from $V$ as arguments, and outputs a scalar. \end{definition} We know that a linear map can be defined pureply by knowing how it transforms a linear space's basis elements (hence why its matrix representation is possible). The same holds for tensors, and it is why some think of tensors as a possibly multidimensional array (even if that explanation doesn't reveal a tensor's behaviour). \begin{proposition}[Tensors as an array] Let $(V,F,+,\cdot)$ be a linear space with a $(p,q)$ tensor $T$. Define the array of scalars $T^{i_1 \hdots i_p}_{j_1 \hdots j_q} = T(\varepsilon^{i_1}, \hdots , \varepsilon^{i_p}, \mathbf{e}_{j_1} , \hdots , \mathbf{e}_{j_q})$. \end{proposition} %We have seen matrixes define linear maps on vectors, where an one dimension of the output vector may be linearly affected by all dimensins of the input vector. Hence the tensors define \emph{multilinear maps}, and can also define \emph{multilinear} equations just like how matrixes define linear equations. \begin{definition}[Tensor algebra] \end{definition} \begin{definition}[Tensor product] For two linear spaces $V,W$, the \emph{tensor product} is the linear space $V \otimes W$ associated with a bilinear map $T : V \times W \to V \otimes W$ \end{definition} We can create an equivalent but more abstract definition of a tensor in this way \begin{definition}[Tensor (tensor product definition)] A $(p,q)$ tensor is an element of a tensor product space $\bigotimes^{p}_{n=1} V^{*} \otimes \bigotimes^{q}_{n=1} V$ \end{definition} \chapter{More eigenequations} minimal polynomial