\chapter{Infinite dimensional linear spaces}


We will explore "infinite" analogues of various fundamental linear algebra concepts that can exist in topological linear spaces.


\section{Infinite linear combinations}

Studying linear combinations is fundamental for analyzing linear spaces since it is a precursor to the notion of a \emph{basis}, which is perhaps the ultimate tool for studying a linear space.




\begin{definition}[Infinite linear combination]
In a TLS and for a set of vectors and set of coefficients, an \emph{infinite linear combination} is a vector series of the following form.
	\[\sum^{\infty}_{k=1} c_k \mathbf{b}_k \]
\end{definition}



\section{Closed linear span}



Given the fact that we are working in a topological linear space, we can and examine all the limit points of this set (i.e the infinite series that could potentially be made.



\begin{definition}[Closed linear span]
Let $V$ be a TLS, a \emph{closed linear span of $S$}  is the closure in $(V, \mathcal{T})$ of the span $\mathrm{span}(S)$
	\[\mathrm{cl}( \mathrm{span}(S))\]
\end{definition}

Due to the continuity of $+,\cdot$, this set is indeed its own linear subspace.

\begin{proposition}
Closed linear spans a linear subspace
\end{proposition}





\section{Schauder basis (functional analysis)}


We motivate our discussion by the following proposition that holds for a Hamel basis in a finite dimensional space; the whole idea of a Schauder basis is to find such a representation in our infinite dimensional spaces.



\begin{proposition}
Let $V$ be an $n$-dimensional linear space with basis $(\mathbf{b}_k )_{k=1}^{n}$, then for every vector $\mathbf{v}$ there exists a unique sequence $(c_k)_{k=1}^n$ such that the following holds
\[ \sum^{n}_{k=1} c_k \mathbf{b}_k \]
\end{proposition}



Because this is a finite linear combination, the order of terms may be permuted without effect since vector addition is commutative. However an infinite linear combination however may not obey the property of unconditional convergence; it is possible that permuting terms of the series can change its limit or even make it diverge! Because of this, we require a countable set of vectors obeying a similar property to be either linearly ordered or a sequence.



\begin{definition}[Schauder basis of a linear space]
	Let $V$ be a TLS, a \emph{Schauder basis of $V$} is a sequence of distinct vectors $( \mathbf{b}_n )$  such that for any vector $\mathbf{v} \in V$ there exists a unique $( c_n )^{\infty}_{n=1}$  such that the following holds.
\[ \mathcal{v} = \sum^{\infty}_{n=1} c_n \mathbf{b}_n \]
\end{definition}



\begin{proposition}
Separable Hilbert spaces have countable Schauder basis.
\end{proposition}






\section{$\ell^p$ spaces}


Our knowledge of TLSs and infinite dimensional linear spaces is sufficient to begin developing some concrete function spaces. One of the most basic examples of infinite dimensional linear spaces are sequence spaces, among which the $\ell^p$ are a special subspace that will be seen to form.



\begin{definition}[Sequence space]
Given a field $F$, the space $F^{\mathbb{N}}$ represents all sequences $\{x_n\}_{n \in \mathbb{N}} \in F$ with vector addition and multiplication defined as such.
$\{x_n\}_{n \in \mathbb{N}} + \{y_n\}_{n \in \mathbb{N}} = \{x_n + y_n\}_{n \in \mathbb{N}}$
$a \{x_n\}_{n \in \mathbb{N}} = \{ a x_n\}_{n \in \mathbb{N}}$
\end{definition}

One can naturally imagine a product topology for this linear space, and one can prove that this is frechet, complete, and metrizable. Unfortunately however, there is no continuous norms that can be made on this space, hence unfotunately it cannot be a normed linear space.

If we work backwards from an ideal norm, reminiscent of the EUclidean norm we are familiar with, perhaps we can find a normed subspace of this space.


\subsection{$p$-norm}
In linear algebra one can study spaces $\mathbb{R}^n$ (or $F^n$) using a finite $p$-norm. When dealing with sequence spaces, we can equip them with similarly with a \emph{$p$-norm}.


\begin{definition}[$p$-norm of $F^{\mathbb{N}}$]
When $p \geq 1$
\[ \|\{x_n\}\|_{\infty} = (\sum^{\infty}_{n=1} |x_n|^{p})^{\frac{1}{p}}\]
\[ \|\mathbf{x}\|_{\infty} = \max_{n \in \mathbb{N}} (|x_{n}|)\]
\end{definition}

It is customary to check that this is actually a norm for $p \geq 1$.


For now, we assume that we are considering sequences such that its $p$-norm in the natural product topology converges to begin with. However notice that the triangle inequality fails to hold for $p < 1$, and hence cannot be a norm for $p<1$. But as it turns out, the $p$-norm is nice enough to act as a norm for $p\geq 1$


\subsection{$\ell^p$ space}

The main issue with the $p$-norm is that of convergence; on a general sequence space, the $p$-norm may very well diverge! Therefore we'll consider a linear subspace for which the $p$-norm does converge; this is the notion of an $\ell^p$ space.

Considering that we want a linear space on sequences that agrees with some $p$-norm as its norm. Our $p$-norm satisfies the definition of a norm with $p \geq 1$, however the $p$-norm doesn't converge for every sequence. Therefore we admit into our subspace the sequences for which the chosen $p$-norm converges, which translates to sequences such that that inner sum $\sum^{\infty}_{k=1} |[\mathbf{x}]_k|^p$ converges.



\begin{definition}[$\ell^p$ space]
An \emph{$\ell^p$ space of $F$} is a linear subspace of $F^{\mathbb{N}}$ that is the normed linear space $(F,+,\cdot,\ell^p(F),\|\cdot\|_p)$ with the following.
\[ \ell^p(F) = \{\{x_n\} \in F^{\mathbb{N}} : \|\{x_n\}\| < \infty \}\]
\end{definition}


Since we have constructed this space with the intention of creating a normed linear space on sequences ona complete field, we immediately know the following, using our construction as a proof.

\begin{proposition}
$p \geq 1$, then $\ell^p$ is a Banach space.
\end{proposition}

What's more is the following.

\begin{proposition}
$\ell^2$ is a Hilbert space.
\end{proposition}

The $\langle \{x_n\} , \{y_n\} \rangle = \sum^{\infty}_{n=1} |x_n y_n| $ is a valid inner product in $\ell^2$ since the fact that if follows the Cauchy-Schwarz inequality can be used to not only guarantees its convergence, but also guarantees it follows the triangle inequality, hence proving it is an inner product.




\section{$L^p$ spaces}


Now that we have familiarity with some infinite dimensional linear spaces, we can begin studying functionals.

In real analysis we consider $\mathbb{R}$ and the functions upon it; functional analysis will consider function spaces and the \emph{functionals} upon it. 



\begin{definition}[Essential supremum]
\[ \mathrm{esssup} f = \inf \{c \in Y : \mu ( \{ x \in X : f(x) \leq c \})=0 \} \]
\end{definition}





\begin{definition}[$L^p$-norm]
\[\|f\|_p = (\int_{X} |f|^{p} d\mu)^{\frac{1}{p}} \]
	\[\|f\|_{\infty} = \mathrm{esssup} |f| \]
\end{definition}



\begin{definition}[$L^p$ space]
	\[L^p (X, \Sigma, \mu) = \{ f : X \to \mathbb{R} : \|f\|_p < \infty \}\]
\end{definition}



To develop some inequalites on $L^p$ spaces, we first prove a simple lemma that requires nothing than basic real analysis and optimization.

\begin{lemma}[Young's inequality]
\[\frac{1}{p}+\frac{1}{p}=1\]
	\[ab \leq \frac{a^p}{p}+\frac{b^q}{q}\]
\end{lemma}


\begin{theorem}[Hölder's inequality]
\[\frac{1}{p}+\frac{1}{p}=1\]
	\[\|fg\|_1 \leq \|f\|_{p} \|g\|_{q}\]
\end{theorem}

The triangle inequality is one of the most powerful tools in mathematical analysis, and indeed it exists on $L^p$ spaces.

\begin{theorem}[Minkowski's inequality]
\[p \in [1,\infty)\cap\{\infty\}\]
\[\|f+g\|_p \leq \|f\|_p +  \|g\|_p\]
\end{theorem}




\subsection{Othogonal Schauder basis of an $L^p$ spaces}
%Legendre polynomials
%Chebyshev polynomials
%Hermite polynomials
%Orthogonality of trigonometric functions



\begin{definition}[Chebyshev polynomials]
The \emph{Chebyshev polynomials (first kind)} are a sequence of polynomials $(T_n)_{n \in \mathbb{N}}$ relating to cosine relations of an angle with factor $n$.
\[ T_n ( \cos(\theta) ) = \cos (n \theta) \]
\begin{itemize}
\item $ T_{0} = 1$
\item $ T_{1} = x$
\item $ T_{n+1} = 2x T_{n} - T_{n-1}$
\end{itemize}


The \emph{Chebyshev polynomials (second kind)} are a sequence of polynomials $(U_n)_{n \in \mathbb{N}}$ relating to sine relations of an angle with factor $n$.
\[  U_{n-1} ( \cos(\theta) ) = \frac{\sin (n \theta)}{\sin (\theta)} \]
\begin{itemize}
\item $ U_{0} = 1$
\item $ U_{1} = 2x $
\item $ U_{n+1} = 2x U_{n} - U_{n-1}$
\end{itemize}

\end{definition}



\begin{definition}[Legendre polynomials]
The \emph{Legendre polynomials} are a sequence of polynomials $(P_n)_{n \in \mathbb{N}}$ forming an orthogonal basis for $L^2 [-1,1]$ with weight 1.
\end{definition}

\begin{definition}[Hermite polynomial (Physicist's)]
	\[H_n (x) = (-1)^n e^{x^2} \frac{d^n}{dx^n}(e^{-x^2})\]
\end{definition}

The Hermite polynomials are an orthogonal Schauder basis for $L^2(\mathbb{R})$



In fact the trigonometric functions form an orthogonal basis for $L^2 (I)$, a result called Carleson's theorem; in this way functional analysis gives the justification for the methods of Fourier analysis!


\subsection{Riesz-Fischer theorems}

The Riesz-Fischer theorem gives the central motivation for studying $L^p$ spaces rather than spaces of Riemann integrable functions, because it gives assurance that the $L^p$ spaces are complete.


\begin{theorem}[Riesz-Fischer theorem I]
	\[p \in [1,\infty) \implies L^p \text{ is a Banach space}\]
\end{theorem}


\begin{theorem}[Riesz-Fischer theorem II]
\[L^2 \text{ is a Hilbert space}\]
\end{theorem}


These results are why $L^p$ spaces are often preferred over spaces of Riemann integrable functions; because they are complete spaces.



\begin{theorem}[Weierstrass approximation theorem ($L^p$ space)]
	Let $I$ be a compact interval of $(\mathbb{R} , \mathcal{E})$, the set of polynomials on $I$ is dense in  $L^2(I)$ and $L^1 (I)$
\end{theorem}

\section{$C^k$ spaces}
$C^0$ space (continuous functions)
$C^k$ space (k times continuously differentiable functions)
$C^{\infty}$ space (infinitely differentiable functions)
$C^{\omega}$ space (analytic functions)


The Weierstrass approximation theorem is often covered in a course on real analysis; and now we will return to it by restating it equivalently by means of 

\begin{theorem}[Weierstrass approximation theorem]
Let $I$ be a compact interval of $(\mathbb{R} , \mathcal{E})$, the set of polynomials on $I$ is dense in  $C(I)$
\end{theorem}