\part{Fundamentals} \chapter{Introduction to ODES} At the beginning of one's mathematical journey, they learn the 4 arithmetic operations and how to do calculations with them. Later on, the 4 operations (and possibly more functions) connect variables in algebraic equations, where one needs to find a number (or set of numbers) that satisfies the equation, possibly restricted to some set. When first solving algebraic equations, students are unfamiliar with complex numbers and hence implicitly restrict themselves to real number solutions. \[ x^3 -6x^2 +11x -6 = 0, x \in \mathbb{R}\] In solving this cubic equation, one finds the following set of solutions on $\mathbb{R}$. \[ x \in \{1,2,3\}\] Similarly when one learns mathematical analysis, they learn about the derivative and integral of functions and the techniques to calculate them. Similarly, we can consider \emph{differential equations} which relate a variable function with its derivatives. Our goal is to find such a function that satisfies the equation. In this part, we will only consider differential equations with solutions that are real, complex, and $\mathbb{R}^n$ functions. \[ y' = y, y \in C^{1}(\mathbb{R})\] It turns out that this differential equation has the following solution set on $C^1(\mathbb{R})$. \[ y \in \{ f \in C^1(\mathbb{R}) : f(x) = ke^{x}, k \in \mathbb{R} \}\] Such equations prove useful in areas of mathematical analysis, but are also heavily used in modelling phenomenon for applied mathematics. Algebraic equations can be represented as $P(x)=0$, where $P$ is some polynomial. In a similar guise we can form a definition for ODEs; \emph{ordinary differential equations}. \begin{definition}[Algebraic equation] An \emph{algebraic equation} is an equation of the following form, defined by some polynomial $P$. \[ P(x) = 0\] Solutions for $x$ are in $\mathbb{C}$ \end{definition} \begin{definition}[Ordinary differential equation (ODE)] An \emph{ordinary differential equation (ODE)} is an equation of the following form, defined by some $f : D \subseteq \mathbb{R}\times \mathbb{R}^n \mathbb{R}$ \[ f(x,y(x),y'(x),\hdots ,y^{(n)}(x)) = 0\] Solutions for $y : I \to \mathbb{R}$ are in $C^{n}(I)$, for some interval $I$. \end{definition} If those are ordinary differential equations, does that mean we have... extraordinary differential equations? The 'ordinary' means that we consider differential equations to be in terms of and solved by univariate functions rather than multivariate functions. Differential equations stated and solved in terms of multivariate functions are known as \emph{partial differential equations}, whose theory becomes immensely more difficult. As if this wasn't enough, there are still many other types of equations relating a univariate function to its derivatives that aren't covered in our definition of an ODE. \begin{itemize} \item Systems of ODES \item ODEs with multiple function variables (like $y_1 y'_2 = y_1 - y_2$) \item Differential equations of univariate functions that employ the idea of fractional derivatives \end{itemize} That said, the power of solving a regular ODE can go quite far in tackling such types of differential equations. \begin{definition}[General solution of an ODE] Given an ODE $y'=f(x,y(x))$, its \emph{general solution} is a unique function that satisfies the DE up to a set of constants (if such a function exists). \end{definition} Not all ODEs can be reduced to a single general solution, however many that we will discuss in this book can. If one constructs and solves an ODE to model some physical phenomenon, they would typically like to ascribe meaning to the otherwise meaningless free constants in the general solution. Suppose that physical restraints of a problem imply that a useful solution must satisfy $y(0)=1$, or $y(2)=0$ or some related condition; we call this an IVP. \begin{definition}[Initial value problem (IVP)] An \emph{initial value problem (IVP)} is an ODE with a specified $(x_0,y_0) \in \mathrm{dom}(f)$, so $y(x_0)=y_0$ \end{definition} IVPs are vital in applications to boil an ODE down to a specific function that work in the context of application, and it is interesting to consider from a pure mathematical perspective for the uniqueness of solutions that this permits (more on this in the advanced chapter). On that note, it would be beneficial to us to have at least a vague inclination on what ODEs will have solutions and better yet, unique solutions. We will tackle this result in the advanced part of the book, but for now we offer a mantra that you'll just have to take on faith for a little while. \section{Types of differential equations} \subsection{Order and degree} \begin{definition}[Order of a differential equation] Highest order derivative \end{definition} \begin{definition}[Degree of a differential equation] Highest power applied to a derivative \end{definition} \subsection{Forms of equations} \begin{definition}[Degree of a differential equation] \end{definition} There exist some more exotic types of differential equatiosn such as fractional differential equations \section{First order separable ODE} Separable ODEs are those that can be written in such a way that it is the , that is, the $y$ and $x$ can be 'separated' as variables of 2 different functions. Separable ODEs of order 1 turn out to be the simplest type of ODEs, having solution method that is elementary and elegant. By doing some algebraic manipulation and using the reverse chain rule, we can make the problem into that of solving an algebraic equation for $y$. \[y' = f(y)g(x)\] \[\frac{y'}{f(y)} = g(x)\] \[ \int \frac{y'}{f(y)}dx = \int g(x) dx\] \[ \int \frac{1}{f(y)}dy = \int g(x) dx\] \begin{theorem}[First order separable ODE] Given a first order separable ODE $y' = f(y)g(x)$, its general solution satisfies the following algebraic equation. \[ \int \frac{1}{f(y)}dy = \int g(x) dx\] \end{theorem} \section{Integrating factor} Integrating factor proves vital in the solutions to first order linear ODEs. Recall what a first order linear ODE looks like \[y' + q_1 (x)y = q_2 (x)\] One may notice that the $y' + q_1 (x)y$ part looks almost like the result of applying the product rule. If we had some $I(x)$ satisfying $I' = q_1(x) I$, then $I(x)y' + q_1 (x) I(x) y$ would be reversible by the product rule. $I' = q_1(x) I$ is separable so we can solve it and discover that $ I(x) = e^{\int q_1 (x) dx}$; here is how multiplying by this factor is useful. \[y' + q_1 (x)y = q_2 (x)\] \[I(x) = e^{\int q_1 (x) dx}\] \[I(x)y' + q_1 (x)I(x)y = I(x) q_2 (x)\] \[ [ I(x)y]' = I(x) q_2 (x)\] \[ I(x) y =\int I(x) q_2 (x) dx\] \[ y(x) =\frac{ \int I(x) q_2 (x) dx}{I(x)}\] The moral of the story is that multiplying by $I$ allows us to reverse the product rule and integrate both sides of the equation, we call this the \emph{integrating factor}, since multiplying our (first order linear) ODE allows us to integrate towards a solution. The general idea of an integrating factor is that it is some 'factor' such that multiplying an ODE by it allows us to use some rule (like the product rule) to integrate to the solution. Integrating factors may be used for other types of ODEs, however they depend on the context of the equation. \begin{theorem}[First order linear ODE] Given a first order linear ODE $y' + q_1 (x)y = q_2 (x)$, its general solution is the following where $I(x) = e^{\int q_1 (x) dx}$. \[ y(x) =\frac{ \int I(x) q_2 (x) dx}{I(x)}\] \end{theorem} \section{Second order linear, homogeneous, constant coefficient ODE} We'll now take a look at the simplest possible 2nd order ODE. It arises often in applications and has a simple general solution. \[y'' + by' +cy = 0 \] \begin{definition}[Characteristic equation] Given an ODE $\sum^{n}_{k=0} a_k y^{(k)} = 0$, its \emph{characteristic equation} is the following algebraic equation. \[ \sum^{n}_{k=0} a_k \lambda^{k} = 0\] \end{definition} \begin{theorem} The ODE $y'' + by' +cy = 0 $ with $a,b \in \mathbb{C}$ has the following general solution if its characteristic equation has 2 distinct zeros. \[ y(x) = K_1 e^{\lambda_1 x} + K_2 e^{\lambda_2 x} \] If the characteristic equation has a zero of multiplicity 2, the ODE has the following general solution. \[ y(x) = (Ax+b)e^{\lambda_1 x} \] \end{theorem} \begin{corollary} The ODE $y'' + by' +cy = 0 $ with $a,b \in \mathbb{C}$ has the following general solution if its characteristic equation has zeros with nonzero imaginary part ($b^2 -4c < 0$). \[ y(x) = e^{\alpha x}[K_1 \cos (\omega x) + K_2 \sin(\omega x)] \] \end{corollary} There are a few ways A basic solution method is to transform second order linear ODES with constant coefficients into a first order linear ODE. There are a few ways we can do this, but using the theory available to us, we chose the method of using an integrating factor. Letting $I(x)= e^{bx + k}$ \[ y'' + by' +cy = 0 \] \[ Iy'' + bIy' +cIy = 0 \] \[ (Iy')'= -cIy \] \[ Iy = I\frac{m}{m+n} y'\] \[ y = \frac{m}{m+n} y'\] \[ y = \frac{m}{m+n} y'+ K_1\] With this f \section{Exact equations} We will now look at an example of a class of 1st order ODEs that isn't necessarily linear. Though it may be non-linear, its form can still be relatively easy to work to. This is due to the idea of a \emph{conservative function}. \[I(x,y) + J(x,y) y' = 0\] \[y' = \frac{I(x,y)}{J(x,y)} \] \[I(x,y)dx + J(x,y)dy = 0\] The last is not 'exactly' rigorous (haha), but offers a hint about one way in whichwe can integrate towards the solution. \subsection{Solving by conservative functions} Imagine there exists some underlying function $V$, satisfying the following. \[V_x (x,y) = I(x,y) \] \[V_y (x,y) = J(x,y) \] Then we are left with the following. \[V_x (x,y) + y' V_y (x,y) =0 \] \[\int [ V_y (x,y) + y' V_x (x,y)] dx =\int 0dx \] \[ 2 V (x,y) +C_1 = C_2 \] \[ V (x,y) = K \] Wouldn't that be nice? The only problem is knowing when such a potential function exists, that is when do we have the following equations? \[V_x (x,y) = J(x,y) \] \[V_y (x,y) = I(x,y) \] By differentiating both equations with respect to the 'other' variable not yet differentiated,we get 2 different forms of $V_{xy}$, and immediate see that we require $J_y= I_x$. \begin{theorem}[Solution to exact equations with conservative function] Let IJ be functions satisfying $J_y= I_x = V_{xy}$ \[I(x,y) + J(x,y) y' = 0\] Then the following algebraic equation holds. $V(x,y)=K$ \end{theorem} \subsection{Solving by homogeneity} Say that we can't find a conservative function, to solve exact equations analytically we'll need to take a new approach, using the alternative form. \begin{theorem}[Solving exact equations with homogeneous functions] We will now study when both $I,J$ are homogeneous of the same degree. \[y' = \frac{I(x,y)}{J(x,y)} \] Then the ansatz $y(x)=xv(x)$ makes the ODE separable. \end{theorem} \section{Bernoulli equations} \[y' + q_1 (x) y = q_2 (x) y^n\] A common technique in differential equations (and even algebraic equations!) is to transform a difficult equation into an equation that can be solved. The Bernoulli equation can be transformed into a first order linear equation in the following way. \begin{itemize} \item Make the substitution $u = y^{1-n}$ \item Make the substitution $u' = (1-n) y'y^{-n}$ \item Solve the first order linear equation \end{itemize} Fortunately, we are able to calculate analytic solutions for first order linear differential equations, so we can therefore analytically solve Bernoulli equations! However, sometimes we cannot transform our DE into something so nice. \section{Riccati equations} \[q_1 (x)y' + q_2 (x) y + q_3 (x) y^2 = q_4 (x)\] Notice that when $q_4(x)=0$ we have a Bernoulli equation, Riccati equations are perhaps useful to view as possibly non-homogeneous Bernoulli equations. \begin{itemize} \item Make the substitution $y(x) = \frac{A(x)u'(x)}{u(x)}$ \item Determine the $A(x)$ required to cancel nonlinear terms \item Solve the second order linear ODE \end{itemize} Second order linear ODEs are generally much harder to solve; we have studied the case of constant coefficients but a general solution is not yet available to us. We will soon study various techniques to solve such problems. An interesting sidenote is the following formula for generating a new solution for the Ricatti equation using 3 knowl solutions. \begin{proposition}[Nonlinear superposition of Riccati solutions] \[ y_4 = \frac{y_1 (y_3 - y_2) + a y_2 (y_1 -y_3)}{y_3 -y_2 + a(y_1 - y_3)} \] \end{proposition} Proving such a proposition requires the machinery of Lie theory; this will be covered in the advanced part of this book. \section{Basic tools for solving ODEs} \section{Method of undetermined coefficients} Method of forming an ansatz to reduce differential equations into algebraic equations \[ ke^{ax} \iff Ke^{ax} \] \[ k \cos (ax) + m \sin (ax) \iff K \cos (ax) + M \sin (ax) \] \[ k x^n \iff \sum^{n}_{i=0} K_i x^i \] \section{Differential operators} %

Differential operator

%

Unary operator (function) defined in terms of functions and differentials

% \section{Annihilator method} Similar to the method of undetermined coefficients, however the first goal is to find some differential operator called the \emph{annihilator} which when composed with the ODE from the left makes the RHS equal to 0. \section{Picard iteration} It is used as a tool in the Picard-Lindelöf theorem (proven and explained in the advanced part), however like in many constructive proofs, the tool itself is actually of considerable value in determining an analytic solution. The algorithm is a specific example of the Banach fixed-point theorem in action. \chapter{Linear ODEs} Linear ODEs are those of the following form. \[\sum^{n}_{k=0} a_k(x) y^{(k)} = b(x)\] Specifically, homogeneous linear ODEs will be the center of our concentration; the central idea is that solution functions to homogeneous linear ODEs form a linear space! The application of linear algebra is the motivation behind many of the ideas presented in this chapter. Even for inhomogeneous linear ODEs, a general solution still requires the evaluation \begin{proposition} The general solution to an inhomogeneous linear ODE is the homogeneous variant's general solution plus some function called the \emph{particular solution}. \end{proposition} From a result called the Picard-Lindeloef theorem, we are assured of the following proposition. \begin{proposition} Consider the following linear ODE where the $a_k : I_k \to \mathbb{R}$ and $b : I_b \to \mathbb{R}$ are continuous on their domain \[\sum^{n}_{k=0} a_k(x) y^{(k)} = b(x)\] This ODE has a general solution on $I_b cup [ \bigcup_{k=0}^{n} I_k] $ The general solution of a linear ODE of degree $n$ is the linear combination of $n$ distinct functions. \end{proposition} \begin{corollary} Consider the following linear ODE where the $a_k : I_k \to \mathbb{R}$ and $b : I_b \to \mathbb{R}$ are continuous on their domain \[\sum^{n}_{k=0} a_k(x) y^{(k)} = b(x)\] Let $F = \{f \in C^{n}(I) : \forall x \in I [ \sum^{n}_{k=1} q_k (x) f^{(k)}(x) = 0 ] \}$, then $(F,\mathbb{R},+,\cdot)$ forms a linear space over $\mathbb{R}$ (where addition and scalar multiplication are usual addition and scaling of functions). \end{corollary} The fact that solutions to inhomogeneous linear ODEs form a linear space is central, and questions about the properties of this linear space will in turn provide information of the solutions to the ODE. Due to the solution space being a linear space, we are led to believe that its general solution will be the linear combination of linearly independent solutions. This begs some questions; what is the dimension of this linear space? How can we find linearly independent solutions? \section{Wronskian} \begin{definition}[Wronskian] Given a vector of real or complex functions $\mathbf{y}$, their \emph{Wronskian} is the function \(W\) which is the determinant of the matrix where each column is a vector with a solution to a linear DE and its \(n-1\) derivatives. \[ W(\mathbf{y})(x) = \begin{vmatrix} y_1 & y_2 & \cdots & y_n \\ y'_1 & y'_2 & \cdots & y'_n \\ \vdots & \vdots & \ddots & \vdots \\ y^{(n-1)}_{1} & y^{(n-1)}_{2} & \cdots & y^{(n-1)}_{n} \end{vmatrix} \] Where $\mathbf{y} = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix}$ is a vector of solutions to a linear DE \end{definition} \[\mathbf{y} \text{ are linearly dependent on } I \implies W(I) =\{0\} \] Where $ \mathbf{y} = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix}$ is a vector of solutions to a linear DE \[\mathbf{y} \text{ are analytic on } I \implies [ \mathbf{y} \text{ are linearly dependent on } I \iff W(I) =\{0\} ] \] \[\mathbf{y} \text{ satisfy } y''+q_1 (x)y'+q_2(x)=0 \text{ on } I \implies [ \mathbf{y} \text{ are linearly dependent on } I \iff W(I) =\{0\} ] \] \subsection{Abel's formula} \begin{theorem}[Abel's formula] Let $y_1,y_2$ be functions satisfying $y''+ a_1 (x)y' + a_2 (x) y=0$, then $W(y_1 , y_2) = K_{12}e^{-\int q_1 (x) dx}$ %\[ y_1,y_2 \text{ satisfy } y'' + q_1 (x) y' + q_2 (x) y = 0 \implies W(y_1 , y_2) = K_{12}e^{-\int q_1 (x) dx}\] \end{theorem} Now that we have a way of calculating the Wronskian in a way that is independent on knowing both functions, we can do something interesting; we can derive the second function purely from the first! \begin{corollary} Let $y_1$ be a nonvanishing function satisfying $y''+ a_1 (x)y' + a_2 (x) y=0$, then the following function $y_2$ is linearly independent to $y_1$ and also satisfies the ODE. \[ y_2 = y_1 \int \frac{e^{-\int q_1 (x) dx}}{y_1 (x)^2} dx\] %\[ y_1 \text{ satisfies } y'' + q_1 (x) y' + q_2 (x) y = 0 \land \neg ( \forall x [y_1(x)=0] ) \implies y_2 = y_1 \int \frac{e^{-\int q_1 (x) dx}}{y_1 (x)^2} dx \text{ satisfies } y'' + q_1 (x) y' + q_2 (x) y = 0 \land y_1 , y_2 \text{ are linearly independent} \] \end{corollary} As wonderful as it is that we can abuse the linear space formed by the linear ODE to generate novel solutions using known solutions, this still requires that we know some solutions for the ODE. To get an initial foothold on the problem, the Annihilator method or Cauchy-Euler method can be used to obtain a first set of solutions before expanding our solutions using Abel's formula. Unfortunately such methods are not always possible to apply due to assumptions not being met, but there exists some more potent methods for evaluating solutions to linear ODEs. \section{Variation of Parameters (VOP)} A general technique to solve inhomogeneous linear ODEs is VOP. VOP is essentially just assuming the ansatz $y(x) = u(x)y_1(x) + v(x) y_2 (x)$ and $u' y_1 + v' y_2 = 0$ \section{Series methods} One powerful idea is to assume that the solution to the linear ODE is an analytic function, meaning that we assume the ansatz $y(x) = \sum^{\infty}_{n=0} a_n x^n$. It's quite a simple idea, yet incredibly effetive since the class of analytic functions is quite rich indeed. Often the solutions to an ODE are nonelementary, so this method comes in clutch where others fail. For differential equations such that $y^{(n)} + \sum q_k (x) y^{(k)} = 0 \land q_k \text{ are analytic on }I \land 0 \in I$, solutions are always analytic and hence has a Taylor series representation and hence a power series representation. This is due to considering the DE on a complex neighborhood and by noting that holomorphic functions are analytic, see Complex Analysis. \begin{itemize} \item Assume the ansatz $y(x) = \sum^{\infty}_{n=0} a_n x^n$ \item Substitute the ansatz into the DE and manipulate into one single sum $ [ \sum^{\infty}_{n=0} a_n x^n ]^{(k)} = \sum^{\infty}_{n=k} a_n \frac{n!}{(n-k)!} x^{n-k}$
  • Since the RHS equals zero, equate the coefficient of the new sum to 0<
  • Solve recursive formula for $a_n$ \begin{definition}[Ordinary point] \[ x_0 \text{ is an ordinary point of } \sum q_n (x) y^{n} (x) = 0 \iff \forall k \in \{0,1,..,n\}a_k \text{ is analytic at }x_0) \] \end{definition} Translating from a recurrence relation into a more recognizable notation can be a bit laborious, however a few elementary identities could help out. \[ \prod^{n}_{k=1} 2k = 2^n n! \] \[ \prod^{n}_{k=1} (2k-1) = \frac{(2n-1)!}{2^(n-1) (n-1)!}\] \begin{definition}[Singular point] \[x_0 \text{ is a singular point of } \sum q_n (x) y^{n} (x) = 0 \iff \neg [ x_0 \text{ is a singular point of } \sum q_n (x) y^{n} (x) = 0]\] \end{definition} All coefficient function analytic at a point means all DEs solutions taylor series equal function itself Series method Method of Frobenius Additionally, the Laplace transform can be used to transform ODEs into algebraic equations! Here is the outline of the method. \begin{itemize} \item Compute the Laplace transform for both sides of the equation and equate them \item Solve algebraically for $y_1$ \item Compute $\mathcal{L}^{-1}\{y_1\}(x)$ \end{itemize} \begin{definition}[Confluent hypergeometric functions] Linearly independent solutions to the ODE \(zw''+(b-z)w'-aw=0\) (Kummer's equation) First kind \[ M(a,b,z)= \sum^{\infty}_{n=0} \frac{a^{(n)} z^n }{ b^{(n)} n! } \] \begin{itemize} \item $x^{(n)}=\prod^{n-1}_{k=0}(x+k)$ is the rising factorial \item $ (x)_{n}=\prod^{n-1}_{k=0}(x-k)$ is the falling factorial \end{itemize} Second kind \[ U(a,b,z) \] \end{definition} \chapter{PDEs} \begin{definition}[Partial differential equation (PDE)] A \emph{partial differential equation (PDE)} is an equation of the following form, defined by some $f : S \subseteq \mathbb{R}^n \times \mathbb{R}^m$ and $D^k \mathbf{y}$ representing the matrix of possible partial derivatives of order $k$. \[ f(\mathbf{x},y(\mathbf{x}), D y(\mathbf{x}) , \hdots , D^m y(\mathbf{x})) = 0\] Solutions for $y : U \to \mathbb{R}$ are (usually, depending on whether all variables can be differentiated $m$ times) in $C^{m}(U)$, for some set $U$ open in (Euclidean topology) $\mathbb{R}^m$. \end{definition} The introduction of a multivariable function already seriously complicates the formal definition from an ODE, and complication is generally the name of the game with PDEs. This chapter aims to look at solution methods on the more elementary side that one can apply in the study of PDEs, generally mapping a PDE to multiple ODEs. \section{Separation of variables} The most elementary solution method I know of is \emph{separation of variables}, which basically makes a general yet bold ansatz that is an attempt to split our PDE into ODEs with respect to one variable each, then prays to the math gods that things work out. This solution method has a chance at being succesful when the RHS (or any 1 of the sides) of the PDE reduces to a constant after we make our ansatz because this allows us to split the PDE into multiple ODEs. Here is the general method. \begin{itemize} \item Assume the ansatz $f(x,y) = X(x)Y(y)$ and substitute into equation \item Separate variables to both sides and equate both sides to a constant $\pm \lambda$ \item Solve each problem using SL theory or otherwise and then substitute the solutions into the original ansatz; the general solution is a linear combination of these eigenfunctions \item Equate the general solution to the final boundary condition and find the projection. \end{itemize} \section{Fourier transform method} Using the ansatz (guess) that a solution can be represented as the Fourier transform of some function $u$ is quite effective. Fourier studied a PDE now hailed as the heat equation, and he tried to find new solutions to it based on the trivial , which were sinudoids. His idea was basically to consider convolutions of some unknown function with these elementary solutions to find further solutions. Fourier conjectured that (sufficiently nice) periodic functions could always be represented as something called a Fourier series, and that non-periodic function (again, sufficiently nice ones, whatever that means) could still be represented by a 'Fourier integral'. Though such statements could not be verified until much later, Fourier's steadfast beliefs ultimately paid off, producing legitimate solutions to the heat equation. Fourier analysis warrant much attention due to its usefulness in PDEs as well as signal processing, and though we do not discuss the theory here, you can read about it in Fourier Analysis. To those already familiar with the theory, I offer you a neat way to solve PDEs with Fourier analysis. \section{Qualitative theorems} \subsection{Maximum principle} \subsection{Harnack's theorem}