Chapter 8

Supporting notes

Scott Alexander

Frame 8

We have
cov((Subscript[X, t]),(Subscript[X, t+h])) = cov((∑_(j=0)^∞)(ϕ^j)(Subscript[Z, t-j]),(∑_(k=0)^∞)(ϕ^k)(Subscript[Z, t+h-k]))
  = (∑_(j=0)^∞)(∑_(k=0)^∞)cov((ϕ^j)(Subscript[Z, t-j]),(ϕ^k)(Subscript[Z, t+h-k]))    (should be justified)
  = (∑_(j=0)^∞)(∑_(k=0)^∞)(ϕ^j)(ϕ^k)cov((Subscript[Z, t-j]),(Subscript[Z, t+h-k])).

Note that by independence of the noise process
cov((Subscript[Z, t-j]),(Subscript[Z, t+h-k])) = 0
unless
t-j = t+h-k    ⟹    k = j+h.

So
cov((Subscript[X, t]),(Subscript[X, t+h])) = (∑_(j=0)^∞)(∑_(k=0)^∞)(ϕ^j)(ϕ^k)cov((Subscript[Z, t-j]),(Subscript[Z, t+h-k]))
  = (∑_(j=0)^∞)(ϕ^j)(ϕ^(j+h))cov((Subscript[Z, t-j]),(Subscript[Z, t-j])) = (∑_(j=0)^∞)(ϕ^j)(ϕ^(j+h))(σ^2) = (σ^2)(ϕ^h)(∑_(j=0)^∞)(ϕ^(2j))
  = (σ^2)(ϕ^h)(1/(1-(ϕ^2)))
as long as
|(ϕ^2)|<1    ⟹    |ϕ|<1.

So 𝔼[(Subscript[X, t])] is constant and cov((Subscript[X, t]),(Subscript[X, t+h])) is function of h.
Therefore (Subscript[X, t]) is weakly stationary.

(∑_(j=0)^∞)((((ϕ^2)))^j)

Frame 9

Let    X=(Subscript[((Subscript[X, t])), t∈ℤ])    and     Z=(Subscript[((Subscript[Z, t])), t∈ℤ]).

Using (1) from page 7
(Subscript[X, t])-ϕ (Subscript[X, t-1]) = (Subscript[X, t])-ϕ(Subscript[(B X), t]) =  (Subscript[Z, t])
or more generally
X-ϕ B X = (1-ϕ B)X = Z
which gives
X = (((1-ϕ B))^(-1))Z.

Using (2) from page 7
(Subscript[X, t]) = (∑_(j=0)^∞)(ϕ^j)(Subscript[Z, t-j]) = (∑_(j=0)^∞)(ϕ^j)(Subscript[((B^j)Z), t])
or more generally
X = (∑_(j=0)^∞)(((ϕ B))^j)Z.

Comparing we have
(((1-ϕ B))^(-1))Z = (∑_(j=0)^∞)(((ϕ B))^j)Z
or
(((1-ϕ B))^(-1)) = (∑_(j=0)^∞)(((ϕ B))^j).

Frame 18

(Subscript[Y, t]) = (∑_(j∈ℤ))(Subscript[ψ, j])(Subscript[X, t-j]) = (∑_(j∈ℤ))(Subscript[ψ, j])(Subscript[((B^j)X), t])
or more generally
Y = (∑_(j∈ℤ))(Subscript[ψ, j])(B^j)X = ((∑_(j∈ℤ))(Subscript[ψ, j])(B^j))X = ψ(B)X
or again at just t
(Subscript[Y, t]) = (Subscript[(((∑_(j∈ℤ))(Subscript[ψ, j])(B^j))X), t]) = (Subscript[(ψ(B)X), t]).

Frame 20-21

Note that
(ψ'ψ)(z) ≡ ψ'(z)ψ(z) = ((∑_(j∈ℤ))(Subscript[ψ, j]^('))(z^j))((∑_(k∈ℤ))(Subscript[ψ, k]) (z^k))
  = (∑_(j∈ℤ))(∑_(k∈ℤ))(Subscript[ψ, j]^('))(Subscript[ψ, k])(z^j) (z^k)
  =(∑_(j∈ℤ))(∑_(k∈ℤ))(Subscript[ψ, j]^('))(Subscript[ψ, k])(z^(j+k))
  =(∑_(l∈ℤ))(∑_(j+k∈l))(Subscript[ψ, j]^('))(Subscript[ψ, k])(z^l).    (note outer sum takes care of the powers l)

So we have
(ψ'ψ)(B) = (∑_(l∈ℤ))(∑_(j+k∈l))(Subscript[ψ, j]^('))(Subscript[ψ, k])(B^l)
and
((ψ'ψ)(B))X = (∑_(l∈ℤ))(∑_(j+k∈l))(Subscript[ψ, j]^('))(Subscript[ψ, k])(B^l)X
or at time t
(Subscript[((ψ'ψ)(B)X), t]) = (∑_(l∈ℤ))(∑_(j+k∈l))(Subscript[ψ, j]^('))(Subscript[ψ, k])(Subscript[((B^l)X), t])
  = (∑_(l∈ℤ))(∑_(j+k=l))(Subscript[ψ, j]^('))(Subscript[ψ, k])(Subscript[X, t-l]).    (A)

Now observe
ψ'(B)(ψ(B)X) = (∑_(j∈ℤ))(Subscript[ψ, j]^('))(B^j)ψ(B)X    (expanding filter ψ')
  = (∑_(j∈ℤ))(Subscript[ψ, j]^('))(B^j)(∑_(k∈ℤ))(Subscript[ψ, k])(B^k)X    (expanding filter ψ)
  = (∑_(j∈ℤ))(∑_(k∈ℤ))(Subscript[ψ, j]^('))(Subscript[ψ, k])(B^j)(B^k)X
  = (∑_(j∈ℤ))(∑_(k∈ℤ))(Subscript[ψ, j]^('))(Subscript[ψ, k])(B^(j+k))X
  = (∑_(l∈ℤ))(∑_(j+k=l))(Subscript[ψ, j]^('))(Subscript[ψ, k])(B^l)X
or at time t
(Subscript[(ψ'(B)(ψ(B)X)), t]) =  (∑_(l∈ℤ))(∑_(j+k=l))(Subscript[ψ, j]^('))(Subscript[ψ, k])(Subscript[((B^l)X), t])
  =(∑_(l∈ℤ))(∑_(j+k=l))(Subscript[ψ, j]^('))(Subscript[ψ, k])(Subscript[X, t-l])
  =(Subscript[((ψ'ψ)(B)X), t])
using (A) from above.

That is
ψ'(B)(ψ(B)X) = ((ψ'ψ)(B))X ≡ (ψ'(B)ψ(B))X.

Frame 22

ParametricPlot[{Cos[θ],Sin[θ]},{θ,0,2π},AxesLabel->{"Re(z)","Im(z)"}]

ParametricPlot[{{1.1Cos[θ],1.1Sin[θ]},{1/1.1Cos[θ],1/1.1Sin[θ]}},{θ,0,2π},PlotStyle->Dashed,AxesLabel->{"Re(z)","Im(z)"}]

Frame 23-27

Existence

Step 1.
Suppose
ϕ(z) = 1-(Subscript[ϕ, 1])z-(Subscript[ϕ, 2])(z^2)-\[CenterEllipsis]--(Subscript[ϕ, p])(z^p) ≠ 0
for all z∈𝕋 (the complex unit circle).

Then for    (1/r)<\[LeftBracketingBar]z\[RightBracketingBar]<r    with     r>1,    the Laurent series about z=0
((1+(Subscript[θ, 1])z+(Subscript[θ, 2])(z^2)+\[CenterEllipsis]+(Subscript[θ, q])(z^q))/(1-(Subscript[ϕ, 1])z-(Subscript[ϕ, 2])(z^2)-\[CenterEllipsis]-(Subscript[ϕ, p])(z^p))) = ((θ(z))/(ϕ(z))) = (∑_(j∈ℤ))(Subscript[ψ, j])(z^j)
converges absolutely, i.e.
(∑_(j∈ℤ))\[LeftBracketingBar](Subscript[ψ, j])(z^j)\[RightBracketingBar] < ∞.

This means the coefficients (Subscript[ψ, j]) are absolutely summable because when z∈𝕋
(∑_(j∈ℤ))\[LeftBracketingBar](Subscript[ψ, j])\[RightBracketingBar] = (∑_(j∈ℤ))\[LeftBracketingBar](Subscript[ψ, j])\[RightBracketingBar]\[LeftBracketingBar]1\[RightBracketingBar] = (∑_(j∈ℤ))\[LeftBracketingBar](Subscript[ψ, j])\[RightBracketingBar]\[LeftBracketingBar](z^j)\[RightBracketingBar] = (∑_(j∈ℤ))\[LeftBracketingBar](Subscript[ψ, j])(z^j)\[RightBracketingBar] < ∞.

Step 2.
Use Proposition 1 (page 14).

This is justified becasue the coefficients (Subscript[ψ, j]) are absolutely summable.

Using these coefficients as a filter on the zero-mean stationary process (Subscript[Z, t]) gives
(Subscript[X, t]) = (∑_(j∈ℤ))(Subscript[ψ, j])(Subscript[Z, t-j]).

By this proposition the filtered process (Subscript[X, t]) is also a zero-mean stationary process.

Step 3.
Recall notation from Proposition 2 (page 19)
ψ(z) = (∑_(j∈ℤ))(Subscript[ψ, j])(z^j) = ((θ(z))/(ϕ(z))) = (θ/ϕ)(z)
so
ψ(B) = (∑_(j∈ℤ))(Subscript[ψ, j])(B^j) = ((θ(B))/(ϕ(B))) = (θ/ϕ)(B).

Then
(Subscript[X, t]) = (∑_(j∈ℤ))(Subscript[ψ, j])(Subscript[Z, t-j]) = (∑_(j∈ℤ))((Subscript[ψ, j])(B^j))(Subscript[Z, t]) = (Subscript[(ψ(B)Z), t]) = (Subscript[(((θ(B))/(ϕ(B)))Z), t]) ≡ (Subscript[((θ/ϕ)(B)Z), t])
or more generally
X = (θ/ϕ)(B)Z.

Step 4.
Use Proposition 2 (page 19).

This is justified becasue the filter    (θ/ϕ)=ψ    has coefficients (Subscript[ψ, j]) that are absolutely summable.

Now apply ϕ(B) to both sides of the last to give
ϕ(B)X = ϕ(B)((θ/ϕ)(B)Z)
  = ((ϕ(θ/ϕ))(B))Z    (using the shift calculus of this proposition)
  = θ(B)Z.

At time t
(Subscript[(ϕ(B)X), t]) =  (Subscript[(θ(B)Z), t])
which by page 10 is
(Subscript[X, t])-(Subscript[((Subscript[ϕ, 1])(B^1)X), t])-(Subscript[((Subscript[ϕ, 2])(B^2)X), t])-\[CenterEllipsis]-(Subscript[((Subscript[ϕ, p])(B^p)X), t]) = (Subscript[Z, t])+(Subscript[((Subscript[θ, 1])(B^1)Z), t])+(Subscript[((Subscript[θ, 2])(B^2)Z), t])+\[CenterEllipsis]+(Subscript[((Subscript[θ, q])(B^q)Z), t])
or
(Subscript[X, t])-(Subscript[ϕ, 1])(Subscript[Z, t-1])-(Subscript[ϕ, 2])(Subscript[Z, t-2])-\[CenterEllipsis]--(Subscript[ϕ, p])(Subscript[X, t-p]) = (Subscript[Z, t])+(Subscript[θ, 1])(Subscript[Z, t-1])+(Subscript[θ, 2])(Subscript[Z, t-2])+\[CenterEllipsis]+(Subscript[θ, q])(Subscript[Z, t-q]).

This is the ARMA(p,q) equation (3) from page 10.

Uniqueness

Step 1.
Take an arbitrary ARMA(p,q) process
ϕ(B)X = θ(B)Z
where
ϕ(z) = 1-(Subscript[ϕ, 1])z-(Subscript[ϕ, 2])(z^2)-\[CenterEllipsis]--(Subscript[ϕ, p])(z^p) ≠ 0
for all z∈𝕋 (the complex unit circle).

We have shown above that such a process is stationary.

Step 2.
For    (1/r)<\[LeftBracketingBar]z\[RightBracketingBar]<r    with     r>1,    the Laurent series about z=0
(1/(1-(Subscript[ϕ, 1])z-(Subscript[ϕ, 2])(z^2)-\[CenterEllipsis]-(Subscript[ϕ, p])(z^p)))  = (1/(ϕ(z))) =  (∑_(j∈ℤ))(Subscript[ψ, j])(z^j)
converges absolutely, i.e.
(∑_(j∈ℤ))\[LeftBracketingBar](Subscript[ψ, j])(z^j)\[RightBracketingBar] < ∞.

This means the coefficients (Subscript[ψ, j]) are absolutely summable because when z∈𝕋
(∑_(j∈ℤ))\[LeftBracketingBar](Subscript[ψ, j])\[RightBracketingBar] = (∑_(j∈ℤ))\[LeftBracketingBar](Subscript[ψ, j])\[RightBracketingBar]\[LeftBracketingBar]1\[RightBracketingBar] = (∑_(j∈ℤ))\[LeftBracketingBar](Subscript[ψ, j])\[RightBracketingBar]\[LeftBracketingBar](z^j)\[RightBracketingBar] = (∑_(j∈ℤ))\[LeftBracketingBar](Subscript[ψ, j])(z^j)\[RightBracketingBar] < ∞

Step 3.
Use Proposition 1 (page 14).

This is justified becasue the coefficients (Subscript[ψ, j]) are absolutely summable.

We take the staionary process
ϕ(B)X = θ(B)Z
and apply the filter (1/ϕ)to give
(1/ϕ)(B)(ϕ(B)X) = (1/ϕ)(B)(θ(B)Z)    (B)
which gives another stationary process.

Step 4.
Use Proposition 2(page 19).

Using the calculus of shift operator the LHS of (B) becomes
(1/ϕ)(B)(ϕ(B)X) = (((1/ϕ)ϕ)(B))X ≡ (((ϕ(B))/(ϕ(B))))X = X
and the RHS of (B) becomes
(1/ϕ)(B)(θ(B)Z) = (((1/ϕ)θ)(B))Z = (θ/ϕ)(B)Z.

That is
X = (θ/ϕ)(B)Z
showing the solution found in "Existence" part of proof is unique.

Frame 36

TT=100;
data=RandomFunction[ARMAProcess[{0.5},{0.4},1],{0,TT}];
ListLinePlot[data,AxesLabel->{"t","(Subscript[X, t])"}]

Frame 38-41

ARMA Price

price=FinancialData["TSLA","Close",{{2021,1,1},{2024,02,14},"Daily"},
Method->"Legacy"];
DateListPlot[price,PlotLabel->"Tesla daily close 01/01/2021-14/02/2024"]

(*
(*Export["C:/Users/Scott/Desktop/2025/Notes_StochasticProcecesses/Other/tsla.csv",price];*)
Export["C:/Users/127858/OneDrive - UTS/Desktop/2025/Notes_StochasticProcecesses/Other/tsla.csv",price];
*)

(*
price=Import["C:/Users/Scott/Desktop/2025/Notes_StochasticProcecesses/Other/tsla.csv"];
(*price=Import["C:/Users/127858/OneDrive - UTS/Desktop/2025/Notes_StochasticProcecesses/Other/tsla.csv"];*)
DateListPlot[price,PlotLabel->"Tesla daily close 01/01/2021-14/02/2024"]
*)

tsmPrice=TimeSeriesModelFit[price[[All,2]],{"ARMA"}]
tsmPrice["BestFit"]

ARMA Return

n=Length[price]; (* sample size *)
return=Transpose[{price[[2;;n,1]],Differences[Log[price[[All,2]]],1]}];
DateListPlot[return,PlotLabel->"Tesla daily log-return 01/01/2021-14/02/2024"]

tsmReturn=TimeSeriesModelFit[return[[All,2]],"ARMA"]
tsmReturn["BestFit"]

Frame 46-47

Elimination of trend

n=250;
z=Table[RandomReal[NormalDistribution[0,1]],{n}];
c0=0;c1=0.002;c2=0.0003;
x=Table[c0+c1*i+c2*(i^2)+z[[i]],{i,1,n}];
x=Prepend[x,0];
ListPlot[x,AxesLabel->{"t","(Subscript[X, t])"}]

xB=Differences[x,1];
ListPlot[xB,AxesLabel->{"t","(1-B)(Subscript[X, t])"}]

xBB=Differences[x,2];
ListPlot[xBB,AxesLabel->{"t","(1-B)^2(Subscript[X, t])"}]

Detecting unit roots

Frame 46 example

(Subscript[H, 0]): unit root present in data
(Subscript[H, A]): no unit root present in data

A unit root in the data cannot be ruled out (p-value 0.56833146333228`).

There is strong evidence of no unit root in once-differenced and twice-differenced data
(p-values 5.052542808944158`*^-14, 2.0220290229019908`*^-14).

(Note these p-values will change if model is re-run.)

UnitRootTest[x]

UnitRootTest[xB]

UnitRootTest[xBB]

Frame 50-51

Tesla data unit roots

DateListPlot[price,PlotLabel->"Tesla daily close 01/01/2021-14/02/2024"]

(Subscript[H, 0]): unit root present in Tesla data
(Subscript[H, A]): no unit root present in Tesla data

A unit root in the Tesla price data cannot be ruled out (p-value 0.5276049722898593`).

UnitRootTest[price[[All,2]]]

Tesla log-returns unit roots

DateListPlot[return,PlotLabel->"Tesla daily log-return 01/01/2021-14/02/2024"]

(Subscript[H, 0]): unit root present in Tesla log-returns
(Subscript[H, A]): no unit root present in Tesla log-returns

There is strong evidence of no unit root in the Tesla log-return data (p-value 3.511964691244515`*^-16).

UnitRootTest[return[[All,2]]]

Once-differenced Tesla data unit roots

n=Length[price];
priceB=Transpose[{price[[2;;n,1]],Differences[price[[All,2]],1]}];
DateListPlot[priceB,PlotLabel->"Tesla daily close once-differenced 01/01/2021-14/02/2024"]

(Subscript[H, 0]): unit root present in once-differenced Tesla data
(Subscript[H, A]): no unit root present in once-differenced Tesla data

There is strong evidence of no unit root in once-differenced Tesla price data (p-value 1.712268578670526`*^-14).

UnitRootTest[priceB[[All,2]]]

Tesla ARIMA model

Using ARIMA

tsmPriceARIMA=TimeSeriesModelFit[price[[All,2]],{"ARIMA"}]
tsmPriceARIMA["BestFit"]

Differencing and using ARMA

tsmPriceBARIMA=TimeSeriesModelFit[priceB[[All,2]],{"ARMA"}]
tsmPriceBARIMA["BestFit"]