37363 - Lab/Tutorial 1 Solutions

Scott Alexander

Q1

(a)

Integrating against density

var(X) = 𝔼[(X^2)]-((𝔼[X])^2).

For the moments we have
𝔼[X] = (∫_(-∞)^∞)x (Subscript[f, X])(x)dx  = (∫_0^∞)x λ (e^(-λ x))dx    ((Subscript[f, X])(x)=0 for x<0)
  = (∫_0^∞)x (d/(dx))(-(e^(-λ x)))dx = -(Subscript[[x (e^(-λ x))], 0]^∞)+(∫_0^∞)(e^(-λ x))(d/(dx))xdx    (integration by parts)
  = -(Subscript[[x (e^(-λ x))], 0]^∞)-(Subscript[(1/λ)[(e^(-λ x))], 0]^∞)
  = -(lim_(x → ∞))(x/(e^(λ x))) +0-(lim_(x → ∞))(1/λ)(e^(-λ x))+(1/λ)
  = -(lim_(x → ∞))(((d/(dx))x)/((d/(dx))(e^(λ x)))) +0-(lim_(x → ∞))(1/λ)(e^(-λ x))+(1/λ)
  = -(lim_(x → ∞))(1/(λ (e^(λ x)))) +0-(lim_(x → ∞))(1/λ)(e^(-λ x))+(1/λ)    (L'Hospital's Rule)
  = 0+0-0+(1/λ)
  = (1/λ)
and
𝔼[(X^2)] = (∫_(-∞)^∞)(x^2) (Subscript[f, X])(x)dx  = (∫_0^∞)(x^2) λ (e^(-λ x))dx    ((Subscript[f, X])(x)=0 for x<0)
  = (∫_0^∞)(x^2) (d/(dx))(-(e^(-λ x)))dx = -(Subscript[[(x^2) (e^(-λ x))], 0]^∞)+(∫_0^∞)(e^(-λ x))(d/(dx))(x^2)dx    (integration by parts)
  = -(Subscript[[(x^2) (e^(-λ x))], 0]^∞)+2(∫_0^∞)x (e^(-λ x))dx = -(Subscript[[(x^2) (e^(-λ x))], 0]^∞)+(2/λ) 𝔼[X]
  =-(Subscript[[(x^2) (e^(-λ x))], 0]^∞)+(2/(λ^2))    (from above)
  = -(lim_(x → ∞))((x^2)/(e^(λ x)))+0+(2/(λ^2))
  = 0+0+(2/(λ^2))    (application of L'Hospital's Rule omitted)
  = (2/(λ^2)).

So
var(X) = 𝔼[(X^2)]-((𝔼[X])^2) = (2/(λ^2))-(1/(λ^2)).

Using Gamma function

Recalling
Γ(z) = (∫_0^∞)(y^(z-1))(e^(-y))dy
we have
𝔼[X] = (∫_0^∞)x λ (e^(-λ x))dx = (1/λ)(∫_0^∞)y (e^(-y))dy    (y=λ x)
  = (1/λ)Γ(2) = (1/λ)1! = (1/λ)
and
𝔼[(X^2)] = (∫_0^∞)(x^2) λ (e^(-λ x))dx = (1/λ)(∫_0^∞)(x^2) (λ^2) (e^(-λ x))dx
 = (1/(λ^2))(∫_0^∞)(y^2) (e^(-y))dy    (y=λ x)
  = (1/(λ^2))Γ[3] = (1/(λ^2))2! = (2/(λ^2))
with the rest following as above.

Using Mathematica function “Integrate”

f[x_]:=PDF[ExponentialDistribution[λ],x];
f[x]
Assuming[λ>0,Integrate[(x^2)f[x],{x,0,∞}]-(((Integrate[x f[x],{x,0,∞}]))^2)]    (* note the space between x and f[x] *)

f[x_]:=PDF[ExponentialDistribution[λ],x];
Assuming[λ>0,(∫_0^∞)(x^2)f[x]\[DifferentialD]x-((((∫_0^∞)x f[x]\[DifferentialD]x))^2)]  (* note the space between x and f[x] *)

Using Mathematica function “Variance”

Variance[ExponentialDistribution[λ]]

Using Mathematica function “Expectation”

Expectation[(x^2),x\[Distributed]ExponentialDistribution[λ]]-((Expectation[x,x\[Distributed]ExponentialDistribution[λ]])^2)

Expectation[(x^2),Distributed[x,ExponentialDistribution[λ]]]-((Expectation[x,Distributed[x,ExponentialDistribution[λ]]])^2)

(b)

We have
(Subscript[f, X,Y])(x,y) = (1/(2π (Subscript[σ, X])(Subscript[σ, Y])(Sqrt[1-(ρ^2)])))(∫_(-∞)^∞)exp(-(1/(2(1-(ρ^2))))(((((x-(Subscript[μ, X])))^2)/(Subscript[σ, X]^2))+((((y-(Subscript[μ, Y])))^2)/(Subscript[σ, Y]^2))-((2ρ(x-(Subscript[μ, X]))(y-(Subscript[μ, Y])))/((Subscript[σ, X])(Subscript[σ, Y])))))dy.

Hint. Use change of variables
u=((x-(Subscript[μ, X]))/(Subscript[σ, X]))    and    v=((y-(Subscript[μ, Y]))/(Subscript[σ, Y]))
and then complete the square.

Let
u=((x-(Subscript[μ, X]))/(Subscript[σ, X]))    and    v=((y-(Subscript[μ, Y]))/(Subscript[σ, Y]))
so that
(Subscript[f, X])(x) = (1/(2π (Subscript[σ, X])(Subscript[σ, Y])(Sqrt[1-(ρ^2)])))(∫_(-∞)^∞)exp(-(1/(2(1-(ρ^2))))((u^2)+(v^2)-2ρ u v))(Subscript[σ, Y])dv.

Note that by completing the square
(u^2)+(v^2)-2ρ u v = (((v-ρ u))^2)+(u^2)(1-(ρ^2))
so that
(Subscript[f, X])(x) = (1/(2π (Subscript[σ, X])(Subscript[σ, Y])(Sqrt[1-(ρ^2)])))(∫_(-∞)^∞)exp(-(1/(2(1-(ρ^2))))((((v-ρ u))^2)+(u^2)(1-(ρ^2))))(Subscript[σ, Y])dv
  = (1/(2π (Subscript[σ, X])(Sqrt[1-(ρ^2)])))exp(-((u^2)/2))(∫_(-∞)^∞)exp(-((((v-ρ u))^2)/(2(1-(ρ^2)))))dv
  = (1/((Subscript[σ, X])(Sqrt[2π])))exp(-((u^2)/2))(∫_(-∞)^∞)(1/(Sqrt[2π (1-(ρ^2))]))exp(-((((v-ρ u))^2)/(2(1-(ρ^2)))))dv.
  = (1/((Subscript[σ, X])(Sqrt[2π])))exp(-((u^2)/2))    (integrand is density function of N(ρ u,1-(ρ^2)))
  = (1/((Subscript[σ, X])(Sqrt[2π])))exp(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2)))).

Using Mathematica function “Integrate”

f[x_,y_]:=(1/(2π σX σY(Sqrt[1-(ρ^2)])))Exp[-(1/(2(1-(ρ^2))))(((((x-μX))^2)/((σX)^2))+((((y-μY))^2)/((σY)^2))-((2ρ(x-μX)(y-μY))/(σX σY)))];
Assuming[{-1<ρ<1,σX>0,σY>0},Integrate[f[x,y],{y,-∞,∞}]]

Assuming[{-1<ρ<1,σX>0,σY>0},(∫_(-∞)^∞)f[x,y]\[DifferentialD]y]

(c)

The expected value
𝔼[X] = (∫_(-∞)^∞)x (Subscript[f, X])(x)dx = (∫_(-∞)^∞)x (1/((Sqrt[2π])(Subscript[σ, X])))(e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2)))))dx
  = (1/((Sqrt[2π])(Subscript[σ, X])))(∫_(-∞)^∞)x (e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2)))))dx.

Note that
(d/(dx))(e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2))))) = (e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2)))))(d/(dx))(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2))))
 = -(((x-(Subscript[μ, X])))/(Subscript[σ, X]^2))(e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2)))))
or
(x-(Subscript[μ, X]))(e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2))))) = -(Subscript[σ, X]^2)(d/(dx))(e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2)))))
which gives
x (e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2))))) = (Subscript[μ, X]) (e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2)))))-(Subscript[σ, X]^2)(d/(dx))(e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2))))).

So
𝔼[X] = (1/((Sqrt[2π])(Subscript[σ, X])))(∫_(-∞)^∞)x (e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2)))))dx
  = (1/((Sqrt[2π])(Subscript[σ, X])))(∫_(-∞)^∞)((Subscript[μ, X]) (e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2)))))-(Subscript[σ, X]^2)(d/(dx))(e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2))))))dx
  = (Subscript[μ, X])(∫_(-∞)^∞) (1/((Sqrt[2π])(Subscript[σ, X])))(e^(-((((x-μ))^2)/(2(Subscript[σ, X]^2)))))dx-(1/((Sqrt[2π])(Subscript[σ, X])))(Subscript[σ, X]^2)(∫_(-∞)^∞)(d/(dx))(e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2)))))dx
  = (Subscript[μ, X])-(1/(Sqrt[2π]))(Subscript[σ, X])((lim_(x->∞))(e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2)))))-(lim_(x->-∞))(e^(-((((x-(Subscript[μ, X])))^2)/(2(Subscript[σ, X]^2))))))    (density functions integrate to 1, fundamental theorem calculus)
  = (Subscript[μ, X]).

(d)

𝔼[(X^2)] = (1/(i^2))((d^2)/(d(ω^2)))(Subscript[φ, X])(ω)(Subscript[|, ω=0])  = (1/(i^2))((d^2)/(d(ω^2)))(e^(i μ ω-(1/2)(σ^2)(ω^2)))(Subscript[|, ω=0])
  = (1/(i^2))(d/(dω))(i μ-(σ^2)ω )(e^(i μ ω-(1/2)(σ^2)(ω^2)))(Subscript[|, ω=0])
  = (1/(i^2))((((i μ-(σ^2)ω ))^2)(e^(i μ ω-(1/2)(σ^2)(ω^2)))-(σ^2)(e^(i μ ω-(1/2)(σ^2)(ω^2))))(Subscript[|, ω=0])
  = (1/(i^2))((((i μ))^2)-(σ^2)) = -(-(μ^2)+(σ^2))
  = (μ^2)+(σ^2)

Using Mathematica function “D”

(1/(I^2))D[(E^(I μ ω - (1/2)(σ^2)(ω^2))),{ω,2}]/.ω->0//Simplify

Using Mathematica function “Expectation”

Expectation[(x^2),x\[Distributed]NormalDistribution[μ,σ]]

Expectation[(x^2),Distributed[x,NormalDistribution[μ,σ]]]

(d)

(∫_(-∞)^∞)|(e^(-i ω x))(Subscript[φ, X])(ω)|dω = (∫_(-∞)^∞)|(e^(-i ω x))(e^(i (Subscript[μ, X])ω-(1/2)(Subscript[σ, X]^2)(ω^2)))|dω
  = (∫_(-∞)^∞)|(e^(i ((Subscript[μ, X])-x)ω))|(e^(- (1/2)(Subscript[σ, X]^2)(ω^2)))dω    (positivity of exponential function)
  = (∫_(-∞)^∞)|cos(((Subscript[μ, X])-x)ω)+i sin(((Subscript[μ, X])-x)ω)|(e^(- (1/2)(Subscript[σ, X]^2)(ω^2)))dω
  = (∫_(-∞)^∞)(Sqrt[(cos^2)(((Subscript[μ, X])-x)ω)+(sin^2)(((Subscript[μ, X])-x)ω)])(e^(- (1/2)(Subscript[σ, X]^2)(ω^2)))dω
  = (∫_(-∞)^∞)(e^(- (1/2)(Subscript[σ, X]^2)(ω^2)))dω = (1/(Subscript[σ, X]))(∫_(-∞)^∞)(e^(- (1/2)(z^2)))dz    (z=(Subscript[σ, X])ω)
  = ((Sqrt[2π])/(Subscript[σ, X]))(∫_(-∞)^∞)(1/(Sqrt[2π]))(e^(- (1/2)(z^2)))dz    (integrand is density of N(0,1) RV)
  = ((Sqrt[2π])/(Subscript[σ, X]))    (densities integerate to 1)
  < ∞
if 0<(Subscript[σ, X])<∞.

Using Mathematica function “InverseFourierTransform”

φX[ω_]:=Exp[I μX ω-(1/2)((σX)^2)(ω^2)]
InverseFourierTransform[φX[ω],ω,x,FourierParameters->{1,1}]//FullSimplify

Q2

(a)

(x^2)((dy)/(dx))+x(x+2)y = 0
⟹    (1/y)dy = -((x(x+2))/(x^2))dx
⟹    ∫(1/y)dy = -∫(1+(2/x))dx
⟹    ln(y) = -x-2ln(x)+(Subscript[c, 1])
⟹    y(x) = (e^(-x-2ln(x)+(Subscript[c, 1]))) = ((e^(Subscript[c, 1]))/((e^x)(x^2))) = ((Subscript[c, 2])/((e^x)(x^2)))

Using Mathematica function “DSolve”

DSolve[(x^2) y'[x]+x(x+2)y[x]==0,y[x],x]

(b)

Recall that the solution to
((dy)/(dx))+P(x)y=f(x)
can be found by integrating both sides of
(d/(dx))((e^(∫P(x)dx))y) = (e^(∫P(x)dx))f(x).

In this case
(x^2)((dy)/(dx))+x(x+2)y = (e^x)
or
((dy)/(dx))+(1+(2/x))y = ((e^x)/(x^2)).

The integrating factor is
(e^(∫(1+(2/x))dx)) = (e^(x+2ln(x)+(Subscript[c, 1]))) = (e^(Subscript[c, 1]))(x^2)(e^x) = (Subscript[c, 2])(x^2)(e^x)

Therefore
(d/(dx))((Subscript[c, 2])(x^2)(e^x)y) = (Subscript[c, 2])(x^2)(e^x)((e^x)/(x^2)) = (Subscript[c, 2])(e^(2x))
which after integrating both sides
∫(d/(dx))((x^2)(e^x)y)dx = ∫(e^(2x))dx
becomes
(x^2)(e^x)y = (1/2)(e^(2x))+(Subscript[c, 3])
so
y(x) = ((e^x)/(2(x^2)))+((Subscript[c, 3])/((x^2)(e^x))).

Using Mathematica function “DSolve”

DSolve[(x^2) y'[x]+x(x+2)y[x]==(E^x),y[x],x]

Q3

(a)

The system is
(({{2, 1, 11, 18}, {4, 3, 25, 40}, {0, 1, 3, 4}})) ~ (({{4, 2, 22, 36}, {4, 3, 25, 40}, {0, 1, 3, 4}})) ~ (({{4, 2, 22, 36}, {0, 1, 3, 4}, {0, 1, 3, 4}})) ~ (({{4, 0, 16, 28}, {0, 1, 3, 4}, {0, 0, 0, 0}})) ~ (({{1, 0, 4, 7}, {0, 1, 3, 4}, {0, 0, 0, 0}}))
or
(({{1, 0, 4}, {0, 1, 3}, {0, 0, 0}}))(({{(Subscript[x, 1])}, {(Subscript[x, 2])}, {(Subscript[x, 3])}})) = (({{7}, {4}, {0}}))
which gives
(Subscript[x, 1])+4(Subscript[x, 3])=7
(Subscript[x, 2])+3(Subscript[x, 3])=4
or in vector form
x = (({{(Subscript[x, 1])}, {(Subscript[x, 2])}, {(Subscript[x, 3])}})) = (({{7}, {4}, {0}}))+(({{-4}, {-3}, {1}}))(Subscript[x, 3]).

Using Mathematica function “LinearSolve”

AA={{2,1,11},{4,3,25},{0,1,3}};
b={18,40,4};
LinearSolve[AA,b]

As this is an under-determined system, Mathematica only returns one solution (in this case with (Subscript[x, 3])=0).

(b)

The eingenvalue/eignevector equation is
A x = λ x
or
(A-λ I)x = 0.

Note that if    (A-λ I)    has an inverse then
(A-λ I)x = 0    ⟹    (((A-λ I))^(-1))(A-λ I)x = (A-λ I)0
⟹    x = 0.

A non-trivial solution exists if    (A-λ I)    is not invertible, which is the case if
det(A-λ I)=0.

Proceeding
det(({{7-λ, 4, 16}, {2, 5-λ, 8}, {-2, -2, -5-λ}}))
  = (7-λ)((5-λ)(-5-λ)+16)-4(2(-5-λ)+16)+16(-4+2(5-λ))
  = 9-15 λ+7 (λ^2)-(λ^3).

It is clear the polynomial above equals zero when λ=1.

After polynomial division (details omitted -- hard to typeset) we have
9-15 λ+7 (λ^2)-(λ^3) = -(λ-1)((λ^2)-6λ+9) = -(λ-1)(((λ-3))^2)
which equals zero when the eingenvalues are λ=1, λ=3 (with multiplicity 2).

Eigenvector for λ=1.
(({{6, 4, 16, 0}, {2, 4, 8, 0}, {-2, -2, -6, 0}})) ~ (({{4, 0, 8, 0}, {2, 4, 8, 0}, {0, 2, 2, 0}})) ~ (({{1, 0, 2, 0}, {0, 1, 1, 0}, {0, 1, 1, 0}})) ~ (({{1, 0, 2, 0}, {0, 1, 1, 0}, {0, 0, 0, 0}})) ~ (({{1, 0, 2, 0}, {0, 1, 1, 0}, {0, 0, 0, 0}}))
So 
x = (({{(Subscript[x, 1])}, {(Subscript[x, 2])}, {(Subscript[x, 3])}})) = (({{-2}, {-1}, {1}}))(Subscript[x, 3])=(({{-2}, {-1}, {1}}))
making the arbitrary choice (Subscript[x, 3])=1.

Eigenvector for λ=3.
(({{4, 4, 16, 0}, {2, 2, 8, 0}, {-2, -2, -8, 0}})) ~ (({{1, 1, 4, 0}, {0, 0, 0, 0}, {0, 0, 0, 0}}))
So
x = (({{(Subscript[x, 1])}, {(Subscript[x, 2])}, {(Subscript[x, 3])}})) = (({{-1}, {1}, {0}}))(Subscript[x, 2])+(({{-4}, {0}, {1}})) (Subscript[x, 3])= (({{-4}, {0}, {1}}))
after making the arbitrary choice (Subscript[x, 2])=0, (Subscript[x, 3])=1 and
x = (({{(Subscript[x, 1])}, {(Subscript[x, 2])}, {(Subscript[x, 3])}})) = (({{-1}, {1}, {0}}))(Subscript[x, 2])+(({{-4}, {0}, {1}}))(Subscript[x, 3])=(({{-1}, {1}, {0}}))
after making the arbitrary choice (Subscript[x, 2])=1, (Subscript[x, 3])=0.

Polynomial division

Clear[λ];
((9-15 λ+7 (λ^2)-(λ^3))/(λ-1))//FullSimplify

Using Mathematica function “Eigensystem”

m={{7,4,16},{2,5,8},{-2,-2,-5}};
Eigensystem[m]//MatrixForm

Q4

(a)

The expansion is given by
f(x) ≈ f(a)+f'(a)(x-a)+(1/2)f''(a)(((x-a))^2).

First note that
(d/(dx))(cos(x)(e^(2x))) = 2cos(x)(e^(2x))-sin(x)(e^(2x))
and
((d^2)/(d(x^2)))(cos(x)(e^(2x))) = (d/(dx))(2cos(x)(e^(2x)))-(d/(dx))(sin(x)(e^(2x)))
  = 4cos(x)(e^(2x))-2sin(x)(e^(2x))-2sin(x)(e^(2x))-cos(x)(e^(2x))
  = 3cos(x)(e^(2x))-4sin(x)(e^(2x))
so the expansion about a using three terms is
t(x|a) ≈ cos(a)(e^(2a))+(2cos(a)-sin(a))(e^(2a))(x-a)
                       +(1/2)(3cos(a)-4sin(a))(e^(2a))(((x-a))^2)

Using Mathematica function “Series”

f[x_]:=Cos[x](E^(2x));
Series[f[x],{x,a,2}]
Normal[Series[f[x],{x,a,2}]]    (* drops Big-O terms *)

(b)

t[x_,a_]:=(E^(2 a)) Cos[a]+(1/2) (E^(2 a)) (((x-a))^2) (3 Cos[a]-4 Sin[a])+(E^(2 a)) (x-a) (2 Cos[a]-Sin[a]);
Plot[{t[x,-1],t[x,0],t[x,1],f[x]},{x,-2,2},
PlotLegends->{"t(x;-1)","t(x;0)","t(x;1)","f(x)"},
PlotStyle->{Blue,Darker[Green],Purple,Red}]