Chapter 3

Supporting notes

Scott Alexander

Frame 6

Consider the sequences of RVs
(Subscript[ξ, 1]), (Subscript[ξ, 2]), …, (Subscript[ξ, n]), …
and
(Subscript[η, 1]), (Subscript[η, 2]), …, (Subscript[η, n]), ….

If
(Subscript[ξ, 1]), (Subscript[ξ, 2]), …, (Subscript[ξ, n]), … (Overscript[→, P]) ξ
and
(Subscript[η, 1]), (Subscript[η, 2]), …, (Subscript[η, n]), … (Overscript[→, P]) η
then
(Subscript[ξ, 1])+(Subscript[η, 1]), (Subscript[ξ, 2])+(Subscript[η, 2]), …, (Subscript[ξ, n])+(Subscript[η, n]), … (Overscript[→, P]) ξ+η
and
(Subscript[ξ, 1])(Subscript[η, 1]), (Subscript[ξ, 2])(Subscript[η, 2]), …, (Subscript[ξ, n])(Subscript[η, n]), … (Overscript[→, P]) ξ η.

Frame 9

First inequality.
If    \[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar]>ϵ    then
𝔼[((\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar])^2)I(\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar]>ϵ)] = 𝔼[((\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar])^2)].

If    \[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar]≤ϵ    then
𝔼[((\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar])^2)I(\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar]>ϵ)] = 0
so
𝔼[((\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar])^2)] ≥ 𝔼[((\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar])^2)I(\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar]>ϵ)].


Second inequality.
If    \[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar]>ϵ    then
𝔼[((\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar])^2)I(\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar]>ϵ)] = 𝔼[((\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar])^2)] > 𝔼[(ϵ^2)] = 𝔼[(ϵ^2)I(\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar]>ϵ)].

If    \[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar]≤ϵ
𝔼[((\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar])^2)I(\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar]>ϵ)] = 0
and
𝔼[(ϵ^2)I(\[LeftBracketingBar](Subscript[ξ, n])-ξ\[RightBracketingBar]>ϵ)] = 0.

Frame 12

To see that
var((1/n)(∑_(i=1)^n)(Subscript[X, i])) = (1/(n^2))(∑_(i=1)^n)var((Subscript[X, i]))
note that the (Subscript[X, i]) are uncorrelated and use argument Frame 17-20 below.

Frame 14

CLT Example

(Subscript[ξ, n])-statistic when N(5,2)

m=5;
σ=2;
myDist=NormalDistribution[m,σ];
NN=10000;

n=4;
Xsample=Table[RandomReal[myDist],{NN},{n}];
histogramData=Table[((Mean[Xsample[[i]]]-m)/(σ/(Sqrt[n]))),{i,1,NN}];
meanPlot=Histogram[histogramData,{-4,4,(8/100)},"PDF"];
normalPlot=Plot[PDF[NormalDistribution[],x],{x,-4,4}];
p1=Show[meanPlot,normalPlot,PlotLabel->"n="<>ToString[n]];

n=30;
Xsample=Table[RandomReal[myDist],{NN},{n}];
histogramData=Table[((Mean[Xsample[[i]]]-m)/(σ/(Sqrt[n]))),{i,1,NN}];
meanPlot=Histogram[histogramData,{-4,4,(8/100)},"PDF"];
normalPlot=Plot[PDF[NormalDistribution[],x],{x,-4,4}];
p2=Show[meanPlot,normalPlot,PlotLabel->"n="<>ToString[n]];
graphicsNormal=GraphicsRow[{p1,p2}];(* Export size "ImageSize->800" works well with Beamer/LaTeX "width=100mm" For Beamer images export - keep ratio 8:1*)
graphicsNormal

(Subscript[ξ, n])-statistic when (Subscript[X, i])~Exp(2)

λ=2;
myDist=ExponentialDistribution[λ];
m=Mean[myDist];
σ=StandardDeviation[myDist];
NN=10000;

n=4;
Xsample=Table[RandomReal[myDist],{NN},{n}];
histogramData=Table[((Mean[Xsample[[i]]]-m)/(σ/(Sqrt[n]))),{i,1,NN}];
meanPlot=Histogram[histogramData,{-4,4,(8/100)},"PDF"];
normalPlot=Plot[PDF[NormalDistribution[],x],{x,-4,4}];
p1=Show[meanPlot,normalPlot,PlotLabel->"n="<>ToString[n]];

n=40;
Xsample=Table[RandomReal[myDist],{NN},{n}];
histogramData=Table[((Mean[Xsample[[i]]]-m)/(σ/(Sqrt[n]))),{i,1,NN}];
meanPlot=Histogram[histogramData,{-4,4,(8/100)},"PDF"];
normalPlot=Plot[PDF[NormalDistribution[],x],{x,-4,4}];
p2=Show[meanPlot,normalPlot,PlotLabel->"n="<>ToString[n]];
graphicsNormal=GraphicsRow[{p1,p2}];(* Export size "ImageSize->800" works well with Beamer/LaTeX "width=100mm" For Beamer images export - keep ratio 8:1*)
graphicsNormal

(Subscript[ξ, n])-statistic when (Subscript[X, i]) is dice RV

myDist=EmpiricalDistribution[{1.,2.,3.,4.,5.,6.}];
m=Mean[myDist];
σ=StandardDeviation[myDist];
NN=10000;

n=4;
Xsample=Table[RandomChoice[{1.,2.,3.,4.,5.,6.}],{NN},{n}];
histogramData=Table[((Mean[Xsample[[i]]]-m)/(σ/(Sqrt[n]))),{i,1,NN}];
meanPlot=Histogram[histogramData,{-4,4,(8/100)},"PDF"];
normalPlot=Plot[PDF[NormalDistribution[],x],{x,-4,4}];
p1=Show[meanPlot,normalPlot,PlotLabel->"n="<>ToString[n]];

n=40;
Xsample=Table[RandomChoice[{1.,2.,3.,4.,5.,6.}],{NN},{n}];
histogramData=Table[((Mean[Xsample[[i]]]-m)/(σ/(Sqrt[n]))),{i,1,NN}];
meanPlot=Histogram[histogramData,{-4,4,(8/100)},"PDF"];
normalPlot=Plot[PDF[NormalDistribution[],x],{x,-4,4}];
p2=Show[meanPlot,normalPlot,PlotLabel->"n="<>ToString[n]];
graphicsDice=GraphicsRow[{p1,p2}];
graphicsDice

Frame 17-20

Estimating G = 𝔼[g (X)] using law of large numbers

Let X be a RV with density function (Subscript[f, X]).

Suppose we wish to estimate    G:=𝔼[g(X)] = (∫_(-∞)^∞)g(x)(Subscript[f, X])(x)dx.

Let    (Subscript[G, n])=(1/n)(∑_(i=1)^n)g((Subscript[X, i]))    where (Subscript[X, i]) are "independent and identically distributed" (iid) as X.
Note that    𝔼[(Subscript[G, n])]=𝔼[(1/n)(∑_(i=1)^n)g((Subscript[X, i]))]=(1/n)(∑_(i=1)^n)𝔼[g((Subscript[X, i]))]=(1/n)n 𝔼[g((Subscript[X, i]))] = G.    ((Subscript[G, n]) is an unbiased estimator of G)

The are two versions of the LLN.
The weak version is by Markov (see page 11 Chapter 3 Notes).
The strong version is by Kolmogorov (not in notes).

By the weak version
(lim_(n->∞)) (Subscript[G, n ])= (lim_(n->∞))(1/n)(∑_(i=1)^n)g((Subscript[X, i])) (Overscript[=, P]) 𝔼[g((Subscript[X, i]))] = 𝔼[g(X)]    (X,(Subscript[X, i]) are iid)
  = G
which is what we are trying to estimate.

So if we sample n copies of (Subscript[X, i]) and calculate the sum above we have a MC estimate of G=𝔼[g(X)], i.e. for large n
(Subscript[G, n]) = (1/n)(∑_(i=1)^n)g((Subscript[X, i]))≈ 𝔼[g(X)] = G.

Finding error of MC estimate using CLT

Let
(σ^2)(g):=var(g(X)) 
and note that
var((Subscript[G, n]))=var((1/n)(∑_(i=1)^n)g((Subscript[X, i]))) = cov((1/n)(∑_(i=1)^n)g((Subscript[X, i])),(1/n)(∑_(j=1)^n)g((Subscript[X, j])))
  = 𝔼[(1/n)(∑_(i=1)^n)g((Subscript[X, i]))(1/n)(∑_(j=1)^n)g((Subscript[X, j]))] - 𝔼[(1/n)(∑_(i=1)^n)g((Subscript[X, i]))]𝔼[(1/n)(∑_(j=1)^n)g((Subscript[X, j]))]
  = (1/(n^2))((∑_(i=1)^n)(∑_(j=1)^n)𝔼[g((Subscript[X, i]))g((Subscript[X, j]))] - (∑_(i=1)^n)(∑_(j=1)^n)𝔼[g((Subscript[X, i]))]𝔼[g((Subscript[X, j]))])
  = (1/(n^2))(∑_(i=1)^n)(∑_(j=1)^n)(𝔼[g((Subscript[X, i]))g((Subscript[X, j]))] - 𝔼[g((Subscript[X, i]))]𝔼[g((Subscript[X, j]))])
  = (1/(n^2))(∑_(i=1)^n)(∑_(j=1)^n)cov(g((Subscript[X, i])),g((Subscript[X, j])))
  = (1/(n^2))((∑_(i=1)^n)cov(g((Subscript[X, i])),g((Subscript[X, i]))) +2 (∑_(i=1)^n)(∑_(j=i+1)^n)cov(g((Subscript[X, i])),g((Subscript[X, j]))))
  = (1/(n^2))(∑_(i=1)^n)var(g((Subscript[X, i])))    ((Subscript[X, i]),(Subscript[X, j]) are independent so have zero covariance)
  = (1/(n^2))(∑_(i=1)^n)var(g(X))    ((Subscript[X, i]) all follow distribution of X)
  = (1/(n^2))n var(g(X))
  = (((σ^2)(g))/n).

By the CLT
(lim_(n->∞))(((Subscript[G, n]) - G)/(Sqrt[var((Subscript[G, n]))])) = (lim_(n->∞))(((Subscript[G, n]) - G)/(σ(g)/(Sqrt[n]))) ~ N(0,1).

So for large n we have the approximation
(((Subscript[G, n]) - G)/(Sqrt[var((Subscript[G, n]))])) ~ N(0,1).

Error bound and CI of true value

Let Φ be the N(0,1) CDF.

Then by CLT
P(((|(Subscript[G, n])-G|)/(Sqrt[var((Subscript[G, n]))])) ≤ z) = P(-z ≤ (((Subscript[G, n])-G)/(Sqrt[var((Subscript[G, n]))])) ≤ z) ≈ Φ(z) - Φ(-z)= Φ(z)-(1-Φ(z)) = 2Φ(z)-1
or
P(|(Subscript[G, n])-G| ≤ z (Sqrt[var((Subscript[G, n]))])) ≈ 2Φ(z)-1    (A)
where
var((Subscript[G, n])) = ((var(g(X)))/n) = (((σ^2)(g))/n).

For example, if we take z=3 then    P(|(Subscript[G, n])-G| ≤ ((3 σ(g))/(Sqrt[n])))≈99.73%.    (see code below)

That is, with approximately 99.73% probability we have the error bound
|(Subscript[G, n])-G| ≤ ((3 σ(g))/(Sqrt[n]))
or CI (confidence interval) of the true value
-((3 σ(g))/(Sqrt[n]))+(Subscript[G, n]) ≤ G ≤ (Subscript[G, n])+((3 σ(g))/(Sqrt[n])).

Interpretation of last?
If we constructed 10000 MC estimates we would expect 9973 of the CIs to contain G.

2CDF[NormalDistribution[],3]-1//N

Frame 21-22

Convergence of MC estimator (Subscript[G, n]).
We have
|(Subscript[G, n])-G| =O((n^(-1/2)))    as    n->∞.

This means that there exists a c>0 and an (Subscript[n, 0])>0 such that
|(Subscript[G, n])-G| ≤ (c/(n^(1/2)))    for all    n>(Subscript[n, 0]).

Convergence of Simpson's quadrature estimator (Subscript[H, n]).
We have
|(Subscript[H, n])-G| =O((n^(-4/m)))    as    n->∞.

This means that there exists a c>0 and an (Subscript[n, 0])>0 such that
|(Subscript[H, n])-G| ≤ (c/(n^(4/m)))    for all    n>(Subscript[n, 0]).

Note that    (1/(n^(1/2)))<(1/(n^(4/m)))    when m>8 and n>1.

Plot[{(n^(-4/8)),(n^(-4/9)),(n^(-4/10))},{n,0,10},PlotLegends->"Expressions"]

Frame 23

Let    X~U(0,1)    which has PDF
(Subscript[f, X])(x) = I(0≤x≤1) = ({{{, ({{1,, 0≤x≤1}, {0,, otherwise}})}}).

Setting
g(x) = x (e^(-x))
we can write
G = (∫_0^1)x (e^(-x))dx = (∫_(-∞)^∞)x (e^(-x))I(0≤x≤1)dx
  = (∫_(-∞)^∞)g(x)(Subscript[f, X])(x)dx = 𝔼[g(X)].

SeedRandom[1234];
n=10^5;
g[x_]:=x (E^(-x));
rv=RandomReal[UniformDistribution[{0,1}],n];
rv=Map[g,rv];
Gn=Mean[rv];
varGn=Variance[rv]/n;
sdGn=(Sqrt[varGn]);
{Gn,varGn,sdGn}

Frame 24

Let    X~Exp(1)    which has PDF
(Subscript[f, X])(x) = (e^(-x))I(x≥0) = ({{{, ({{(e^(-x)),, 0≤x≤1}, {0,, otherwise}})}}).

Setting
g(x) = x I(x≤1)
we can write
G = (∫_0^1)x (e^(-x))dx = (∫_(-∞)^∞)x I(x≤1)(e^(-x))I(x≥0)dx
  = (∫_(-∞)^∞)g(x)(Subscript[f, X])(x)dx = 𝔼[g(X)].

SeedRandom[1234];
n=10^5;
g[x_]:=If[x≤1,x,0];
rv=RandomReal[ExponentialDistribution[1],n];
rv=Map[g,rv];
Gn=Mean[rv];
varGn=Variance[rv]/n;
sdGn=(Sqrt[varGn]);
{Gn,varGn,sdGn}

PDF[ExponentialDistribution[1],x]

Frame 27

(lim_(n->∞)) (Subscript[(OverTilde[G]), n]) = (lim_(n->∞)) ((Subscript[G, n])-a((Subscript[Q, n])-Q))
  = (lim_(n->∞)) (Subscript[G, n])-a(lim_(n->∞)) (Subscript[Q, n])+a Q
  (Overscript[=, P])  G-a Q+a Q
  = G.

Frame 28

Finding var((Subscript[(OverTilde[G]), n]))

var((Subscript[(OverTilde[G]), n])) = var((Subscript[G, n])-a((Subscript[Q, n])-Q)) = var((Subscript[G, n])-a (Subscript[Q, n]))
  = cov((Subscript[G, n])-a (Subscript[Q, n])(Subscript[G, n])-a (Subscript[Q, n]))
  = cov((Subscript[G, n]),(Subscript[G, n]))-cov((Subscript[G, n]),a (Subscript[Q, n]))-cov(a (Subscript[Q, n]),(Subscript[G, n]))+cov(a (Subscript[Q, n]),a (Subscript[Q, n]))
  = var((Subscript[G, n]))+(a^2)var((Subscript[Q, n]))-2a cov((Subscript[G, n]),(Subscript[Q, n]))
 =(1/n)(σ^2)(g)+(a^2)var((1/n)(∑_(i=1)^n)q((Subscript[X, i])))-2a cov((1/n)(∑_(i=1)^n)g((Subscript[X, i])),(1/n)(∑_(j=1)^n)q((Subscript[X, j])))
 = (1/n)(σ^2)(g)+(a^2)(1/(n^2))cov((∑_(i=1)^n)q((Subscript[X, i])),(∑_(j=1)^n)q((Subscript[X, j])))-2a (1/(n^2))cov((∑_(i=1)^n)g((Subscript[X, i])),(∑_(j=1)^n)q((Subscript[X, j])))
 = (1/n)(σ^2)(g)+(a^2)(1/(n^2))(∑_(i=1)^n)(∑_(j=1)^n)cov(q((Subscript[X, i])),q((Subscript[X, j])))-2a (1/(n^2))(∑_(i=1)^n)(∑_(j=1)^n)cov(g((Subscript[X, i])),q((Subscript[X, j])))
 = (1/n)(σ^2)(g)+(a^2)(1/(n^2))(∑_(i=1)^n)cov(q((Subscript[X, i])),q((Subscript[X, i])))-2a (1/(n^2))(∑_(i=1)^n)cov(g((Subscript[X, i])),q((Subscript[X, i])))    (independence)
 = (1/n)(σ^2)(g)+(a^2)(1/(n^2))(∑_(i=1)^n)var(q((Subscript[X, i])))-2a (1/(n^2))(∑_(i=1)^n)cov(g((Subscript[X, i])),q((Subscript[X, i])))
 = (1/n)(σ^2)(g)+(a^2)(1/(n^2))(∑_(i=1)^n)var(q(X))-2a (1/(n^2))(∑_(i=1)^n)cov(g(X),q(X))
 = (((σ^2)(g)+(a^2)(σ^2)(q)-2a cov(g(X),q(X)))/n)
  = (((σ^2)(g-a q))/n)

Frame 30-31

Taking the derivative
(d/(da))(((σ^2)(g-a q))/n) = (1/n)(d/(da))(σ^2)(g-a q)
  = (1/n)(d/(da))((σ^2)(g)+(a^2)(σ^2)(q)-2a cov(g(X),q(X)))
  = (1/n)(2a (σ^2)(q)-2 cov(g(X),q(X))).

Setting to zero gives
(SuperStar[a]) = ((cov(g(X),q(X)))/((σ^2)(q)))
so the minimum variance estimator (Subscript[(OverTilde[G]), n]) using (SuperStar[a])has
var((Subscript[(OverTilde[G]), n])) = (((σ^2)(g-(SuperStar[a]) q))/n)
  = (1/n)((σ^2)(g)+((((SuperStar[a])))^2)(σ^2)(q)-2(SuperStar[a]) cov(g(X),q(X)))
  = (1/n)((σ^2)(g)+(((((cov(g(X),q(X)))/((σ^2)(q)))))^2)(σ^2)(q)-2((cov(g(X),q(X)))/((σ^2)(q))) cov(g(X),q(X)))
  = (1/n)((σ^2)(g)+((cov(((g(X),q(X)))^2))/((σ^2)(q)))-2((cov(((g(X),q(X)))^2))/((σ^2)(q))) )
  = (1/n)((σ^2)(g)-((cov(((g(X),q(X)))^2))/((σ^2)(q))))
  = (1/n)((σ^2)(g)-(σ^2)(g)((cov(((g(X),q(X)))^2))/((σ^2)(g)(σ^2)(q))))
  = (((σ^2)(g))/n)(1-((cov(((g(X),q(X)))^2))/((σ^2)(g)(σ^2)(q))))
  = (((σ^2)(g))/n)(1-(Subscript[ρ, g,q]^2))
  = var((Subscript[G, n]))(1-(Subscript[ρ, g,q]^2))
where
(Subscript[ρ, g,q]) = corr(g(x),q(x)) = ((cov(g(X),q(X)))/(σ(g)σ(q))).

That is
var((Subscript[(OverTilde[G]), n])) = var((Subscript[G, n]))(1-(Subscript[ρ, g,q]^2)).

Frame 35

SeedRandom[1];
n=10^5;
g[x_]:=x (E^(-x));
q[x_]:=x;
QQ=Expectation[q[x],x\[Distributed]UniformDistribution[{0,1}]];
rv=RandomReal[UniformDistribution[{0,1}],n];
grv=g[rv];
qrv=q[rv];
aStar=Covariance[grv,qrv]/Variance[qrv];
rv=grv-aStar(qrv-QQ);
GnTilde=Mean[rv];
varGnTilde=Variance[rv]/n;
sdGnTilde=(Sqrt[varGnTilde]);
{GnTilde,varGnTilde,sdGnTilde}

Frame 36

How to find antithetic variate?

Set    Y=2E[X]-X.

Note that
𝔼[Y] = 𝔼[2E[X]-X] = 2E[X] - 𝔼[X]    (E[X] is non-random)
  = 𝔼[X],
var(Y) = var(2E[X]-X) = var(-X)    (E[X] is non-random)
  = 𝔼[(((-X))^2)] - 𝔼[-X]𝔼[-X]
  = 𝔼[(X^2)]-𝔼[X]𝔼[X]
  = var(X)
and
cov(X,Y) = cov(X,2 𝔼[X]-X) = -cov(X,X)
 = -var(X) < 0.

So Y and X have the same mean and variance and negative covariance.

If X is uniform or normal (there are other cases), then Y will follow the same distribution.

To see this note that the CF of Y
𝔼[(e^(i u Y))] = 𝔼[(e^(i u (2E[X]-X)))] =(e^(2i u  E[X]))𝔼[(e^(-i u X))].

So if X~Unif(a,b)
𝔼[(e^(i u X))] = ((i (-(e^(i a u))+(e^(i b u))))/((a-b) u))
and
𝔼[(e^(i u Y))] = ((i (-(e^(i a u))+(e^(i b u))))/((a-b) u))    (see code below).

Also, if X~N(μ,σ)
𝔼[(e^(i u X))] = (e^(i u μ-(((u^2) (σ^2))/2)))
and
𝔼[(e^(i u Y))] = (e^(i u μ-(((u^2) (σ^2))/2)))    (see code below).

Clear[a,b];
dist=UniformDistribution[{a,b}];
(Subscript[μ, X])=Mean[dist];
CharacteristicFunction[dist,u]//FullSimplify
(E^(2 I u (Subscript[μ, X])))CharacteristicFunction[dist,-u]//FullSimplify

Clear[μ,σ];
dist=NormalDistribution[μ,σ];
(Subscript[μ, X])=Mean[dist];
CharacteristicFunction[dist,u]//FullSimplify
(E^(2 I u (Subscript[μ, X])))CharacteristicFunction[dist,-u]//FullSimplify

Frame 38

Finding var((Subscript[(OverHat[G]), n]))

var((Subscript[(OverHat[G]), n])) = var((1/(2n))(∑_(i=1)^n)((Subscript[g, 1])((Subscript[X, i]))+(Subscript[g, 2])((Subscript[X, i]))))
  = (1/(4(n^2)))var((∑_(i=1)^n)((Subscript[g, 1])((Subscript[X, i]))+(Subscript[g, 2])((Subscript[X, i]))))
  = (1/(4(n^2)))(var((∑_(i=1)^n)(Subscript[g, 1])((Subscript[X, i])))+var((∑_(i=1)^n)(Subscript[g, 2])((Subscript[X, i])))+2cov((∑_(i=1)^n)(Subscript[g, 1])((Subscript[X, i])),(∑_(i=1)^n)(Subscript[g, 2])((Subscript[X, i]))))
  = (1/(4(n^2)))((∑_(i=1)^n)var((Subscript[g, 1])((Subscript[X, i])))+ (∑_(j=1)^n)var((Subscript[g, 2])((Subscript[X, i])))+2(∑_(i=1)^n)cov(((Subscript[g, 1])((Subscript[X, i]))),(Subscript[g, 2])((Subscript[X, i]))))    (independence)
  = (1/(4(n^2)))((∑_(i=1)^n)var((Subscript[g, 1])(X))+ (∑_(j=1)^n)var((Subscript[g, 2])(X))+2(∑_(i=1)^n)cov(((Subscript[g, 1])(X)),(Subscript[g, 2])(X)))
  = (1/(4n))((σ^2)((Subscript[g, 1]))+ (σ^2)((Subscript[g, 2]))+2cov(((Subscript[g, 1])(X)),(Subscript[g, 2])(X)))
  = (((σ^2)((Subscript[g, 1])+(Subscript[g, 2])))/(4n))

Frame 38

SeedRandom[1];
n=10^5;
g[x_]:=x (E^(-x));
μX=Expectation[x,x\[Distributed]UniformDistribution[{0,1}]];
rv=RandomReal[UniformDistribution[{0,1}],n];
g1rv=g[rv];
g2rv=g[2μX-rv];
rv=(g1rv+g2rv)/2;
GnHat=Mean[rv];
varGnHat=Variance[rv]/n;
sdGnHat=(Sqrt[varGnHat]);
{GnHat,varGnHat,sdGnHat}

Frame 41-44

Let (Subscript[F, X]^(-1)) be the inverse of (Subscript[F, X]) so that (Subscript[F, X]^(-1))((Subscript[F, X])(x))=x.

Note that for any RV X, its distribution function (Subscript[F, X])(X)~Uniform(0,1).

To see this define U=(Subscript[F, X])(X) and note that
(Subscript[F, U])(u) = P(U≤u) = P((Subscript[F, X])(X)≤u) = P((Subscript[F, X]^(-1))((Subscript[F, X])(X))≤(Subscript[F, X]^(-1))(u))
  = P(X≤(Subscript[F, X]^(-1))(u)) = (Subscript[F, X])((Subscript[F, X]^(-1))(u)) = u
which is CDF of Uniform(0,1) RV.    (see https://en.wikipedia.org/wiki/Probability_integral_transform)

To show X=(Subscript[F, X]^(-1))(U) has distribution function (Subscript[F, X]) note that
P(X≤x) = P((Subscript[F, X]^(-1))(U)≤x) = P((Subscript[F, X])((Subscript[F, X]^(-1))(U))≤(Subscript[F, X])(x)) = P(U≤(Subscript[F, X])(x))
  = (Subscript[F, X])(x)
as P(U≤u) = (Subscript[F, U])(u)=u for Uniform(0,1) RVs.

Exponential Example.
(Subscript[F, X])(x) = (∫_(-∞)^x)(Subscript[f, X])(z)dz = (∫_0^x)λ (e^(-λ z))dz
  = -(Subscript[[(e^(-λ z))], 0]^x) = 1-(e^(-λ x)).

Now set
u = (Subscript[F, X])(x)
  = 1-(e^(-λ x))
which is
(e^(-λ x)) = 1-u
or
x = -((ln(1-u))/λ).

That is we have
(Subscript[F, X]^(-1))(u) = (Subscript[F, X]^(-1))((Subscript[F, X])(x)) = x =  -((ln(1-u))/λ).

Therefore for U~Unif(0,1)
(Subscript[F, X]^(-1))(U) = (Subscript[F, X]^(-1))((Subscript[F, X])(X)) = X =  -((ln(1-U))/λ) ~ -((ln(U))/λ) ~ Exp(λ).

Example - N (2, 4) random number generation

F(x) = (∫_(-∞)^x)f(z)dz = (∫_(-∞)^x)(1/(Sqrt[2π (σ^2)]))(e^((-(({z - μ})^2))/(2(σ^2))))dz.

Using inverse function method

Module[{n,μ,σ,rv,histogram,density},
n=10000;
μ=2;σ=4;
rv=RandomReal[UniformDistribution[{0,1}],n];
rv=InverseCDF[NormalDistribution[μ,σ],rv];
histogram=Histogram[rv,Automatic,"PDF"];
density=Plot[PDF[NormalDistribution[μ,σ],x],{x,μ-3σ,μ+3σ},PlotRange->All,PlotStyle->{Blue,Thick}];
Print[Show[{histogram,density}]];
{{"μ",μ},
{"σ",σ},
{"(Subscript[μ, n])",Mean[rv]},
{"(Subscript[σ, n])",StandardDeviation[rv]},
{"(Subscript[σ, n])/(Sqrt[n])",StandardDeviation[rv]/(Sqrt[n])}}//TableForm
]

Using Box-Muller method

Module[{n,μ,σ,rv,histogram,density},
n=10000;
μ=2;σ=4;
rv1=RandomReal[UniformDistribution[{0,1}],n];
rv2=RandomReal[UniformDistribution[{0,1}],n];
rv=(Sqrt[-2Log[rv1]])Cos[2π rv2];
rv=μ+σ rv;
histogram=Histogram[rv,Automatic,"PDF"];
density=Plot[PDF[NormalDistribution[μ,σ],x],{x,μ-3σ,μ+3σ},PlotRange->All,PlotStyle->{Blue,Thick}];
Print[Show[{histogram,density}]];
{{"μ",μ},
{"σ",σ},
{"(Subscript[μ, n])",Mean[rv]},
{"(Subscript[σ, n])",StandardDeviation[rv]},
{"(Subscript[σ, n])/(Sqrt[n])",StandardDeviation[rv]/(Sqrt[n])}}//TableForm
]

Using Mathematica functions

Module[{n,μ,σ,rv,histogram,density},
n=10000;
μ=2;σ=4;
rv=RandomReal[NormalDistribution[μ,σ],n];
histogram=Histogram[rv,Automatic,"PDF"];
density=Plot[PDF[NormalDistribution[μ,σ],x],{x,μ-3σ,μ+3σ},PlotRange->All,PlotStyle->{Blue,Thick}];
Print[Show[{histogram,density}]];
{{"μ",μ},
{"σ",σ},
{"(Subscript[μ, n])",Mean[rv]},
{"(Subscript[σ, n])",StandardDeviation[rv]},
{"(Subscript[σ, n])/(Sqrt[n])",StandardDeviation[rv]/(Sqrt[n])}}//TableForm
]