Proving variance of b0(hat)

s-z

New Member
#1
i have a question asking to prove (attached image):


I'm completely stuck and any help would be appreciated. thanks
 

Dason

Ambassador to the humans
#2
Hi! :welcome: We are glad that you posted here! This looks like a homework question though. Our homework help policy can be found here. We mainly just want to see what you have tried so far and that you have put some effort into the problem. I would also suggest checking out this thread for some guidelines on smart posting behavior that can help you get answers that are better much more quickly.
 

staassis

Active Member
#3
This is most easily proven in the matrix form. Let the model be

Y = X * Beta + Epsilon,

where all elements of Epsilon have mean 0 and variance sigma^2. Then the vector of coefficient estimates equals

Beta_Hat = (X' * X)^{-1} * X' * Y

=====>

Beta_Hat - Beta = (X' * X)^{-1} * X' * Epsilon

=====>

Cov(Beta_Hat) = E[ (Beta_Hat - Beta) * (Beta_Hat - Beta)' ] = (X' * X)^{-1} * X' * E[Epsilon * Epsilon'] * X * (X' * X)^{-1} =

= (X' * X)^{-1} * X' * sigma^2 * I * X * (X' * X)^{-1} = sigma^2 * (X' * X)^{-1}.

Your formula is just the 1st element on the diagonal.
 

s-z

New Member
#4
ok so i think i got it but not sure.. sorry about the format also, wasn't sure how else to insert it:

Let ∑_(i=1)^n▒〖(x_i-x ̅ )^2=SST_x 〗

Prove β ̂_1=β_1+∑_(i=1)^n▒〖w_i u_i 〗 where; w_i=d_i/(SST_x ) , d_i=x_i-x ̅
β ̂_1=β_1+(∑_(i=1)^n▒〖(x_i-x ̅)u_i 〗)/(SST_x )
=β_1+(∑_(i=1)^n▒〖d_i u_i 〗)/(SST_x )
=β_1 ∑_(i=1)^n▒〖w_i u_i 〗

Prove E[(β ̂_1-β_1 ) u ̅ ]=0 to show β ̂_1 and u ̅ are uncorrelated:

E[(β ̂_1-β_1 ) u ̅ ]=E[u ̅ ∑_(i=1)^n▒〖w_i u_i 〗]
=∑_(i=1)^n▒〖〖E(w〗_i 〖u ̅ u〗_i 〗)
=∑_(i=1)^n▒〖w_i E(〖u ̅ u〗_i 〗)
=1/n ∑_(i=1)^n▒〖w_i E(〖 u〗_i ∑▒u_j 〗)
=1/n ∑_(i=1)^n▒〖w_i [E(〖 u〗_i 〖 u〗_1 )+⋯E(〖 u〗_i 〖 u〗_j )+⋯+E(〖 u〗_i 〖 u〗_n )]〗
=1/n ∑_(i=1)^n▒〖w_i E(u^2 〗)
=1/n ∑_(i=1)^n▒〖w_i [var(〖 u〗_i )+E(〖 u〗_i )E(〖 u〗_i )]〗
=1/n ∑_(i=1)^n▒〖w_i σ^2 〗
=σ^2/n ∑_(i=1)^n▒w_i
=σ^2/(n . SST_x ) ∑_(i=1)^n▒〖(x_i-x ̅)〗
=σ^2/(n . SST_x )

Prove β ̂_0=β_0+u ̅-x ̅(β ̂_1-β_1)
β ̂_0=y ̅-β ̂_1 x ̅
=(β_0+β_1 x ̅+u ̅ )-β ̂_1 x ̅
=β_0+u ̅-x ̅(β ̂_1-β_1)

Prove 〖var(β ̂_0 )=σ^2/n+(σ^2 (〖x ̅)〗^2)/〖SST〗_x 〗_
var(β ̂_0 )=var[β_0+u ̅-x ̅(β ̂_1-β_1 )]
=var(u ̅ )+(-u ̅ )^2 var(β ̂_1-β_1)
=(σ^2/n)+(x ̅ )^2 var(β ̂_1)
=(σ^2/n)+(σ^2 (x ̅ )^2)/〖SST〗_x

Therefore;
var(β ̂_0 )=(σ^2/n)+(σ^2 (x ̅ )^2)/〖SST〗_x
=(σ^2.〖SST〗_x)/(〖SST〗_x.n)+(σ^2 (x ̅ )^2)/SST
=σ^2/〖SST〗_x (1/n ∑_(i=1)^n▒〖x_i^2-(x ̅ )^2)+(σ^2 (x ̅ )^2)/〖SST〗_x 〗
=(σ^2 n^(-1) ∑_(i=1)^n▒x_i^2 )/(∑_(i=1)^n▒(x_i-x ̅ )^2 )