simple regression question (I think)

#1
Hi there,

Suppose I have the simple libear regression model:

Y=a0+a1*X+ε where ε~N(0,σ^2)

Suppose that via least squares I estimate a0 (a0_hat) and a1 (a1_hat).

Can anyone tell me what is the asymptotic distribution of the quantity:

sum((y(i)-a0_hat-a1_hat*x(i))^2) ??? (1)

What I am saying is that if I make a code that generates data from a simple model like this, then estimate the parameters, then plug them back in (1), repeat the procedure 10000 and make a histogramm.

1. Generate data from a simple model like the above.

2. Estimate a0 and a1

3. Calculate value of formula (1).

4. repeat the procedure say 10000 times and make the histogramm.

It is Normal I think but what are the parameters? How can I derive this without the use of a computer (mathematically)?


Thanks in advance!
 

vinux

Dark Knight
#2
I guess you have misconception about asymptotic distribution.

asymptotic distribution means :
suppose Xn is a random variable( in terms of n). it may have some unknown distribution.
and as n is large, the distribution of Xn approaching a known distribution( say normal) very closely. Then we say Xn follows asymptotically normal distribution.
( CLT wil be the best example).
 
#3
I totally agree with you about the asymptotic distribution. And that was exactly what I was asking about the formula (1).

For each iteration of the algorithm I wrote I get one value of the formula. For one iteration I have one value (n=1). For a second iteration I have two (n=2) and so on.

So I was wondering what is the distribution of this quantity as n grows larger (i.e. what is the asymptotic distribution)


By the way since y-a0_hat-a1_hat*x is normally distributed, then the distribution I was asking for is X^2 with n degrees of freedom.

I don't know if I'm wrong in the above. If I am please correct me, you would be very helpful!

Thanks vinux!
 

vinux

Dark Knight
#4
For each iteration of the algorithm I wrote I get one value of the formula. For one iteration I have one value (n=1). For a second iteration I have two (n=2) and so on.

So I was wondering what is the distribution of this quantity as n grows larger (i.e. what is the asymptotic distribution)


By the way since y-a0_hat-a1_hat*x is normally distributed, then the distribution I was asking for is X^2 with n degrees of freedom.

I don't know if I'm wrong in the above. If I am please correct me, you would be very helpful!
Thanks vinux!
Here n indicates the iteration. And
The expression sum((y(i)-a0_hat-a1_hat*x(i))^2) doesn't contain any term of n.

and you are true on this part

y-a0_hat-a1_hat*x ( ε) is normally distributed, then the distribution I was asking for is X^2 with n degrees of freedom.


but it not related to any asymptotic stuff.
 
#5
In the formula i=1,...,n. Now doesn't the asymptotic distribution make sense?

I don't mean to tire you vinux, just trying to be sure I understand everything right!

Thanks again, I appreciate it!
 

vinux

Dark Knight
#6
You must be looking the value of sum((y(i)-a0_hat-a1_hat*x(i))^2) when n =1,2,3...1000(say)

It will be same as 1000 chisquare (with df sample size , not n) random numbers.