Standard error of multiple regression coefficients

davidbinyam

New Member
I was revising my undergraduate courses of Statistics which I took before 20 years. I was exercising how to construct ANOVA table for a regression with two and three predictiors. But I couldn't obtain the standard errors of the coefficients for a test of significance becuase there is no formula in both the books of regression I bought recently. I don't want to buy a third book which again may not contain any such formula. Can anybody who knows these formulas for the standard errors leave me a message here? Thank you!!

Dragan

Super Moderator
I was revising my undergraduate courses of Statistics which I took before 20 years. I was exercising how to construct ANOVA table for a regression with two and three predictiors. But I couldn't obtain the standard errors of the coefficients for a test of significance becuase there is no formula in both the books of regression I bought recently. I don't want to buy a third book which again may not contain any such formula. Can anybody who knows these formulas for the standard errors leave me a message here? Thank you!!

The standard error for a regression coefficients is:

Se(bj) = Sqrt [MSE / (SSXj * TOLj) ]

where MSE is the mean squares for error from the overall ANOVA summaryl, SSXj is the sum of squares for the j-th independent variable, and TOLj is the tolerance associated with the j-th independent variable.

TOLj = 1 - Rj^2, where Rj^2 is determined by regressing Xj on all the other independent variables in the model.

davidbinyam

New Member
The standard error for a regression coefficients is:

Se(bj) = Sqrt [MSE / (SSXj * TOLj) ]

where MSE is the mean squares for error from the overall ANOVA summaryl, SSXj is the sum of squares for the j-th independent variable, and TOLj is the tolerance associated with the j-th independent variable.

TOLj = 1 - Rj^2, where Rj^2 is determined by regressing Xj on all the other independent variables in the model.
Thank you! It works! I was also able to determine the coefficients and their standard error using matrix algebra. That happened to be more efficient. It is advantageous to know two ways of doing that.

alex84

New Member
Can anybody tell me what formula should be used for model with no intercept (i.e. zero constant)? (I know this model requires more attention :yup: and sanity checks).
I've tried following formula Se(bj) = Sqrt [MSE / (SSXj * TOLj) ]
but got results different from Excel and SPPS ones.

I'm searching for a 6 hours - and I see only the formula like above. But it seems in case of model with no intercept there should be other formula.

Thanks in advance for attention :wave:!

Dragan

Super Moderator
Can anybody tell me what formula should be used for model with no intercept (i.e. zero constant)? (I know this model requires more attention :yup: and sanity checks).
I've tried following formula Se(bj) = Sqrt [MSE / (SSXj * TOLj) ]
but got results different from Excel and SPPS ones.

I'm searching for a 6 hours - and I see only the formula like above. But it seems in case of model with no intercept there should be other formula.

Thanks in advance for attention :wave:!

In terms of the calculation of the MSE you're going to have to exclude the intercept term from the degrees of freedom i.e. in simple regression it would be N - 1 instead of N -2.

Also, the SSX term should be expressed as raw sums of squares formula i.e. Sum(X^2) rather than in deviation form (because the intercept is zero).

alex84

New Member
In terms of the calculation of the MSE you're going to

have to exclude the intercept term from the degrees of freedom i.e. in

simple regression it would be N - 1 instead of N -2.

Also, the SSX term should be expressed as raw sums of squares formula i.e.

Sum(X^2) rather than in deviation form (because the intercept is zero).
Thanks a lot, but I still have other value It seems the problem is that

I'm calculating TOLj and Rj2 incorrectly.

E.g. Y={1,2,3}, X={2,3,5}.

I suppose TOL1=0 since I have no more indep. variables? Is this correct?

Dragan

Super Moderator
Thanks a lot, but I still have other value It seems the problem is that

I'm calculating TOLj and Rj2 incorrectly.

E.g. Y={1,2,3}, X={2,3,5}.

I suppose TOL1=0 since I have no more indep. variables? Is this correct?

Yes. The tolerance, TOL, would be ONE.

Last edited:

alex84

New Member
Yes. The tolerance, TOL, would be zero.
Thanks again for prompt reply!

But in both cases - TOL=0 and TOL=1 I'm getting incorrect results.

For Y={1,2,3}, X={2,3,5} easily I've got
Sum(Xi^2,i=1..3) = 38
Sum(XiYi,i=1..3) = 23.
Beta=23/38=0.605263158 (exactly as SPSS 17 and Excel's ATP)
(skipped)
SSres=0.078947368
MSE=0.039473684 (exactly as SPSS 17 and Excel's ATP)

For TOL=1: SQRT(MSE / SUM_X^2/(1-0)) = 0.091970901
For TOL=0: can't be calculated

But both SPSS 17 and Excel ATP give me
Beta = 0.605263158 Beta Standard Error 0.032230128 Last edited:

Dragan

Super Moderator
Thanks again for prompt reply!

But in both cases - TOL=0 and TOL=1 I'm getting incorrect results.

For Y={1,2,3}, X={2,3,5} easily I've got
Sum(Xi^2,i=1..3) = 38
Sum(XiYi,i=1..3) = 23.
Beta=23/38=0.605263158 (exactly as SPSS 17 and Excel's ATP)
(skipped)
SSres=0.078947368
MSE=0.039473684 (exactly as SPSS 17 and Excel's ATP)

For TOL=1: SQRT(MSE / SUM_X^2/(1-0)) = 0.091970901
For TOL=0: can't be calculated

But both SPSS 17 and Excel ATP give me
Beta = 0.605263158 Beta Standard Error 0.032230128 I’m sorry I meant to say you should ignore (or disregard) the Tolerance because you only have 1 independent variable (X) in the model.

Anyway, the correct standard error (SE) is:

SE(B) = Sqrt [ MSE / Sum(X^2) ] = Sqrt [0.039473684 / 38 ] = 0.032230128

And, that should do it alex84

New Member
Oops. I've forgot about
the SSX term should be expressed as raw sums of squares formula i.e. Sum(X^2) rather than in deviation form
Thanks a lot!

alex84

New Member
Dragan,
Could you kindly suggest me formula for R2ij? I'm writing the code for my thesis and I'm wondering if there is some simpler formula, that doesn't require to make a lot of calculations. Because to get Rj^2 for each indep. variable I need to regress Xj on all the other independent variables in the model, i.e. run regression procedure N -1 times, where N is independent variables count.

Dragan

Super Moderator
Dragan,
Could you kindly suggest me formula for R2ij? I'm writing the code for my thesis and I'm wondering if there is some simpler formula, that doesn't require to make a lot of calculations. Because to get Rj^2 for each indep. variable I need to regress Xj on all the other independent variables in the model, i.e. run regression procedure N -1 times, where N is independent variables count.