F-Statistic Question

#1
I'm sorry for probably asking a foolish question, but my searches have yielded very limited results (as well as my reading) in regards to the F-Statistic. Is there another term for this? I'm really just trying to understand how this value relates to the rest of the regression analysis in general. Can anyone point me in the right direction?

:confused: :confused:
 

Dragan

Super Moderator
#2
I'm sorry for probably asking a foolish question, but my searches have yielded very limited results (as well as my reading) in regards to the F-Statistic. Is there another term for this? I'm really just trying to understand how this value relates to the rest of the regression analysis in general. Can anyone point me in the right direction?

:confused: :confused:

I am going to assume you are referring to the F-statistic associated with the ANOVA summary for the full regression model.

With that in mind, this F-statistic tests the following null hypothesis:

H[0]: Beta1 = Beta2 = ... = BetaK = 0.

That is, the population parameters associated with the independent variables are all simultaneously equal to zero. In words, under H[0], it's states that you will get no help in predicting Y (the dependent variable) from the independent variables and that the best predictor of Y is the mean of Y (YBar).

That said, if you're referring to some other F-Statistic, then you're going to have be more specific and explain what statistic you're referring to (e.g. an F statistic associated with a change in R^2, for example).
 
#3
Hmm... Thanks Dragon for the insight. I suppose it would also help for me to post the regression results here:

Residual standard error: 365.4 on 18 degrees of freedom
Multiple R-squared: 0.9933, Adjusted R-squared: 0.9914
F-statistic: 531.2 on 5 and 18 DF, p-value: < 2.2e-16

This is the F-statistic I am looking for. I believe it needs to show a linear relationship between the y and x variables. If it doesn’t then the model is not appropriate for multiple regression. Or at least that is what I have read. Any insight into this?
 

Dragan

Super Moderator
#4
Hmm... Thanks Dragon for the insight. I suppose it would also help for me to post the regression results here:

Residual standard error: 365.4 on 18 degrees of freedom
Multiple R-squared: 0.9933, Adjusted R-squared: 0.9914
F-statistic: 531.2 on 5 and 18 DF, p-value: < 2.2e-16

This is the F-statistic I am looking for. I believe it needs to show a linear relationship between the y and x variables. If it doesn’t then the model is not appropriate for multiple regression. Or at least that is what I have read. Any insight into this?


What this tells me is that you're running a regression with 5 independent variables with N=24 data points.

Your sample size is too small! With this sample size you should not have any more than 1 or 2 (max) independent variables. Or, is this a homework assignment???

Next, the important statistics are the t-statistics associated with each of the regression coefficients. Look at the p-values for each. In so doing, try and reduce your I.V.'s.

Note: The overall F-statistic is of little use here. It's essentially demonstrating that the R^2 is signicantly greater than zero.
 
#5
What this tells me is that you're running a regression with 5 independent variables with N=24 data points.

Your sample size is too small! With this sample size you should not have any more than 1 or 2 (max) independent variables. Or, is this a homework assignment???

Next, the important statistics are the t-statistics associated with each of the regression coefficients. Look at the p-values for each. In so doing, try and reduce your I.V.'s.

Note: The overall F-statistic is of little use here. It's essentially demonstrating that the R^2 is signicantly greater than zero.
Unfortunately, no. It's not homework, but I just pulled a small amount of data together for an example of the F-statistic. Right now, you are talking a little over my head with some of that stuff since I am learning this as I go in order to understand how to show that certain coefficients are not (hopefully) predictors of salary for work.

What I'm most interested in here is what I need to be concerned about with the F-Statistic when running regression analyses for the future. I feel like this is something I need to be aware of so I don't make a poor model that is not actually showing what my goals are, and I'm concerned about what I don't know that I don't know.... Thanks.
 

Dr.D

New Member
#6
Unfortunately, no. It's not homework, but I just pulled a small amount of data together for an example of the F-statistic. Right now, you are talking a little over my head with some of that stuff since I am learning this as I go in order to understand how to show that certain coefficients are not (hopefully) predictors of salary for work.

What I'm most interested in here is what I need to be concerned about with the F-Statistic when running regression analyses for the future. I feel like this is something I need to be aware of so I don't make a poor model that is not actually showing what my goals are, and I'm concerned about what I don't know that I don't know.... Thanks.

Hi

The F-statistic must be interpreted with its p-value. The F-statistics tells us whether the overall regression (all the independent variables combined in the model) is statistically significant (there is a significant joint relationship). If the p-value is less than .05, as in your case, it supports this. After that you must look at the individual regression coefficients (their corresponding p-value) of each predictor variable to determine which variable is statistically significant. Although the F-statistic is significant, it doesn't mean that all variables would be significant - it just measures the joint effect of those variables. It is really saying that at least one of those predictors are significant in the model; so you have to determine individual significant effects, after inspecting the F-statistic. If the F-statistic is non-significant ( p > .05), you just conclude that the overall model is not significant - and there is no relationship whatsover.
 
#7
Dr. D, you are officially my hero! Thanks for that great explanation on the F-Statistic. It's exactly what I was looking for. Thanks again to everyone who contributed, I have certainly learned a lot!
 
Last edited: