Predictors insignificant & F-stat is highly significant!!

#1
I am investigating if levels of cultural index effect the relationship between two variables. For this purpose I have made two groups (high vs low). Data is from 13 countries with more than 1000 observations.
Also, there is no multicollinearity and have controlled for hetero, auto and cross sectional dependence through Driscoll and Kraay SEs. Due to endogenous focal variable, 2sls is used.

Group low on index has few variables including the focal one, significant. However, group with high cultural index has significant F-stat but all variables in the model are highly insignificant. Although my hypothesis is supported but I am worried about the statistical side of this model behaviour.

What’s the solution???
 

Karabiner

TS Contributor
#2
I am investigating if levels of cultural index effect the relationship between two variables. For this purpose I have made two groups (high vs low).
Ususally you should not make 2 groups, since it is regardesd as artifical, logically unsound,
usually unnecessary and often harmful. Why don't you just use the original variable?

Group low on index has few variables including the focal one, significant. However, group with high cultural index has significant F-stat but all variables in the model are highly insignificant.
Does that mean that you performed 2 separate analyses in order to investigate your
hypothesis, one for each artifical group? That would not be the common strategy.
If you want to investigate the effect of variable z on the relationship between variables
x and y, then probably you assume a moderator effect of z.
This woud be modeled by y = x + z + x*z .

With kind regards

Karabiner
 

Karabiner

TS Contributor
#3
I am investigating if levels of cultural index effect the relationship between two variables. For this purpose I have made two groups (high vs low).
Ususally you should not make 2 groups, since it is regardesd as artifical, logically unsound,
usually unnecessary and often harmful. Why don't you just use the original variable?

Group low on index has few variables including the focal one, significant. However, group with high cultural index has significant F-stat but all variables in the model are highly insignificant.
Does that mean that you performed 2 separate analyses in order to investigate your
hypothesis, one for each artifical group? That would not be the common strategy.
If you want to investigate the effect of variable z on the relationship between variables
x and y, then probably you assume a moderator effect of z.
This would be modeled by y = x + z + x*z .

With kind regards

Karabiner
 
#4
Ususally you should not make 2 groups, since it is regardesd as artifical, logically unsound,
usually unnecessary and often harmful. Why don't you just use the original variable?


Does that mean that you performed 2 separate analyses in order to investigate your
hypothesis, one for each artifical group? That would not be the common strategy.
If you want to investigate the effect of variable z on the relationship between variables
x and y, then probably you assume a moderator effect of z.
This would be modeled by y = x + z + x*z .

With kind regards

Karabiner
Many thanks for your detailed reply.

Moderation Analysis:

Initially, I have applied moderation (interaction effect) analysis. But, no relationship was observed.
I went back to literature and saw groups are also made frequently (since culture e.g. power distance, a dimension of national culture, is a relative index, it’s either high, low or medium for any specific country with respect to others).
Moreover, I applied chow test to confirm if difference among groups exist for this variable or I have to pool it (like ur suggesting).
Chow test shows making groups is more appropriate. That’s how I came to conclude that I have to make groups.
 
#5
Ususally you should not make 2 groups, since it is regardesd as artifical, logically unsound,
usually unnecessary and often harmful. Why don't you just use the original variable?


Does that mean that you performed 2 separate analyses in order to investigate your
hypothesis, one for each artifical group? That would not be the common strategy.
If you want to investigate the effect of variable z on the relationship between variables
x and y, then probably you assume a moderator effect of z.
This would be modeled by y = x + z + x*z .

With kind regards

Karabiner
2sls:
After making groups based on the mean values of a specific national culture dimension (e.g. power distance). I have used by sort command in stata and have run 2SLS (since I have endogeneity issue in the model). It showed, that relationship is stronger in low level and no relationship seen in high level ( but all control variables were insignificant with F-Stat sig).

What are your thoughts??
 

Karabiner

TS Contributor
#6
My thought is that if you want to know whether there is a moderation effect,
then you have to test it directly. Artificially forming 2 groups, performing the
same analysis in each group, and then comparing p-values is inappropriate,
as far as I know. The difference between "significant" and "not significant"
is not itself significant. http://www.stat.columbia.edu/~gelman/research/published/signif4.pdf

As to your initial question why there is a statistically significant F for the
regression model, but no regression weight is statistically significant:
F test statistic, regression weights with their standard errrors, and exact
p-values would be useful to know. The sample size for this analysis was
about 500, I supose?

With kind regards

Karabiner
 
#7
My thought is that if you want to know whether there is a moderation effect,
then you have to test it directly. Artificially forming 2 groups, performing the
same analysis in each group, and then comparing p-values is inappropriate,
as far as I know. The difference between "significant" and "not significant"
is not itself significant. http://www.stat.columbia.edu/~gelman/research/published/signif4.pdf

As to your initial question why there is a statistically significant F for the
regression model, but no regression weight is statistically significant:
F test statistic, regression weights with their standard errrors, and exact
p-values would be useful to know. The sample size for this analysis was
about 500, I supose?

With kind regards

Karabiner
Many thanks again.

Sample size is 1144.
 

noetsi

Fortran must die
#10
I can't imagine why no predictor would be significant and the model is other than Multicolinearity. Sometimes when running contrast and running a model power is different, but I don't know if that applies to t test versus the model f test.