only one predictor significant in multiple regression

[joshkafisha]

Hello dear forum,

I have the following problem:

I am using multiple regressions to test the influence of different factors on trust in 4 Actors, one multiple regression per person. To check which variables I include, I calculated bivariate correlations and included what is significant (p < .05). Now I have the problem with the regressions that for 3 out of 4 Actors only one of the variables (always the same one) becomes significant, the others do not. the predictors differ somewhat depending on the Actor, e.g. sex, living state, educational attainment, political orientation. The variable in question is conspiracy mindedness (was recorded with 4 items and then summarized). Since I have little experience here so far, I am not sure how to handle/report this.

I am grateful for any help and hints.
Best regards

Edit: R-squared seems relatively high for social studies with e.g. .302 for the whole model

 

[Karabiner]

I am using multiple regressions to test the influence of different factors on trust in 4 Actors, one multiple regression per person.

Four separate regressions? Why do you think this is necessary?
You could consider one analysis for all these data. But of course,
we do not know the context and the research question and the
research design.

To check which variables I include, I calculated bivariate correlations and included what is significant (p < .05).

Never do this. It distorts the subsequent analyses, since you base
your selection on (doubtful) statistical criteria, of which the regression
analysis does not "know", and can therefore not take it into account.
How many variables were there and how large is your sample size?

Now I have the problem with the regressions that for 3 out of 4 Actors only one of the variables (always the same one) becomes significant, the others do not.

I guess that something like this happening was very probable from the start.
What does "significant/not significant" mean, by the way - 0.05 versus 0.049?
Or maybe 0.0001 versus 0.99?

the predictors differ somewhat depending on the Actor, e.g. sex, living state, educational attainment, political orientation. The variable in question is conspiracy mindedness (was recorded with 4 items and then summarized). Since I have little experience here so far, I am not sure how to handle/report this.

IMO you need to perform a multilevel model. If not, then you should use the
same set of predictors in each regression. You should not select these co-variables
based on a statistical criterion, but rather based on substantial considerations
and/or reference literature.

But regardless of all of this, what is your specific problem with the pattern of
results? It is not quite clear to me.

Edit: R-squared seems relatively high for social studies with e.g. .302 for the whole model

It should be so. You preselection exploited possible chance associations and "optimized"
the results.

With kind regards

Karabiner

 

[joshkafisha]

Hey Karabiner,

thanks for your answer!

I would like to check with the help of the regression how much influence the variables have in each case on the confidence in different actors. I have correlated a total of 10 variables, of which 4-6 per actor became significant in the bivariate correlation. The sample differs slightly per actor, but is approximately N = 850 per actor. Unfortunately there is not too much literature about trust in these actors, so i thought i should use a statistical criterion.
Regarding the significance I have attached a screenshot.


All the Best
J.

 

[Karabiner]

With such a large sample size, there is no need for a pre-selection.
And your research question seems to require that you perform
the same analysis for each actor.

I still suppose that there are not 4*850 participants, but 850?

The best approach would be a multilevel regression, in my opinion,
but if you want to perform 4 separate regressions, then you
should consider the coefficients in the first place, not just the
p-values (with n=850, the standard errors are small, except if
you have multicollinearity in your model, i.e. very highly
intercorrelated predictors).

Again the question, what is the specific problem with the results?
This is not completely clear to me.

With kind regards

Karabiner

 

[joshkafisha]

With such a large sample size, there is no need for a pre-selection.
And your research question seems to require that you perform
the same analysis for each actor.

Thank you so far! I will try that later!

Again the question, what is the specific problem with the results?

What is unclear to me is that for one of the actors 3/4 predictors are significant (<.001, <.001 and .038) and for the others just conspiracy thinking, I would have expected otherwise. But maybe I'm just thinking wrong here.

The best approach would be a multilevel regression, in my opinion, but if you want to perform 4 separate regressions, then you should consider the coefficients in the first place, not just then p-values

Since I am designing the evaluations for the 4 actors separately (I am also doing a qualitative evaluation afterwards), it makes sense to calculate them separately as well, doesn't it? I am also not quite sure whether I can calculate a multilevel regression since I have no experience with this yet and would have to read in completely again.

Trust was also measured with a 5-point Likert scale (trust - partial trust - neutral - vice versa), would you stick with a multiple linear regression or rather switch to a logistic one?

Thanks a lot for your help!