Change of statistical significance when excluding heterogenic studies?

[Tom4534]

Hi all,

I have performed a meta-analysis and found a certain pooled effect with 95% CI that was not statistically significant (p>0.05). Consequently, I have done a sensitivity analysis for which I have excluded studies that contributed most to heterogeneity.
In order to do so I used the Leave-one-out method to determine which studies contributed most and have excluded studies so heterogeneity (I^2) would decrease to 35% or less.
However, the pooled effect remained nearly the same but the 95% CI became narrower and my result is now statistically significant (p<0.05).
I think this is because heterogeneity is linked to the standard error of each study, and when excluding studies with large standard errors the overall 95% CI becomes narrower to the point where zero is no longer included in the confidence interval making the result statistically significant.
Is my rationale correct?

Thanks in advance,

Tom

 

[fed2]

I think in these meta-analsis type tests there is a contribution both from the standard error within and between studies( ie heterogeneity). It sounds like the narrowing of the confidence interval may be linked more to the reduction in heterogeneity than the exclusion of studies with large standard error.

 

[Tom4534]

So, you mean the absolute size of the SE of each individual study and its relative size compared to the other studies (which would be the heterogeneity)?
Is there a way to find out which factor contributes more to my result becoming statistically significant?

 

[fed2]

My reason for saying that the change in the p-value was best explained by a reduction in heterogeneity is because you 'excluded studies that contributed most to heterogeneity'. So it just sort of follows. What was the I^2 before it reduced to 35%?

 

[Tom4534]

I get it now, thank you!
For each assessment, we kept leaving out studies one by one until heterogeneity dropped below 35% as we found this to be acceptable.
The example I described in my first post started with I^2 = 93%, but was reduced to 0% while the pooled estimate became statistically significant.