My approach was to conduct 2 way MANCOVA with the two IVs, covariate on the 8 DVs. However, a reviewer questioned the problem of multiple t-tests and wondered about the simpler and more valid method.

Could anyone help me on this issue?

- Thread starter joonha35
- Start date
- Tags group differences t-tests

My approach was to conduct 2 way MANCOVA with the two IVs, covariate on the 8 DVs. However, a reviewer questioned the problem of multiple t-tests and wondered about the simpler and more valid method.

Could anyone help me on this issue?

Is my approach wrong? The reviewer says it just "feels" that this was not good and asked me to get advice from an expert. If you want, I'm happy to send the manuscript and comments. Thanks!!

Also age was effective and so for the interaction between culture and collection method on a few DVs.

composed from 8 variables. What else was analysed as dependent variables?

But in order to test the hypothesis

I suppose the hypothesis was not introducedvafter you had analysed the data?

With kind regards

Karabiner

As I wrote in the first posting, there were 8 DVs I wanted to check "country" (my interest) and "collection method" (not my interest) effects on each. But the MANCOVA test found "country" was not significant overall whereas "collection method" was significant for some of the variables. There were also significant interaction effects between "age" and "collection method" on some variables. "Age" was not independent from "collection method" in the study design (average age was higher in online sample than classroom sample and age variance was greater in online sample too). Anyway so in order to test my hypothesis that the mean value of a target DV would be different between countries, I conducted separate sets of independent sample t-tests by collection method (between classroom samples and between online samples). The hypothesis was supported in classroom samples only.

And the hypothesis is introduced beforehand in Introduction.

Can you understand what I did? Don't you mind if I sent the materials to you in case? I think that'd be easier to understand. Thanks!!

As I wrote in the first posting, there were 8 DVs I wanted to check "country" (my interest) and "collection method" (not my interest) effects on each. But the MANCOVA test found "country" was not significant overall whereas "collection method" was significant for some of the variables.

With kind regards

Karabiner

With kind regards

Karabiner

thanks for the comment. 4 out of the 8 variables can be said to represent a hypothetical construct (i.e., individual self, collective self and two types of relational self) whereas the other four variables are different constructs. In this case, may I conduct MANCOVA with only 4 DVs as a single construct?

As far as I know and from what I have learned, MANOVA is used if you have a set of dependent variables which jointly represent a hypothetical construct.

However, take for example, the DVs of body weight, mean arterial pressure, and LDL cholesterol measured on each patient as a function of some treatment. There is a clear rationale to support a relationship between these three DVs, so a MANOVA may be an appropriate test early on. However, if there is sufficient evidence of a difference in the mean vectors across treatment groups, you can use post-hoc type analyses to see where the differences exist between treatment groups-- ANCOVAs are often applicable, after the significant MANOVA, in this case to still account for the underlying covariance structure of the 3 DVs. If you are investigating body weight in one ANCOVA, you'd also include the mean arterial pressure and LDL cholesterol variables on the RHS as covariates, while still looking at the treatment as your independent variable of interest.

This all is highly dependent on your research questions, of course.

Hope this helps.

Thank you Karabiner,

it is very helpful. Maybe I could consider the four different self variables (variables A, B, C, D) as a single construct? To rephrase my hypothesis, it was that the degrees of variable C would be different between two countries.

Following your suggestion, I first inserted only "collection method" and "country" as IVs and the four self variables as DVs (a single construct) without "Age" covariate. It was found that "collection method" and "collection method x country" but not "country" were significant (I mainly read Wilk's Lambda throughout the result tables although it is not so different from other tests). Then I conducted MANCOVA including "Age" as a covariate and this time found that only "collection method x country" was significant. "Age" could have done some role here but the single effect itself was not significant.

Did I do right so far?

it is very helpful. Maybe I could consider the four different self variables (variables A, B, C, D) as a single construct? To rephrase my hypothesis, it was that the degrees of variable C would be different between two countries.

Following your suggestion, I first inserted only "collection method" and "country" as IVs and the four self variables as DVs (a single construct) without "Age" covariate. It was found that "collection method" and "collection method x country" but not "country" were significant (I mainly read Wilk's Lambda throughout the result tables although it is not so different from other tests). Then I conducted MANCOVA including "Age" as a covariate and this time found that only "collection method x country" was significant. "Age" could have done some role here but the single effect itself was not significant.

Did I do right so far?

Last edited:

Could you please have a look if this approach would be valid? And in this case what should I do further? separate ANCOVA in each collection method?

A 3-way (self, country, and collection method) repeated measures ANCOVA was conducted with age as a covariance. Mauchly’s test x2(5) =136.62, p < .001 indicated that sphericity was not assumed. The following results are based on Huynh-Feldt, as Greenhouse-Geisser Epsilon was .81, which was above the criterion value (.75, Field, 2013; Howell, 2002). There was a self main effect, F(2.5, 1221)=12.59, p < .001 and interaction effect between self and age, F(2.5, 1221)= 12.83, p < .001, indicating that dominant self-construals varied across age. Interaction between self and country was not significant, F(2.5, 1221) = .1.5, p = .22, indicating that there was no significant difference between Japanese and Korean groups in ratings on different selves. However, there was a marginal interaction effect between self and collection method, F(2.5, 1221) = .2.17, p = .10 and also a three-way interaction between self, country, and collection method, F(2.5, 1221) = .2.63, p = .06.

Field, A. (2013). Discovering Statistics with IBM SPSS Newbury Park, CA: Sage.

Howell, D.C. (2002). Statistical Methods for Psychology (5th ed.). Pacific Grove CA: Duxbury.

Field, A. (2013). Discovering Statistics with IBM SPSS Newbury Park, CA: Sage.

Howell, D.C. (2002). Statistical Methods for Psychology (5th ed.). Pacific Grove CA: Duxbury.