Which ANOVA? Or series of independent samples t-tests?

[nsyd2020]

Hello everyone,

I am looking at how native speakers (NS) and non-native speakers (NNS) use lexical repetition in argumentative essays.
I would like to compare the mean frequency and density of lexical repetition between the two groups. And also examine whether a participant's use of lexical repetition is affected by/related to an interaction between their group membership and proficiency level.

The data looks like this:

​

IV: Group
- NSs n = 18
- NNSs n = 18

Each participant wrote an argumentative essay (total 36 essays). For each essay/participant I obtained:

• Holistic measure of writing proficiency (e.g., Advanced-High, Superior), transformed into a numerical value from 1 (= least proficient) to 5 (most proficient).
• Measures of lexical repetition frequency and density
  • DV 1: How many times a participant repeats the same word across sentences, e.g. education/education
  • DV 2: How many times a participant repeats a word in different form, e.g. education/educator
  • DV 3: Density of same-word repetition (a factor of DV1)
  • DV 4: Density of modified word repetition (a factor of DV2)
  • DV 5: Ratio of same-word to modified word repetition (a factor of DV1/DV2)

As you can see the DVs are
- observations of the same participant in one sitting (not over time)
- are related

Questions:

1. Should I do a series of independent samples t-tests since the DVs are related? If yes, is there a way I could examine the effect or interaction of proficiency level?
2. Would repeated measures ANOVA be (more) appropriate even when the DVs are related?
3. Is the measure of writing proficiency a second DV?
4. Neither test is appropriate? Alternatives?

Many thanks!

Nisreen

 

[katxt]

Have you considered a one way MANOVA (multivariate anova) which tests all the variables at once and takes into account the correlation between them? Then you can sort things out afterwards with post hoc tests (t tests in this case, assuming the data is suitable). kat

 

[nsyd2020]

Have you considered a one way MANOVA (multivariate anova) which tests all the variables at once and takes into account the correlation between them? Then you can sort things out afterwards with post hoc tests (t tests in this case, assuming the data is suitable). kat

Thanks, Kat!

I read up on MANOVA and one guide says that one of the assumptions is independence of observations = no relationship between the observations in each group of the independent variable. My DVs are all related.

Also not sure if MANOVA would help me answer the questions regarding proficiency level effect. Do you think a two-away MANOVA would be more appropriate? 2 IVs (group + proficiency level) and 5 DVs?

 

[katxt]

Your DVs are related, but that's OK. I think what they mean is that participants within a group must be independent. Participant[ant 1 and Participant 2 should not be brothers, for instance, because their answers may tend to be similar. This is the same restriction as for t tests.
If you consider Proficiency level a predictor rather than a DV, then you could do a MANCOVA which allows for a covariate (a numerical variable rather than a group variable). Sort of an ANOVA with a regression thrown in.
My own choice would probably be a GLM (General Linear Model) with 5 DVs and two predictors, Group as a factor, and Proficiency as a continuous variable. This assumes that Proficiency is not assessed from the DV data. kat

 

[nsyd2020]

Excellent! Thanks for your advice.