I have a reviewer of a manuscript requesting corrections for multiple comparisons in a type of analysis that is not correlational or group comparisons and so I am uncertain whether this is appropriate. The analysis is looking at contingencies between two speakers (i.e., the probability that given speaker A says X, speaker B says Y sort of thing) at two time points. The two time points were combined in a single table for efficiency and I believe the unit of analysis would not be the individual (there were 50 pairs), but the contingency between each pair of speech acts? There were about 2500 pairs of codes at one time and 3200 at another. So my question is, IF corrections for multiple comparisons are warranted is the number of comparisons the total across the two time points or just each time individually, and IS this warranted? Any references to this issue would be much appreciated as everything I am reading focuses on correlations or group comparisons and does not address the issue of complimentary sets of analyses like this. I'm not asking for the correct test to use, I've got that, but on whether there SHOULD be a correction made and if so, how to define the "number" of comparisons that were made.
Correct for multiple comparisons with two time points and contingent relations
- Thread starter vltompkins
- Start date
- Tags multiple comparisons
Thanks for your reply. Yes, I get the basics about what the number of comparisons is and why to do it. My question is more nuanced than that. How do you determine what the number of comparisons is? I'll give a more concrete example. I have two time points--let's say I'm looking at 10 correlations at Time 1 and the same variables at Time 2. So is the number of comparisons 20 or 10 considering that the two time points are not part of the same analysis (i.e., correlation matrix)?
The number of sub-analyses is the number of p-values you generate. Does not matter where. If
1] for each of the correlations, you calculate a p-value to decide whether the correlation is significant
and
2] all these correlations are used to shed light on the same research question
sum all these cases up.
1] for each of the correlations, you calculate a p-value to decide whether the correlation is significant
and
2] all these correlations are used to shed light on the same research question
sum all these cases up.