Comparing 2 lists of stimuli - what test should I use?

#1
Hello :)

I am looking to find out whether 2 different lists of words I am using as stimuli are significantly different from each other, in terms of their length (in letters) and their frequency (i.e. how frequently they are used in the English language).
The words were pulled from a database that lists each word alongside its length and frequency. I am not sure how to test whether the 2 lists differ as no participants provided the the length nor frequency of the words, those things just *exist*. I have done statistical tests before but they have all been used to compare data that participants have provided, and I'm at a bit of a loss as to what to do here. At first I did a t-test for both length and frequency but quickly realised I didn't actually know what test to use, so I've scrapped that lol.

There is probably a simple answer to this but I can't get my head around it, any help would be really appreciated :p Thanks for reading!!
 

Karabiner

TS Contributor
#2
Not sure whether any test of significance would be necessary or at least useful here at all.
Why exactely are you interested in the comparison of the 2 lists?

With kind regards

Karabiner
 
#3
I'm testing whether reaction time to these words is different because of a separate characteristic that is different between the two lists. So I'm trying to experimentally control for other things such as their length and frequency i.e. they should not be significantly different between the two lists. :)
 

Karabiner

TS Contributor
#4
If it is about your experimental procedure, then tests of statistical significance are useless.
Statistical significance does not tell you whether differences are "significant" in the sense of
"important" or "influential". You are not interested in the statistical null hypothesis (e.g. "in the
population of words from which my words were sampled, the mean length between lists
differs 0.0000000000..."). Rather, you are interested in whether in your actual experiment
there are marked differences which could influence results. A significant result won't tell you
whether the difference between your samples was large enough to bias the experiment.
A non-significant result , on the other hand, could be due to small sample size and would
not protect you from bias (but I don't know whether you want to compare word lists with
just a few words, or maybe very long lists). You'll have to judge whether the differences
are meaningful by using the descriptive statistics.

With kind regards

Karabiner