Indepentent-samples t-test or paired?

#1
For my bachelor's thesis I've collected data of 23 persons who's been sleeping 5 consecutive weekdays with their phone outside of bedroom and 5 consecutive weekdays with the phone by their bed (in reach) and daily filling in a sleep diary (the consensus sleep diary). The participants were randomized to start with either setting (phone outside or inside bedroom). The data I'm analyzing consists of self-reported sleep quality (1-5 likerts), sleep latency, hours of sleep, time in bed spent not sleeping, number of wake-ups during night and time awake during these wake-ups. I'm using SPSS to analyze the data.

I'm unsure if I should treat every night as a case and do independent t-test with outside/by the the bed as the grouping variable? This would give me 209 nights of data after removing outliers etc. Other option is to do a paired samples t-test with every participants mean on their data points - it has to be a mean since it's an uneven distribution between the two settings after removing outliers, missed nights etc.
 

Karabiner

TS Contributor
#2
You can perform a repeated-measures analysis of variance with the
repeated-measures factor "condition" (phone vs. no phone) and "day"
(1 to 5). In addition, there's a between-subjects factor (grouping factor)
"sequence" (first with phone vs. first without phone). This
Analysis requires complete data of a participant, though. So in order
not to loose a participant for the analysis, you will have to reconsider
so-called outliers, and/or will have to impute missing values. Alternatively,
you can consider a multilevel model where observations are "nested"
within individuals. Such a model does not require complete data
from each participant.

With kind regards

Karabiner
 

obh

Well-Known Member
#3
How do you do outlier on a Likert 5 scale ...? I don't think you need to remove outliers ...

Hi @Karabiner, isn't the sample size too small (23) for the Likert -5 to be considered similar to normal to do the repeated measure ANOVA?
Isn't Wilcoxon sign rank on the averages of each person better??
 

Karabiner

TS Contributor
#4
How do you do outlier on a Likert 5 scale ...? I don't think you need to remove outliers ...

Hi @Karabiner, isn't the sample size too small (23) for the Likert -5 to be considered similar to normal to do the repeated measure ANOVA?
Isn't Wilcoxon sign rank on the averages of each person better??
I supposed (or hoped) that the OP did use real multi-item Likert scales,
not just single Likert-type items as outcome measures.

By the way, whether a multi-item Likert scale is considered normal is
not a matter of sample size. Or never a matter at all. It is true that non-normality
(of residuals) can probably be ignored if sample size is large enough. But a single
Likert-item is ordinal, regardless of sample size. And therefore it cannot be analysed
using signed rank test (which requires interval scale). A repeated-measures analysis
for ordinal scales can be carried out using generalized estimating equations
(EEG). EDIT: I mean GEE

With kind regards

Karabiner
 
Last edited:
#5
Thanks for the responses!

Would it be wrong to do independent t-tests?

How would I go about doing a multilevel model otherwise, if you dont mind me asking?

The outliers are on the other items, such as time in bed etc. Feels wrong to use a night's data when someone has slept on 2 hours. Also a few of them added notes such as 'slept away', 'sick' etc and I want to have the data as clean as possible + only targeting healthy sleepers. Am I off here and should include everything you guys think? And a Wilcoxon sign rank - what is this?
 

Karabiner

TS Contributor
#6
Would it be wrong to do independent t-tests?
Yes. The observations of the same person with phone and without phone are dependent.
If you aggregate e.g. sleep latency across 5 days in each condition, then you have 2 values
for each person: mean latency with phone, and mean latecy without phone. If you want
to compare these values, you have to use a test for dependent samples, not independent
samples.
The outliers are on the other items, such as time in bed etc. Feels wrong to use a night's data when someone has slept on 2 hours.
If you observe 230 nights, then naturally someone will sleep very short
or very long in a few of them. Seems like valid observations to me, unless
you know that it was recorded erronously.
Also a few of them added notes such as 'slept away', 'sick' etc and I want to have the data as clean as possible + only targeting healthy sleepers.
So you make up your criteria posthoc (which is the opposite of clean),
or is there a study protocol with clear inclusion and exclusion criteria?

Apart from that, since you aggregate 5 values within each condition
for each person, then it will probably not make a huge difference.
And a Wilcoxon sign rank - what is this?
A test for dependent variables, which is "nonparametric", and therefore
has less assumptions than a t-test; the test is particularly useful in case of
small sample size (such as yours).

With kind regards

Karabiner
 
#7
Yes. The observations of the same person with phone and without phone are dependent....
Karabiner
Again, many thanks for response!

As for the nights I want to remove, it's two cases of waking in the middle of the night to follow the election and returning to sleep 2-3 hours later and 4 cases of sickness (noted by the participants) that created outlier-like night data. There are also a few simply just missed nights in both settings, so unless I remove more cases to compensate there will be incomplete data from some participants.

I want to study healthy sleepers in normal conditions, at least this was my intent collecting the data. I rejected two persons with clinical sleep problems and rejected data from two persons who stated that they were feeling unease during the experiment-weeks (for different reasons) and slept different than they usually do.

I'll look into the Wilcoxon sign rank test tomorrow! Thanks a lot again.
 

obh

Well-Known Member
#8
I supposed (or hoped) that the OP did use real multi-item Likert scales,
not just single Likert-type items as outcome measures.

By the way, whether a multi-item Likert scale is considered normal is
not a matter of sample size. Or never a matter at all. It is true that non-normality
(of residuals) can probably be ignored if sample size is large enough. But a single
Likert-item is ordinal, regardless of sample size. And therefore it cannot be analysed
using signed rank test (which requires interval scale). A repeated-measures analysis
for ordinal scales can be carried out using generalized estimating equetions
(EEG).

With kind regards

Karabiner
Hi @Karabiner

Sorry, of course, you can't use the Wilcoxon sign rank, but why not the Sign test?
ps, you mean GEE?