Indepentent-samples t-test or paired?

[Erikk]

For my bachelor's thesis I've collected data of 23 persons who's been sleeping 5 consecutive weekdays with their phone outside of bedroom and 5 consecutive weekdays with the phone by their bed (in reach) and daily filling in a sleep diary (the consensus sleep diary). The participants were randomized to start with either setting (phone outside or inside bedroom). The data I'm analyzing consists of self-reported sleep quality (1-5 likerts), sleep latency, hours of sleep, time in bed spent not sleeping, number of wake-ups during night and time awake during these wake-ups. I'm using SPSS to analyze the data.

I'm unsure if I should treat every night as a case and do independent t-test with outside/by the the bed as the grouping variable? This would give me 209 nights of data after removing outliers etc. Other option is to do a paired samples t-test with every participants mean on their data points - it has to be a mean since it's an uneven distribution between the two settings after removing outliers, missed nights etc.

 

[Karabiner]

You can perform a repeated-measures analysis of variance with the
repeated-measures factor "condition" (phone vs. no phone) and "day"
(1 to 5). In addition, there's a between-subjects factor (grouping factor)
"sequence" (first with phone vs. first without phone). This
Analysis requires complete data of a participant, though. So in order
not to loose a participant for the analysis, you will have to reconsider
so-called outliers, and/or will have to impute missing values. Alternatively,
you can consider a multilevel model where observations are "nested"
within individuals. Such a model does not require complete data
from each participant.

With kind regards

Karabiner

 

[obh]

How do you do outlier on a Likert 5 scale ...? I don't think you need to remove outliers ...

Hi @Karabiner, isn't the sample size too small (23) for the Likert -5 to be considered similar to normal to do the repeated measure ANOVA?
Isn't Wilcoxon sign rank on the averages of each person better??

 

[Karabiner]

How do you do outlier on a Likert 5 scale ...? I don't think you need to remove outliers ...

Hi @Karabiner, isn't the sample size too small (23) for the Likert -5 to be considered similar to normal to do the repeated measure ANOVA?
Isn't Wilcoxon sign rank on the averages of each person better??

I supposed (or hoped) that the OP did use real multi-item Likert scales,
not just single Likert-type items as outcome measures.

By the way, whether a multi-item Likert scale is considered normal is
not a matter of sample size. Or never a matter at all. It is true that non-normality
(of residuals) can probably be ignored if sample size is large enough. But a single
Likert-item is ordinal, regardless of sample size. And therefore it cannot be analysed
using signed rank test (which requires interval scale). A repeated-measures analysis
for ordinal scales can be carried out using generalized estimating equations
(EEG). EDIT: I mean GEE

With kind regards

Karabiner

 

[Erikk]

Thanks for the responses!

Would it be wrong to do independent t-tests?

How would I go about doing a multilevel model otherwise, if you dont mind me asking?

The outliers are on the other items, such as time in bed etc. Feels wrong to use a night's data when someone has slept on 2 hours. Also a few of them added notes such as 'slept away', 'sick' etc and I want to have the data as clean as possible + only targeting healthy sleepers. Am I off here and should include everything you guys think? And a Wilcoxon sign rank - what is this?

 

[Karabiner]

Would it be wrong to do independent t-tests?

Yes. The observations of the same person with phone and without phone are dependent.
If you aggregate e.g. sleep latency across 5 days in each condition, then you have 2 values
for each person: mean latency with phone, and mean latecy without phone. If you want
to compare these values, you have to use a test for dependent samples, not independent
samples.

The outliers are on the other items, such as time in bed etc. Feels wrong to use a night's data when someone has slept on 2 hours.

If you observe 230 nights, then naturally someone will sleep very short
or very long in a few of them. Seems like valid observations to me, unless
you know that it was recorded erronously.

Also a few of them added notes such as 'slept away', 'sick' etc and I want to have the data as clean as possible + only targeting healthy sleepers.

So you make up your criteria posthoc (which is the opposite of clean),
or is there a study protocol with clear inclusion and exclusion criteria?

Apart from that, since you aggregate 5 values within each condition
for each person, then it will probably not make a huge difference.

And a Wilcoxon sign rank - what is this?

A test for dependent variables, which is "nonparametric", and therefore
has less assumptions than a t-test; the test is particularly useful in case of
small sample size (such as yours).

With kind regards

Karabiner

 

[Erikk]

Yes. The observations of the same person with phone and without phone are dependent....
Karabiner

Again, many thanks for response!

As for the nights I want to remove, it's two cases of waking in the middle of the night to follow the election and returning to sleep 2-3 hours later and 4 cases of sickness (noted by the participants) that created outlier-like night data. There are also a few simply just missed nights in both settings, so unless I remove more cases to compensate there will be incomplete data from some participants.

I want to study healthy sleepers in normal conditions, at least this was my intent collecting the data. I rejected two persons with clinical sleep problems and rejected data from two persons who stated that they were feeling unease during the experiment-weeks (for different reasons) and slept different than they usually do.

I'll look into the Wilcoxon sign rank test tomorrow! Thanks a lot again.

 

[obh]

I supposed (or hoped) that the OP did use real multi-item Likert scales,
not just single Likert-type items as outcome measures.

By the way, whether a multi-item Likert scale is considered normal is
not a matter of sample size. Or never a matter at all. It is true that non-normality
(of residuals) can probably be ignored if sample size is large enough. But a single
Likert-item is ordinal, regardless of sample size. And therefore it cannot be analysed
using signed rank test (which requires interval scale). A repeated-measures analysis
for ordinal scales can be carried out using generalized estimating equetions
(EEG).

With kind regards

Karabiner

Hi @Karabiner

Sorry, of course, you can't use the Wilcoxon sign rank, but why not the Sign test?
ps, you mean GEE?

 

[Karabiner]

Sorry, of course, you can't use the Wilcoxon sign rank, but why not the Sign test?

Yes, this works for ordinal paired data.

ps, you mean GEE?

Oops. thank you.

With kind regards

Karabiner