Methods and Data Analysis for Experimental Study Design

CupcakeKat · Jun 2, 2016

Good day! I am working on my methods section for my dissertation proposal and want to ensure I am thinking through my methods and data analysis correctly and using the appropriate statistical tests. I plan to collect data this fall.

Thank you!

Purpose Statement
This purpose of this research is to assess the effect of a text message based educational intervention aimed at improving sleep quality and sleep hygiene behaviors in freshman undergraduate college students

Research Questions
1. What effect do educational text messages have on sleep quality in freshman undergraduate college students as measured by the Pittsburgh Sleep Quality Index (PSQI) (Buysse, et al., 1989)?
2. What effect do educational text messages have on sleep hygiene behaviors in freshman undergraduate college students as measured by the Sleep Hygiene Index (SHI) (Mastin, et al., 2006)?

Hypothesis #1
The sleep quality of freshman undergraduate college students who will participate in the text message educational intervention will be better than the sleep quality of college students who will participate in the placebo or control group.
Null Hypothesis #1
There will be no significant difference in sleep quality of freshman undergraduate college students who will participate in the text message educational intervention and those who will participate in the placebo or control group.
Hypothesis #2
The sleep hygiene behaviors of freshman undergraduate college students who will participate in the text message educational intervention will be better than the sleep hygiene behaviors of college students who will participate in the placebo or control group.
Null Hypothesis #2
There will be no significant difference in sleep hygiene behaviors of freshman undergraduate college students who will participate in the text message educational intervention and those who will participate in the placebo or control group.

3 group pre-posttest experimental design
Participants will be randomized into the three groups. Participants will take the same pre and posttests, experiment will last 6 weeks. Both groups receiving text messages will get them at the same time, frequency, dose, etc., just different text content. Participants will also take pre and posttest on sleep knowledge. I think the sleep knowledge component would act as a mediator?

IV: Experimental (receives sleep texts), Placebo (receives neutral texts), Control group (receives no texts)
Covariates: gender (male/female), race (white/nonwhite) Need to control for gender and race
DV: sleep quality (continuous global score with higher scores meaning poorer sleep quality, cut off score >5 often indicates poor sleep quality
DV: sleep hygiene behavior (continuous global score with higher scores equaling more maladaptive sleep behaviors)
Mediator: sleep knowledge (researcher created survey based on sleep text messages to assess what they know about sleep at pre and posttest; unsure if this is really a mediator; behavior rarely changes without new knowledge, but new knowledge doesn't guarantee behavior change)

I believe I need to do ANOVA's to test for between group differences (experimental, placebo and control) at pretest for the 2 DV's and basic demographic variables (age, gender, race, etc) to ensure randomization was effective and there are no significant group differences at pretest.

After posttest data is collected, I believe I need to run a Repeated Measures MANOVA.

Does this data analysis make sense for my design? Thank you! I appreciate your time to read my lengthy post!

CupcakeKat

Karabiner · Jun 3, 2016

There is a gap between the intervention and sleep quality, as it seems.
Wouldn't a mediator model perhaps be more adequate (education -> behaviour -> quality)?
To test the effect on behaviour would be a realistic goal, as it seems. Or can relevant changes
in sleep quality be expected here within 6 weeks? How large will your sample be?

Freshman undergraduate college students aren't your target group.
Freshman undergraduate college students with poor sleep quality (or rather,
those with poor behaviour) could be a target group. Why should one carry out an
intervention on people who don't need one? You cannot observe improvements for
them.

Something like: education -> knowledge -> behaviour -> sleep quality.

Why? You have a randomized design. No effect of both variables will be expected
on results, if randomization works well. You can stratify by gender and race during
the randomization process.

This makes no sense. If you have randomized allocation to groups, all
"statistically significant" differences MUST be false-positive results!
As mentioned before, if you want to ensure that gender and race
are distributed equally between groups, you can use stratification.

Maybe.

With kind regards

K.

CupcakeKat · Jun 3, 2016

Thank you K. I think I understand your points and appreciate your time and responses.

Other studies have shown statistically significant improvements and clinical improvements in sleep quality within 6 weeks, which is why I chose that time frame.

education -> knowledge -> behavior -> quality makes a lot of sense to me. I guess in my head I believe that a change in behavior would lead to a change in quality which is why I wanted to measure both.

Previous studies suggest over 70% of college students have poor sleep quality but fewer studies have looked at sleep behaviors, though they suggest they have little sleep knowledge and poor behaviors.

I mentioned race and gender as previous research suggest females and whites consistently report poorer sleep quality compared to their counterparts.

I see your point about the redundancy of randomization and checking pretest group differences.

Thank you again for your time.

Methods and Data Analysis for Experimental Study Design

CupcakeKat

New Member

Karabiner

TS Contributor

CupcakeKat

New Member