Mistake in design? No control group...!

C

campsych

New Member
#1
#1
Hi all,

I am concerned that I have made an error in my design and need to present stats soon and fear looking like a fool :(

Essentially what I wanted to look at was whether a treatment type had a positive influence on clients. Unfortunately, there was no ability to have a control group due to the nature of the program and an inability to measure those who were not attending.

There are 4 DVs (measure pre and post) and the IV being 'Attended treatment'. There are a few other IVs that I have collected, though they are irrelevant for the current study.

Clients are measured on Depression, Anxiety and Stress on Intake and Discharge and there is a significant difference there. My concern is that not having a control group means that the stats are irrelevant. I am worried that there are too many confounding factors that may explain recovery making the findings redundant.

Any help on how on can report these statistics would be greatly appreciated.
 
Jake

Jake

Cookie Scientist
#2
#2
Well you at least have pre-treatment and post-treatment measurements, so that's better than nothing. It's certainly true it would be better if you had pre- and post-scores for both a treatment and a control group, but if you are able to show that the treatment at least that the treatment influences scores relative to what they they were prior, that is still something at least.
 
rogojel

rogojel

TS Contributor
#3
#3
hi,
this would be an observational study. The question is of course how the results could be generalized to larger populations, for example. If your sample was not randomly selected then the generalization would be questionable in any case, so the lack of the control group is IMHO not a crucial difficulty.

You could address the question of bias, for example by looking at all the other variables you measured and checking that the sample is heterogenous in all those respects (if that was the case) or if not, then looking at how the homogenity of the sample influences the generalization.

E.g. if al your samples are males above 50 then you could address the question whether and why older males might react more positively to the treatment , This will still not make the study equivalent to an experimant, but would add to the weight of the conclusion..

What you cannot prove though, is thatbthe effect is not placebo. For that you would need a control group I guess.

regards
rogojel
 
C

campsych

New Member
#4
#4
Hey guys,

Thanks for the feedback!

I realise that I made a crucial error. There was other data that was previously collected on the program before I collected mine, when it was a different treatment altogether. As a result, I will using that data as the Control in a quasi-experimental type approach to the study.

I have no sig differences between the two treatment types, which is fine - the old treatment was a dominant approach to the problem, the one I am studying has similar outcomes and is a new treatment. So just as effective is good in this case.

In this case what is the best way to compare the data, should I have a measure of change (ie discharge minus intake scores) which I compare treatments against each other, and then what analysis do I use?

Cheers
 
CB

CB

Super Moderator
#5
#5
campsych said:
I have no sig differences between the two treatment types, which is fine - the old treatment was a dominant approach to the problem, the one I am studying has similar outcomes and is a new treatment. So just as effective is good in this case.
Click to expand...
Be careful how you describe your findings when writing them up. No significant difference does not directly imply that the treatments are equally effective. It simply means you do not have enough evidence to reject a null hypothesis that they have the same effect. This could be because the treatments do in reality have the same effect, or it could mean that they do have different effects but you didn't have enough power to detect the difference.
 
You must log in or register to reply here.