Question About a Seemingly Invalid Sampling

#1
Hi, I was wondering if someone can help me with the following question:

Suppose that we were interested in the effects of a prep course on SAT Math scores. Would it be appropriate to employ a research design where we took a random sample of 25 students already enrolled in the course? Why or why not?

My answer is no, but I'm not sure if my line of reasoning is correct. For one, the 25 students who are already in the course may be systematically different those that had never taken the course. This seems like a good reason, but my initial line of thinking was that to conduct a research design, you need to compare multiple groups.

But if the characteristic that you think will have a causal effect on the dependent variable (SAT Math) is shared by the 25 students, then it seems like you are "controlling" for the variable. In other words, assuming that these 25 students make up the sample under study, any additional variables that differentiate among the participants are useless because we are keeping constant the one variable of especial interest (whether or not the course was taken). But am I missing the point in that perhaps the researchers are comparing the results of this to another measure such as national SAT average? With this additional assumption, would the research method be deemed valid?

Any comments or feedback would be helpful. Thank you in advance.
 

Masteras

TS Contributor
#2
take also another sample of students not enrolled in the course and see for both samples their SAT score and then compare these two with t-test. or even better gather some more info from all students and see how the rest info relates to their scores. multiple regression