Simple Stats for Pre vs Post Sample Comparison

#1
Hello,

I want to be sure I am using the correct statistics for pre vs post-test data. Any help is greatly appreciated. I need to know if the % correct post is statistically significant from the % correct pre and the effect size.

Example: Multiple choice question with 3 response (A,B &C). Answer B is the correct response. Here is the data that I have:
Number of responses to each answer choice in the pre-test (N=60)
A- 19
B- 24
C- 17

Number of responses to each answer choice in the post-test (N=55)
A- 5
B- 44
C- 6

The data cannot be matched from pre to post and the sample sizes is different between the two.

What is the best statistical test to compare pre vs post with the hypothesis that post-test scores will be higher? And how do I calculate an effect size? I do not have any statistical software so will need to use Excel or an online calculator.

Thank you!
Amanda
 

Miner

TS Contributor
#2
Since this appears to be homework, I will provide clues to guide you.

Hypothesis tests are usually classified as:
  • 1-sample or 2-sample. How many samples (groups, not n) do you have?
  • If they are 2-samples, they can be paired or unpaired. Is your data paired or unpaired?
Next, is your hypothesis may be 1-sided or 2-sided?
  • 1-sided means that you want to test whether your sample mean is larger (or smaller)
  • 2-sided means that you want to test whether your sample mean is different.
 
#3
Since this appears to be homework, I will provide clues to guide you.

Hypothesis tests are usually classified as:
  • 1-sample or 2-sample. How many samples (groups, not n) do you have?
  • If they are 2-samples, they can be paired or unpaired. Is your data paired or unpaired?
Next, is your hypothesis may be 1-sided or 2-sided?
  • 1-sided means that you want to test whether your sample mean is larger (or smaller)
  • 2-sided means that you want to test whether your sample mean is different.
This is not homework. I work in continuing medical education where I need to analyze the data for my activities. We use pre-tests and post-tests to measure change in learner behavior from before to after they received the education.
 

Miner

TS Contributor
#4
Pre/Post test would typically indicate the use of a Paired t-test. However, you stated that the data cannot be matched and the sample sizes are unequal. This forces you to use the 2-sample t-test. You also want to know whether the post test is higher than the pre-test, so this dictates a 1-sided test.
 

katxt

Well-Known Member
#5
I presume that you are comparing total scores, not question by question.
This looks to me like a typical 2 sided test, even if you have hypothesized that post will be higher than pre. 1 sided tests are usually reserved for special monitoring or regulatory situations and even then not all agree. kat