Compare means from two parallel tests

#1
Hi,
My goal is to measure environmental attitude (EA). To do that I want to construct a scale and use a Rasch model to estimate item and person scores.
My study design is the following.
1. I want to measure EA as a base
2. I want to introduce a cognitive dissonance intervention (or control group)
3. I want to measure EA again to see if the dissonance intervention changed

My hypothesis is contrary to popular opinion.
I expect to not find a significant difference between the 2 conditions for the second measurement and hence additionally no change over time in neither condition (i.e. people do not chance their EA due to the cognitive dissonance intervention).

I have 2 questions:

1. Am I allowed to use the same test in measure 1 and measure 2 or do I have to create parallel forms consisting of different, but similar items?

2. How can I test this hypothesis statistically?
I know, that I can definitely compare the means of both conditions and also I can test if the means changed over time. But I am only familiar with tests that test a statistic difference but with none, that try to identify a null effect, neither between nor within subjects.

If you could help me out, I would be grateful.

Thank you

PS: If possible please share or cite any papers or books that review this topic
 
Last edited:

katxt

Active Member
#2
It sounds like you are looking for an equivalence test. You can't prove two things the exactly same. You need to decide before you start how far from equality you could get and still be happy to say that there is no significant clinical difference. Then see if the confidence interval for the measure difference lies inside that range.
Try https://en.wikipedia.org/wiki/Equivalence_test for a start. The idea may need to be adapted for a before/after/control/intervention test, and also if the data isn't suitable for t type tests.