Structural Equation Modelling (SEM)

#1
Hi all!

Within my master thesis I have to work with a SEM and I did a horrible mistake. My initial plan was to follow this structure:

1. Hypothesis development
2. Modelling / Operationalization / ...
3. Pretest (one-dimensionality with exploratory factor analysis)
4. Adaptation of items / factors
5. AMOS / Confirmatory factor analysis
6. Results ...
7. ...

Here's my problem: I forgot to make a pretest (step 3) and sent out the questionnaires to all participants. The survey is already finished and I got 150 responses. However, the mess is that the items don't load on the latent variables as predicted. I predicted 5 latent variables. The items only load on three.

What should I do now? Is it possible to adapt my model according to the new insights on factor loadings?

Thx in advance!
Cheers
 

noetsi

No cake for spunky
#3
Here's my problem: I forgot to make a pretest (step 3) and sent out the questionnaires to all participants. The survey is already finished and I got 150 responses. However, the mess is that the items don't load on the latent variables as predicted. I predicted 5 latent variables. The items only load on three.
How would you have done a pretest of the participants if you did not send out a questionaire? What were you pretesting for? I have not seen pretest/post test done much in SEM - where commonly your are looking at pattern of covariance among variables (and sometimes factors) not if something changed. Could you explain why you needed to do a pretest?

If the reason you did a pretest was to develop some theory to test later (or a pilot run), one alternative would be to find something in the literature that justified doing what you did. But as noted above, you need to speak to your advisor.

The items not loading on factors seems to be an entirely different issue than whether you did a pre and post test. It is not uncommon to find your hypothesis of what the factors are to be wrong and to respecify your model to show a new one. So finding that is not a problem, it is an oportunity to revise your research.
 
#4
Thanks for the fast replies! :)

Initially the plan was to conduct some confirmatory factor analysis. Before this, however, I wanted to test for one-dimensionality with the help of exploratory factor analysis.

The main question for me is now, whether I have to continue with my theoretical construct (and the weak factor loadings) or if I should rather try to find a new model before going futher in analyzing...

Concerning the pretest:
I wanted to conduct a pretest in order to check factor loadings in advance before sending out to all paticipants. If this pretest would have shown me that items do not load according to my theory I would have tried to adapt the theory and sent out a new questionnaire.

Thx & cheers,
GabrielBox
 

noetsi

No cake for spunky
#5
I am not sure what you mean by one dimensionality. That there was one latent factor?

If your model does not work well you should definitely work on generating a new model. There are a series of fit parameter (they vary with the specific software) that tell you how well you model fits the observed data. You should use these to determine if model fit improves (if one model is nested in another an alternative is a chi square difference test. I think you can compare with AIC as well for non-nested models).

Some CFA software, not sure about Amos, will make suggestions about ways to improve the model such as the MI values in M plus. Those are worth looking at although they should only be used if they make theoretical sense.

I wanted to conduct a pretest in order to check factor loadings in advance before sending out to all paticipants. If this pretest would have shown me that items do not load according to my theory I would have tried to adapt the theory and sent out a new questionnaire.
Why not just do this with the questionaires you got back? I am not an expert in SEM (I simply took classes in it) so doing a pilot may be common, but I dont think it is required to do what you suggested. One way to address this is to look at what is done in the literature (another way is to talk to your advisor assuming they work in SEM (not all advisors work with what they advice on unfortunately - my supervisory professor did not).

Regardless of your original model if the results of the new survey had generated indications your model did not fit well you would revise it.