Missing information (adjusted residuals) from pooled multiple imputation output

[123Michael456]

I am currently working on a dataset which has had multiple imputation analysis run on it.

When I run any analysis, I have SPSS set up in such a way that my results are presented in the output window by ‘imputation block’, and at the bottom of every section (table) of analysis (whether it be frequency tables or regressions) there is always a final ‘pooled’ result, which is the section of the output that I take my results/findings from.

My problem is that I often find that this final pooled result does not include as much detail as these output tables typically include, or as much as the tables relating to the individual imputation blocks do. For example, in a frequency table the associated percentages don’t appear in the ‘pooled’ result section, as they would usually as part of any frequency analysis.

While I’ve been happy to work out the percentages so far, this issue of missing output is becoming more of an issue. Currently I’m running some chi-squares of my data, and also looking at the adjusted residuals associated with chi-square contingency tables in order to unpack the main effects contributing to each chi-square result.

Unfortunately though, the adjusted residuals are not provided in the ‘pooled ’ section.
Could anyone tell me why this type of information is missing from my pooled results, and if there is any solution to this?

Much appreciated

Michael

 

[hlsmith]

Sounds like you may need to go to the SPSS documentation for that procedures. I am surprised the percentages are not displaying, though I am not a SPSS user. Typically that is all you get, in that programs typically don't create overall counts for the MI final pooled set.


The chi-sq statistic is for the pooled analysis conducted on all of your imputes. Maybe a way to find your desired information is before running the pooled chi-sq, you tell SPSS only to use certain groups. Then you can run a bunch of chi-sqs with just 2x2 tables and surmise, which category is influencing the pooled test. Kind of a post hoc pairwise comparison. I used to do this before more recently using the standardized chi-sq residuals. This is also what individuals using Fisher's exact test would do, I believe.

 

[123Michael456]

Thank you very much for your response hlsmith.

I’ve looked into how to adjust the documentation but this is quite a complex job. SPSS has it’s own quite simplistic coding syntax that you can use to run custom types of analysis, but it doesn’t allow you to be too specific about the kind of information you want – you’re ultimately still bound by the standard functions and output that SPSS usually offers.

As for your suggestion, do you mean I would essentially manually run chi-squares for each imputation block, and then create my own final pooled result for the chi-square? If so, that would make sense to me, as dauntingly labour-intensive as it may be.

 

[hlsmith]

No sure how SPSS works in this regard. I would imagine that you impute your data, then run desired analyses on data.


So you impute your data like before, but now when you run the chi-square after your previous pooled analyses - you do it again but you tell SPSS to just use 2 groups at a time. It then runs a 2x2 chi sq and pools results, then you repeat for the next 2x2 combination, etc. Say you have 3 groups, so you ran the overall chi sq, then ran A vs B, A vs C, and B vs C. I think of it like ANOVA, which is an omnibus test just telling you that at least one group is different. Then you would run pairwise t-tests. So I am talking about doing the same thing, but with overall chi sq, then followed by pairwise chi sq tests.


Maybe I am missing something, because you are using the term "block". What do you mean by this. I had imagined you meant impute. So if I create 10 new datasets with imputed data, I typically call those imputes. So 10 imputes.

 

[123Michael456]

Yes that's how SPSS works, and this helps.

Also, yes by block I'm referring to the same thing as you call imputes.

So effectively after running my main analysis I would run several 1x1 chi-squares to identify which of the 'cells' within that 2x3 table would be significant?

Also, if you're reporting this kind of information, I assume the standard procedure is just to report all of the significant chi-squares?

Thanks again

 

[hlsmith]

Yes.


I am usually over cautious - but if you make a lot of comparisons you may want to think about correcting your level of significance (i.e., alpha), to minimize threats of false discovery.

 

[123Michael456]

Thanks again for your response hlsmith, and sorry for the delayed reply.

What you've said makes sense, I'm definitely going to try taking this approach, and yes I agree it would probably be worth correcting the significance level.

Thanks for your time and your advice, it's very much appreciated

Michael

 

[hlsmith]

No problem, I worked through a similar problem on my own about 8 months ago. So I can relate to your situation.