How and by what magnitude will Multiple Imputation influence my statistical results?

#1
Dear Forum,

Hello and thank you for reading. I am a researcher based at UNSW and I have just received reviewers comments back on a manuscript that I submitted assessing the safety and efficacy of Sativex (a cannabis extract) for managing cannabis withdrawal. I have been able to address most of the comments, but this one below has just stumped me on exactly what type of answer I should give - and what the answer is!! The question relates to my participant drop out, which was significantly higher in the placebo group during the 6 days of blinded 'medication' administration. Following those 6 days, drop out was high in both groups (for three days of medication free 'wash-out'). My primary outcome is withdrawal severity, measured once daily using a validated withdrawal scale, and analysed using Mixed Model for Repeated Measures. I used Multiple Imputation, and here is the reviewers comment:

1. Of the 51 patients enrolled into the study, 24 were randomized to the placebo and 27 subjects to the active drug group and were included into the ITT analysis. Missing data were imputed using the multiple imputation method. The authors should comment on how and with what magnitude this method may influence the statistical results of their primary outcomes.


I think the answer maybe should be that you cannot answer this as we do not know what value the missing data would have taken... but.... not sure - I have had a go at it below - let me know if you think you can improve on this please :)

Dropout was higher in placebo during the medication phase and high in both groups post-medication. Missing data were imputed with MI, which would increase mean values for placebo at peak drop-out (during and post medication phase - days 5 and 6 onwards), and decrease mean values for the nabiximols group post medication, relative to complete case analysis. However parameter estimates are averaged over several plausible datasets, and MI benefits from generating more realistic standard errors than complete case or single imputation methods, giving more accurate confidence intervals and tests of statistical significance.

Many thanks for your time and any input you can give me to get over this final hurdle in my response to reviewers.

Kind regards

Dave
 

spunky

Can't make spagetti
#2
Re: How and by what magnitude will Multiple Imputation influence my statistical resul

I think the answer maybe should be that you cannot answer this as we do not know what value the missing data would have taken

well... that is not entirely true. there are statistics out there that let you measure the ammount of "information" lost in any dataset where non-response (or dropout effects, in your case) is present.

one i'm familiar with is fraction of missing information (the literature calls it \(\lambda_{j}\)) that tells you how missingness affects the estimation of the jth parameter in your model.it measures the inflation in the variance of the parameter estimate relative to what this variance would have been had all the data been observed.

i don't remember off the top of my head how to get it it because i don't usually work with missing data... BUT i know for sure that (a) it's not complicated and (b) how to calculate it is in:

Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. Hoboken, NJ: Wiley.


i believe it would be a good idea for you to maybe report your \(\lambda_{j}s\) and, if they're small, you can always say you don't have evidence to believe the results would have been all that much different, had there been no drop out (or if they're high then you have to warn readers to interpret your results cautiously)