Participant allocation in RCT study

#1
When you collect baseline data during a RCT study and through using the Chi-square test to compare the characteristics of the participants in each group (control and intervention), find that the allocation of participants between groups causes a significant association (p<0.05) between certain characteristics such as gender (male/female) and income level (>20,000, etc). What could you say in regards to the appropriateness of allocation of participants in the study? Is it okay that there is a significant association between male and females between the control and intervention group? Is there something you would do that could improve the allocation into the groups?

If anyone has any insight in to this, I would really appreciate it.

Thanks in advance!

Mal
 
Last edited:

Karabiner

TS Contributor
#2
In a RANDOMIZED controlled trial it makes no sense at all to carry out statistical significance tests
to compare baseline characteristics. Any "statistically significant" group differences must by definition
be type 1 errors. Otherwise you'd have to assume that the randomization procedure was faulty.

What you really are interested in, that is whether your samples differ substantially with regard to
baseline characteristics, in such a way that their different characteristics could cause spurious
results when you compare groups with regard to the outcome variable of interest.

What could you say in regards to the appropriateness of allocation of participants in the study? Is it okay that there is a significant association between male and females between the control and intervention group?
But such a judgement can not be based on statistical tests of significance (which make statements
about populations, not about magnitude of differences between samples). If sample sizes are large,
then irrelevant sample differences could be "statistically significant". And if sample sizes are not large,
then practically relevant differences could be "not statisticaly significant".

One main problem is that there are no "objective" criteria to determine wheter an existing difference
could cause spurious results. It concerns the magnitude of the difference, as well as the influence
of the variable on outcome.

Is there something you would do that could improve the allocation into the groups?
Stratification.

With kind regards

Karabiner


References
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4310023/
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1116277/
 
Last edited:
#3
It actually gets tossed around as a critique of RCT that not enough of them (ie ~5%) have statistically significant baseline difference to support that the computer randomization was bona-fide!
 

hlsmith

Less is more. Stay pure. Stay poor.
#4
@Karabiner hit most of the big stuff. So to confirm, treatment was randomized and there are differences between background covariates? It is correct that these variables may not affect the causal relationship between exposure and outcome, or be large enough to be clinically relevant. If there are known confounders of interest block randomization can be used during treatment assignment. Post assignment options include controlling for known confounders in the statistical model. This is kind of like augmented inverse treatment weights, where you weight the background covariates but also control for them directly in the model. Of note, controlling for them in the model will result in conditional estimates not marginal estimates!