Is Chi square test and binary logistic regression are not reliable due to small expected number?

[NACIRA]

I received a critic from a reviewer stating that the the Chi-square test for some variables is not reliable due to either small expected number in the cells or due to some cell have zero and they should not be entered into binary logistic regression model. And the results obtained from the final model of the logistic regression are not correct.
To my knwoledge, the Chi square test is corrected by Fisher’s exact test when the numbers are too smal or very unequally distributed among the cells of the table.
In addition, the variables whose p-value <0.25 as well as variables biologically plausible should be subjected to multivariable analysis (binary logistic regression) and the goodness of fit of the binary logistic regression model should be assessed by Hosmer and Lemeshow test.
Is the reviewer's argument correct? and if so how?

 

[hlsmith]

The tradition was to USE Fisher's exact test when cell expectancies are small (e.g., <5), though recently people have pointed out that this was never really tested. However, it can't hurt to just use Fisher's test.

When there is sparsity in data (small counts in subgroups), it is preferred to also use exact logistic regression or logistic regression with Firth's correction or Bayesian logistic to ensure the model runs/converges.

You shouldn't base variable inclusion on p<0.25 from bivariate analyses, but based on background context knowledge.

Of note, we have no idea what you presented in your submission or what type of methods and sample sizes you had. So given what you submitted, the reviewer may be right on topic. If you want to read up on these concepts, Frank Harrel has quite a bit publish in the area.

 

[hlsmith]

The tradition was to USE Fisher's exact test when cell expectancies are small (e.g., <5), though recently people have pointed out that this was never really tested. However, it can't hurt to just use Fisher's test.

When there is sparsity in data (small counts in subgroups), it is preferred to also use exact logistic regression or logistic regression with Firth's correction or Bayesian logistic to ensure the model runs/converges.

You shouldn't base variable inclusion on p<0.25 from bivariate analyses, but based on background context knowledge.

Of note, we have no idea what you presented in your submission or what type of methods and sample sizes you had. So given what you submitted, the reviewer may be right on topic. If you want to read up on these concepts, Frank Harrel has quite a bit publish in the area.