External Validation of Cox Model

Jyde

New Member
#1
I have two data set (training and validation) for building and validating a Cox model.

With the training data set I fitted a cox model using stepwise selection method.

The significant variables in the model were the only variables included in the validation model. Is this the right approach?

While validating the model I realized that the variables are not significant in the validation model and also the assumptions of the cox model do not hold (I checked the assumption on the validation data). Should I ignore the fact that the variables are insignificant and go ahead in making corrections for the problem with model assumptions in validation data?

Thirdly,in both training and validation data I have a variable 'treatment' with three groups. In training the groups are Standard, New drug and mixture, while in validation data the groups are Standard, New drug and X (is a treatment which is different from mixture in training data). Is it right to include this variable in both model or should I eliminate the groups that are not match; mixture from training data and X from validation data or should I work with it like that? I am not sure how this affects my analysis.

Thanks for your responses.
 

hlsmith

Less is more. Stay pure. Stay poor.
#4
Systematic bias (selection bias, measurement bias, and possibly confounding) along with chance is gonna mess with you. Yeah, you usually take the coefficients from the test set and score the validation set and see how accurate the model is in the validation set. So you wont rerun the model. I don't use PHREG regularly, so I can't remember the assumptions or if you would care too much about them given the above approach.


What assumptions seem to be broken?
 

Jyde

New Member
#5
Systematic bias (selection bias, measurement bias, and possibly confounding) along with chance is gonna mess with you. Yeah, you usually take the coefficients from the test set and score the validation set and see how accurate the model is in the validation set. So you wont rerun the model. I don't use PHREG regularly, so I can't remember the assumptions or if you would care too much about them given the above approach.


What assumptions seem to be broken?

Proportionality. Though I can stratify the affected variable but then I would loss information on it. I could introduce an interaction but then I don't want to complicate my interpretation.
 

hlsmith

Less is more. Stay pure. Stay poor.
#6
Hmm. I have not cross-validated for PHREG. You could go back and make this change to the original model as well, but then you are letting validation influence testing and it may not even effect the original model, but to take up degrees of freedom.


I think it just comes back to the differences in the datasets. This is an interesting problem, though I don't know enough to move you forward. Can you explain the differences between the datasets or anything that may make them unique. Is it possible to merge them or use a shrinkage approach. However, these may not help you out given the problem.
 

Jyde

New Member
#7
Hmm. I have not cross-validated for PHREG. You could go back and make this change to the original model as well, but then you are letting validation influence testing and it may not even effect the original model, but to take up degrees of freedom.


I think it just comes back to the differences in the datasets. This is an interesting problem, though I don't know enough to move you forward. Can you explain the differences between the datasets or anything that may make them unique. Is it possible to merge them or use a shrinkage approach. However, these may not help you out given the problem.
That was my thought (letting validation influencing testing)!

The only difference about the data set is that one of the treatment arm in validation data set contains a different treatment from we have in testing data set. Though out of curiosity I decided to remove observations from the different arms and used treatment arms that were common to both data set still it couldn't fix the problem.