How should I statistically analyse this study?


I'm a 3rd year medical student studying in Sydney, Australia. I am currently working at one of the hospitals and am conducting a patient study with a number of doctors here. We have finished the study and collected all the data and are now trying to analyse it in the most appropriate way. However we are having difficulty in working out what method of statistical analysis to use.
The study looks at the predictive value of two clinical tools in predicting if a chronic wound will heal. For each clinical tool used in assessing a patients wound (tools A and B), the medical physician predicted on a scale of 1-4 how well the wound will heal. After treatment each wound was re-evaluated and again categorised on the 1-4 scale as to how well they actually did heal.
In order to work out which clinical tool (A or B) was better at predicting the fate of the wound, we must look at what each tool predicted on the scale of 1-4, compared to the outcome that was reached ( on the same scale of 1-4).

This is the scale used to classify the wounds:
1. Healed
2. Significantly improved
3. Modestly improved
4. Not improved or worse

For example,

Using tool A, the doctor predicted that following treatment the wound would be a 1 (healed).
Using tool B, the doctor predicted that following treatment the wound would only be a 2 (significantly improved).

We need to know which of the two tools is the better predictor.
Following treatment the patient is re-assessed and is found to be a 2 (significantly improved). Therefore, you could write the results as follows:

Tool A Tool B Outcome
1. 2 (-1) 2 (-1)

So in the results, we are comparing results of tool B and the outcome, to Tool A (this explains why there is a -1 in brackets, as the prediction of tool B and the outcome of the wound were one category worse than what tool A predicted.)

We have 60 patients in the study, all of whom have results like the example I have supplied.

I don’t know exactly how to analyse the data, as this is quite a complex study. I am asking for your help in analysing this data in order to determine which of the two clinical tools was more successful in predicting the outcome.

I'm sorry for the length of this post, but it is complicated and I need to explain everything in detail. If you are confused or need more information, please ask me!!

I would appreciate any help at all!!!!!


TS Contributor
Would it be useful to just compare the rates at which each tool "correctly" predicted the patient outcome? Here you could just do a z-test to see if one percentage is significantly higher than the other.

How about using a "delta" or difference score on your scale, using "predicted minus outcome" (if it predicted a 2 and the outcome was 2, then the delta would be 0; if it predicted a 1 but the outcome was a 2, then the delta is a -1) and comparing the distributions of the delta scores for each tool. Comparing the percentages of scores that fall into a particular delta category could give you insight to the predictive characteristics of each tool (i.e., does one tool tend to give overly optimistic or pessimistic predictions)

You might also consider which tool has an average delta score that is closest to 0, or one that has the smaller variability (but maybe having a consistent, predictable bias that can be easily adjusted/shifted).

Maybe someone else reading this with more experience in predicting patient outcomes or comparing therapies / tools / methods will chime in.