Gold Standard vs. New Technology

seanw

New Member
#2
25 people had their heights estimated using a New Technology. Their actual heights were then measured using a measuring tape. I want to know if the New Technology can be used to accurately estimate peoples' heights.

Just by looking at the raw data, the New Technology seems to be pretty far off, but how do I test this properly in MiniTab?

I assume the data is paired (matched) because it's the same people in both measurements, and there was only a short amount of time between the measurements being taken (so their heights couldn't have changed).

Should I do a "paired t-test" in this case? Or is correlation more appropriate?

I've been told a Mann-Whitney may be appropriate, but isn't that just for unmatched data?

I've been asked if the relationship is "linear", and to make a probability plot, but I don't see the purpose of this - is it to test if the data is "normal"?

Thank you.
Some considerations:

(1) The data do in fact seem paired (two measurements per person, and we're comparing the two measurements against each other), so you want a test that takes advantage of the pairing. Hence Mann-Whitney U would not be appropriate, since it assumes the data are unmatched.

(2) Height is a continuous variable, so you want a test of means. A matched-pairs t-test would be appropriate. You could do a Wilcoxon signed-rank test (not a Mann-Whitney U-test) but it would be uniformly less powerful than the t-test.

(3) A probability plot is used to visualize the goodness-of-fit of matched pairs of data points. So it is simply a kind of visual supplement to the matched-pairs test.
 

Miner

TS Contributor
#3
Also, consider your null hypothesis. Is the null that there is no difference and you want to prove a difference? Use a paired t-test. Or, is the null that there is a difference and you want to prove there is no difference? Use an equivalence test.
 

hlsmith

Less is more. Stay pure. Stay poor.
#4
You would use receiver operator characteristic curve. It would provide you with the accuracy of the new tech along with 95% confidence intervals.
 

seanw

New Member
#5
I've generated a probability plot for each set of data - the New Technology and the Measuring Tape - and both appear to be U-shaped and give me an AD value of about 1.5 and a P-value of >0.005. I assume this means the data is not normally distributed. Do I have to transform the data before continuing? Can I still use a paired T-test?

Meanwhile, my paired T-test suggested the New Technology COULD be used to estimate height (despite looking massively off on paper), albeit with a very large Confidence Interval.
What you described doing is a normal probability plot; I had thought you would want to do a probability plot to compare the two datasets to one another. It's only required that the difference scores be normally-distributed, and you would ideally test that with a Shapiro-Wilk test, not an A-D test.
 

Karabiner

TS Contributor
#6
My null hypothesis is that there's no significant difference.
The power to reject that hypothesis is low in your
study, due to small sample size. If you fail to reject
the null hypothesis, then you probably will still not be
able to conclude that both instruments do not differ,
or at least don't differ much.

In addition, using dependent sample t-test you just
will be able to say something about the mean
difference between paired measurements. If you
have large positive and also large negative differences
between paired measurements, the mean difference
could still be zero or nearly zero.

Just my 2pence

K.