Intraobserver repeatability

#1
Hi, I'm new to this and wonder if anyone can help me out.
I'm trying to analyse some data from work. A single observer makes 4 repeated measurements for each subject to see if the measurements are repeatable. Which statistical test do I need to apply to the data? If I only had 2 measurements per subject I could use Pearson's r, but does this work for multiple measurements?
 

JohnM

TS Contributor
#2
You need to be more clear on what your objective is. Repeatability implies "low variability" so in the case of 4 measurements, you could say that they appear to be repeatable if their variability is "low."

What is the objective of your work?
 
#3
The object is to assess how good a new piece of equipment is at measuring a certain aspect of a patient's eye. These are measurements which may influence whether they need treatment or not and so we must be able to rely on the measurements.
Is it enough to calculate std dev? How low is good enough? We usually quote p values or confidence intervals.
 

JohnM

TS Contributor
#4
1. Do you have some known standards that you can compare to (i.e., items or objects with known dimensions, to a certain number of decimal places / precision)? or perhaps records of patients that needed / didn't need the treatment, and use that to see if the new equipment correctly diagnoses or correctly classifies?

2. Do you have an older piece of equipment that does the same thing that you can compare to?

With these types of comparisons, you can then set up statistical tests to see if repeatability has significantly improved (#2) with the new equip, or compare departure from the known standard (#1).

I hesitate to recommend just doing a comparison of repeatability (variance) because what if there is a measurement bias in the new equipment - the variance may be smaller, but the average may have shifted.....