1. Do you have some known standards that you can compare to (i.e., items or objects with known dimensions, to a certain number of decimal places / precision)? or perhaps records of patients that needed / didn't need the treatment, and use that to see if the new equipment correctly diagnoses or correctly classifies?

2. Do you have an older piece of equipment that does the same thing that you can compare to?

With these types of comparisons, you can then set up statistical tests to see if repeatability has significantly improved (#2) with the new equip, or compare departure from the known standard (#1).

I hesitate to recommend just doing a comparison of repeatability (variance) because what if there is a measurement bias in the new equipment - the variance may be smaller, but the average may have shifted.....