Statistical power of two dataset

#1
Hi All,


I would like to inquire how to decide whether Dataset A or Dataset B would be reliable based on their coefficient of variation, number of data points and mean value.

What kind of factors could be considered? I am not searching for the direct answer but rather to jump into the relevant part of statistics to learn more.
what is the topic in that I could learn more? is it the reliability or "power" of statistical test?

Like

Dataset A:
-Mean value : 10,6
-CV: 4%
-Number of data : 3500

or

Dataset B:
-Mean value : 9,7
-CV: 2%
-Number of data : 35


Looking forward your answers :)


Best regards,
Peter
 

Karabiner

TS Contributor
#2
What exactely do you mean by "reliable"?

Could you please provide some context? Where are these data from, are they from an empirical study? If yes, what is the topic of the study, what are the research questions? Why do you ask this question, what do you need the answer for?

With kind regards

Karabiner
 
#3
Hi Karabiner,

Thank you for your inquiry.

Ok, I think it is better to clarify the whole question with background information.

There are so called external quality control services for clinical laboratories.

It means that participating labs are getting samples - similar like patient samples from blood- with unknown concentration . All of samples has the same concentration guaranteed by the manufacturer. This is double blind service neither the lab nor the labs dont know the concentrations.

Once all labs reported their results, they get feedback about their performance with a statistical report. Performance means that how lab result deviate from the consensus value=mean of the grouped results. It is expressed with Z-score

There are 3 level of the comparisons.

1. Consensus value is coming from the mean value of all results -> less reliable as there are many measuring method and Instruments. It provides less relevant and accurate results.

2. Consensus value is coming from the mean value of the results from users who are using the same measuring method independently from manufacturer of the lab analyzer

3. Consensus value is coming from the mean value of the results who are using the same instrument and same measuring method. It is called peer Group comparison.

If there is at least 9 user from your peer the the final comparison will be based on your peer Group results like below:

1545771987404.png


It is true that peeer Group comparison can be the most accurate comparison as the CV is the lowest for peer Group. However also, it can be because of the the low number of result.

As a customer point of view, it is not obvious whether why the peer Group comparison is applied. I would say, for me it seems more reliable to have my results compared to 2500 results with CV 6% instead of 12 results with 2%CV?

I hope now my doubts are clear and why am I asked my question prior.
Feel free to ask back for further clarification:)

Thank you,
Best regards,
Peter