Basic Sample Sizing Question

[Jono_Harris]

Hi all,

I haven't done any stats for a long time, I am looking to incorporate some stats into my Ops Risk role to get a little edge. I think I am on the right track with this first little attempt but just wanted to run it by the forum for any feedback.

So at work, we are starting a new program in which we review all critical risks for divisions, we then sample test the controls for these risks to ensure they are working as expected. We currently just sample test 10% but I thought it'd be a good idea to get a statistically significant sample size when testing. The test results are a simple pass/fail.

So to get total accuracy we have to test 100% obviously. Now to get comfort that we are testing an appropriate sample size, the bank needs to agree on a margin of error and a confidence level, which will ultimately determine the sample size. Do you guys agree with this?

Also any tips/suggestions for determining a ME and confidence level when results are a simple pass/fail?

Help much appreciated.

Jono

 

[hlsmith]

Can you just calculate the SE and or confidence interval on the samples and use that? Sorry if I am not understanding your question.

 

[Jono_Harris]

Hi HL, thanks for coming back on this.

Ok lets wind it back, perhaps I was headed down the wrong path.

At work, we test controls on processes, the result is either a pass or fail. The population could be 10 or could be 1000. How do I determine a sample size that is statistically significant. I did a bit more reading and perhaps this is a function on non-parametric statistics due to the qualitative nature of the testing?

 

[hlsmith]

If you have the population size each time, you then just need to know the minium effect size (probably for you a difference in proportions and the reference proportion value) you would like to be able to disscern. You can then do a sample size calculation for proportions by plugging in a power level, alpha level, and the previously mentioned values.

The following looks as good as any:

https://www.stat.auckland.ac.nz/~wild/ChanceEnc/Ch08.psampsize.pdf