Seeking basic analysis help

#1
Seeking analysis help (defect rates)

I have a question regarding an analysis of the number of defects found in products over a certain time period and whether or not a method to fix these defects is working.

The data are split into two time periods: before and after the “fix” was implemented.
The Before period consists of roughly a couple years of data. The After period is significantly less – only about 3-4 months. To make a proper assessment of the level of defects in both periods, I’m sure a proportional sample of the Before period will need to be taken. Unfortunately, I am not able to wait longer for more data to appear in the After period for a stronger analysis.

I am able to retrieve the stats for any data I choose (population mean, sample mean, etc. for any relevant data), so I don’t see how to apply statistical methods to inquire about information I already have. Furthermore, I am almost certain the data are not normally distributed so I guess a transform may be needed.

I guess, in short, my question is: how can I tell if things are actually getting better? All I can think of is to choose a random sample from both periods and compare the weighted averages of defects of both samples (just by quick glance I am inclined to say, “yes” since the average number of defects is significantly lower. Though I understand a great number of factors could be coming into play for this lower defect rate other than simply applying a "fix.") My problem is others tell me they are looking for some degree of certainty or confidence interval and to use a null/alternative hypothesis approach, but I have no idea how to use these in this type of setting. Statistics is not my trade (I studied pure mathematics in school.) I can definitely learn whatever is required but I don’t know what path to take.

I greatly appreciate any help you can provide. Please let me know if you need any additional information. Thanks!
 
Last edited:

Link

Ninja say what!?!
#2
I'd look into survival analysis or rates analyses. In essence, what you're asking is whether the rate of defects is the same now as it was before.

Sorry I can't be more detailed. However, this subject isn't something that I can explain simply in one post. Hope that helps though.