Integrating z-scores from multiple tests

#1
Suppose I take a sample, do some measurements, and calculate a z-score which is highly significant (e.g. 5). Suppose I then repeat this process and this time my z-score is not so significant (e.g. 1.5). Suddenly I'm less confident that I can reject the null hypothesis (should I be?), so I repeat the experiment a third time and get a z-score that is barely significant (e.g. 2).

How do I integrate the results of these three experiments to generate a final z-score or p-value? Similarly, suppose my three z-scores were all 'almost' significant (e.g. 1.9) - could integrating the results of the three trials result in a rejection of the null hypothesis?
 
#2
Suppose I take a sample, do some measurements, and calculate a z-score which is highly significant (e.g. 5). Suppose I then repeat this process and this time my z-score is not so significant (e.g. 1.5). Suddenly I'm less confident that I can reject the null hypothesis (should I be?), so I repeat the experiment a third time and get a z-score that is barely significant (e.g. 2).

How do I integrate the results of these three experiments to generate a final z-score or p-value? Similarly, suppose my three z-scores were all 'almost' significant (e.g. 1.9) - could integrating the results of the three trials result in a rejection of the null hypothesis?
Couldn't you just lump the data together and then use the z-test on the entire collective?
 

JohnM

TS Contributor
#3
You could lump the data together and compute one z-score. This obviously results in a higher sample size, and in general, with all other things held constant, the higher the sample size, the more powerful the test, and therefore the more likely it is to reject the null hypothesis...
 
#4
You could lump the data together and compute one z-score. This obviously results in a higher sample size, and in general, with all other things held constant, the higher the sample size, the more powerful the test, and therefore the more likely it is to reject the null hypothesis...
The nature of the data and the experiments prevents me from lumping the results together; all I have to work with is a set of z-scores from independent trials.
 

JohnM

TS Contributor
#5
Do you have the means and standard deviations that correspond to each z-score? If so, you can:

(1) compute the overall mean and then
(2) get a good estimate of the overall standard deviation by adding up the individual variances, dividing by the number of sets and then taking the square root

If not, then I'm afraid there's not much you can do in terms of combining z-scores.
 
#6
Do you have the means and standard deviations that correspond to each z-score? If so, you can:

(1) compute the overall mean and then
(2) get a good estimate of the overall standard deviation by adding up the individual variances, dividing by the number of sets and then taking the square root

If not, then I'm afraid there's not much you can do in terms of combining z-scores.
I think this might work. I have a different N for each experiment, so I imagine I'll have to calculate a weighted average of the means and variances, right?