software speed

M

MikeMulligan

New Member
#1
#1
Not homework BTW ... I want to measure the processing time (performance) of some software under a set of fixed conditions. I have a means to log the start and stop of the processing. The only possible factors are some uncontrollable (noise) factors, e.g., the CPU and memory usage. Then there is random variation.

I know a little about typical design of experiments to determine the effect of factors. However, in this case, should I just see if there is any correlation of the uncontrollable (noise) factors on the results. I`m not sure yet how to go about that --- determining if the noise factors affect the results over just random variation.

It`s not clear to me how to determine how many runs I need to make for statistically valid results.

If someone could just point me in the right direction, I would appreciate it.

Thanks, Stanley
 
hlsmith

hlsmith

Less is more. Stay pure. Stay poor.
#2
#2
Is there a certain software you are interested in? Is there a certain function / procedure you are interested in? Does a certain sample size requirement need to be met?

Provide a few more details please.
 
M

MikeMulligan

New Member
#3
#3
hlsmith said:
Is there a certain software you are interested in? Is there a certain function / procedure you are interested in? Does a certain sample size requirement need to be met?

Provide a few more details please.
Click to expand...
The software receives an input message at a specified rate, does some processing, and publishes an output message. The input and output messages are time stamped and logged. I need to measure the time from message input to message output, and to understand the variability in this measured time.
 
hlsmith

hlsmith

Less is more. Stay pure. Stay poor.
#4
#4
Some software has a temporary memory that gets built up if not purges or restarted, does your software have this?

So it will be the exact same input and function over and over again?
 
M

MikeMulligan

New Member
#5
#5
It is very possible that there might be some sort of "warm up" effect like this, and it could be complicated. The longer term behavior is of more interest. One thing I should do is determine if there is such an effect.

In general, I suppose I should be looking at time series analysis for each run for that, and then use some method to look at data across runs.

I am still not sure how to determine how many runs I need, even after I have some data to analyze.
 
hlsmith

hlsmith

Less is more. Stay pure. Stay poor.
#7
#7
Well as long as nothing is changing and the process is stable, perhaps some kind of convergence can be established based on some many iterations without substantive change.
 
You must log in or register to reply here.