Models comparison

[Mukund]

I have a time series data of 1000 points for each of the different machines. I tried different forecasting techniques to make a one step prediction. The goal is to find out one common predictive model that could work for all machines.

The forecasting techniques used are,

5 period moving average
10 period moving average
40 period moving average
Single exponential smoothing
Double exponential smoothing
Linear regression
Log transformed linear regression
I tried to compare all the models using Root Mean Squared Error (RMSE). The best model is one with low RMSE.

The problem is different methods have low RMSE and are good for different machines. For example, Linear regression is good for machine 1 and Double exponential smoothing is good for machine 2 etc. Also, the difference in RMSE between methods is very low. But the goal is to have one common reliable method for all machines. How do we do it statistically? Any recommendations?

P.S. tried ANOVA and t-test on SE (squared error) but the results was like there is no significant difference in mean.

 

[Miner]

What type of machines are these and what are you trying to predict? Is this a continuation of the breakdown time series from your other posts?

 

[Mukund]

I am trying to predict utilisation as a percentage of time and breakdown as a percentage of time. And yes. It is a continuation

 

[Miner]

Breakdowns would be better analyzed using reliability, maintainability and availability approaches, specifically repairable systems analysis. SRC is a good source for information on these topics.

Taking another look at your previous posts, try an individuals- moving range control chart. It appears that you have what is essentially a stationary time series with periodic blips. The control chart would isolate what is normal variation due to the stationary part of the time series from the abnormal variation (i.e., blips). The stationary part is self predicting. See https://en.wikipedia.org/wiki/Shewhart_individuals_control_chart

 

[Mukund]

Control chart is a good idea. But I need a metric to compare the models and finalize on one model. Once we finalise the model, a control chart can be used to monitor the performance of the model.

Do you think metrics like RMSE can be used to compare the models?

 

[Miner]

Can you provide some data?