How to improve forecasting

noetsi

Fortran must die
#1
I forecast monthly spending two years into the future. While our error is generally acceptable (5 percent or so a year out) I want to improve on that. I run exponential smoothing models (six of which I pick the best 3, best being defined by a MAPE from a hold out data set). I am trying to add an ARIMA models although that is experimental. I update my estimates once a month, we get new data only once a month.

No one at HQ, probably no one period, knows our spending process very well and there is no interest at HQ (other than by me) to do so. So applying expert judgement to the process is essentially impossible although I have tried. All we have right now is past data.

Any suggestion to improve this process, would be appreciated.
 

Miner

TS Contributor
#2
I think you are doing incredibly well to forecast two years with so little error. My understanding of ARIMA is that you should not use it to forecast more than 3 months. This is not due to a weakness in the method, but due to the inherent unpredictability of cyclical behavior.
 

noetsi

Fortran must die
#3
I think that is a generic problem miner with time series. I keep trying to come with new miracle ways to improve my methods. My theory is, more complex is better (which is not what the literature says actually for time series exponential smoothing is better in the M trials)
 

Miner

TS Contributor
#4
In general, I will use the simplest possible method that provides the results I need. It sounds like your company HQ is satisfied with the 5% error. Even if you got it down to 4%, all it will take is a management decision to launch a new program/product, or a pandemic to hit, and your forecast is no longer relevant.
 

noetsi

Fortran must die
#5
The pandemic has destroyed my models (they were working great until this month). The problem with the exponential models I use is that the disruption for even a few months could cause long term prediction issues. I can treat them as outliers I suppose and replace them in the time series with more reasonable values.
 

noetsi

Fortran must die
#7
Depending on the methods you're using you can certainly model the shocks that were introduced
Exponential smoothing does not model the shock in the same way ARIMA does with moving averages although it does model them. But I am not sure it would matter, because I don't think there is any way to be sure how long this shock will last, if the system will return to normal, and if so how long it will take. It is the same issue hays et el raised with their ARIMA models....