MA() part of ARIMA: How do you get error terms?

[jamesdvance]

I've been through the excellent forecasting book coauthored by Hyndman on otext.org. I'm having trouble understanding the actual algorithm. The MA(x) is supposed to be a moving average of error terms. But where does the error come from? Is it based on the AR(x) term? In that case, what about AR(0)?

Any help would be greatly appreciated.

James

 

[Dason]

I don't quite understand the question.

 

[noetsi]

I think that with an AR term Y is correlated with lags of itself while with a moving average Y is correlated with lagged error terms (so Yt is correlated with et-1). I am fairly certain the calculation of MA does not employ regression because the error term of past lags can not be observed. MA has little substantive meaning in regression.

 

[jamesdvance]

Noetsi, I think that clarifies part of it -- et is not an 'actual' past error term. Is it safe to say it is estimated as a random variable?

Dason, I think my confusion on the subject makes for a confusing question.

 

[snoggla]

Noetsi, I think that clarifies part of it -- et is not an 'actual' past error term. Is it safe to say it is estimated as a random variable?

I guess so. just solve for AR(∞) representation and u will see that is a function of random variables.

MA is simply a weighted average of q consecutive past shocks. and u regress on those q past shocks.
regarding ARMA:
MA-part models dependence on recent shocks, while AR captues dependence on distant shocks.

the thing is, just like noetsi explained. you cant observe those shocks directly, hence you will rely on the AR(∞) for estimation.

 

[noetsi]

I have not seen AR described in terms of shocks, only MA. Whether the past errors are an "actual" error term or not is a fascinating question. It is not an observable error term but population standard deviations are not observable (knowable) either usually - that does not mean they don't exist

I think they solve MA indirectly through working through AR calculations perhaps although it has been a very long time since I read that. MA and AR are related a MA of one form can represent a AR of another form which is why in ARIMA you may use a MA of 1 rather than an AR of 2 in some situations- effectively they are the same and the rule of simplicity comes into play.

Thankfully I don't worry about things such as how algorithms work - not smart enough to understand it anyway. I found the Hyndman book on transfer functions essentially incomprehensible.

If you are learning ARIMA I strongly recommend the Duke website on it. It is truly awesome although designed for practice not theory.

 

[jamesdvance]

I have not seen AR described in terms of shocks, only MA. Whether the past errors are an "actual" error term or not is a fascinating question. It is not an observable error term but population standard deviations are not observable (knowable) either usually - that does not mean they don't exist

I think they solve MA indirectly through working through AR calculations perhaps although it has been a very long time since I read that. MA and AR are related a MA of one form can represent a AR of another form which is why in ARIMA you may use a MA of 1 rather than an AR of 2 in some situations- effectively they are the same and the rule of simplicity comes into play.

Thankfully I don't worry about things such as how algorithms work - not smart enough to understand it anyway. I found the Hyndman book on transfer functions essentially incomprehensible.

If you are learning ARIMA I strongly recommend the Duke website on it. It is truly awesome although designed for practice not theory.

Nice, I will definitely check out the Duke website! I'm looking for practice only, but need a baseline understanding to at least use it correctly. Thanks guys

 

[noetsi]

http://people.duke.edu/~rnau/411arim.htm

There may be more than one, it fills my notes, but that is a start.

As my first instructor of ARIMA said, it is an art form as much as a science.

One thing to be careful about. Different software will handle ARIMA very differently. Also there is a form of multivariate ARIMA (where you predict one time series with another through ARIMA) but that is a bear to do because you have to prewhiten each predictor with ARIMA first - a time consuming process. I think this is what Hyndman calls transfer functions - others call it ARIMAX.

Personally I recommend dynamic regression such as ARDL then doing that. It just look to time intensive and uncertain given the inherent judgment calls in ARIMA.

I think it is worth looking at ESM (Holt Winson etc) rather than ARIMA. It has a good track record in predicting according to statistical test of that, is robust to assumptions, is commonly used, and is far far easier to do than ARIMA. It also does not require you to make the judgment calls as to the form of PDQ - which require expert judgment although there are rules of thumb as on the Duke site.