Detrending data

noetsi

Fortran must die
#1
I am confused what is occurring here.

We can remove the trend component in two easy steps. First, identify the overall trend by using the linear model function, lm

. The model should use the time series index for the x variable and the time series data for the y variable:

m <- lm(coredata(ts) ~ index(ts))

what is the index. The first time point is 1, the second time point is 2 etc?


Second, remove the linear trend from the original data by subtracting the straight line found by
lm

This is easy because we have access to the linear model’s residuals , which are defined by the difference between the original data and the fitted line:

where

r

i

is the

i

th residual and where

β

1

and

β

0



are the model’s slope and intercept, respectively.

We can extract the residuals from the linear model by using the

resid function and then embed the residuals inside a

zoo object:

>

detr <- zoo(resid(m), index(ts))

I don't understand what is occurring in the second step.

https://www.academia.edu/34707463/Detrending_a_Time_Series_Data

Elsewhere it is suggested you subtract a regression line created by regressing the time series on X and then subtracting this from the time series. I think this would give you the residuals, do you then use the residuals as the detrended time series? If you want to use this to predict future values, I assume you have to convert the residuals back after you use them to predict.
 
Last edited:

Dason

Ambassador to the humans
#2
Your question isn't formatted very well. Are you asking what the line
Code:
detr <- zoo(resid(m), index(ts))
is doing?
 

hlsmith

Not a robit
#3
Beyond your above gobbledygook, detrending is usually achieved by differencing. If a transformation is need due to say nonconstant variance, do that prior to differencing since many transformations dont like zeros or very small values.
 

noetsi

Fortran must die
#4
Your question isn't formatted very well. Are you asking what the line
Code:
detr <- zoo(resid(m), index(ts))
is doing?
I copied and pasted it and it was difficult to do Dason. I am asking what that is doing, more generally if what they are suggesting is regressing time on an index where one is the first period 2 is the second period etc, and then subtracting the resulting regression line from the time series data. I think this would generate the residuals of the regression.

Hlsmith what you said is always what I have read until today. But it turns out this is not universally accepted (Tsay for instance disagreed). I can send you some links if you want to see this. What I posted is one alternative to differencing. This thread contains others ... You probably can guess who I am in the conversation :) Note that I think the above is for deterministic trends which differencing never addresses, but even for stochastic trends there are disagreements on differencing.

https://stats.stackexchange.com/questions/405539/stochastic-versus-deterministic-time-series
 

noetsi

Fortran must die
#6
That is what I thought to. But it appears that is no longer best practice. The link I posted above goes into that in some detail.