Weighted Least Squares (WLS) using initial OLS and basis functions.

hob

New Member
#1
Hi All,

I have a question about WLS using basis functions. I am tasked with writing some code to automate a WLS procedure.

Initially I am taking Ordinary Least Squares on some data (X & Y). I achieve this by using a polynomial basis function, resulting in a matrix [X] corresponding to 1,X,X^2.

From this I generate a BLUE using:

b = (X'X)^{-1} X'Y

(so far so good), this gives an regression fit, y = aX^{2} + bX + c.

Now, the problem relates to the automation, it is assumed that the data is heteroskedastic and that WLS should be used instead of OLS.

The various books I have read have suggested using the OLS answer to then weight the X & Y data corresponding to the inverse of each measurement's variance (1/(sigma_{i}^{2})) from the average measurement variance. Creating a weighting matrix ''W'' with the diagonal the inverse of the measurements variance.

Therefore the estimator is given by:

b = (X'WX)^{-1} XW'Y

Such that there is now a weighting towards each measurement.


The problem I encounter is that the OLS provides a fit to the data, the WLS fit, however, does not look correct (the polynomial function is vastly different from the OLS estimate).

Is there a problem with my methodology anywhere?

Many thanks,

Hob
 

Dason

Ambassador to the humans
#2
In what way does it not look correct?

Right now you haven't really provided us much more information than "I did something but I'm not sure it's right". With no data or plots or code the best I can say is that you might be right or you might be wrong.
 

hob

New Member
#3
In what way does it not look correct?

Right now you haven't really provided us much more information than "I did something but I'm not sure it's right". With no data or plots or code the best I can say is that you might be right or you might be wrong.
Apologies, I found the bug in the code (it initialised the matrix with a non-zero value).

I also have another questing, this time regarding GLS under GARCH(1,1) errors.

I have looked in a few books but have not found the answer of dealing with GARCH(p,q) errors in regression. I have attached two images from "Heteroskedasticity in Regression" by R.Kaufman.

The first is the weighting matrix for Hetroskedasticity and the second is for AR(1) errors. For a combination (I am assuming) it is a combined matrix i.e with the variance diagonal of the first inserted in the second matrix?

If this is the required matrix, is the auto correlation term (/rho) calculated by the equation shown in the third image?

I basically have been given a GARCH(1,1) model with coefficients calculated using MLE, I have to construct the code to perform regression accounting for the GARCH errors.


Many thanks,

Hob