generalized least squares

skan

New Member
Hello.
How can I modify least square linear regression method in order to fit a straight line to data with non gaussian error (such as Levy, Poisson or any heavy-tailed distribution).
I guess I must use maximum likelihood or M-estimators but I don't know exactly how.

regards

skan

New Member
why

Hello.

I hope you understand my poor english.

Imagine I know that a straight line is the curve that better fits to these data (*) but they are not normally distributed, their error is a heavy talied fdp and I think least square method gives too much "weight" to nearer data.

(*) Or I can transform them to get it.

skan

New Member
As an example you can read this:
http://arxiv.org/ftp/arxiv/papers/0704/0704.1867.pdf

Or
"In particular, standard methods such as least-squares fitting are known to produce systematically biased estimates of parameters for power-law distributions and should not be used in most circumstances"
"Fitting a line to your log-log plot by least squares is a bad idea. It generally doesn't even give you a probability distribution, and even if your data do follow a power-law distribution, it gives you a bad estimate of the parameters. You cannot use the error estimates your regression software gives you, because those formulas incorporate assumptions which directly contradict the idea that you are seeing samples from a power law"
from http://cscs.umich.edu/~crshalizi/weblog/491.html

I know how to maximize the maximum likelihood but not how to use it to fit data.

Last edited:

kazec

New Member
Skan, you may consider using the generalized Student distribution to characterize your noise (skewness, thick-tailedness can be taken care of). You then need to use numerical algorithm to search for a solution to the regression problem. To the best of my knowledge, least squares doesn't have a generalized version...

skan

New Member
ok

OK thanks.
A friend told me the solution to my problem comes from generalized models, as you say, and maximum likelihhod. I'm investigating it.