Regarding negative correlation and negative variance estimation result

#1
In ANOVA, we sometimes talk about that variance estimation can get negative result when there is negative correlation.

I do not understand what does the "negative correlation" refer to? Or which kind of relationship among data does "negative correlation" model?

Moreover, how to understand the negative variance estimation result?
 
#2
Does negative correlation mean from 0 to -1 (which means as one variable goes up another goes down? Certainly correlations from 0 to negative one exist and have that general meaning.

I don't understand how you can have negative variance. How does one go below zero variation?
 

Dason

Ambassador to the humans
#3
I don't understand how you can have negative variance. How does one go below zero variation?
Depending on the estimation process it's possible to get a negative estimate for the variance. Some method of moments estimators can do this depending the output but it's also possible for software to give a negative estimate when using maximum likelihood. The problem is that sometimes software doesn't include the implicit parameter boundaries when calculating the MLE. Even though we only care about maximums within the parameter space (because otherwise the likelihood is defined to be 0 and it certainly can't be a maximum at 0) it's easier for the software to just straight up maximum the likelihood and the without those restraints the iterative process to find that maximum might wander outside of the support of the parameters and find a maximum out there.
 

spunky

Can't make spagetti
#4
... or i could see a complex-valued random variable get a negative variance because of the imaginary part... or if you end up dealing with those horrible covariance/correlation matrices that are not positive semidefinite, but that's kind of unusual....

i guess Dason's example would be the most sensible reason... and god knows it happens really, really often in multilevel modeling/SEM..
 
#5
Depending on the estimation process it's possible to get a negative estimate for the variance. Some method of moments estimators can do this depending the output but it's also possible for software to give a negative estimate when using maximum likelihood. The problem is that sometimes software doesn't include the implicit parameter boundaries when calculating the MLE. Even though we only care about maximums within the parameter space (because otherwise the likelihood is defined to be 0 and it certainly can't be a maximum at 0) it's easier for the software to just straight up maximum the likelihood and the without those restraints the iterative process to find that maximum might wander outside of the support of the parameters and find a maximum out there.
In either case the finding would be invalid right (that is its a flaw in the method or the software similar to what occurs in factor analysis at times in explaining more than 100 percent of the variation (due to issues like the wrong scale between variables)? One you know is wrong? Or does it have real substantive meaning?

That reminds me of Issac Asimov proposing to his chemistry committee components that disolved before they were put into a solution :)
 

Dason

Ambassador to the humans
#6
In either case the finding would be invalid right (that is its a flaw in the method or the software similar to what occurs in factor analysis at times in explaining more than 100 percent of the variation (due to issues like the wrong scale between variables)? One you know is wrong? Or does it have real substantive meaning?
If we're talking about the method of moments estimators then technically they're not wrong - they are the method of moments estimators - it's just that they're negative which according to our constraints tells us that it's impossible.

In the case of the MLE then yes it is giving us the wrong answer for the question posed to it. The problem lies in that it is giving us a correct answer for maximizing the given function - it's just that whoever took the time to code the function up never put the restriction on the parameter space into the function. So a negative variance component might be a hint that the model isn't quite adequate but then again it might just be that you don't have enough data to adequately estimate the parameters in the model!
 

spunky

Can't make spagetti
#7
In the case of the MLE then yes it is giving us the wrong answer for the question posed to it.
have you ever seen MCMC estimators give you negative variances? now that you talked about the method of moments and MLE you made me wonder how're the bayesians doing on this area of wierd results... my intuitive answer would be "no" but i'm not as proficient in advanced bayesian estimation techniques as i would like to be...
 

Dason

Ambassador to the humans
#8
Well it's possible for an MCMC to give some iterations giving negative results - but that's only if the programmer doesn't take the time to program in the parameter constraints and the model isn't very good and the more I think about it I really can't imagine MCMC giving negative results for a variance component... Any likelihood will specifically prohibit a negative variance so any MCMC algorithm worth its weight won't sample on the negative real line.

Edit: But no - I've never seen an MCMC give negative iterations - but then again I'm the one typically programming my MCMCs...
 

spunky

Can't make spagetti
#9
Edit: But no - I've never seen an MCMC give negative iterations - but then again I'm the one typically programming my MCMCs...
heh, good point... and thanks, it's good to know i can always count with your informed opinion for these things...