# PDF of (nearly) collinear variables

#### Sam Vimes

##### New Member
My question is motivated by a problem with estimation by maximum likelihood. In simple terms, I want to estimate a parameter $$\beta$$ and have three variables, $$X$$, $$Y$$ and $$Z$$, such that $$Z = X+Y$$. Using all three variables is pointless and the joint pdf is singular. Now, suppose that instead of $$Z$$ I have $$U$$ such that $$U = Z + \epsilon$$, where $$\epsilon$$ is independent from $$X$$ and $$Y$$ and its distribution does not depend on $$\beta$$. The joint pdf of $$X$$, $$Y$$ and $$U$$ is not singular. Seems to me obvious that $$U$$ does not contain any additional information about $$\beta$$, given $$X$$ and $$Y$$. So, I think it must be true that, in terms of the likelihood function, $$L(\beta;X,Y,U)=L(\beta;X,Y)*const$$. If I am correct, what would be the best way to prove it? Thanks.

Last edited:

#### BGM

##### TS Contributor
The question is equivalent to show that the conditional pdf

$$f_{U|X=x,Y=y}(u|x,y) = \frac {f_{U,X,Y}(u,x,y)} {f_{X,Y}(x,y)}$$

is independent of the parameter $$\beta$$

Consider the conditional CDF

$$F_{U|X=x,Y=y}(u|x,y)$$

$$= \Pr\{U \leq u|X = x, Y = y\}$$

$$= \Pr\{X + Y + \epsilon \leq u |X = x, Y = y\}$$

$$= \Pr\{\epsilon \leq u - x - y|X = x, Y = y\}$$

$$= \Pr\{\epsilon \leq u - x - y\}$$ (by independence)

$$= F_\epsilon(u - x - y)$$

By assumption, the distribution of $$\epsilon$$ is independent of $$\beta$$, and therefore

$$F_{U|X=x,Y=y}(u|x,y) = F_\epsilon(u - x - y)$$

is independent of the beta, and similar for its derivative:

$$f_{U|X=x,Y=y}(u|x,y) = \frac {\partial F_{U|X=x,Y=y}(u|x,y)} {\partial u}$$