parameter estimation based on observed survival probabilities

[schleprock]

Say 100 samples have been pulled from a Normal distribution with unknown mean and variance. All I know about the results are the following:

• 23 out of 100 are greater than 200
• 9 out of 100 are greater than 300
• 2 out of 100 are greater than 400

How would I find the best estimate of the mean and SD of the distribution based on this information?

Thanks in advance!

 

[Dason]

Do you know anything about maximum likelihood estimation?

 

[schleprock2]

Yes, but I'm not clear how that would apply in this situation. Individual observations aren't available, so it doesn't seem like MLE applied to the PDF is helpful. The CDF has no closed form solution. I did think about using MLE and a binomial distribution (combined w/ an approximation to the Normal CDF) to solve for the mu and sigma, but it's unwieldy, and honestly I feel like I'm over-complicating things. Seems like a straightforward problem, but for some reason the answer isn't obvious to me. Thanks for your help!

 

[Dason]

If I tell you mu and sigma can you tell me the probability of an observation being less than 200? Greater than 200 but less than 300?

 

[schleprock2]

If I tell you mu and sigma can you tell me the probability of an observation being less than 200? Greater than 200 but less than 300?

Of course. But if I tell you Pr(X<200), can you tell me mu and sigma? No, because there is not a unique solution.

If I also tell you Pr(X>=200 AND X<300), then -- with some effort -- you could give me mu and sigma. But if I also give you Pr(X>=300 AND X<400), and those probabilities are based on observed results (not a Normal distribution w/ known parameters), then I think there probably is no mu and sigma which would describe that. I would be interested to know how you would solve for the most likely mu and sigma in that case.

 

[Dason]

I think you're forgetting how maximum likelihood works though. You basically say "if mu = 50 and sigma = 30 what is the probability of observing the data" (the likelihood) and your goal is to find the values of mu and sigma that maximize that. So pretend that you know mu and sigma for a second - can you write out the joint distribution of the observed data? Then it becomes a task of finding which values maximize that.

 

[schleprock2]

It's funny you say that, because I thought the problem might be that I'm too stuck on EXACTLY how MLE works in normal circumstances! Normally, we'd be dealing with individual observations, not some result like "23 of 100 obs are >200". Given individual observations, the path is to use the probability density function to establish a likelihood function, and then maximize the likelihood with respect to the individual parameters.

But what is the likelihood function here?? We're dealing with a cdf, not a pdf, and the Normal cdf is not closed form.

 

[Dason]

Well it might have been normally distributed originally but that's not what you see now. Now all you have is binned data but given the parameters you can find the probabilities of the bins.

Forget the original problem exists and pretend that you're trying to solve this problem:

The probability of an observation being Red is \(\alpha\), the probability of an observation being Blue is \(\beta\) and the probability of it being neither is \(1-\alpha-\beta\). If you observe 32 reds, 23 blues, and 17 neithers then what are the MLEs for \(\alpha\) and \(\beta\).

 

[schleprock2]

Well it might have been normally distributed originally but that's not what you see now. Now all you have is binned data but given the parameters you can find the probabilities of the bins.

Forget the original problem exists and pretend that you're trying to solve this problem:

The probability of an observation being Red is \(\alpha\), the probability of an observation being Blue is \(\beta\) and the probability of it being neither is \(1-\alpha-\beta\). If you observe 32 reds, 23 blues, and 17 neithers then what are the MLEs for \(\alpha\) and \(\beta\).

I think you're hitting on one of the complications, which is that the information given speaks more directly to a binomial distribution than a Normal distribution. In your example, I don't think any MLE is needed; the best estimate for \(\alpha\) is 44.4% (32/72) and for \(\beta\) it's 31.9% (23/72).

But that doesn't really get me anywhere, does it? I want to know something about the Normal distribution that underlies the percent of observations falling into the various "bins".

 

[Dason]

I don't think any MLE is needed

That's not a good way to think about this. Take a step back and ask yourself how would you find the MLE in this case.

 

[schleprock2]

In the example you gave -- what are the MLEs of alpha (Prob of an observation being red) and beta (Prof of an observation being blue) -- you're talking about a binomial distribution. k successes in n trials. This page (https://onlinecourses.science.psu.edu/stat504/node/28) walks through the calculation better than I could in this quick reply, but the conclusion is that the MLE of alpha is k/n, or number of successes (32) divided by number of trials (72), or 44.4%.

So that's solved. But it doesn't appear to get me any closer to using that information to understand the underlying Normal distribution.

 

[Dason]

No it's technically not a binomial distribution. It would be a multinomial distribution (with a binomial there are only two possibilities).

Plus it's not solved. The whole point of it was to get you think about what the likelihood function actually is. There is a direct connection between the problem I gave you and the the problem you're trying to do.

 

[GretaGarbo]

Is the conditions here that:
Say 100 samples have been pulled from a Normal distribution
23 out of 100 are greater than 200
9 out of 100 are greater than 300
2 out of 100 are greater than 400?

OR is it that:

66 out of 100 are less than 200
23 out of 100 are greater than 200 and less than 300
9 out of 100 are greater than 300 and less than 400
2 out of 100 are greater than 400?
And that all are statistically independent?

 

[schleprock2]

No it's technically not a binomial distribution. It would be a multinomial distribution (with a binomial there are only two possibilities).

Plus it's not solved. The whole point of it was to get you think about what the likelihood function actually is. There is a direct connection between the problem I gave you and the the problem you're trying to do.

Yes, it's technically multinomial. However, it's just as easily stated as a binomial ("blue" or "not blue").

Thanks for the feedback, but I don't think this is really getting me anywhere. I was really looking for suggested solutions, not additional problems related to the original question.

 

[schleprock2]

Is the conditions here that:
Say 100 samples have been pulled from a Normal distribution
23 out of 100 are greater than 200
9 out of 100 are greater than 300
2 out of 100 are greater than 400?

OR is it that:

66 out of 100 are less than 200
23 out of 100 are greater than 200 and less than 300
9 out of 100 are greater than 300 and less than 400
2 out of 100 are greater than 400?
And that all are statistically independent?

Hi Greta-

I think the problem could be equivalently stated that way, but I don't think those outcomes are statistically independent, since the number of samples is fixed. More outcomes less than 200 means that (statistically) fewer will be greater than 400.

 

[Dason]

Thanks for the feedback, but I don't think this is really getting me anywhere. I was really looking for suggested solutions, not additional problems related to the original question.

So you don't really see how your original problem is actually a multinomial? I wasn't giving you more work for the hell of it. Like I said it was *directly* related to your original problem. Take a minute and try to find the connection.

 

[schleprock2]

So you don't really see how your original problem is actually a multinomial? I wasn't giving you more work for the hell of it. Like I said it was *directly* related to your original problem. Take a minute and try to find the connection.

I believe I already agreed that it was multinomial, though I also said I felt it could be stated as binomial. After all, the former is just a generalization of the latter.

To be honest, I'm not sure you're understanding the problem. This may be my fault for not explaining it clearly. So let me try a slightly different angle.

You suggested thinking about the problem in terms of a multinomial (binomial) -- which I agreed with -- and suggested applying MLE -- which I also agreed with. However, ultimately I'm not interested in just solving for p (probability of success, or probability of "blue" in your example). I’m interested in solving for the parameters of the underlying distribution that drives p.

So yes, I can do as you suggested and establish the likelihood function for a multinomial distribution, take the natural log, take the derivative with respect to p, and solve for the p that maximizes log-likelihood. Then what? I’m left with an estimate of p, which doesn’t get me anywhere because p is a function of mu and sigma. My first thought was to substitute the Normal CDF as p – in other words include mu and sigma in the likelihood function explicitly – but again the CDF has no closed form solution, so this didn’t seem to be a viable approach.

I don’t know if this is coming across, but in any case I don’t think the problem is as easy as you are making it out to be. If I’m mistaken about any of the above, please let me know where.

 

[Dason]

I completely understand the problem. What I'm trying to get you to do is set up the **** likelihood! You just seem too stubborn to do that.

So you agree that it's multinomial given the bin probabilities right? And how do you find the bin probabilities? Well they're just probabilities derived from the normal distribution (so these probabilities are functions of mu and sigma). So there you have it - that's how you get your likelihood.

Trust me I completely understand the problem - I was trying to get you to make that last step but I think you were just stopping at "oh I guess it could be a multinomial but I don't care about the ps" when you should have taken it one step further and realized the ps are functions of mu and sigma.

 

[schleprock2]

I completely understand the problem. What I'm trying to get you to do is set up the **** likelihood! You just seem too stubborn to do that.

Rather than laboriously type out the likelihood function in a response -- finding the right symbol codes and whatnot -- I linked yesterday to a web page that showed the likelihood function (admittedly, for a binomial) that I would've ended up typing. Here it is again: https://onlinecourses.science.psu.edu/stat504/node/28. Regardless, you can safely assume for the purposes of this and future discussions that I understand MLE, how to set up a likelihood function, etc.

So you agree that it's multinomial given the bin probabilities right? And how do you find the bin probabilities? Well they're just probabilities derived from the normal distribution (so these probabilities are functions of mu and sigma). So there you have it - that's how you get your likelihood.

I think this is where we diverge. I've agreed several times that these probabilities are driven by mu and sigma. But the function is not closed-form. The PDF is, but the CDF isn't...it's a definite integration of the PDF from -infinity to x. So if you could please explain to me how to use that integral (i.e. the Normal CDF) in place of p such that I can maximize likelihood relative to mu and sigma, that would be fantastic. Or if there's a simpler way to do it, that would be even better. Either way, I hope you can see where the maximum likelihood function for a multinomial isn’t the piece of the puzzle that I’m missing.

Trust me I completely understand the problem - I was trying to get you to make that last step but I think you were just stopping at "oh I guess it could be a multinomial but I don't care about the ps" when you should have taken it one step further and realized the ps are functions of mu and sigma.

See above. I fully realize that the ps are functions of mu and sigma. What I don’t yet understand – but am hoping you do – is how to employ the CDF of a Normal in the multinomial likelihood function.

 

[Dason]

Rather than laboriously type out the likelihood function in a response -- finding the right symbol codes and whatnot -- I linked yesterday to a web page that showed the likelihood function (admittedly, for a binomial) that I would've ended up typing. Here it is again: https://onlinecourses.science.psu.edu/stat504/node/28. Regardless, you can safely assume for the purposes of this and future discussions that I understand MLE, how to set up a likelihood function, etc.

I think this is where we diverge. I've agreed several times that these probabilities are driven by mu and sigma. But the function is not closed-form. The PDF is, but the CDF isn't...it's a definite integration of the PDF from -infinity to x. So if you could please explain to me how to use that integral (i.e. the Normal CDF) in place of p such that I can maximize likelihood relative to mu and sigma, that would be fantastic. Or if there's a simpler way to do it, that would be even better. Either way, I hope you can see where the maximum likelihood function for a multinomial isn’t the piece of the puzzle that I’m missing.

See above. I fully realize that the ps are functions of mu and sigma. What I don’t yet understand – but am hoping you do – is how to employ the CDF of a Normal in the multinomial likelihood function.

Just use the CDF. There isn't a closed form for it but we do have a symbol for the function and software can evaluate the CDF. Just like how you might not be able to easily tell me what the square root of 23423421 is without using a calculator we can do the same thing for the normal CDF.