Estimating the mean (mu) and stdev (sigma) of a non-truncated normal using data from a left-truncated normal.

rjasg

New Member
#1
Hi everyone. I am trying to estimate the mean (mu) and stdev (sigma) of a non-truncated normal using data from a left-truncated normal.
I am getting values of mu and sigma to far off from the reals. Apparently there are numerical issues in the estimation process. Can anyone give me a hand?
 

obh

Active Member
#2
First I don't understand your question:

of a non-truncated normal using data from a left-truncated normal.
If the data is left-truncate why do you want to calculate "non-truncated"?

Generally, ther are simple formulas to calculate the average and the sample standard deviation.

Is it possible to paste the data and the results?
 

rjasg

New Member
#3
Hello. The left truncated distribution has the same parameters of the normal (non-truncated) mu and sigma and also a third parameter that is the truncation point that is known. I am interested in estimating the mu and the sigma parameters using a sample of simulated data from a left truncated normal. I hope this made it clear.
 

obh

Active Member
#4
How come "the left truncated distribution has the same parameters as the normal" ?
Can you please elaborate on what is the left distribution? and what is normal distribution?

Is it possible to paste the data and the results?
 

rjasg

New Member
#5
There are known formulas for the mu and sigma for this case however those formulas are given as a set of 2 nonlinear equations.
I have tried to find the roots but apparently there are numerical problems and because of that I am getting solutions distant from those I want.
I am attaching the left truncated data from which I wante to estimate mu and sigma.
 

Attachments

obh

Active Member
#6
Can you please elaborate on what is the left distribution? and what is normal distribution?

is the text file contains 2 numbers per row?
 

rjasg

New Member
#7
Hello again. Normal is the same as Gaussian for me and, as you know, it has two parameters, mu and sigma that are respectively the mean and standard deviation (stdev). Left truncation means that there is a fixed left limit for the density function meaning that is impossible to have observations below that point to which I call a left truncation point. So the Left truncated Gaussian is a whole new different distribution and its mean and stdev are no longer the mu and sigma. Altough these values are not moments anymore they still are parameters of the left truncated Gaussian distribution. These are precisely the parameters I am trying to estimate from the left truncated gaussian distribution.
 

Attachments

rjasg

New Member
#8
I am Sorry, I have sent you a y.txt file instead of z.txt file that are random numbers of the left-truncated normal.
The truncation point is -10/3.
 

Attachments

Dason

Ambassador to the humans
#12
What program are you using for the estimation? And can you maybe provide a complete example. It seems like you've shown the data you're interested in but haven't given output for that. You've shown output for simulated data but didn't show the actual data if I understand it correctly. You haven't told us what program or whatever you're using for the estimation. Ideally you could show the code you used for the simulation and the code you used for the estimation along with the output.
 
#13
I am working in Matlab that, as you know, is the same as Octave, a free programing software that you can install easily.
The simulation program has the name=Generate_Left_truncated_normal.txt, that generates PN and Left-truncated normal random numbers.
The estimation program has the name: Maximize_MLfunction.txt. This last program calls the routine function name=estimate_mu_sigma_given_lambda.txt that gives a solution (using Matlab function fsolve for the nonlinear system whose solution are the real mu and sigma). The estimation program has a do while cycle that performs the Newton-Raphson iteration method inside a do for cycle for the parameter lambda that varies from 0.2 to 1 ( I am only interested in the case 0<lambda<1. The ML function and derivative is calculated on routine function calculate_logML.txt. The programs are set to perform on the example mu=5 sigma=10 and lambda=0.3. Please let me know if it worked.
 

Attachments

Dason

Ambassador to the humans
#14
Going back to the Results.txt file - why are there two columns for lambda? What sample size did you have in your data? Could you talk about lambda more - is that the parameter that tells the left truncation point?
 

Dason

Ambassador to the humans
#15
And would you be willing to post the code in those files directly in a post using code tags? It would make it a lot easier to look over.
 
#16
The starting point of mu and sigma is mean(z) and std(z) where z is the left truncated data.
I am sorry about the 2 lambda values.
Results.txt is a table generated by the estimation program. It is composed of 5 columns,
starting value for lambda, estimated value of mu, estimated value of sigma, value of lambda after logML maximization , logML.
Yes I will put the code tags.