How to scale the Gamma distribution when its probability is more than 1?

#1
Let us assume we have a gamma distribution with parameters
alpha (or theta) = 1.33,
beta (or k) = 0.0124

Some values of this gamma distribution have probability of more than 1, some even reach up to 35

I tried the following code in matlab, which plots the pdf of the gamma distribution

b = 0:0.01:1
prob = gampdf(b,1.33,0.0124)
plot(prob)

So is there any mistake, or are we suppose to scale it down to less than 1?
 
#2
Remember that for a continuous random variable, like one with the gamma distribution, the value of the pdf at x is NOT the probability that the RV is x. That's only true for discrete RVs.

Because a Gamma RV is contnuous, and so you have an infinite number of possible values for x, you have to integrate the pdf over some region. The value of that definite integral is the probability that the RV takes a value in that region.
 

Dragan

Super Moderator
#3
Let us assume we have a gamma distribution with parameters
alpha (or theta) = 1.33,
beta (or k) = 0.0124

Some values of this gamma distribution have probability of more than 1, some even reach up to 35

I tried the following code in matlab, which plots the pdf of the gamma distribution

b = 0:0.01:1
prob = gampdf(b,1.33,0.0124)
plot(prob)

So is there any mistake, or are we suppose to scale it down to less than 1?
I suspect that you're confusing a probability value with a value of the height of the gamma probability density function. For example, at X=0.01, the height of the pdf is 37.54.
 
#4
So how do I get the probability of a continuous value, for e.g. x=0.01? For a known function such as Gamma distribution, can I just subtract the cumulative distribution function for x=0.01 and the cumulative distribution function for x=0.009?

And what if the distribution is unknown, and the probability density function is constructed based on kernel density estimation? Is there a way to get the probability for a x value?

Thank you so much for your help!
 
#5
The probability for ONE exact value in a continuous distribution is zero... When working with continuous distributions, you gotta work with ranges or intervals. As you know, the probability that a continuous random variable X will lie between a and b is equal to the integral of f(x)dx from a to b. Now you can see that if a = b, then the intergral will be zero, thus zero probability for a specific value.

This may seem weird, but actually it makes a lot of sence. If all values are possible, including 0.01, 0.011, 0.009, 0.0099, 0.010000001, and so on.... (because we're dealing with continuous) then P(x=a) = 0, since it is soooo very unlikely that one specific value will occur.

So anyways... If you want the probability of x=0.01, you would work with P(0.009<X<0.011) or something like that... Which you can get with the cumulative distribution functions...
 
#6
Of I'll say something about how to do this but read on afterwards--I think it might not be a good idea in general. The best way to do it might be to decide first what level of precision you're after, and then calculate over the interval that would round to that value. So if you really want to know the probability of observing a value of .01, given that any "true" value in [.0095, .0105) would round to .01, then calculate that integral. But if you're rounding more precisely, you might want to look at [.00995, .01005) or whatever. Of course this is a smaller interval, and the probability will be smaller.

This sort of thing is done when using the normal distribution, which is continuous, to approximate the binomial distribution, which is discrete and takes only positive integer values. We might approximate the chance that the variable takes value 500 by integrating the normal pdf from 495.5 to 500.5.

The problem with doing this to find probabilities of pseudo-discrete RVs, like rounded values of continuous RVs, is that the more precise you get, the smaller the interval is and the smaller the probability gets, eventually shrinking to 0. Suppose you have a standard normal RV. The pdf is highest at the mean, 0. But if you calculate the probability of X=0 by integrating the pdf from .99995 to 0.00005, it'll be really tiny. You need to be careful in this case to realize that it's still the most likely value, even though all values have probability 0, and the estimates all go to 0 as well. This is a little confusing, but something you need to accept in probability very early on is that intuition is not to be trusted. There are very specific definitions of probability, and it's perfectly acceptable to say that the probability of a continuous RV taking value x is 0 for every x, even though it has to take some value.
 
#7
Thank you so much for your help guys, I understand what is going on now.

I have another problem, if I have a probability distribution that is derived from kernel density estimation, where the pdf is obtained by

p(x) = 1/(Nh) * \sum^N_{i=1} K(x - x_i / h), with K as a Gaussian function

I know now the the correct way to find the probability of x is to integrate p(x) from x - \delta to x + \delta, where \delta is a small value.

This way is tedious and I will have to calculate the integrate, which may be complex.

Can I do a simpler way, by
1. normalizing the data into the range [0,1],
2. for each step i from 0 to 1, I will increment the step by \delta, and calculate p(i),
3. sum all p(i) up.
4. The probability of x will then be p(x) / ( sum_i p(i) * \delta )

What do you guys think?
 
Last edited:
#8
I can't help you on that one since I have no experience with kernels... Making this post basically useless. If you have some good references on kernels... maybe I can help you with this...

But a kernel, isn't that always a function with different parameters? So the integral wil always be a function with those paramters too... So maybe there is a standard form for the integral?
 
#9
I think numerical integration, e.g. monte carlo methods et al, are the way to go here. But I'd still shy away from saying anything like "the probablity of x is (anything but 0)" for a continuous RV. It's just not really done. Stick with probabilities of intervals, or just talk about the cdf.
 
#10
I can't help you on that one since I have no experience with kernels... Making this post basically useless. If you have some good references on kernels... maybe I can help you with this...

But a kernel, isn't that always a function with different parameters? So the integral wil always be a function with those paramters too... So maybe there is a standard form for the integral?
Yes, it is possible to integrate kernel, but the whole matters become complicated when x becomes a vector. So I agree with AtlasFrysmith that numerical integration is the way to go, which my colleague also suggested, maybe simpson's rule or trapezium rule.

The main problem will be in determining the size of the intervals, I think. The aim of using kernel is to smooth the function, but in the end I am using intervals to slice up the function, an irony. :)