Central limit theorem - Approximation the Sum of Bernoullis to Normal Distribution

#1
Hello,
I'm new in this forum and I would like to ask for your help.
I have a set of Independent Bernoulli random variables, but they are not identically distributed, with different success probabilities.
I would like to approximate the Sum of this random variables, but since they are not identically distributed, I do not know which conditions should be met in order to perform this approximation. I've seen in many places that "given certain conditions" this approximation is possible, but I do not know what are the conditions.
Could anyone please give me some advice?

Thank you.
 
Last edited:

BGM

TS Contributor
#2
Re: Central limit theorem - Approximation the Sum of Bernoullis to Normal Distributio

I think you can first check the Lyapunov's condition:

http://en.wikipedia.org/wiki/Central_limit_theorem#Lyapunov_CLT

Let say you have a sequence of independent random variables

\( X_i \sim \text{Bernoulli}(p_i), i = 1, 2, 3, \ldots \)

Then

\( s_n^2 = \sum_{i=1}^n p_i(1 - p_i) \)

If you check the \( \delta = 1 \) case,

\( E[|X_i - p_i|^3] = p_i(1 - p_i)^3 + (1 - p_i)p_i^3 = p_i(1 - p_i)[p_i^2 + (1 - p_i)^2] \)

You need to ensure

\( \lim_{n \to +\infty} \frac {\displaystyle \sum_{i=1}^n p_i(1 - p_i)[p_i^2 + (1 - p_i)^2]} {\displaystyle \left[\sum_{i=1}^n p_i(1 - p_i)\right]^{\frac {3} {2}}} = 0 \)

If this is satisfied, you have the following result:

\( \frac {1} {s_n} \sum_{i=1}^n (X_i - p_i) \to \mathcal{N}(0, 1) \)

There are more relaxed result but lets check this first.
 

Dason

Ambassador to the humans
#3
Re: Central limit theorem - Approximation the Sum of Bernoullis to Normal Distributio

Note that if that condition isn't satisfied that doesn't mean you don't have an asymptotic normal distribution. But Lyapunov's condition is a lot easier to check than some of the other conditions so it's a good place to start.
 
#4
Re: Central limit theorem - Approximation the Sum of Bernoullis to Normal Distributio

Thank you BGM,
I appreciate much your response.

I'm trying to find a way to verify the limit you suggested, but the problem is that those probabilities may vary in the interval [0;1]. Moreover, I have a finite set of random variables and I do not know what kind of conditions should I consider, in order to verify that limit.
I'm using Mathematica do calculate the limit and, considering the same \(p_i \) for every random variable, the condition is verified except for p=0 or p=1, but that Sum is the Binomial distribution and the result is already known.
Do you have any idea how could I verify that limit for different success probabilities?
Thank you for your help.
 
Last edited:

Dason

Ambassador to the humans
#5
Re: Central limit theorem - Approximation the Sum of Bernoullis to Normal Distributio

Do you have a distribution on the probabilities themselves?
 
#6
Re: Central limit theorem - Approximation the Sum of Bernoullis to Normal Distributio

Sorry Dason,

Maybe I did not explain properly.

I have a set of Bernoulli random variables with different success probabilities, varying from 0 to 1, and I want the normal approximation of the sum.
 
Last edited:

Dason

Ambassador to the humans
#7
Re: Central limit theorem - Approximation the Sum of Bernoullis to Normal Distributio

Yeah - but I was asking about the distribution of those success probabilities. How are these success probabilities being generated?

For the most part I doubt you're dealing with some weird case where the sum isn't asymptotically normal but it's still fun to try to figure it out.

Also if we're dealing with a finite sample size it really doesn't matter too much if it really is asymptotically normal - the important part for you (I'm guessing) is just if the finite sum is approximately normal. You could easily investigate this via simulation.
 
#8
Re: Central limit theorem - Approximation the Sum of Bernoullis to Normal Distributio

OK, thanks. That's it, the important part is just if the finite sum is approximately normal.

The success probabilities are obtained in the following manner:

I have a set of independent Uniform random variables

\( X_i \sim \text{Unif}(R_i, FF_i), i = 1, \ldots, n\)

To get the success probabilities I do:


\( Z_i \sim \text{Bernoulli}(p_i), i = 1, \ldots, n\)


\( p_i = P(Z_i = 1) = P(X_i > r_i), i = 1, \ldots, n \)


I think we may say that those success probabilities are random, because \( R_i \) , \( FF_i \) and \( r_i \) are random.

Thank you Dason for your suggestion.

What kind of simulation should I try?
 
Last edited:

Dason

Ambassador to the humans
#9
Re: Central limit theorem - Approximation the Sum of Bernoullis to Normal Distributio

How are you obtaining \(R_i, FF_i, \text{ and } r_i\)?
 
#10
Re: Central limit theorem - Approximation the Sum of Bernoullis to Normal Distributio

Those variables are obtained as a result of an experiment and each of them may take values between \( V_{min} \) and \( V_{max} \). I think we may consider that those variables are Uniformly distributed in the interval \( [V_{min}, V_{max}] \).