Log In Start studying!

Select your language

Suggested languages for you:
StudySmarter - The all-in-one study app.
4.8 • +11k Ratings
More than 3 Million Downloads
Free
|
|

Sum of Independent Random Variables

Sum of Independent Random Variables

Suppose two people assemble widgets in a factory, and they work independently. You know the average time it takes each person to assemble a widget. Can you find the average time it would take them to work together to assemble a widget if all you know is their separate average assembly time? In some cases, you can! The key question is whether or not your random variables are independent. So read on to learn about the sum of independent random variables!

Meaning of the sums of independent random variables

You have probably seen how to find the probability generating function of a random variable already. In fact, you have more than likely even looked at what happens when you add two of them together without even realising it!

Let's look at a quick example.

Say you have two bags of balls which are labelled with numbers. One bag has three balls labelled with the number \(0\), and \(2\) balls labelled with the number \(1\). You can represent this by the random variable \(X\) where \(x=0,1\).

The second bag has four balls labelled with the number \(2\), and \(1\) ball labelled with the number \(3\). You can represent this by random variable \(Y\) where \(y=2,3\).

Then the tables for the probability distributions are:

Table 1 - Probability distribution for \(X\)

\(x\)
\(0\)
\(1\)
\(P(X=x)\)
\(\frac{2}{5}\)
\(\frac{3}{5}\)

Table 2 - Probability distribution for \(Y\)

\(y\)
\(2\)
\(3\)
\(P(Y=y)\)
\(\frac{4}{5}\)
\(\frac{1}{5}\)

You then have the probability generating functions

\[G_X(t)=\frac{2}{5}+\frac{3}{5}t\]

and

\[G_Y(t)=\frac{4}{5}t^2+\frac{1}{5}t^3.\]

Now you can easily see the probability of choosing a given ball from either bag.

What if you wanted to find the sum of their probability distributions? How would you go about finding the probability generating function for \(Z=X+Y\)? One method would be to write out the probability distribution table of \(Z\):

Table 3 - Probability distribution for \(Z=X+Y\)

\(z\)
\(P(Z=z)\)
\(2\)
\((P(X=0))(P(Y=2))=\dfrac{8}{25}\)
\(3\)
\((P(X=0))(P(Y=3))+(P(X=1))(P(Y=2))=\dfrac{14}{25}\)
\(4\)
\((P(X=1))(P(Y=3))=\dfrac{3}{25}\)

Hence the probability generating function of \(Z\) would be

\[G_Z(t)=\frac{8}{25}t^2+\frac{14}{25}t^3 +\frac{3}{25}t^4 .\]

Finding the probability generating function for \(Z=X+Y\) wasn't too bad in the previous example because the individual probability generating functions weren't that complicated. But in a case with more complex probability generating functions this can get complicated very quickly!

In two special cases, there is a much quicker way to find the sum of two probability generating function.

  • The first case is where you have two independent discrete random variables \(X\) and \(Y\) and you are asked to find \(Z\) where \(Z=X+Y\).

  • The second case is where you are asked to find \(Z\) where \(Z\) is a linear function of discrete random variable \(X\) (i.e. \(Z=aX+b\)).

Let's look at each case.

Finding the probability generating function of \(Z=X+Y\)

There is a very important theorem that covers this case called the Convolution Theorem.

Convolution Theorem: Suppose two independent discrete random variables \(X\) and \(Y\) have probability generating functions \(G_X(t)\) and \(G_Y(t)\). The probability generating function of \(Z=X+Y\) is

\[G_Z(t)=G_X(t)G_Y(t).\]

Let's look at an application.

In the previous example you found two probability generating functions

\[G_X(t)=\frac{2}{5}+\frac{3}{5}t\]

and

\[G_Y(t)=\frac{4}{5}t^2+\frac{1}{5}t^3,\]

and then constructed a table to find that for \(Z=X+Y\),

\[G_Z(t)=\frac{8}{25}t^2+\frac{14}{25}t^3 +\frac{3}{25}t^4 .\]

Do you get the same answer using the Convolution Theorem?

Solution:

Using the formula \(G_Z(t)=G_X(t)G_Y(t) \), you have:

\[\begin{align} G_Z(t) &= \left(\frac{2}{5} +\frac{3}{5}t\right) \left(\frac{4}{5}t^2+\frac{1}{5}t^3\right)\\&= \frac{8}{25}t^2+\frac{2}{25}t^3+\frac{12}{25}t^3+\frac{3}{25}t^4 \\&=\frac{8}{25}t^2+\frac{14}{25}t^3 +\frac{3}{25}t^4.\end{align}\]

So you can see that you get the same answer by constructing the table as you do using the Convolution Theorem.

The main benefit of the Convolution Theorem is that it gives you a way to find the probability generating function without constructing a table, which leads to fewer opportunities for errors.

Finding the probability generating function of \(Z=aX+b\)

Let's take a quick look at how you could build the probability generating function of \(Z=nX\) where \(n\) is a natural number from the probability generating function of \(X\). Starting with \(n=2\),

\[ Z = 2X = X+X\]

so you can use the Convolution Theorem to get that

\[G_Z(t)=G_X(t)G_X(t) = (G_X(t))^2.\]

Then you could use proof by induction to show that for any natural number \(n\), if \(Z=nX\) then

\[G_Z(t)=\underbrace{G_X(t)G_X(t)\cdots G_X(t)}_{n \text{ times}} = (G_X(t))^n.\]

As you know, there are more than just the natural numbers out there, and you still need to account for that '\(+b\)' too. So it helps to look at the alternate definition of the probability generating function:

\[G_Z(t) = \text{E}(t^Z).\]

Then you can use properties of the expected value function to get the following:

\[\begin{align} G_Z(t) &= \text{E}(t^Z) \\ &= \text{E}(t^{aX+b}) \\ &= \text{E}(t^{aX}t^b) \\ &= t^b\text{E}(t^{aX}) \\ &= t^b\text{E}\left((t^a)^X\right) \\ &= t^bG_X(t^a) . \end{align}\]

While this property doesn't have a fancy name, it is worth stating separately.

If \(X\) is a discrete random variable and has a probability generating function of \(G_X(t)\), the probability generating function of \(Z\) where \(Z=aX+b\) is:

\[G_Z(t)=t^bG_X(t^a).\]

Let's take a look at a quick example.

Find the probability generating function of \(Z=2X+3\) where \(X\) has the probability generating function

\[G_X(t)=\frac{2}{5}+\frac{3}{5}t.\]

Solution:

While you could construct a table to find \(G_Z(t)\), it is much easier to use the property discussed above. Then

\[G_Z(t)=t^3G_X(t^2),\]

and so you have

\[\begin{align} G_Z(t)&=t^3G_X(t^2)\\&=t^3\left(\frac{2}{5}+\frac{3}{5}t^2\right)\\ &= \frac{2}{5}t^3+\frac{3}{5}t^5. \end{align}\]

Expectation of the sum of independent random variables

Just like all random variables, sums of independent random variables also have an expectation or mean. You can use the Convolution Theorem and the alternate definition of the probability generating function to find the expectation of the sum of independent random variables, along with the formulas:

  • \(G'_X(1) = E(X)\);

  • \(\text{E}(aX+b) = a\text{E}(X) + b\); and

  • \(\text{E}(X+Y) = \text{E}(X) + \text{E}(Y)\).

For a reminder on where the formulas come from see the article Mean and Variance of Discrete Probability Distributions.

Let's look at an example.

Suppose you know that the independent random variables \(X\) and \(Y\) have probability generating functions

\[G_X(t)=\frac{1}{27}(1+2t)^3\]

and

\[G_Y(t)=\frac{1}{3}+\frac{2}{3}t .\]

Find \(\text{E}(X)\), \(\text{E}(Y)\), and \(\text{E}(X+Y)\).

Solution:

First let's find \(\text{E}(X)\). Taking the derivative,

\[G'_X(t)=\frac{6}{27}+\frac{24}{27}t+\frac{24}{27}t^2,\]

so

\[\begin{align} \text{E}(X) &=G'_X(1) \\ &=\frac{6}{27}+\frac{24}{27}+\frac{24}{27} \\ &=2.\end{align}\]

Similarly for \(\text{E}(Y)\),

\[G'_Y(t)=\frac{2}{3}\]

so

\[\text{E}(Y)=\frac{2}{3}.\]

Then

\[\begin{align} \text{E}(X+Y) &=\text{E}(X)+\text{E}(Y)\\ &= 2 + \frac{2}{3} \\ &=\frac{8}{3}. \end{align}\]

Variance of the sum of independent random variables

Just as you have found the mean above, you can also find the variance of sums of independent random variables. To do this you will need the formulas:

  • \(\text{Var}(aX+b) = a^2\text{Var}(X)\); and
  • \(\text{Var}(Z)= G''_Z(1)+G'_Z(1)-(G'_Z(1))^2\).

Let's look at an example.

Suppose the discrete independent random variables \(X\) and \(Y\) have probability generating functions

\[G_X(t)=0.5+0.5t^2\]

and

\[G_Y(t)=0.1+0.9t^4.\]

Find the variance of \(Z=X+Y\).

Solution:

Given that \(G_Z(t)=G_X(t)G_Y(t)\) from the Convolution Theorem you have:

\[\begin{align} G_Z(t) &= (0.5+0.5t^2) (0.1+0.9t^4) \\&=0.05 +0.45t^4 + 0.05t^2+0.45t^6 .\end{align} \]

Then taking the derivative gives you

\[ G'_Z(t) = 1.8t^3 + 0.1t + 2.7t^5,\]

so

\[\begin{align} \text{E}(Z)&= G'_Z(1) \\&= 1.8 + 0.1 + 2.7 \\&=4.6 .\end{align}\]

To find the variance of \(Z\) you will need the second derivative evaluated at \(t=1\):

\[ G''_Z(t) = 5.4t^2+0.1+13.5t^4,\]

therefore

\[\begin{align} G''_Z(1)&= 5.4+0.1+13.5\\&= 19 .\end{align}\]

That means the variance of \(Z\) is

\[ \begin{align} \text{Var}(Z)&= G''_Z(1)+G'_Z(1)-(G'_Z(1))^2 \\&= 19+4.6-(4.6)^2=2.44 .\end{align}\]

Sum of independent random variables examples

You have already seen some examples of funding the sum of independent random variables, along with their mean and variance. However for specific kinds of distributions, like the binomial distribution and the uniform distribution, looking at them, in particular, can be illuminating. So keep going for the specifics!

Sum of independent binomial random variables

Suppose you have two independent random variables that follow binomial distributions. In other words, \(X \sim \text{Bin}(n_X, p_X)\) and \(Y \sim \text{Bin}(n_Y, p_Y)\). You already know that

\[G_X(t) = (1-p_X+p_Xt)^{n_X}\]

and

\[G_Y(t) = (1-p_Y+p_Yt)^{n_Y}.\]

For more information, see Probability Generating Functions and the Binomial Distribution.

So using the Convolution Theorem, if \(Z = X+Y\) then

\[ \begin{align} G_Z(t) &= G_X(t) G_Y(t) \\ &= (1-p_X+p_Xt)^{n_X} (1-p_Y+p_Yt)^{n_Y}. \end{align}\]

Let's take an example.

What is the probability generating function for the sum of \(X\sim \text{Bin}(5,0.5)\) and \(Y\sim \text{Bin}(15,0.2)\)?

Solution:

Here

\[G_X(t) = (1-0.5+0.5t)^5 = ( 0.5 + 0.5t)^5,\]

and

\[G_Y(t) = (1-0.2+0.2t)^{15} = (0.8+0.2t)^{15} ,\]

so

\[\begin{align}G_{X+Y}(t)&=G_X(t)G_Y(t) \\&=( 0.5 + 0.5t)^5 (0.8+0.2t)^{15} .\end{align} \]

Sum of independent uniform random variables

Remember that a discrete uniform random variable takes on equal probabilities for each possible outcome. So if distribution \(X\) has events occurring with probability \(\dfrac{1}{n}\), then

\[ G_X(t) = \frac{t(1-t^n)}{n(1-t)}.\]

So if \(Y\) is a second discrete uniform random distribution that has events occurring with probability \(\dfrac{1}{m}\), and \(Z = X+Y\), then

\[ \begin{align} G_Z(t) &= G_X(t) G_Y(t) \\ &= \left(\frac{t(1-t^n)}{n(1-t)} \right)\left(\frac{t(1-t^m)}{m(1-t)}\right) \\ &= \frac{t^2(1-t^n)(1-t)^m}{nm(1-t)^2} .\end{align}\]

Let's take an example.

Suppose you have two \(4\)-sided dice:

  • die \(X\) has faces reading \(1\), \(2\), \(3\), and \(4\); and
  • die \(Y\) has faces reading \(1\), \(1\), \(2\), and \(2\).

Find the probability generating function of \(Z=X+Y\).

Solution:

For die \(X\), \(n=4\) so

\[G_X(t)=\frac{t(1-t^4)}{4(1-t)},\]

and for die \(Y\) you have \(m=2\), so

\[G_Y(t)=\frac{t(1-t^2)}{2(1-t)}.\]

Then

\[ \begin{align} G_Z(t)&=G_X(t)G_Y(t) \\ &= \left(\frac{t(1-t^4)}{4(1-t)} \right)\left(\frac{t(1-t^2)}{2(1-t)} \right) \\ &= \frac{t^2(1-t^4)(1-t^2)}{(4)(2)(1-t)^2} \\ &= \frac{t^2}{8}(1+t)^2(1+t^2). \end{align}\]

It actually isn't necessary to memorise the different formulas for the various kinds of discrete probability distributions as long as you keep in mind the Convolution Theorem!

Sum of Independent Random Variables - Key takeaways

  • Suppose that discrete independent random variables \(X\) and \(Y\) have probability generating functions \(G_X(t)\) and \(G_Y(t)\). The probability generating function of \(Z\) (where \(Z=X+Y\)) is \(G_Z(t)=G_X(t)G_Y(t)\).
  • If \(X\) is a discrete random variable and has a probability generating function of \(G_X(t)\), the probability generating function of \(Z\) where \(Z=aX+b\) is \(G_Z(t)=t^bG_X(t^a)\)
  • To find the variance of \(Z=aX+b\) where \(X\) is a discrete random probability distribution, remember that:
    • \(\text{Var}(aX+b) = a^2\text{Var}(X)\); and
    • \(\text{Var}(Z)= G''_Z(1)+G'_Z(1)-(G'_Z(1))^2\).
  • To find the mean of the sum of discrete independent random variables remember the formulas:
    • \(G'_X(1) = E(X)\);

    • \(\text{E}(aX+b) = a\text{E}(X) + b\); and

    • \(\text{E}(X+Y) = \text{E}(X) + \text{E}(Y)\).

Frequently Asked Questions about Sum of Independent Random Variables

To find the mean and/or variance of the sum of independent random variables, first find the probability generating function of the sum of the random variables and derive the mean/variance as normal.

You can find this by working from first principles or by multiplying together the probability generating functions of the random variables.

You can find the distribution of the sum of independent random variables by working from first principles or by multiplying together the probability generating functions of the random variables.

Yes, the sum of two independent random variables is  independent.

You can find the distribution of the sum of independent random variables by working from first principles or by multiplying together the probability generating functions of the random variables.

You can find the distribution of the sum of independent random variables by working from first principles or by multiplying together the probability generating functions of the random variables.

Final Sum of Independent Random Variables Quiz

Question

If discrete random variables \(X\) and \(Y\) have probability generating functions \(G_X(t)\) and \(G_Y(t)\), what is the probability generating function of \(Z\)?

Show answer

Answer

\(G_Z(t)=G_X(t)G_Y(t)\).

Show question

Question

If \(X\) is a discrete random variable and has a probability generating function of \(G_X(t)\), what is the probability generating function of \(Y\) where \(Y=aX+b\)?

Show answer

Answer

\(G_Y(t)=t^bG_X(t^a)\).

Show question

Question

If \(X \sim \text{Po}(\lambda)\), what is the probability generating function of \(2X\)?

Show answer

Answer

\(G_X(t)=e^{2\lambda (t-1)}\).

Show question

Question

If \(X \sim \text{Geo}(p)\), what is the probability generation function of \(2X\)?

Show answer

Answer

\((G_X)^2\).

Show question

Question

If \(X\sim \text{Bin}(n,p)\) and \(Y\sim \text{Bin}(2n,p)\), what is the probability generating function of \(Z=X+Y\)?

Show answer

Answer

\(G_Z(t)=(1-p+pt)^{3n}\).

Show question

Question

If \(X\sim \text{Pois}(\lambda)\), what is the probability generating function of \(Y=aX+b\)?

Show answer

Answer

\(G_Y(t)=t^b  e^{\lambda t^a -1}\).

Show question

Question

If \(X\) and \(Y\) are independent discrete random variables, and \(Z=X+Y\), what can you use to find the probability generating function \(G_Z(t)\)?

Show answer

Answer

The Convolution Theorem.

Show question

Question

Say you have \(X\sim \text{Geo}(p)\) and \(Y\sim \text{Geo}(q)\). What is the probability generating function of \(Z=X+Y\)?

Show answer

Answer

\(G_Z(t) = G_X(t)+G_Y(t)\).

Show question

Question

How do you find the mean of a distribution, \(\text{E}(X)\) from the probability generating function for \(X\)?

Show answer

Answer

\(G'_X(1)\).

Show question

Question

Which of the following formulas do you use to find the variance of a probability distribution from the probability generating function?

Show answer

Answer

\(G''_X(1)+G'_X(1)-(G'_X(1))^2\).

Show question

Question

If \(X\) is a discrete random variable and has a probability generating function of \(G_X(t)\), the probability generating function of \(Z\) where \(Z=aX+b\) is ____.

Show answer

Answer

\(G_Z(t)=t^bG_X(t^a)\).

Show question

Question

To use the Convolution Theorem, what needs to be true about \(X\) and \(Y\)?

Show answer

Answer

The both need to be discrete random probability distributions.

Show question

Question

What is \(\text{E}(X^2)\) in terms of the probability generating function of \(X\) if \(X\) is a discrete random probability distribution?

Show answer

Answer

\(\text{E}(X^2)=G''_X(1)+G'_X(1)\).

Show question

Question

If \(Z=X+Y\) where \(X\) and \(Y\) are independent random probability distributions, what is the expectation of \(Z\) in terms of \(X\) and \(Y\)?

Show answer

Answer

\(\text{E}(Z)=\text{E}(X)+\text{E}(Y)\).

Show question

Question

If \(Y=5X+3\), what is the probability generating function of \(Y\)?

Show answer

Answer

\(G_Y(t)=t^3 G_X(t^5)\).

Show question

More about Sum of Independent Random Variables
60%

of the users don't pass the Sum of Independent Random Variables quiz! Will you pass the quiz?

Start Quiz

Discover the right content for your subjects

No need to cheat if you have everything you need to succeed! Packed into one app!

Study Plan

Be perfectly prepared on time with an individual plan.

Quizzes

Test your knowledge with gamified quizzes.

Flashcards

Create and find flashcards in record time.

Notes

Create beautiful notes faster than ever before.

Study Sets

Have all your study materials in one place.

Documents

Upload unlimited documents and save them online.

Study Analytics

Identify your study strength and weaknesses.

Weekly Goals

Set individual study goals and earn points reaching them.

Smart Reminders

Stop procrastinating with our study reminders.

Rewards

Earn points, unlock badges and level up while studying.

Magic Marker

Create flashcards in notes completely automatically.

Smart Formatting

Create the most beautiful study materials using our templates.

Sign up to highlight and take notes. It’s 100% free.