29  Conditional Variance

Conditional expectations can also be used to compute the variance of a random variable. \[\text{Var}(X) = \text E[X^2] - (\text E[X])^2\] and then use conditioning to obtain both \(\text E[X]\) and \(\text E[X^2]\).

Example: Independent trials, each resulting in a success with probability \(p\), are performed in sequence. Let \(N\) be the trial number of the first success. Find \(\text{Var}(N)\).

Solution: Let \(Y = 1\) if the first trial results in a success, and \(Y = 0\) otherwise. Conditioning \(\text E[N^2]\) and \(\text E[N]\) on \(Y\), we have: \[\text E[N^2] = \text E[\text E[N^2|Y]]\] However, \[\text E[\text E[N^2|Y=1]] = 1, \qquad \text E[\text E[N^2|Y=0]] = \text E[(N+1)^2]\] If the first trial results in a success, then clearly \(N = 1\) and so \(N^2 = 1\). On the other hand, if the first trial results in a failure, then the total number of trials necessary for the first success will equal one (the first trial that results in failure) plus the necessary number of additional trials. Since this latter quantity has the same distribution as \(N\), we get that \(\text E[N^2|Y = 0] = \text E[(N+1)^2]\). Hence, we see that \[\begin{align}\text E[N^2] &= \text E[\text E[N^2|Y=0]]P\{Y=0\}+ \text E[\text E[N^2|Y=1]]P\{Y=1\} \\ &= p + \text E[N^2+2N+1](1-p) \\ &= 1 + (1-p) \text E[N^2+2N]\end{align}\] Since \(\text E[N] = 1/p\), this yields \[\text E[N^2] = 1 + \frac{2(1-p)}{p} + (1-p)\text E[N^2] \implies \text E[N^2] = \frac{2-p}{p}\] Therefore, \[\text{Var}(N) = \text E[N^2] - (\text E[N])^2 = \frac{2-p}{p} - \frac{1}{p^2} = \frac{1-p}{p^2}\]

Proposition: Letting \(\text{Var}(X|Y)\) denote that function of \(Y\) whose value when \(Y = y\) is \(\text{Var}(X|Y = y)\), we have \[\text{Var}(X) = \text E\Big[\text{Var}(X|Y)\Big] + \text{Var}\Big(\text E[X|Y]\Big)\]

Example: Let \(X_1, X_2,...\) be independent and identically distributed random variables with mean \(\mu\) and variance \(\sigma^2\), and assume that they are independent of the nonnegative integer valued random variable \(N\). The random variable \(S = \sum_{i=1}^N X_i\) is called a compound random variable. Compute \(\text{Var}(S)\).

Solution: We have \[\text{Var}(S|N=n) = \text{Var}\bigg(\sum_{i=1}^n X_i | N=n\bigg) = \text{Var}\bigg(\sum_{i=1}^n X_i\bigg) = n\sigma^2\] By the same reasoning, \[\text E[S|N=n] = n\mu\] Therefore, \[\begin{align}\text{Var}(S|N) &= N\sigma^2 \\ \text E[S|N] &= N\mu\end{align}\] and the conditional variance formula gives \[\text{Var}(N) = \sigma^2 \text E[N] + \mu^2 Var(N)\]