31  Limit Theorems

31.1 Important Inequalities

Proposition (Markov’s Inequality): If \(X\) is a random variable that takes only nonnegative values, then for any value \(a > 0\): \[P\{X\ge a\} \le \frac{\text E[X]}{a}\] As a corollary, we obtain the following.

Proposition (Chebyshev’s Inequality): If \(X\) is a random variable with mean \(\mu\) and variance \(\sigma^2\), then for any value \(k > 0\): \[P\{|X-\mu| \ge k\}\le \frac{\sigma^2}{k^2}\]

Example: Suppose we know that the number of items produced in a factory during a week is a random variable with mean 500. * What can be said about the probability that this week’s production will be at least 1000? * If the variance of a week’s production is known to equal 100, then what can be said about the probability that this week’s production will be between 400 and 600?

Solution: Let \(X\) be the number of items that will be produced in a week. * By Markov’s Inequality, \[P\{X \ge 1000\} \le \frac{\text E[X]}{1000} = \frac{1}{2}\] * By Chebyshev’s Inequality, \[P\{|X-500| \ge 100\} \le \frac{\sigma^2}{100^2} = \frac{1}{100} \implies P\{|X-500|<100\} \ge 0.99\] ## Strong Law of Large Number Let \(X_1, X_2, ...\) be a sequence of independent, identically distributed random variables, and let \(\text E[X_i] = \mu\). Then, with probability 1 (a.s), \[\frac{X_1 + ... + X_n}{n}\to \mu \quad \text{as} \quad n\to\infty\]

If the summands \(X_1, ..., X_n\) are not identically distributed, then with probability 1, \[\frac{X_1 + ... + X_n - \text E[\overline X_n ]}{n} \to 0 \quad \text{as} \quad n \to \infty\] provided \[ \text E[X_i^2] < \infty \text{ and } \sum_{i=1}^\infty \frac{\text{Var}(X_k)}{k^2} < \infty\] ## Central Limit Theorem Let \(X_1, X_2, ...\) be a sequence of independent, identically distributed random variables, each with mean \(\mu\) and variance \(σ^2\). Then \[P\bigg\{\frac{X_1+X_2...+X_n - n\mu}{\sigma \sqrt{n}} \le a\bigg\} \to \frac{1}{\sqrt{2\pi}}\int_{-\infty}^a e^{-x^2}{2}\] Note that this theorem holds for any distribution of the \(X_i\)’s.

Example: The lifetime of a special type of battery is a random variable with mean 40 hours and standard deviation 20 hours. A battery is used until it fails, at which point it is replaced by a new one. Assuming a stockpile of 25 such batteries, the lifetimes of which are independent, approximate the probability that over 1100 hours of use can be obtained.

Solution: If we let \(X_i\) denote the lifetime of the i-th battery to be put in use, then we want to calculate \[\begin{align}P\{X_1 + · · · + X_{25} > 1100\} &= P\bigg\{\frac{(\sum_{i=1}^{25} X_i) - 1000}{20\sqrt{25}} > \frac{1100-1000}{20\sqrt{25}}\bigg\} \\ &\approx P\{N(0,1) > 1\}\\&\approx 0.1587\end{align}\]