25  Beta and Dirichlet Distribution

25.1 Beta Distribution

We introduce the Beta function: \[B(\alpha, \beta) = \int_0^1 x^{\alpha - 1}(1-x)^{\beta - 1} dt\] Proposition: \[B(\alpha, \beta) = \frac{\Gamma(\alpha + \beta)}{\Gamma(\alpha) \Gamma(\beta)}\] The Beta distribution is a family of continuous distributions defined on the interval \([0, 1]\) in terms of parameters \(\alpha,\beta > 0\) controlling the shape of the distribution. Its pdf is: \[f(x) = \frac{1}{B(\alpha, \beta)}x^{\alpha-1}(1-x)^{\beta-1}, \qquad 0\le x \le 1\] and its cdf is \(F(x) = \frac{B(x, \alpha, \beta)}{B(\alpha, \beta)}\) where \(B(x, \alpha, \beta)\) is the incomplete Beta function.

Properties: * \(\text E[X] = \frac{a}{a+b}\) * \(\text{Var}(X) = \frac{ab}{(a+b)^2(a+b+1)}\) ## Dirichlet Distribution We define the Multivariate Beta function: \[B(\alpha_1, ..., \alpha_K) = \frac{\Gamma\bigg(\sum_{k=1}^K \alpha_k\bigg)}{\prod_{k=1}^K \Gamma(\alpha_k)} \] The Dirichlet distribution is a generalization of the Beta distribution, with parameter \(\boldsymbol \alpha = (\alpha_1, ..., \alpha_K)\). Its pdf is: \[f(x_1, ..., x_K) = \frac{1}{B(\boldsymbol \alpha)} \prod_{k=1}^K x_k^{\alpha_{k-1}},\hskip 4em x\ge0,\sum_{k=1}^K x_k = 1\]

Properties: Denote \(\alpha_0 = \textbf 1^T \alpha\). We have * \(\text E[X_i] = \frac{\alpha_i}{\alpha_0}\) * \(\text{Var}(X_i) = \frac{\alpha_i(\alpha_0 - \alpha_i)}{\alpha_0^2(\alpha_0+1)}\) * \(\text{Cov}(X_i, X_j) = \frac{-\alpha_i\alpha_j}{\alpha_0^2(\alpha_0+1)}\) if \(i \ne j\).

The Beta and the Dirichlet distributions are important due to the fact that they are conjugate priors of important distributions, with the Beta being the conjugate priors of the Bernoulli, binomial, negative binomial and geometric, and the Dirichlet being the conjugate priors of the categorical and multinomial distribution.