Multinomial distribution

From Wikipedia, the free encyclopedia

In probability theory, the multinomial distribution is a generalization of the binomial distribution.

The binomial distribution is the probability distribution of the number of "successes" in n independent Bernoulli trials, with the same probability of "success" on each trial. In a multinomial distribution, each trial results in exactly one of some fixed finite number k of possible outcomes, with probabilities p₁, ..., p_k (so that p_i ≥ 0 for i = 1, ..., k and $\sum_{i=1}^k p_i = 1$ ), and there are n independent trials. Then let the random variables $X i$ indicates the number of times outcome number i was observed over the n trials. $X=(X_1,\ldots,X_k)$ follows a multinomial distribution with parameters n and p.

1 Specification
- 1.1 Probability mass function
2 Properties
3 Related distributions
4 See also

[edit] Specification

[edit] Probability mass function

The probability mass function of the multinomial distribution is:

$f(x_1,\ldots,x_k;n,p_1,\ldots,p_k) = \begin{cases}{n! \over x_1!\cdots x_k!}p_1^{x_1}\cdots p_k^{x_k} \quad & \mbox{when } \sum_{i=1}^k x_i=n \\ 0 & \mbox{otherwise.} \end{cases}$

for non-negative integers x₁, ..., x_k.

[edit] Properties

The expected value is

$\operatorname{E}(X_i) = n p_i.$

The covariance matrix is as follows. Each diagonal entry is the variance of a binomially distributed random variable, and is therefore

$\operatorname{var}(X_i)=np_i(1-p_i).$

The off-diagonal entries are the covariances:

$\operatorname{cov}(X_i,X_j)=-np_i p_j$

for i, j distinct.

All covariances are negative because for fixed N, an increase in one component of a multinomial vector requires a decrease in another component.

This is a k × k nonnegative-definite matrix of rank k − 1.

The off-diagonal entries of the corresponding correlation matrix are

$\rho(X_i,X_j) = -\sqrt{\frac{p_i p_j}{ (1-p_i)(1-p_j)}}.$

Note that the sample size drops out of this expression.

Each of the k components separately has a binomial distribution with parameters n and p_i, for the appropriate value of the subscript i.

The support of the multinomial distribution is the set : $\{(n_1,...,n_k)\in \mathbb{N}^{k}| n_1+...+n_k=n\}.$ Its number of elements is : ${n+k-1 \choose k-1}.$

[edit] Related distributions

When k=2, the multinomial distribution is the binomial distribution.
The Dirichlet distribution is the conjugate prior of the multinomial in Bayesian statistics.

[edit] See also

Multinomial theorem

	Probability distributions [ view • talk • edit ]
	Univariate	Multivariate
Discrete:	Benford • Bernoulli • binomial • Boltzmann • categorical • compound Poisson • degenerate • Gauss-Kuzmin • geometric • hypergeometric • logarithmic • negative binomial • parabolic fractal • Poisson • Rademacher • Skellam • uniform • Yule-Simon • zeta • Zipf • Zipf-Mandelbrot	Ewens • multinomial • multivariate Polya
Continuous:	Beta • Beta prime • Cauchy • chi-square • Dirac delta function • Erlang • exponential • exponential power • F • fading • Fisher's z • Fisher-Tippett • Gamma • generalized extreme value • generalized hyperbolic • generalized inverse Gaussian • Half-Logistic • Hotelling's T-square • hyperbolic secant • hyper-exponential • hypoexponential • inverse chi-square (scaled inverse chi-square)• inverse Gaussian • inverse gamma (scaled inverse gamma) • Kumaraswamy • Landau • Laplace • Lévy • Lévy skew alpha-stable • logistic • log-normal • Maxwell-Boltzmann • Maxwell speed • normal (Gaussian) • normal inverse Gaussian • Pareto • Pearson • polar • raised cosine • Rayleigh • relativistic Breit-Wigner • Rice • shifted Gompertz • Student's t • triangular • type-1 Gumbel • type-2 Gumbel • uniform • Variance-Gamma • Voigt • von Mises • Weibull • Wigner semicircle • Wilks' lambda	Dirichlet • inverse-Wishart • Kent • matrix normal • multivariate normal • multivariate Student • von Mises-Fisher • Wigner quasi • Wishart
Miscellaneous:	Cantor • conditional • exponential family • infinitely divisible • location-scale family • marginal • maximum entropy • phase-type • posterior • prior • quasi • sampling • singular