Normal-Wishart distribution

Template:Probability distribution In probability theory and statistics, the normal-Wishart distribution (or Gaussian-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and precision matrix (the inverse of the covariance matrix).

Definition

Suppose

$\mathit{\mu }|{\mathit{\mu }}_0,\lambda ,\mathrm{\Lambda }\sim 𝒩({\mathit{\mu }}_0,(\lambda \mathrm{\Lambda }{)}^{-1})$

has a multivariate normal distribution with mean ${\mathit{\mu }}_0$ and covariance matrix $(\lambda \mathrm{\Lambda }{)}^{-1}$, where

$\mathrm{\Lambda }|𝐖,\nu \sim 𝒲(\mathrm{\Lambda }|𝐖,\nu )$

has a Wishart distribution. Then $(\mathit{\mu },\mathrm{\Lambda })$ has a normal-Wishart distribution, denoted as

$(\mathit{\mu },\mathrm{\Lambda })\sim \mathrm{N}\mathrm{W}({\mathit{\mu }}_0,\lambda ,𝐖,\nu ).$

Characterization

Probability density function

$f(\mathit{\mu },\mathrm{\Lambda }|{\mathit{\mu }}_0,\lambda ,𝐖,\nu )=𝒩(\mathit{\mu }|{\mathit{\mu }}_0,(\lambda \mathrm{\Lambda }{)}^{-1})𝒲(\mathrm{\Lambda }|𝐖,\nu )$

Properties

Marginal distributions

By construction, the marginal distribution over $\mathrm{\Lambda }$ is a Wishart distribution, and the conditional distribution over $\mathit{\mu }$ given $\mathrm{\Lambda }$ is a multivariate normal distribution. The marginal distribution over $\mathit{\mu }$ is a multivariate t-distribution.

Posterior distribution of the parameters

After making $n$ observations ${𝒙}_1,\dots ,{𝒙}_n$, the posterior distribution of the parameters is

$(\mathit{\mu },\mathrm{\Lambda })\sim \mathrm{N}\mathrm{W}({\mathit{\mu }}_n,{\lambda }_n,{𝐖}_n,{\nu }_n),$

where

${\lambda }_n=\lambda +n,$

${\mathit{\mu }}_n=\frac{\lambda {\mathit{\mu }}_0+n\overline{𝒙}}{\lambda +n},$

${\nu }_n=\nu +n,$

${𝐖}_n^{-1}={𝐖}^{-1}+\sum _{i=1}^n({𝒙}_i-\overline{𝒙})({𝒙}_i-\overline{𝒙}{)}^T+\frac{n\lambda }{n+\lambda }(\overline{𝒙}-{\mathit{\mu }}_0)(\overline{𝒙}-{\mathit{\mu }}_0{)}^T.$

Generating normal-Wishart random variates

Generation of random variates is straightforward:

1. Sample $\mathrm{\Lambda }$ from a Wishart distribution with parameters $𝐖$ and $\nu$
2. Sample $\mathit{\mu }$ from a multivariate normal distribution with mean ${\mathit{\mu }}_0$ and variance $(\lambda \mathrm{\Lambda }{)}^{-1}$

Related distributions

• The normal-inverse Wishart distribution is essentially the same distribution parameterized by variance rather than precision.
• The normal-gamma distribution is the one-dimensional equivalent.
• The multivariate normal distribution and Wishart distribution are the component distributions out of which this distribution is made.

References

• Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media.