Normal-inverse-gamma distribution

Template:Probability distribution In probability theory and statistics, the normal-inverse-gamma distribution (or Gaussian-inverse-gamma distribution) is a four-parameter family of multivariate continuous probability distributions. It is the conjugate prior of a normal distribution with unknown mean and variance.

Definition

Suppose

$x\mid {\sigma }^2,\mu ,\lambda \sim \mathrm{N}(\mu ,{\sigma }^2/\lambda )$

has a normal distribution with mean $\mu$ and variance ${\sigma }^2/\lambda$, where

${\sigma }^2\mid \alpha ,\beta \sim {\mathrm{\Gamma }}^{-1}(\alpha ,\beta )$

has an inverse-gamma distribution. Then $(x,{\sigma }^2)$ has a normal-inverse-gamma distribution, denoted as

$(x,{\sigma }^2)\sim \text{N-}{\mathrm{\Gamma }}^{-1}(\mu ,\lambda ,\alpha ,\beta ).$

($\text{NIG}$ is also used instead of $\text{N-}{\mathrm{\Gamma }}^{-1}.$)

The normal-inverse-Wishart distribution is a generalization of the normal-inverse-gamma distribution that is defined over multivariate random variables.

Characterization

Probability density function

$f(x,{\sigma }^2\mid \mu ,\lambda ,\alpha ,\beta )=\frac{\sqrt{\lambda }}{\sigma \sqrt{2\pi }}\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{\left(\frac{1}{{\sigma }^2}\right)}^{\alpha +1}\exp (-\frac{2\beta +\lambda (x-\mu {)}^2}{2{\sigma }^2})$

For the multivariate form where $𝐱$ is a $k\times 1$ random vector,

$f(𝐱,{\sigma }^2\mid \mu ,{𝐕}^{-1},\alpha ,\beta )=|𝐕{|}^{-1/2}(2\pi {)}^{-k/2}\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{\left(\frac{1}{{\sigma }^2}\right)}^{\alpha +1+k/2}\exp (-\frac{2\beta +(𝐱-\mathit{\mu }{)}^T{𝐕}^{-1}(𝐱-\mathit{\mu })}{2{\sigma }^2}).$

where $|𝐕|$ is the determinant of the $k\times k$ matrix $𝐕$. Note how this last equation reduces to the first form if $k=1$ so that $𝐱,𝐕,\mathit{\mu }$ are scalars.

Alternative parameterization

It is also possible to let $\gamma =1/\lambda$ in which case the pdf becomes

$f(x,{\sigma }^2\mid \mu ,\gamma ,\alpha ,\beta )=\frac{1}{\sigma \sqrt{2\pi \gamma }}\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{\left(\frac{1}{{\sigma }^2}\right)}^{\alpha +1}\exp (-\frac{2\gamma \beta +(x-\mu {)}^2}{2\gamma {\sigma }^2})$

In the multivariate form, the corresponding change would be to regard the covariance matrix $𝐕$ instead of its inverse ${𝐕}^{-1}$ as a parameter.

Cumulative distribution function

$F(x,{\sigma }^2\mid \mu ,\lambda ,\alpha ,\beta )=\frac{e^{-\frac{\beta }{{\sigma }^2}}{\left(\frac{\beta }{{\sigma }^2}\right)}^{\alpha }(\mathrm{erf}\left(\frac{\sqrt{\lambda }(x-\mu )}{\sqrt{2}\sigma }\right)+1)}{2{\sigma }^2\mathrm{\Gamma }(\alpha )}$

Properties

Marginal distributions

Given $(x,{\sigma }^2)\sim \text{N-}{\mathrm{\Gamma }}^{-1}(\mu ,\lambda ,\alpha ,\beta ).$ as above, ${\sigma }^2$ by itself follows an inverse gamma distribution:

${\sigma }^2\sim {\mathrm{\Gamma }}^{-1}(\alpha ,\beta )$

while $\sqrt{\frac{\alpha \lambda }{\beta }}(x-\mu )$ follows a t distribution with $2\alpha$ degrees of freedom.

In the multivariate case, the marginal distribution of $𝐱$ is a multivariate t distribution:

$𝐱\sim t_{2\alpha }(\mathit{\mu },\frac{\beta }{\alpha }𝐕)$

Scaling

Suppose

$(x,{\sigma }^2)\sim \text{N-}{\mathrm{\Gamma }}^{-1}(\mu ,\lambda ,\alpha ,\beta ).$

Then for $c>0$,

$(cx,c{\sigma }^2)\sim \text{N-}{\mathrm{\Gamma }}^{-1}(c\mu ,\lambda /c,\alpha ,c\beta ).$

Proof: To prove this let $(x,{\sigma }^2)\sim \text{N-}{\mathrm{\Gamma }}^{-1}(\mu ,\lambda ,\alpha ,\beta )$ and fix $c>0$. Defining $Y=(Y_1,Y_2)=(cx,c{\sigma }^2)$, observe that the PDF of the random variable $Y$ evaluated at $(y_1,y_2)$ is given by $1/c^2$ times the PDF of a $\text{N-}{\mathrm{\Gamma }}^{-1}(\mu ,\lambda ,\alpha ,\beta )$ random variable evaluated at $(y_1/c,y_2/c)$. Hence the PDF of $Y$ evaluated at $(y_1,y_2)$ is given by :$f_Y(y_1,y_2)=\frac{1}{c^2}\frac{\sqrt{\lambda }}{\sqrt{2\pi y_2/c}}\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{\left(\frac{1}{y_2/c}\right)}^{\alpha +1}\exp (-\frac{2\beta +\lambda (y_1/c-\mu {)}^2}{2y_2/c})=\frac{\sqrt{\lambda /c}}{\sqrt{2\pi y_2}}\frac{(c\beta {)}^{\alpha }}{\mathrm{\Gamma }(\alpha )}{\left(\frac{1}{y_2}\right)}^{\alpha +1}\exp (-\frac{2c\beta +(\lambda /c)(y_1-c\mu {)}^2}{2y_2}).$

The right hand expression is the PDF for a $\text{N-}{\mathrm{\Gamma }}^{-1}(c\mu ,\lambda /c,\alpha ,c\beta )$ random variable evaluated at $(y_1,y_2)$, which completes the proof.

Exponential family

Normal-inverse-gamma distributions form an exponential family with natural parameters ${\theta }_1=\frac{-\lambda }{2}$, ${\theta }_2=\lambda \mu$, ${\theta }_3=\alpha$, and ${\theta }_4=-\beta +\frac{-\lambda {\mu }^2}{2}$ and sufficient statistics $T_1=\frac{x^2}{{\sigma }^2}$, $T_2=\frac{x}{{\sigma }^2}$, $T_3=\log \left(\frac{1}{{\sigma }^2}\right)$, and $T_4=\frac{1}{{\sigma }^2}$.

Kullback–Leibler divergence

Measures difference between two distributions.

Posterior distribution of the parameters

See the articles on normal-gamma distribution and conjugate prior.

Interpretation of the parameters

See the articles on normal-gamma distribution and conjugate prior.

Generating normal-inverse-gamma random variates

Generation of random variates is straightforward:

1. Sample ${\sigma }^2$ from an inverse gamma distribution with parameters $\alpha$ and $\beta$
2. Sample $x$ from a normal distribution with mean $\mu$ and variance ${\sigma }^2/\lambda$

Related distributions

• The normal-gamma distribution is the same distribution parameterized by precision rather than variance
• A generalization of this distribution which allows for a multivariate mean and a completely unknown positive-definite covariance matrix ${\sigma }^2𝐕$ (whereas in the multivariate inverse-gamma distribution the covariance matrix is regarded as known up to the scale factor ${\sigma }^2$) is the normal-inverse-Wishart distribution

See also

• Compound probability distribution

References

• Denison, David G. T.; Holmes, Christopher C.; Mallick, Bani K.; Smith, Adrian F. M. (2002) Bayesian Methods for Nonlinear Classification and Regression, Wiley. Template:ISBN
• Koch, Karl-Rudolf (2007) Introduction to Bayesian Statistics (2nd Edition), Springer. Template:ISBN