View source for Logit-normal distribution

{{Short description|Probability distribution}}
{{Probability distribution
  | name      = Logit-normal
  | type      = continuous
  | pdf_image = [[Image:LogitnormalPDF.svg|325px|Plot of the Logitnormal PDF]]
  | cdf_image = [[Image:LogitnormCdfGrid2.svg|325px|Plot of the Logitnormal PDF]]
  | notation  = <math>P( \mathcal{N}(\mu,\,\sigma^2) )</math>
  | parameters= ''σ''<sup>2</sup> > 0 — squared scale (real),<br /> ''μ'' ∈ '''R''' — location
  | support   = ''x'' ∈ (0, 1)
  | pdf       = <math>\frac{1}{\sigma \sqrt{2 \pi}}\, e^{-\frac{(\operatorname{logit}(x) - \mu)^2}{2\sigma^2}}\frac{1}{x (1-x)}</math>
  | cdf       = <math>\frac12\Big[1 + \operatorname{erf}\Big( \frac{\operatorname{logit}(x)-\mu}{\sqrt{2\sigma^2}}\Big)\Big]</math>
  | mean      = no analytical solution
  | median    = <math>P(\mu)\,</math>
  | mode      = no analytical solution
  | variance  = no analytical solution
  | skewness  = 
  | kurtosis  = 
  | entropy   = 
  | mgf       = no analytical solution
  | char      = 
  | fisher    = 
  }}

In [[probability theory]], a '''logit-normal distribution''' is a [[probability distribution]] of a [[random variable]] whose [[logit]] has a [[normal distribution]].  If ''Y'' is a random variable with a normal distribution, and ''t'' is the standard [[logistic function]], then ''X''&nbsp;=&nbsp;''t''(''Y'') has a logit-normal distribution; likewise, if ''X'' is logit-normally distributed, then ''Y''&nbsp;=&nbsp;[[logit]](''X'')= log (''X''/(1-''X'')) is normally distributed.  It is also known as the '''logistic normal distribution''',<ref name="AitchisonShen1980">{{cite journal|last1=Aitchison|first1=J.|last2=Shen|first2=S. M.|title=Logistic-normal distributions: Some properties and uses |journal=Biometrika |volume=67|issue=2|year=1980|pages=261 |issn=0006-3444| doi=10.2307/2335470|jstor=2335470}}</ref> which often refers to a multinomial logit version (e.g.<ref>http://people.csail.mit.edu/tomasz/papers/huang_hln_tech_report_2006.pdf {{Bare URL PDF|date=March 2022}}</ref><ref>Peter Hoff, 2003.  [https://www.stat.washington.edu/research/tech-reports/nonparametric-modelling-hierarchically-exchangeable-data Link]</ref><ref>{{cite web|url=http://brenocon.com/blog/2011/05/log-normal-and-logistic-normal-terminology/|title=Log-normal and logistic-normal terminology - AI and Social Science – Brendan O'Connor|website=brenocon.com|access-date=18 April 2018}}</ref>).

A variable might be modeled as logit-normal if it is a proportion, which is bounded by zero and one, and where values of zero and one never occur.

== Characterization ==

=== Probability density function ===
The [[probability density function]] (PDF) of a logit-normal distribution, for 0 < ''x'' < 1, is:

: <math>f_X(x;\mu,\sigma) = \frac{1}{\sigma\sqrt{2 \pi}}\,\frac{1}{x (1-x)}\,
e^{-\frac{(\operatorname{logit}(x) - \mu)^2}{2\sigma^2}}</math>

where ''μ'' and ''σ'' are the [[mean]] and [[standard deviation]] of the variable’s [[logit]] (by definition, the variable’s logit is normally distributed).

The density obtained by changing the sign of ''μ'' is symmetrical, in that it is equal to f(1-x;-''μ'',''σ''), shifting the mode to the other side of 0.5 (the midpoint of the (0,1) interval).

[[Image:LogitnormDensityGrid.svg|thumb|left|420px|Plot of the Logitnormal PDF for various combinations of ''μ'' (facets) and ''σ'' (colors)]]{{clear}}

=== Moments ===
The moments of the logit-normal distribution have no analytic solution. The moments can  be estimated by [[numerical integration]], however numerical integration can be prohibitive when the values of <math display="inline">\mu, \sigma^2</math>are such that the density function diverges to infinity at the end points zero and one. An alternative is to use the observation that the logit-normal is a transformation of a normal random variable. This allows us to approximate the <math>n</math>-th moment via the following quasi Monte Carlo estimate <math>E[X^n] \approx \frac{1}{K-1} \sum_{i=1}^{K-1} \left( P\left(\Phi^{-1}_{\mu,\sigma^2}(i/K)\right) \right)^n,</math>

where <math display="inline">P</math> is the standard logistic function, and <math display="inline">\Phi^{-1}_{\mu,\sigma^2}</math> is the inverse cumulative distribution function of a normal distribution with mean and variance <math display="inline">\mu, \sigma^2</math>. When <math>n=1</math>, this corresponds to the mean.

=== Mode or modes ===
When the derivative of the density equals 0 then the location of the mode x satisfies the following equation:

:<math>\operatorname{logit}(x) = \sigma^2(2x-1)+\mu .</math>
For some values of the parameters there are two solutions, i.e. the distribution is [[bimodal]].

== Multivariate generalization==

The '''logistic normal distribution''' is a generalization of the logit–normal distribution to D-dimensional probability vectors by taking a logistic transformation of a multivariate normal distribution.<ref name="AitchisonShen1980">{{cite journal|last1=Aitchison|first1=J.|last2=Shen|first2=S. M.|title=Logistic-normal distributions: Some properties and uses |journal=Biometrika |volume=67|issue=2|year=1980|pages=261 |issn=0006-3444| doi=10.2307/2335470|jstor=2335470}}</ref><ref name="stat_anal_comp_data">J. Atchison. "The Statistical Analysis of Compositional Data."  Monographs on Statistics and Applied Probability, Chapman and Hall, 1986. [https://www.amazon.com/The-Statistical-Analysis-Compositional-Data/dp/1930665784 Book]</ref><ref name="CometsRosalsky2011">{{cite book|editor1-last=Lovric|editor1-first=Miodrag|last=Hinde|first=John |author-link=John Hinde (statistician)|chapter=Logistic Normal Distribution |title=International Encyclopedia of Statistical Sciences| year=2011|pages=754–755 |doi=10.1007/978-3-642-04898-2_342|publisher=Springer|isbn=978-3-642-04897-5}}</ref>

=== Probability density function ===

The [[probability density function]] is:

: <math>f_X( \mathbf{x}; \boldsymbol{\mu} , \boldsymbol{\Sigma} ) = \frac{1}{ | (2 \pi)^{D-1} \boldsymbol{\Sigma} |^\frac{1}{2} } \, \frac{1}{ \prod\limits_{i=1}^D x_i } \, e^{- \frac{1}{2} \left\{ \log \left( \frac{ \mathbf{x}_{-D} }{ x_D } \right) - \boldsymbol{\mu} \right\}^\top \boldsymbol{\Sigma}^{-1} \left\{ \log \left( \frac{ \mathbf{x}_{-D} }{ x_D } \right) - \boldsymbol{\mu} \right\} } \quad ,  \quad \mathbf{x} \in \mathcal{S}^D \;\; , </math>

where <math>\mathbf{x}_{-D}</math> denotes a vector of the first (D-1) components of <math>\mathbf{x}</math> and <math>\mathcal{S}^D</math> denotes the [[Simplex#Probability|simplex]] of D-dimensional probability vectors. This follows from applying the '''additive logistic transformation''' to map a [[Multivariate normal distribution|multivariate normal]] random variable <math>\mathbf{y} \sim \mathcal{N} \left( \boldsymbol{\mu} , \boldsymbol{\Sigma} \right) \; , \; \mathbf{y} \in \mathbb{R}^{D-1}</math> to the simplex:

: <math>\mathbf{x} = \left[ \frac{ e^{ y_1 } }{ 1 + \sum_{i=1}^{D-1} e^{ y_i } } , \dots , \frac{ e^{ y_{D-1} } }{ 1 + \sum_{i=1}^{D-1} e^{ y_i } } , \frac{ 1 }{ 1 + \sum_{i=1}^{D-1} e^{ y_i } } \right]^\top</math>
[[File:Gaussian and Logistic Normal pdfs.pdf|thumb|upright=1.5|Gaussian density functions and corresponding logistic normal density functions after logistic transformation.]]

The unique inverse mapping is given by:

: <math>\mathbf{y} = \left[ \log \left( \frac{ x_1 }{ x_D } \right) , \dots , \log \left( \frac{ x_{D-1} }{ x_D } \right) \right]^\top</math>.

This is the case of a vector '''x''' which components sum up to one. In the case of '''x''' with sigmoidal elements, that is, when
: <math>\mathbf{y} = \left[ \log \left( \frac{ x_1 }{ 1-x_1 } \right) , \dots , \log \left( \frac{ x_D }{ 1-x_D } \right) \right]^\top</math>
we have  
: <math>f_X( \mathbf{x}; \boldsymbol{\mu} , \boldsymbol{\Sigma} ) = \frac{1}{ | (2 \pi)^{D-1} \boldsymbol{\Sigma} |^\frac{1}{2} } \, \frac{1}{ \prod\limits_{i=1}^D \left(x_i(1-x_i)\right) } \, e^{- \frac{1}{2} \left\{ \log \left( \frac{ \mathbf{x} }{ 1-\mathbf{x} } \right) - \boldsymbol{\mu} \right\}^\top \boldsymbol{\Sigma}^{-1} \left\{ \log \left( \frac{ \mathbf{x}}{ 1-\mathbf{x} } \right) - \boldsymbol{\mu} \right\} } </math>
where the log and the division in the argument are taken element-wise. This is because the Jacobian matrix of the transformation is diagonal with elements <math>\frac{ 1 }{ x_i(1-x_i)}</math>.

=== Use in statistical analysis ===

The logistic normal distribution is a more flexible alternative to the [[Dirichlet distribution]] in that it can capture correlations between components of probability vectors. It also has the potential to simplify statistical analyses of [[compositional data]] by allowing one to answer questions about log-ratios of the components of the data vectors. One is often interested in ratios rather than absolute component values.

The probability simplex is a bounded space, making standard techniques that are typically applied to vectors in <math>\mathbb{R}^n</math> less meaningful. Statistician [[John Aitchison]] described the problem of spurious negative correlations when applying such methods directly to simplicial vectors.<ref name="stat_anal_comp_data"/> However, mapping compositional data in <math>\mathcal{S}^D</math> through the inverse of the additive logistic transformation yields real-valued data in <math>\mathbb{R}^{D-1}</math>. Standard techniques can be applied to this representation of the data. This approach justifies use of the logistic normal distribution, which can thus be regarded as the "Gaussian of the simplex".

=== Relationship with the Dirichlet distribution ===

[[File:Logistic Normal approximation to Dirichlet distribution.pdf|thumb|upright=0.65|Logistic normal approximation to Dirichlet distribution]]

The [[Dirichlet distribution|Dirichlet]] and logistic normal distributions are never exactly equal for any choice of parameters. However, Aitchison described a method for approximating a Dirichlet with a logistic normal such that their [[Kullback–Leibler divergence]] (KL) is minimized:

: <math>K(p,q) = \int_{\mathcal{S}^D} p \left( \mathbf{x} \mid \boldsymbol{\alpha} \right) \log \left( \frac{ p \left( \mathbf{x} \mid \boldsymbol{\alpha} \right) }{ q \left( \mathbf{x} \mid \boldsymbol{\mu} , \boldsymbol{\Sigma} \right) } \right) \, d \mathbf{x}</math>

This is minimized by:

: <math>\boldsymbol{\mu}^* = \mathbf{E}_p \left[ \log \left( \frac{ \mathbf{x}_{-D} }{ x_D } \right) \right] \quad , \quad \boldsymbol{\Sigma}^* = \textbf{Var}_p \left[ \log \left( \frac{ \mathbf{x}_{-D} }{ x_D } \right) \right]</math>

Using moment properties of the Dirichlet distribution, the solution can be written in terms of the [[Digamma function|digamma]] <math>\psi</math> and [[Trigamma function|trigamma]] <math>\psi'</math> functions:

: <math>\mu_i^* = \psi \left( \alpha_i \right) - \psi \left( \alpha_D \right) \quad , \quad i = 1 , \ldots , D-1</math>
: <math>\Sigma_{ii}^* = \psi' \left( \alpha_i \right) + \psi' \left( \alpha_D \right) \quad , \quad i = 1 , \ldots , D-1</math>
: <math>\Sigma_{ij}^* = \psi' \left( \alpha_D \right) \quad , \quad i \neq j</math>

This approximation is particularly accurate for large <math>\boldsymbol{\alpha}</math>. In fact, one can show that for <math>\alpha_i \rightarrow \infty , i = 1 , \ldots , D</math>, we have that <math>p \left( \mathbf{x} \mid \boldsymbol{\alpha} \right) \rightarrow q \left( \mathbf{x} \mid \boldsymbol{\mu}^* , \boldsymbol{\Sigma}^* \right)</math>.

== See also ==
* [[Beta distribution]] and [[Kumaraswamy distribution]], other two-parameter distributions on a bounded interval with similar shapes

== References ==
{{reflist}}

== Further reading ==
* Frederic, P. & Lad, F. (2008) [http://www.tandfonline.com/doi/abs/10.1080/03610910801983178#.U1LER61dW1E Two Moments of the Logitnormal Distribution.] ''Communications in Statistics-Simulation and Computation''. 37: 1263-1269
* {{Cite journal| last1 = Mead | first1 = R. | title = A Generalised Logit-Normal Distribution | jstor = 2528553 | journal = Biometrics | volume = 21 | issue = 3 | pages = 721–732 | year = 1965 | doi = 10.2307/2528553| pmid = 5858101 }}

==External links==
* [http://logitnorm.r-forge.r-project.org/ logitnorm package] for [[R (programming language)|R]]

{{ProbDistributions|continuous-bounded}}

{{DEFAULTSORT:Logit-normal distribution}}
[[Category:Continuous distributions]]