Generalized Pareto distribution

In statistics, the generalized Pareto distribution (GPD) is a family of continuous probability distributions. It is often used to model the tails of another distribution. It is specified by three parameters: location $\mu$, scale $\sigma$, and shape $\xi$. Sometimes it is specified by only scale and shape and sometimes only by its shape parameter. Some references give the shape parameter as $\kappa =-\xi$. This parameterization was introduced by James Pickands III .

Definition

The cumulative distribution function of $X\sim \text{GPD}(\mu ,\sigma ,\xi )$ ($\mu \in ℝ$, $\sigma >0$, and $\xi \in ℝ$) is

$F_{\mu ,\sigma ,\xi }(x)=\{\begin{array}{cc}1-{(1+\xi \frac{x-\mu }{\sigma })}^{-1/\xi }& \text{for }\xi \ne 0,\\ 1-\exp (-\frac{x-\mu }{\sigma })& \text{for }\xi =0,\end{array}$

where the support of Template:Mvar is $x\ge \mu$ when $\xi \ge 0$, and $\mu \le x\le \mu -\sigma /\xi$ when $\xi <0$.

The probability density function (pdf) of $X\sim \text{GPD}(\mu ,\sigma ,\xi )$ is

$f_{\mu ,\sigma ,\xi }(x)=\{\begin{array}{cc}\frac{1}{\sigma }{(1+\xi \frac{x-\mu }{\sigma })}^{-1-1/\xi }& \text{for }\xi \ne 0,\\ \frac{1}{\sigma }\exp (-\frac{x-\mu }{\sigma })& \text{for }\xi =0,\end{array}$

again, for $x\ge \mu$ when $\xi \ge 0$, and $\mu \le x\le \mu -\sigma /\xi$ when $\xi <0$.

The GPD survival function (sf), $\overline{F}(x)=1-F(x)$, is the solution to the following nonlinear ordinary differential equation:

${\overline{F}}^{\prime }(x)+\frac{1}{\sigma }\overline{F}(x{)}^{1+\xi }=0,\overline{F}(\mu )=1$.

Thus, the tail shape $\xi$ quantifies the nonlinear deviation from linear systems and the scale $\sigma$ is associated with the linear source of uncertainty.

The standard cumulative distribution function (cdf) of the GPD is defined using $z=\frac{x-\mu }{\sigma }$ (or, equivalently, setting $\mu =0$ and $\sigma =1$):

$F_{\xi }(z)=\{\begin{array}{cc}1-{(1+\xi z)}^{-1/\xi }& \text{for }\xi \ne 0,\\ 1-e^{-z}& \text{for }\xi =0.\end{array}$

where the support is $z\ge 0$ for $\xi \ge 0$ and $0\le z\le -1/\xi$ for $\xi <0$.

The corresponding probability density function (pdf) is

$f_{\xi }(z)=\{\begin{array}{cc}{(1+\xi z)}^{-1-1/\xi }& \text{for }\xi \ne 0,\\ e^{-z}& \text{for }\xi =0.\end{array}$

Related distributions

• $\mathrm{G}\mathrm{P}\mathrm{D}(\mu ,\sigma ,\xi =0)\sim \mu +\mathrm{E}\mathrm{x}\mathrm{p}(\lambda =1/\sigma )$ (shifted exponential distribution).
• $\mathrm{G}\mathrm{P}\mathrm{D}(\mu ,\sigma ,\xi >0)\sim (\mu -\sigma /\xi )+\mathrm{P}\mathrm{a}\mathrm{r}\mathrm{e}\mathrm{t}\mathrm{o}(x_{\mathrm{m}}=\sigma /\xi ,\alpha =1/\xi )$ (shifted Pareto distribution).
• $\mathrm{G}\mathrm{P}\mathrm{D}(\mu ,\sigma ,\xi <0)\sim (\mu -\sigma /\xi )-\mathrm{P}\mathrm{o}\mathrm{w}\mathrm{e}\mathrm{r}(x_{\mathrm{m}}=-\sigma /\xi ,\alpha =-1/\xi )$ (flipped power function distribution over vertical line $x=(\mu -\sigma /\xi )/2$).
  • $\mathrm{G}\mathrm{P}\mathrm{D}(\mu ,\sigma ,\xi =-1)\sim \mathrm{U}(\mu ,\mu +\sigma )$ (continuous uniform distribution).
  • $\mathrm{G}\mathrm{P}\mathrm{D}(\mu ,\sigma ,\xi =-1/2)\sim \mathrm{T}\mathrm{r}\mathrm{i}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{u}\mathrm{l}\mathrm{a}\mathrm{r}(a=\mu ,b=\mu +2\sigma ,c=\mu )$ (triangular distribution).
• $\log (\mathrm{G}\mathrm{P}\mathrm{D}(\mu =0,\sigma ,\xi ))\sim \mathrm{e}\mathrm{x}\mathrm{G}\mathrm{P}\mathrm{D}(\sigma ,\xi )$ (exponentiated generalized Pareto distribution).
• $\mathrm{G}\mathrm{P}\mathrm{D}(\mu ,\sigma ,\xi )\sim \mu +\mathrm{B}\mathrm{u}\mathrm{r}\mathrm{r}(c=1,k=1/\xi ,\lambda =\sigma /\xi )$ (shifted Burr distribution with forced parameter Template:Mvar).

Prediction

• It is often of interest to predict probabilities of out-of-sample data under the assumption that both the training data and the out-of-sample data follow a GPD.
• Predictions of probabilities generated by substituting maximum likelihood estimates of the GPD parameters into the cumulative distribution function ignore parameter uncertainty. As a result, the probabilities are not well calibrated, do not reflect the frequencies of out-of-sample events, and, in particular, underestimate the probabilities of out-of-sample tail events.
• Predictions generated using the objective Bayesian approach of calibrating prior prediction have been shown to greatly reduce this underestimation, although not completely eliminate it. Calibrating prior prediction is implemented in the R software package fitdistcp.

Generating generalized Pareto random variables

Generating GPD random variables

If U is uniformly distributed on Template:Open-closed, then

$$X=\mu +\frac{\sigma (U^{-\xi }-1)}{\xi }\sim \mathrm{G}\mathrm{P}\mathrm{D}(\mu ,\sigma ,\xi \ne 0)$$ and $$X=\mu -\sigma \ln (U)\sim \mathrm{G}\mathrm{P}\mathrm{D}(\mu ,\sigma ,\xi =0).$$

Both formulas are obtained by inversion of the cdf.

The Pareto package in R and the gprnd command in the Matlab Statistics Toolbox can be used to generate generalized Pareto random numbers.

GPD as an Exponential-Gamma Mixture

A GPD random variable can also be expressed as an exponential random variable, with a Gamma distributed rate parameter.$$X\mid \mathrm{\Lambda }\sim \mathrm{E}\mathrm{x}\mathrm{p}(\mathrm{\Lambda })$$ and $$\mathrm{\Lambda }\sim \mathrm{G}\mathrm{a}\mathrm{m}\mathrm{m}\mathrm{a}(\alpha ,\beta )$$ then $$X\sim \mathrm{G}\mathrm{P}\mathrm{D}(\mu =0,\sigma =\beta /\alpha ,\xi =1/\alpha )$$

Template:Collapse top Let $\text{Gamma}(x;\alpha ,\beta )$ be the density of a gamma distribution with parameters $\alpha$ and $\beta$, and $\text{Exp}(x;\mathrm{\Lambda })$ be the density of an exponential distribution with parameter $\mathrm{\Lambda }$. We begin by writing the marginal density of the latent-variable mixture for $\theta =[\alpha ,\beta ,\mathrm{\Lambda }{]}^{\mathrm{\top }}$ and expand $$\begin{array}{cc}{p}_{\theta }(x)& ={\int }_0^{\mathrm{\infty }}\text{Gamma}(\mathrm{\Lambda };\alpha ,\beta )\text{Exp}(x;\mathrm{\Lambda })d\mathrm{\Lambda }\\ & ={\int }_0^{\mathrm{\infty }}\left[\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{\mathrm{\Lambda }}^{\alpha -1}{e}^{-\beta \mathrm{\Lambda }}\right]\left(\mathrm{\Lambda }{e}^{-\mathrm{\Lambda }x}\right)d\mathrm{\Lambda }\\ & ={\int }_0^{\mathrm{\infty }}\frac{\mathrm{\Lambda }{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{\mathrm{\Lambda }}^{\alpha -1}{e}^{-\mathrm{\Lambda }(\beta +x)}d\mathrm{\Lambda }\\ & =\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{\int }_0^{\mathrm{\infty }}{\mathrm{\Lambda }}^{\alpha }{e}^{-\mathrm{\Lambda }(\beta +x)}d\mathrm{\Lambda }\\ & =\frac{{\beta }^{\alpha }}{\mathrm{\Gamma }(\alpha )}\frac{\mathrm{\Gamma }(\alpha +1)}{(\beta +x{)}^{\alpha +1}}\\ & =\frac{\alpha {\beta }^{\alpha }}{(\beta +x{)}^{\alpha +1}}\\ & =\text{GPD}(0,\frac{\beta }{\alpha },\frac{1}{\alpha })\end{array}$$ For the third from the last equality, we have used the definition of the gamma function, $$\mathrm{\Gamma }(x)={\int }_0^{\mathrm{\infty }}{t}^{x-1}{e}^{-t}dt$$ and a change of variables $t=c\mathrm{\Lambda }$, to observe that $$\begin{array}{cc}\mathrm{\Gamma }(\alpha +1)& ={\int }_0^{\mathrm{\infty }}(c\mathrm{\Lambda }{)}^{\alpha }{e}^{-c\mathrm{\Lambda }}d(c\mathrm{\Lambda })\\ & =c^{\alpha +1}{\int }_0^{\mathrm{\infty }}{\mathrm{\Lambda }}^{\alpha }{e}^{-c\mathrm{\Lambda }}d\mathrm{\Lambda }\end{array}$$ where $c=\beta +x$. Template:Collapse bottom

The Exponential-Gamma mixture explains the importance of the generalized Pareto distribution parameterization, which correspondence with the mean and relative variance of the fluctuations of exponential scale, $\mathrm{\Lambda }$. Notice however, that since the parameters for the Gamma distribution must be greater than zero, we obtain the additional restrictions that $\xi$ must be positive.

In addition to this mixture (or compound) expression, the generalized Pareto distribution can also be expressed as a simple ratio. Concretely, for $Y\sim \mathrm{E}\mathrm{x}\mathrm{p}(1)$ and $Z\sim \mathrm{G}\mathrm{a}\mathrm{m}\mathrm{m}\mathrm{a}(1/\xi ,1),$ we have $\mu +\frac{\sigma Y}{\xi Z}\sim \mathrm{G}\mathrm{P}\mathrm{D}(\mu ,\sigma ,\xi ).$ This is a consequence of the mixture after setting $\beta =\alpha$ and taking into account that the rate parameters of the exponential and gamma distribution are simply inverse multiplicative constants.

Exponentiated generalized Pareto distribution

The exponentiated generalized Pareto distribution (exGPD)

The pdf of the $\mathrm{e}\mathrm{x}\mathrm{G}\mathrm{P}\mathrm{D}(\sigma ,\xi )$ (exponentiated generalized Pareto distribution) for different values $\sigma$ and $\xi$.

If $X\sim \mathrm{G}\mathrm{P}\mathrm{D}(\mu =0,\sigma ,\xi )$, then $Y=\log (X)$ is distributed according to the exponentiated generalized Pareto distribution, denoted by $Y\sim \mathrm{e}\mathrm{x}\mathrm{G}\mathrm{P}\mathrm{D}(\sigma ,\xi )$.

The probability density function(pdf) of $Y\sim \mathrm{e}\mathrm{x}\mathrm{G}\mathrm{P}\mathrm{D}(\sigma ,\xi )(\sigma >0)$ is

$$g_{(\sigma ,\xi )}(y)=\{\begin{array}{c}\frac{e^y}{\sigma }(1+\frac{\xi e^y}{\sigma }{)}^{-1/\xi -1}\text{for }\xi \ne 0,\\ \frac{1}{\sigma }{e}^{y-e^y/\sigma }\text{for }\xi =0,\end{array}$$ where the support is $-\mathrm{\infty }<y<\mathrm{\infty }$ for $\xi \ge 0$, and $-\mathrm{\infty }<y\le \log (-\sigma /\xi )$ for $\xi <0$.

For all $\xi$, the $\log \sigma$ becomes the location parameter. See the right panel for the pdf when the shape $\xi$ is positive.

The exGPD has finite moments of all orders for all $\sigma >0$ and $-\mathrm{\infty }<\xi <\mathrm{\infty }$.

The variance of the $\mathrm{e}\mathrm{x}\mathrm{G}\mathrm{P}\mathrm{D}(\sigma ,\xi )$ as a function of $\xi$. Note that the variance only depends on $\xi$. The red dotted line represents the variance evaluated at $\xi =0$, that is, ${\psi }^{\prime }(1)={\pi }^2/6$.

The moment-generating function of $Y\sim \mathrm{e}\mathrm{x}\mathrm{G}\mathrm{P}\mathrm{D}(\sigma ,\xi )$ is $$M_Y(s)=\mathrm{E}\left[e^{sY}\right]=\{\begin{array}{cccc}-\frac{1}{\xi }{(-\frac{\sigma }{\xi })}^{s}B(s+1,-1/\xi ),& \text{for }& -1<s<\mathrm{\infty },& \xi <0,\\ \frac{1}{\xi }{\left(\frac{\sigma }{\xi }\right)}^{s}B(s+1,1/\xi -s)& \text{for }& -1<s<1/\xi ,& \xi >0,\\ {\sigma }^s\mathrm{\Gamma }(1+s),& \text{for }& -1<s<\mathrm{\infty },& \xi =0,\end{array}$$ where $B(a,b)$ and $\mathrm{\Gamma }(a)$ denote the beta function and gamma function, respectively.

The expected value of $Y\sim \mathrm{e}\mathrm{x}\mathrm{G}\mathrm{P}\mathrm{D}(\sigma ,\xi )$ depends on the scale $\sigma$ and shape $\xi$ parameters, while the $\xi$ participates through the digamma function: $$\mathrm{E}[Y]=\{\begin{array}{cc}\log (-\frac{\sigma }{\xi })+\psi (1)-\psi (-1/\xi +1)& \text{for }\xi <0,\\ \log \sigma -\log \xi +\psi (1)-\psi (1/\xi )& \text{for }\xi >0,\\ \log \sigma +\psi (1)& \text{for }\xi =0.\end{array}$$ Note that for a fixed value for the $\xi \in (-\mathrm{\infty },\mathrm{\infty })$, the $\log \sigma$ plays as the location parameter under the exponentiated generalized Pareto distribution.

The variance of $Y\sim \mathrm{e}\mathrm{x}\mathrm{G}\mathrm{P}\mathrm{D}(\sigma ,\xi )$ depends on the shape parameter $\xi$ only through the polygamma function of order 1 (also called the trigamma function): $$\mathrm{Var}[Y]=\{\begin{array}{cc}{\psi }^{\prime }(1)-{\psi }^{\prime }(-1/\xi +1)& \text{for }\xi <0,\\ {\psi }^{\prime }(1)+{\psi }^{\prime }(1/\xi )& \text{for }\xi >0,\\ {\psi }^{\prime }(1)& \text{for }\xi =0.\end{array}$$ See the right panel for the variance as a function of $\xi$. Note that ${\psi }^{\prime }(1)={\pi }^2/6\approx 1.644934$.

Note that the roles of the scale parameter $\sigma$ and the shape parameter $\xi$ under $Y\sim \mathrm{e}\mathrm{x}\mathrm{G}\mathrm{P}\mathrm{D}(\sigma ,\xi )$ are separably interpretable, which may lead to a robust efficient estimation for the $\xi$ than using the $X\sim \mathrm{G}\mathrm{P}\mathrm{D}(\sigma ,\xi )$. The roles of the two parameters are associated each other under $X\sim \mathrm{G}\mathrm{P}\mathrm{D}(\mu =0,\sigma ,\xi )$ (at least up to the second central moment); see the formula of variance $Var(X)$ wherein both parameters are participated.

The Hill estimator

Assume that $X_{1:n}=(X_1,\cdots ,X_n)$ are $n$ observations (which need not be i.i.d.) from an unknown heavy-tailed distribution $F$ such that its tail distribution is regularly varying with the tail-index $1/\xi$ (hence, the corresponding shape parameter is $\xi$). To be specific, the tail distribution is described as $$\overline{F}(x)=1-F(x)=L(x)\cdot x^{-1/\xi },\text{for some }\xi >0,\text{where }L\text{ is a slowly varying function.}$$ It is of a particular interest in extreme value theory to estimate the shape parameter $\xi$, especially when $\xi$ is positive (so-called heavy-tailed distribution).

Let $F_u$ be their conditional excess distribution function. The Pickands–Balkema–de Haan theorem (Pickands, 1975; Balkema and de Haan, 1974) states that for a large class of underlying distribution functions $F$, and large $u$, $F_u$ is well approximated by the generalized Pareto distribution (GPD), which motivated Peaks Over Threshold (POT) methods to estimate $\xi$: the GPD plays the key role in the POT approach.

A renowned estimator using the POT methodology is the Hill estimator. Technical formulation of the Hill estimator is as follows. For $1\le i\le n$, write $X_{(i)}$ for the $i$-th largest value of $X_1,\cdots ,X_n$. With this notation, the Hill estimator based on the $k$ upper order statistics is defined as $${\hat{\xi }}_k^{\text{Hill}}={\hat{\xi }}_k^{\text{Hill}}(X_{1:n})=\frac{1}{k-1}\sum _{j=1}^{k-1}\log \left(\frac{X_{(j)}}{X_{(k)}}\right),\text{for }2\le k\le n.$$ In practice, the Hill estimator is used as follows. First, calculate the estimator ${\hat{\xi }}_k^{\text{Hill}}$ at each integer $k\in \{2,\cdots ,n\}$, and then plot the ordered pairs $\{(k,{\hat{\xi }}_k^{\text{Hill}}){\}}_{k=2}^n$. Then, select from the set of Hill estimators $\{{\hat{\xi }}_k^{\text{Hill}}{\}}_{k=2}^n$ those which are roughly constant with respect to $k$: these stable values are regarded as reasonable estimates for the shape parameter $\xi$. If $X_1,\cdots ,X_n$ are i.i.d., then the Hill estimator is a consistent estimator for the shape parameter $\xi$. Note that the Hill estimator ${\hat{\xi }}_k^{\text{Hill}}$ makes a use of the log-transformation for the observations $X_{1:n}=(X_1,\cdots ,X_n)$. (The Pickands estimator ${\hat{\xi }}_k^{\text{Pickand}}$ also employs the log-transformation, but in a slightly different way.)

See also

• Burr distribution
• Pareto distribution
• Generalized extreme value distribution
• Pickands–Balkema–de Haan theorem

Further reading

• Template:Cite journal
• Template:Cite journal
• Template:Cite book Chapter 20, Section 12: Generalized Pareto Distributions.
• Template:Cite book
• Template:Cite book

External links

• Mathworks: Generalized Pareto distribution