Doob's h-transform

In mathematics, more specifically in stochastics, Doob's h-transform is a method to transform a Markov process into a new Markov process, which exhibits certain properties. Most prominently, for a homogeneous Markov process $X$ with (random) lifetime $T$, the reverse process $(X_{T-t}{)}_{t\ge 0}$ and the conditioned process $X\mid X_T=x$ on terminating at a single point $x$, can be expressed as an $h$-transform of the original process via some deterministic function $h$. The method was first introduced by J. L. Doob in 1957 to analyze quotients $f/h$ of superharmonic functions $f$ and (positive) harmonic functions $h$ from a probabilistic point of view.Template:Sfn

The key advantage of Doob's $h$-transform is that the transition probabilities of the transformed process are explicitly given as functions of $h$. If $X$ is an Itô diffusion, for example a Brownian motion, then under relatively mild conditions, the $h$-transformed process is again an Itô diffusion with an explicit drift term depending on $\mathrm{\nabla }\log h$.Template:Sfn

Motivation

Consider a $d$-dimensional Brownian motion $W$ started in $x_0$ and stopped at the time $T$, when it first hits the unit ball $B$, and one is interested in the process $W^x$ of $W$ conditioned on hitting some point $x\in \partial B$ at time $T$. In law, $W^x$ is given by$${\mathbb{P}}^{x_0}(W^x\in A):={\mathbb{P}}^{x_0}(W\in A\mid W_T=x)=\mathbb{P}(W\in A\mid W_T=x,W_0=x_0),$$for any Borel set $A\subset \{f\in C([0,\mathrm{\infty }),{ℝ}^d):f(0)=x_0\}$.

According to Bayes formula, for some fixed $t\ge 0$, conditioning $W_t$ on $W_T$ can be expressed in terms of the distribution of $W_T$ given $W_t$ and the distribution of $W_t$, namely for $M\in ℬ({ℝ}^d)$$${\mathbb{P}}^{x_0}(W_t\in M\mid W_T=x)=\frac{1}{q(x\mid x_0)}{\int }_{M}q(x\mid y){\mathbb{P}}^{x_0}(W_t\in \mathrm{d}\mathrm{y}),$$with$$q(x\mid y):=\frac{\mathrm{d}\mathbb{P}(W_T\in \cdot \mid W_t=y)}{\mathrm{d}\sigma }(x)=\frac{\mathrm{d}{\mathbb{P}}^y(W_T\in \cdot )}{\mathrm{d}\sigma }(x),$$where $\sigma$ denotes the uniform distribution on the sphere. In this context, the function $q$ happens to be the well-known Poisson kernel, which has the explicit formula$$q(x\mid y)=\frac{1}{\sigma (\partial B)}\frac{1-\Vert y{\Vert }^2}{\Vert x-y{\Vert }^d},x\in \partial B,y\in B.$$The distribution of $W_t$ under ${\mathbb{P}}^{x_0}$ is explicit as well, since the Brownian Motion is a Gaussian process and so the Bayes formula for $W_t^x$ gives an explicit way to compute the transition probabilities. The Bayes formula from above is precisely an $h$-transform of $W$ with $h=q(x\mid \cdot )$.

For the simulation of $W^x$, one is interested in a stochastic differential equation for the latter. With the Bayes formula from above, the infinitesimal generator ${𝒜}^x$ of $W^x$ can be related to the generator $𝒜=\frac{1}{2}\mathrm{\Delta }$ (the Laplace operator) of Brownian motion: For $f\in C^2({ℝ}^d)$, and by denoting the Feller semigroups of $W$ and $W^x$ by $(P_t{)}_{t\ge 0}$ and $(P_t^x{)}_{t\ge 0}$ respectively, it holds for $y\in {ℝ}^d$Template:Sfn$$\begin{array}{cc}{𝒜}^{x}f(y)& =\underset{t\to 0}{lim}\frac{P_t^{x}f(y)-f(y)}{t}\\ & =\frac{1}{q(x\mid y)}\underset{t\to 0}{lim}\frac{P_t(q(x\mid \cdot )f)(y)-q(x\mid y)f(y)}{t}\\ & =\frac{1}{q(x\mid y)}𝒜(q(x\mid \cdot )f)(y)\\ & =\frac{1}{q(x\mid y)}{\mathrm{\nabla }}_{y}q(x\mid y)\cdot \mathrm{\nabla }f(y)+\frac{1}{2}\mathrm{\Delta }f(y),\end{array}$$because $\mathrm{\Delta }q(x\mid \cdot )=0$. Comparing ${𝒜}^x$ with the generator of a general Itô diffusion, one can conclude that$$\mathrm{d}{W}_t^x=\mathrm{\nabla }\log q(x\mid W_t)\mathrm{d}t+\mathrm{d}{W}_t,W_0=x_0.$$

Definition

Homogeneous Markov processes

Let $X$ be a homogeneous Markov process with state space $E$, which is often assumed to be a locally compact Polish space to guarantee the existence of regular conditional expectations, and denote its transition semigroup by $(P_t{)}_{t\ge 0}$. Let $h:E\to [0,\mathrm{\infty }]$ be an excessive function, that is,Template:Sfn$$\begin{array}{cc}\mathrm{\forall }t\ge 0:& {\int }_{E}h(y)P_t(x,\mathrm{d}y)\le h(x),\\ \underset{t\to 0}{lim}& {\int }_{E}h(y)P_t(x,\mathrm{d}y)=h(x).\end{array}$$Then the $h$-transform $X^h$ is defined as the Markov process with transition semigroup $(P_t^h{)}_{t\ge 0}$ defined by$$P_t^h(x,A):=\{\begin{array}{cc}\frac{1}{h(x)}{\int }_{A}h(y)P_t(x,\mathrm{d}y),& x\in E\setminus \{h=0\text{ or }h=\mathrm{\infty }\}\\ 0,& x\in \{h=0\text{ or }h=\mathrm{\infty }\}\end{array}$$for all $t\ge 0$ and $A\in ℬ(E)$.

This definition implies that in general $P_t^h(x,E)\le 1$, which means that the process need not staying inside its state space for all times. For such processes, one usually defines an additional cemetery state $\partial \notin E$ that the process enters with probability $1-P_t(x,E)$ for all $x\in E$ and which it never leaves again, i.e., $P_t(\partial ,\{\partial \})=1$. In other words, the process is allowed to be killed at some finite time. On the extended state space $E_{\partial }=E\cup \{\partial \}$, the $P_t^h(x,\cdot )$ with the previous definitions are probability measures for all $x\in E_{\partial }$ by construction.

The points $x\in \{h=0\text{ or }h=\mathrm{\infty }\}$ are so-called branching points, for which it holds $P_0(x,\{x\})=0$ and $P_0(x,\{\partial \})=1$. Thus, the $h$-transformed process never enters the set $\{h=0\text{ or }h=\mathrm{\infty }\}$, and it is killed at the random time $T^h=inf\{t\ge 0:X_t^h=\partial \}$.

Inhomogeneous Markov processes

Let $X$ be an inhomogeneous Markov process with values in ${ℝ}^d$, inhomogeneous transition probabilities $(P_{t,s}{)}_{t\ge s\ge 0}$ and let $h:[0,\mathrm{\infty })\times {ℝ}^d\to (0,\mathrm{\infty })$ be space-time regular, that is, for $s>0$, $t\ge 0$ and $x\in {ℝ}^d$Template:SfnTemplate:Sfn$$h(t,x)={\int }_{{ℝ}^d}h(t+s,y)P_{t+s,s}(x,\mathrm{d}y).$$Then the $h$-transform $X^h$ is defined as the (inhomogeneous) Markov process with transition probabilities $(P_{t,s}^h{)}_{t\ge s\ge 0}$ defined by$$P_{t+s,s}^h(x,A):=\frac{1}{h(t,x)}{\int }_{A}h(t+s,y)P_{t+s,s}(x,\mathrm{d}y),\mathrm{\forall }x\in E,A\in ℬ(E).$$

Properties

In the theory of $h$-transforms for homogeneous Markov processes $X$ on a filtered probability space $(\mathrm{\Omega },ℱ,({ℱ}_t{)}_{t\ge 0},\mathbb{P})$ it is assumed thatTemplate:Sfn

1. $X_0$ is distributed according to some distribution $\mu$,
2. $X$ has a lifetime $T$, that is, a random variable (not necessarily a stopping time) such that $X_t(\omega )\in E$ for $t>T(\omega )$ and $X_t(\omega )=\partial$ for $t\ge T(\omega )$,
3. $X$ is a strong Markov process with càdlàg paths on $(0,T)$.

For simplicity, the underlying probability space is usually chosen as the canonical space of càdlàg paths with values in $(E,ℬ(E))$ and $X$ as the identity.

Probabilities

The transition semigroup $(P_t^h{)}_{t\ge 0}$ generates a family of probability measures $({\mathbb{P}}_h^x{)}_{x\in E}$ on the canonical space of càdlàg paths. For any stopping time $\tau$, it holds for $A\in {ℱ}_{\tau }$ and $x\in E\setminus \{h=0\text{ or }h=\mathrm{\infty }\}$Template:Sfn

$${\mathbb{P}}_h^x(A,T>\tau )=\frac{1}{h(x)}{𝔼}^x[1_{A}h(X_{\tau })].$$

For a family $(u_s{)}_{s\in S}$ of excessive functions on $E$ over an index set $S$ (and measurable w.r.t. the product $\sigma$-algebra on $E\times S$), if $h$ is of the form

$$h(x)={\int }_S{u}_s(x)\mathrm{d}\beta (s)$$

with some finite measure $\beta$ on $S$, then for $A\in ℱ$ the probability simplifies toTemplate:Sfn

$${\mathbb{P}}_h^x(A)=\frac{1}{h(x)}{\int }_S{u}_s(x){\mathbb{P}}_{u_s}^x(A)\mathrm{d}\beta (s).$$

Infinitesimal generator

As an almost direct consequence of the above, if $X$ has infinite lifetime ($T=\mathrm{\infty }$) and has generator $A$, then the generator of its $h$-transformed process $X^h$ is

$\begin{array}{cc}{A}^{h}f(x)=\underset{t\to 0}{lim}\frac{{𝔼}^x[f(X_t^h)]-f(x)}{t}& =\frac{1}{h(x)}\underset{t\to 0}{lim}\frac{{𝔼}_h^x[f(X_t)h(X_t)]-f(x)h(x)}{t}\\ & =\frac{1}{h(x)}A(fh)(x)\end{array}$

for all $x\in E$ and all functions $f$ such that $fh$ is in the domain of $A$.

Killing

If $\tau$ is a co-optional time, that is, if for any $t\ge 0$ it holds $\tau \circ {\theta }_t=(\tau -t{)}^{+}$, where $({\theta }_s{)}_{s\ge 0}$ is the shift-semigroup, then one can define$$\begin{array}{cc}& h(x):={\mathbb{P}}^x(\tau >0),x\in E\\ & X_t^{\tau }:=\{\begin{array}{cc}{X}_t,& t<\tau \\ \partial ,& t\ge \tau \end{array}\end{array}$$for which it holds that $X^{\tau }$ is an $h$-transform of $X$.Template:Sfn

Time-reversal

The time-reversal $\overleftarrow{X}$ of $X$ is defined as

$$\overleftarrow{X_t}(\omega )=\{\begin{array}{cc}{X}_{T(\omega )-t},& t\le T(\omega )\\ \partial ,& t>T(\omega )\text{ or }T(\omega )=\mathrm{\infty }\end{array}.$$

$\overleftarrow{X}$ is again a homogeneous Markov process, however now with left-continuous paths.Template:Sfn

If $X^h$ has initial measure $\nu \propto h\cdot \mathrm{d}\mu$, then the reverses of $X$ and $X^h$ have the same transition probabilities. In particular, for a co-optional time $\tau$ (for example a last hitting time), if $X$ starts in a fixed point almost surely, then the reverses of $X$ and $X^{\tau }$ have the same transition probabilities.Template:Sfn

Moreover, if $X$ possesses a dual process $\hat{X}$ (defined below), then its reverse can be expressed as an $h$-transform of its dual. To formulate this statement, we define for $X$ and $\hat{X}$ respectively the transition semigroups $(P_t{)}_{t\ge 0}$ and $({\hat{P}}_t{)}_{t\ge 0}$, and, for $p\ge 0$, the resolvents

$$\begin{array}{c}{U}_p:={\int }_0^{\mathrm{\infty }}{e}^{-pt}{P}_t\mathrm{d}t,\\ {\hat{U}}_p:={\int }_0^{\mathrm{\infty }}{e}^{-pt}{\hat{P}}_t\mathrm{d}t.\end{array}$$

While for $p>0$ the resolvents are always finite, $U_0$ can become infinite.

As the dual of $X$, $\hat{X}$ must fulfill for some measure $\nu$

1. $\hat{X}$ is a strong Markov process with càdlàg paths,
2. For all $p\ge 0$ and $f,g\ge 0$ measurable: ${\int }_E{\int }_{E}f(y)U_p(x,\mathrm{d}y)g(x)\mathrm{d}\nu (x)={\int }_{E}f(x){\int }_{E}g(y){\hat{U}}_p(x,\mathrm{d}y)\mathrm{d}\nu (x),$
3. For all $p\ge 0$ and $x\in E$, both $U_p(x,\cdot )$ and ${\hat{U}}_p(x,\cdot )$ are absolutely continuous w.r.t. $\nu$.

By setting

$$h(y):={\int }_E\frac{\mathrm{d}{U}_0(x,\cdot )}{\mathrm{d}\nu }(y)\mathrm{d}\mu (x):={\int }_E{u}_0(x,y)\mathrm{d}\mu (x),y\in E,$$

if $0<h<\mathrm{\infty }$, then $\overleftarrow{X}$ and ${\hat{X}}^h$ have the same transition probabilities.Template:Sfn

Conditioning

Doob's $h$-transform can also be used to condition the process $X$ on hitting a distinct point or, more generally, to attain a pre-determined distribution at its lifetime. For simplicity, it is supposed that ${\hat{P}}_0(x,\{x\})=1$ for all $x\in E$, that is, the dual process $\hat{X}$ has no branching points.

Let $y\in E$ and $U_0$ be the resolvent of $X$ for $p=0$ as defined above. Then with

$$h(x)=u_0(x,y),$$

it holds that $X_{T-}^h=\underset{t\uparrow T}{lim}{X}_t^h=y$ almost surely on $\{T<\mathrm{\infty }\}$.Template:Sfn

If $\mu ={\delta }_{x_0}$, that is, $X$ starts in $x_0$ almost surely, and $\alpha$ is a distribution on $E$ such that

$$\beta (A)={\int }_A\frac{1}{u_0(x,y)}\mathrm{d}\alpha (y),A\in ℬ(E),$$

is a $\sigma$-finite measure, then one can achieve $X_{T-}^h\sim \alpha$ by choosingTemplate:Sfn

$$h(x):={\int }_E{u}_0(x,y)\mathrm{d}\beta (y)={\int }_E\frac{u_0(x,y)}{u_0(x_0,y)}\mathrm{d}\alpha (y).$$

Applications

Generative modelling

Doob's $h$-transform has begun to find applications in generative modelling, for example for point clouds, climate data, graphs and image segmentation. The method has also been proposed for faster conditional training and sampling of diffusion models.

See also

• Markov property
• Markov chain
• Feller process

References

• Template:Cite book
• Template:Cite book
• Template:Cite book
• Template:Cite book