Stationary Distributions (Continuous Time)

We quickly recap the results known for discrete time that also hold in continuous time. (The slight advantage from a modeling perspective is that in continuous time, transitions occur distinctly, which gives more chances for reversibility to hold)

Stationary Distributions

The stationary distribution of an irreducible continuous-time Markov chain is the unique probability distribution $\pi$ satisfying the full balance equation

$\displaystyle \pi Q = 0$

That is,

$\displaystyle \sum_{y \ne x} \pi(y)\, q(y,x) = \sum_{y \ne x} \pi(x)\, q(x,y) \quad \forall x \in \mathcal{X}$

and, since $\pi$ is a probability distribution, we require

$\displaystyle \sum_{x \in \mathcal{X}} \pi(x) = 1$

Theorem. An irreducible continuous-time Markov chain is positive recurrent if and only if there exists a solution to the balance equations.

Time Reversal

If $X = (X(t) : t \in \mathbb{R})$ is a continuous-time Markov chain with $X(0) \sim \pi$ , we define its time reversal by

$\displaystyle X^R = (X(-t) : t \in \mathbb{R})$

Theorem. If $X$ is a continuous-time Markov chain with Q-matrix $Q = (q(x,y) : x,y \in \mathcal{X})$ and stationary distribution $\pi$ , then the reversed process $X^R$ is also a continuous-time Markov chain with Q-matrix