Markov chain

The concept of a Markov chain was developed by Andrey Andreyevich Markov. A Markov chain is a sequence of random variables with the property that it is forgetful of all but its immediate past. For a process ${\mathbf {\Phi } }$ evolving on a space ${\mathsf {X}}$ and governed by an overall probability law ${\mathsf {P}}$ to be a time-homogeneous Markov chain there must be a set of "transition probabilities" $\{P^{n}(x,A),x\in {\mathsf {X}},A\subset {\mathsf {X}}\}$ for appropriate sets $A$ such that for times $n,m$ in ${\mathbb {Z} }_{+}$ (Ref. 1 Eq. 1.1)

${\mathsf {P}}(\Phi _{n+m}\in A\vert \Phi _{j},j\leq m;\Phi _{m}=x)=P^{n}(x,A);$

that is $P^{n}(x,A)$ denotes the probability that a chain at x will be in the set A after n steps, or transitions. The independence of $P^{n}$ on the values of $\Phi _{j},j\leq m$ is the Markov property, and the independence of $P^{n}$ and m is the time-homogeneity property.

References

1. S. P. Meyn and R. L. Tweedie "Markov Chains and Stochastic Stability", Springer-Verlag, London (1993)
2. Ruichao Ren and G. Orkoulas "Parallel Markov chain Monte Carlo simulations", Journal of Chemical Physics 126 211102 (2007)