Markov chain

From SklogWiki
Jump to navigation Jump to search

The concept of a Markov chain was developed by Andrey Andreyevich Markov. A Markov chain is a sequence of random variables with the property that it is forgetful of all but its immediate past. For a process evolving on a space and governed by an overall probability law to be a time-homogeneous Markov chain there must be a set of "transition probabilities" for appropriate sets such that for times in (Ref. 1 Eq. 1.1)

that is denotes the probability that a chain at x will be in the set A after n steps, or transitions. The independence of on the values of is the Markov property, and the independence of and m is the time-homogeneity property.


  1. S. P. Meyn and R. L. Tweedie "Markov Chains and Stochastic Stability", Springer-Verlag, London (1993)
  2. Ruichao Ren and G. Orkoulas "Parallel Markov chain Monte Carlo simulations", Journal of Chemical Physics 126 211102 (2007)