Markov chain: Difference between revisions
Carl McBride (talk | contribs) No edit summary |
Carl McBride (talk | contribs) No edit summary |
||
| Line 1: | Line 1: | ||
The concept of a '''Markov chain''' was developed by Andrey Andreyevich Markov. | The concept of a '''Markov chain''' was developed by Andrey Andreyevich Markov. A Markov chain is a sequence of random variables with the property that it is forgetful of all but its immediate past. | ||
For a process <math>{\mathbf \Phi}</math> evolving on a space <math>{\mathsf X}</math> and governed by an overall probability law <math>{\mathsf P}</math> to be a time-homogeneous Markov chain there must be a set of "transition probabilities" <math>\{P^n (x,A), x \in {\mathsf X}, A \subset {\mathsf X}\}</math> for appropriate sets <math>A</math> such that | |||
for times <math>n,m</math> in <math>{\mathbb Z}_+</math> (Ref. 1 Eq. 1.1) | |||
:<math>{\mathsf P} (\Phi_{n+m} \in A \vert \Phi_j,j \leq m; \Phi_m =x)= P^n(x,A);</math> | |||
that is <math>P^n(x,A)</math> denotes the probability that a chain at ''x'' will be in the set ''A'' after ''n'' steps, or transitions. The independence of <math>P^n</math> on the values of <math>\Phi_j,j \leq m</math> is the Markov property, | |||
and the independence of <math>P^n</math> and ''m'' is the time-homogeneity property. | |||
==References== | ==References== | ||
#[http://probability.ca/MT/ S. P. Meyn and R. L. Tweedie "Markov Chains and Stochastic Stability", Springer-Verlag, London (1993)] | #[http://probability.ca/MT/ S. P. Meyn and R. L. Tweedie "Markov Chains and Stochastic Stability", Springer-Verlag, London (1993)] | ||
Latest revision as of 12:56, 14 August 2007
The concept of a Markov chain was developed by Andrey Andreyevich Markov. A Markov chain is a sequence of random variables with the property that it is forgetful of all but its immediate past. For a process Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {\mathbf \Phi}} evolving on a space Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {\mathsf X}} and governed by an overall probability law Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {\mathsf P}} to be a time-homogeneous Markov chain there must be a set of "transition probabilities" Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \{P^n (x,A), x \in {\mathsf X}, A \subset {\mathsf X}\}} for appropriate sets Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A} such that for times Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n,m} in Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {\mathbb Z}_+} (Ref. 1 Eq. 1.1)
- Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {\mathsf P} (\Phi_{n+m} \in A \vert \Phi_j,j \leq m; \Phi_m =x)= P^n(x,A);}
that is Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P^n(x,A)} denotes the probability that a chain at x will be in the set A after n steps, or transitions. The independence of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P^n} on the values of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Phi_j,j \leq m} is the Markov property, and the independence of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P^n} and m is the time-homogeneity property.