Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in or
create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
| Latest revision |
Your text |
| Line 1: |
Line 1: |
| The concept of a '''Markov chain''' was developed by Andrey Andreyevich Markov. A Markov chain is a sequence of random variables with the property that it is forgetful of all but its immediate past. | | The concept of a '''Markov chain''' was developed by Andrey Andreyevich Markov. |
| For a process <math>{\mathbf \Phi}</math> evolving on a space <math>{\mathsf X}</math> and governed by an overall probability law <math>{\mathsf P}</math> to be a time-homogeneous Markov chain there must be a set of "transition probabilities" <math>\{P^n (x,A), x \in {\mathsf X}, A \subset {\mathsf X}\}</math> for appropriate sets <math>A</math> such that
| |
| for times <math>n,m</math> in <math>{\mathbb Z}_+</math> (Ref. 1 Eq. 1.1)
| |
| | |
| :<math>{\mathsf P} (\Phi_{n+m} \in A \vert \Phi_j,j \leq m; \Phi_m =x)= P^n(x,A);</math>
| |
| | |
| that is <math>P^n(x,A)</math> denotes the probability that a chain at ''x'' will be in the set ''A'' after ''n'' steps, or transitions. The independence of <math>P^n</math> on the values of <math>\Phi_j,j \leq m</math> is the Markov property,
| |
| and the independence of <math>P^n</math> and ''m'' is the time-homogeneity property.
| |
| ==References== | | ==References== |
| #[http://probability.ca/MT/ S. P. Meyn and R. L. Tweedie "Markov Chains and Stochastic Stability", Springer-Verlag, London (1993)] | | #[http://probability.ca/MT/ S. P. Meyn and R. L. Tweedie "Markov Chains and Stochastic Stability", Springer-Verlag, London (1993)] |