10: Markov Chains
In this chapter, you will learn to:
- Write transition matrices for Markov Chain problems.
- Explore some ways in which Markov Chains models are used in business, finance, public health and other fields of application
- Find the long term trend for a Regular Markov Chain.
- Solve and interpret Absorbing Markov Chains.
-
- 10.1: Introduction to Markov Chains
- We will now study stochastic processes, experiments in which the outcomes of events depend on the previous outcomes; stochastic processes involve random outcomes that can be described by probabilities. Such a process or experiment is called a Markov Chain or Markov process. The process was first studied by a Russian mathematician named Andrei A. Markov in the early 1900s.
Thumbnail: A diagram representing a two-state Markov process, with the states labelled E and A. Each number represents the probability of the Markov process changing from one state to another state, with the direction indicated by the arrow. For example, if the Markov process is in state A, then the probability it changes to state E is 0.4, while the probability it remains in state A is 0.6. (CC BY-SA 3.0; Joxemai4 via Wikipedia)