# Tag Archives: markov process

It is said that Markov originally invented Markov processes to understand how some letters follow other letters in poetry. Recall that a Markov process is a probability random process that models moving from one state to another state, where the possible states is some set. There is a fixed probability from moving from each state […]

## Cereal box prizes and transition matrices

If you don't know what a transition matrix is, you might want to read the transition matrix post before reading this one. Transition matrices can be used to solve some classic probability problems. For example, consider the following problem: Suppose in each cereal box you buy there is one number in the set $\{1,2,3,4,5\}$. You […]

## Transition matrices

Imagine $n$ states of a system in a discrete-time stochastic system. For each pair of states $i$ and $j$, there is a probability $p_{ij}$ of moving to state $j$ in the next time step, given that the system is in state $i$. Each of these probabilities can be put in a matrix, known as the […]