Category Archives: probability

It is said that Markov originally invented Markov processes to understand how some letters follow other letters in poetry. Recall that a Markov process is a probability random process that models moving from one state to another state, where the possible states is some set. There is a fixed probability from moving from each state […]

## Cereal box prizes and transition matrices

If you don't know what a transition matrix is, you might want to read the transition matrix post before reading this one. Transition matrices can be used to solve some classic probability problems. For example, consider the following problem: Suppose in each cereal box you buy there is one number in the set $\{1,2,3,4,5\}$. You […]

## Transition matrices

Imagine $n$ states of a system in a discrete-time stochastic system. For each pair of states $i$ and $j$, there is a probability $p_{ij}$ of moving to state $j$ in the next time step, given that the system is in state $i$. Each of these probabilities can be put in a matrix, known as the […]

## Expected iterations for a finite random walk

Consider three cells as so: A player (the blue disc) starts out in the left-most cell, and discrete time starts. At each step in time, the player has a 1/2 probability of moving left and a 1/2 probability of moving right. If the player chooses to move left but cannot because it is in the […]