A Markov chain is a random process with the property that given the values of the process from time zero up through the current time, the conditional probability of the value of the process at any ...