Good book on markov chains


Ambassador to the humans
But as Spunky said, how are you going to use them. The primary use I can think of is for Bayesian Statistics. Would you want Bayesian book suggestions?
Not necessarily. I think what you're referring to is MCMC but markov chains have a much wider range of uses than just MCMC.


Omega Contributor
Yes, get MC gets used for a wide array of topics. I typically think about them getting at expectations of complex high dimensional targets and that was my point. My recommendations will be different than others.


Fortran must die
I plan to use them to model how things work in our work processes. For example, this was not done as a markov chain, I built a tool years ago that showed how many would be on the waiting list given specific releases. A wide range of factors influenced this, I created it utilizing my understanding of the process and historic data.

But I suspect you could do this a lot better using Markov models. MCMC I already know about and can run. Its not what I am interested in.
Noetsi, so are you interested in

1) an introductory book where all major properties of discrete-time and continuous-time Markov chains are nicely laid out


2) a more advanced book showing applications of Markov chains to real-life processes?

Or are you interested in most general Markov processes, where the state space is continuous?


Fortran must die
An introductory book to start with. I have had a few courses in operations research, but almost nothing on markov processes (except in the context of algorithms for missing values).
Chapters 4-7 of

Ross, S. M. (2009). Introduction to Probability Models (10th ed). Academic Press.

will serve as a rigorous introduction into Markov chains and closely related stochastic processes. Later on, when you solve actual problems, you may fall back on

Lawler, G. F. (1995). Introduction to Stochastic Processes. New York: Chapman and Hall/CRC.

who has some of the properties compiled more concisely. For an interesting example of applying Markov chains to credit rating migrations (Moody's, S&P), see

Duffie, D., & Singleton, K. (2003). Credit Risk: Pricing, Measurement, and Management. Princeton University Press.

The book also contains applications in the area of default modeling.