Markov Chains

I. Discrete Parameter.- § 1. Fundamental definitions.- § 2. Transition probabilities.- § 3. Classification of states.- § 4. Recurrence.- § 5. Criteria and examples.- § 6. The main limit theorem.- § 7. Various complements.- § 8. Repetitive pattern and renewal process.- § 9. Taboo probabilities.- § 10. The generating function.- § 11. The moments of first entrance time distributions.- § 12. A random walk example.- § 13. System theorems.- § 14. Functionals and associated random variables.- § 15. Ergodic theorems.- § 16. Further limit theorems.- § 17. Almost closed and sojourn sets.- II. Continuous Parameter.- § 1. Transition matrix: basic properties.- § 2. Standard transition matrix.- § 3. Differentiability.- § 4. Definitions and measure-theoretic foundations.- § 5. The sets of constancy.- § 6. Continuity properties of sample functions.- § 7. Further specifications of the process.- § 8. Optional random variable.- § 9. Strong Markov property.- § 10. Classification of states.- § 11. Taboo probability functions.- § 12. Last exit time.- § 13. Ratio limit theorems; discrete approximations.- § 14. Functionals.- § 15. Post-exit process.- § 16. Imbedded renewal process.- § 17. The two systems of differential equations.- § 18. The minimal solution.- § 19. The first infinity.- § 20. Examples.