Markov Chain Process: Theory and Cases is designed for students of natural and formal sciences. It explains the fundamentals related to a stochastic process that satisfies the Markov property. It presents 10 structured chapters that provide a comprehensive insight into the complexity of this subject by presenting many examples and case studies that will help readers to deepen their acquired knowledge and relate learned theory to practice.
This book is divided into four parts. The first part thoroughly examines the definitions of probability, independent events, mutually (and not mutually) exclusive events, conditional probability, and Bayes’ theorem, which are essential elements in Markov’s theory. The second part examines the elements of probability vectors, stochastic matrices, regular stochastic matrices, and fixed points. The third part presents multiple cases in various disciplines: Predictive computational science, Urban complex systems, Computational finance, Computational biology, Complex systems theory, and Computational Science in Engineering. The last part introduces learners to Fortran 90 programs and Linux scripts.
To make the comprehension of Markov Chain concepts easier, all the examples, exercises, and case studies presented in this book are completely solved and given in a separate section.
This book serves as a textbook (either primary or auxiliary) for students required to understand Markov Chains in their courses, and as a reference book for researchers who want to learn about methods that involve Markov Processes.
This book is divided into four parts. The first part thoroughly examines the definitions of probability, independent events, mutually (and not mutually) exclusive events, conditional probability, and Bayes’ theorem, which are essential elements in Markov’s theory. The second part examines the elements of probability vectors, stochastic matrices, regular stochastic matrices, and fixed points. The third part presents multiple cases in various disciplines: Predictive computational science, Urban complex systems, Computational finance, Computational biology, Complex systems theory, and Computational Science in Engineering. The last part introduces learners to Fortran 90 programs and Linux scripts.
To make the comprehension of Markov Chain concepts easier, all the examples, exercises, and case studies presented in this book are completely solved and given in a separate section.
This book serves as a textbook (either primary or auxiliary) for students required to understand Markov Chains in their courses, and as a reference book for researchers who want to learn about methods that involve Markov Processes.
Table of Contents
1 Probability1.1 Introduction
1.2 Basic Definitions
1.3 Axiomatic Construction
1.4 Properties
1.5 Conditional Probability
1.6 Random Event Types
1.7 P(A ∩ B) and P(A|B)
1.8 Independent Random Events
1.9 Dependent Random Events
1.10 Bayes’ Theorem
1.11 Conclusions
1.12 Exercises
2 Matrix Models
2.1 Introduction
2.2 Malthus Model
2.3 Leslie Model
2.4 Lefkovitch Model
2.5 Discrete-Time Markov Chain Process
2.6 Continuous-Time Markov Chain Process
2.7 Stability Matrix Models
2.8 Conclusions
2.9 Exercises
3 Random Walks
3.1 Introduction
3.2 Random Walk
3.3 One-Dimensional Random Walk
3.4 Two-Dimensional Random Walk
3.5 Three-Dimensional Random Walk
3.6 Gaussian-Dimensional Random Walk
3.7 Markov-Dimensional Random Walk
3.8 Conclusions 25 Ii Basic Concepts
4 Markov Chain Process
4.1 Introduction
4.2 Definition
4.3 Matrix of Transition Probabilities
4.4 Regular Matrix
4.5 Absorbing Matrix
4.6 Vector of Initial Conditions
4.7 Markov Chain Process
4.8 Discrete and Continuous-Time Markov Chain Process
4.9 Conclusions
4.10 Exercises
5 Discrete-Time Markov Chain Process
5.1 Introduction
5.2 Definition
5.3 Calculating the Stationary Distribution
5.3.1 Steady-State Vector
5.3.2 Eigenvalues and Eigenvectors
5.4 Classification of States
5.4.1 Equivalence Relation
5.4.2 Irreducible Form
5.4.3 Recurrent Form
5.4.4 Transient Form
5.5 Law of Large Numbers
5.6 Metropolis-Hastings Algorithm
5.7 Conclusions
5.8 Exercises
6 Continuous-Time Markov Chain Process
6.1 Introduction
6.2 Distribution Functions
6.2.1 Exponential Distribution
6.2.2 Poisson Distribution
6.3 Markov Matrix
6.3.1 Irreducible Matrix
6.3.2 Aperiodic Matrix
6.3.3 Ergodicity
6.4 Stationary Distribution
6.5 Steady-State Vector
6.5.1 System of Linear Equations
6.5.2 Transition Matrix Diagonalisation
6.6 Conclusions
6.7 Exercises 62 Iii Cases
7 Computational Urban Issues
7.1 Frequent Mobility Routes-Discrete-Time Case
7.1.1 Introduction
7.1.2 Materials and Methods
7.1.3 Results
7.1.4 Discussion
7.1.5 Conclusions
7.2 Frequent Mobility Routes-Continuous-Time Case
7.2.1 Introduction
7.2.2 Materials and Methods
7.2.3 Results
7.2.4 Discussion
7.2.5 Conclusions
7.3 Remarks
8 Computational Biology Issues
8.1 Protein Structure - Discrete-Time Case
8.1.1 Introduction
8.1.2 Materials and Methods
8.1.3 Results
8.1.4 Discussion
8.1.5 Conclusions
8.2 Protein Structure - Continuous-Time Case
8.2.1 Introduction
8.2.2 Materials and Methods
8.2.3 Results
8.2.4 Discussion
8.2.5 Conclusions
8.3 Remarks
9 Computational Financial Issues
9.1 Prediction of Market Trends-Discrete-Time Case
9.1.1 Instroduction
9.1.2 Materials and Methods
9.1.3 Results
9.1.4 Discussion
9.1.5 Conclusions
9.2 Prediction of Market Trends-Continuous-Time Case
9.2.1 Introduction
9.2.2 Materials and Methods
9.2.3 Results
9.2.4 Discussion
9.2.5 Conclusions
9.3 Remarks
10 Computational Science Issues
10.1 Hierarchical Markov Chain Process - Discrete-Time Case
10.1.1 Introduction
10.1.2 Materials and Methods
10.1.3 Results
10.1.4 Discussion
10.1.5 Conclusions
10.2 Hierarchical Continuous-Time Markov Chain Process Case
10.2.1 Introduction
10.2.2 Materials and Methods
10.2.3 Results
10.2.4 Discussion
10.2.5 Conclusions
10.3 Remarks
11 Computational Medicine Issues
11.1 Pandemic Spread Rate-Discrete-Time Case
11.1.1 Instroduction
11.1.2 Materials and Methods
11.1.3 Results
11.1.4 Discussion
11.1.5 Conclusions
11.2 Pandemic Spread Rate-Continuous-Time Case
11.2.1 Introduction
11.2.2 Materials and Methods
11.2.3 Results
11.2.4 Discussion
11.2.5 Conclusions
11.3 Remarks
12 Computational Social Sciences Issues
12.1 Natural Language Recognition - Discrete-Time Case
12.1.1 Instroduction
12.1.2 Materials and Methods
12.1.3 Results
12.1.4 Discussion
12.1.5 Conclusions
12.2 Natural Language Recognition - Continuous-Time Case
12.2.1 Introduction
12.2.2 Materials and Methods
12.2.3 Discussion
12.2.4 Conclusions
12.3 Remarks
13 Computational Operations Research Issues
13.1 Queuing Theory-Discrete-Time Case
13.1.1 Instroduction
13.1.2 Materials and Methods
13.1.3 Results
13.1.4 Discussion
13.1.5 Conclusions
13.2 Queuing Theory-Continuous-Time Case
13.2.1 Introduction
13.2.2 Materials and Methods
13.2.3 Results
13.2.4 Discussion
13.2.5 Conclusions
13.3 Remarks
14 Computational Information System Issues
14.1 Pagerank System - Discrete-Time Case
14.1.1 Instroduction
14.1.2 Materials and Methods
14.1.3 Discussion
14.1.4 Conclusions
14.2 Pagerank System-Continuous-Time Case
14.2.1 Introduction
14.2.2 Materials and Methods
14.2.3 Discussion
14.2.4 Conclusions
14.3 Remarks
15 Future Uses
15.1 Scope
15.2 Main Possibilities
16 Solutions 121 a Computational Programs
16.1 One-Dimensional Random Walk Program
16.2 Two-Dimensional Random Walk Program
16.3 Three-Dimensional Random Walk Program
16.4 Gaussian-Dimensional Random Walk Program
16.5 Calculation of Row-Vector of Initial Conditions
16.6 Calculation of U = Upn Over 3 × 3 Matrix
16.7 Calculation of U = Upn Over 5 × 5 Matrix
16.8 Calculation of U = Upn Over 5 × 5 Matrix
16.9 Calculation of U = Upn Over 4 × 4 Matrix
16.10 Urban Issues Program
16.11 Biology Issues Program
16.12 Financial Issues Program
16.13 Medicine Issues Program
16.14 Social Issues Program
16.15 Operations Research Issues Program
Author
- Carlos Polanco
