THE COMPLETE COLLECTION NECESSARY FOR A CONCRETE UNDERSTANDING OF PROBABILITY
Written in a clear, accessible, and comprehensive manner, the Handbook of Probability presents the fundamentals of probability with an emphasis on the balance of theory, application, and methodology. Utilizing basic examples throughout, the handbook expertly transitions between concepts and practice to allow readers an inclusive introduction to the field of probability.
The book provides a useful format with self-contained chapters, allowing the reader easy and quick reference. Each chapter includes an introduction, historical background, theory and applications, algorithms, and exercises. The Handbook of Probability offers coverage of:
- Probability Space
- Probability Measure
- Random Variables
- Random Vectors in Rn
- Characteristic Function
- Moment Generating Function
- Gaussian Random Vectors
- Convergence Types
- Limit Theorems
The Handbook of Probability is an ideal resource for researchers and practitioners in numerous fields, such as mathematics, statistics, operations research, engineering, medicine, and finance, as well as a useful text for graduate students.
Table of Contents
List of Figures xv
Preface xvii
Introduction xix
1 Probability Space 1
1.1 Introduction/Purpose of the Chapter 1
1.2 Vignette/Historical Notes 2
1.3 Notations and Definitions 2
1.4 Theory and Applications 4
1.4.1 Algebras 4
1.4.2 Sigma Algebras 5
1.4.3 Measurable Spaces 7
1.4.4 Examples 7
1.4.5 The Borel _-Algebra 9
1.5 Summary 12
Exercises 12
2 Probability Measure 15
2.1 Introduction/Purpose of the Chapter 15
2.2 Vignette/Historical Notes 16
2.3 Theory and Applications 17
2.3.1 Definition and Basic Properties 17
2.3.2 Uniqueness of Probability Measures 22
2.3.3 Monotone Class 24
2.3.4 Examples 26
2.3.5 Monotone Convergence Properties of Probability 28
2.3.6 Conditional Probability 31
2.3.7 Independence of Events and _-Fields 39
2.3.8 Borel–Cantelli Lemmas 46
2.3.9 Fatou’s Lemmas 48
2.3.10 Kolmogorov’s Zero–One Law 49
2.4 Lebesgue Measure on the Unit Interval (01] 50
Exercises 52
3 Random Variables: Generalities 63
3.1 Introduction/Purpose of the Chapter 63
3.2 Vignette/Historical Notes 63
3.3 Theory and Applications 64
3.3.1 Definition 64
3.3.2 The Distribution of a Random Variable 65
3.3.3 The Cumulative Distribution Function of a Random Variable 67
3.3.4 Independence of Random Variables 70
Exercises 71
4 Random Variables: The Discrete Case 79
4.1 Introduction/Purpose of the Chapter 79
4.2 Vignette/Historical Notes 80
4.3 Theory and Applications 80
4.3.1 Definition and Basic Facts 80
4.3.2 Moments 84
4.4 Examples of Discrete Random Variables 89
4.4.1 The (Discrete) Uniform Distribution 89
4.4.2 Bernoulli Distribution 91
4.4.3 Binomial (n p) Distribution 92
4.4.4 Geometric (p) Distribution 95
4.4.5 Negative Binomial (r p) Distribution 101
4.4.6 Hypergeometric Distribution (N m n) 102
4.4.7 Poisson Distribution 104
Exercises 108
5 Random Variables: The Continuous Case 119
5.1 Introduction/Purpose of the Chapter 119
5.2 Vignette/Historical Notes 119
5.3 Theory and Applications 120
5.3.1 Probability Density Function (p.d.f.) 120
5.3.2 Cumulative Distribution Function (c.d.f.) 124
5.3.3 Moments 127
5.3.4 Distribution of a Function of the Random Variable 128
5.4 Examples 130
5.4.1 Uniform Distribution on an Interval [ab] 130
5.4.2 Exponential Distribution 133
5.4.3 Normal Distribution (_ _2) 136
5.4.4 Gamma Distribution 139
5.4.5 Beta Distribution 144
5.4.6 Student’s t Distribution 147
5.4.7 Pareto Distribution 149
5.4.8 The Log-Normal Distribution 151
5.4.9 Laplace Distribution 153
5.4.10 Double Exponential Distribution 155
Exercises 156
6 Generating Random Variables 177
6.1 Introduction/Purpose of the Chapter 177
6.2 Vignette/Historical Notes 178
6.3 Theory and Applications 178
6.3.1 Generating One-Dimensional Random Variables by Inverting the Cumulative Distribution Function (c.d.f.) 178
6.3.2 Generating One-Dimensional Normal Random Variables 183
6.3.3 Generating Random Variables. Rejection Sampling Method 186
6.3.4 Generating from a Mixture of Distributions 193
6.3.5 Generating Random Variables. Importance Sampling 195
6.3.6 Applying Importance Sampling 198
6.3.7 Practical Consideration: Normalizing Distributions 201
6.3.8 Sampling Importance Resampling 203
6.3.9 Adaptive Importance Sampling 204
6.4 Generating Multivariate Distributions with Prescribed Covariance Structure 205
Exercises 208
7 Random Vectors in Rn 210
7.1 Introduction/Purpose of the Chapter 210
7.2 Vignette/Historical Notes 210
7.3 Theory and Applications 211
7.3.1 The Basics 211
7.3.2 Marginal Distributions 212
7.3.3 Discrete Random Vectors 214
7.3.4 Multinomial Distribution 219
7.3.5 Testing Whether Counts are Coming from a Specific Multinomial Distribution 220
7.3.6 Independence 221
7.3.7 Continuous Random Vectors 223
7.3.8 Change of Variables. Obtaining Densities of Functions of Random Vectors 229
7.3.9 Distribution of Sums of Random Variables. Convolutions 231
Exercises 236
8 Characteristic Function 255
8.1 Introduction/Purpose of the Chapter 255
8.2 Vignette/Historical Notes 255
8.3 Theory and Applications 256
8.3.1 Definition and Basic Properties 256
8.3.2 The Relationship Between the Characteristic Function and the Distribution 260
8.4 Calculation of the Characteristic Function for Commonly Encountered Distributions 265
8.4.1 Bernoulli and Binomial 265
8.4.2 Uniform Distribution 266
8.4.3 Normal Distribution 267
8.4.4 Poisson Distribution 267
8.4.5 Gamma Distribution 268
8.4.6 Cauchy Distribution 269
8.4.7 Laplace Distribution 270
8.4.8 Stable Distributions. L´evy Distribution 271
8.4.9 Truncated L´evy Flight Distribution 274
Exercises 275
9 Moment-Generating Function 280
9.1 Introduction/Purpose of the Chapter 280
9.2 Vignette/Historical Notes 280
9.3 Theory and Applications 281
9.3.1 Generating Functions and Applications 281
9.3.2 Moment-Generating Functions. Relation with the Characteristic Functions 288
9.3.3 Relationship with the Characteristic Function 292
9.3.4 Properties of the MGF 292
Exercises 294
10 Gaussian Random Vectors 300
10.1 Introduction/Purpose of the Chapter 300
10.2 Vignette/Historical Notes 301
10.3 Theory and Applications 301
10.3.1 The Basics 301
10.3.2 Equivalent Definitions of a Gaussian Vector 303
10.3.3 Uncorrelated Components and Independence 309
10.3.4 The Density of a Gaussian Vector 313
10.3.5 Cochran’s Theorem 316
10.3.6 Matrix Diagonalization and Gaussian Vectors 319
Exercises 325
11 Convergence Types. Almost Sure Convergence. Lp-Convergence. Convergence in Probability 338
11.1 Introduction/Purpose of the Chapter 338
11.2 Vignette/Historical Notes 339
11.3 Theory and Applications: Types of Convergence 339
11.3.1 Traditional Deterministic Convergence Types 339
11.3.2 Convergence of Moments of an r.v. - Convergence in Lp 341
11.3.3 Almost Sure (a.s.) Convergence 342
11.3.4 Convergence in Probability 344
11.4 Relationships Between Types of Convergence 346
11.4.1 a.s. and Lp 347
11.4.2 Probability and a.s./Lp 351
11.4.3 Uniform Integrability 357
Exercises 359
12 Limit Theorems 372
12.1 Introduction/Purpose of the Chapter 372
12.2 Vignette/Historical Notes 372
12.3 Theory and Applications 375
12.3.1 Weak Convergence 375
12.3.2 The Law of Large Numbers 384
12.4 Central Limit Theorem 401
Exercises 409
13 Appendix A: Integration Theory. General Expectations 421
13.1 Integral of Measurable Functions 422
13.1.1 Integral of Simple (Elementary) Functions 422
13.1.2 Integral of Positive Measurable Functions 424
13.1.3 Integral of Measurable Functions 428
13.2 General Expectations and Moments of a Random Variable 429
13.2.1 Moments and Central Moments. Lp Space 430
13.2.2 Variance and the Correlation Coefficient 431
13.2.3 Convergence Theorems 433
14 Appendix B: Inequalities Involving Random Variables and Their Expectations 434
14.1 Functions of Random Variables. The Transport Formula 441
Bibliography 445
Index 447
