Probability and Stochastic Processes - A Friendly Introduction for Electrical and Computer Engineers, Fourth Edition serves as an accessible guide for engineering students delving into the realms of probability theory and stochastic processes.This text strikes a balance between rigorous mathematical exposition and clear, intuitive explanations, ensuring that students grasp the fundamental concepts essential for applying mathematics to real-world engineering challenges. Enhanced with the practical MATLAB applications. The book offers students valuable hands-on experienceto reinforce the theoretical material.
This International adaptation has been thoroughly revised and updated. Notably, it includes a new chapter on Probabilistic Inequalities and Bounds. The sections on Stochastic Processes and Sums of Random Variables have been comprehensively enhanced to encompass additional topics, aligning with the latest curriculum requirements. With an array of new and updated examples, quizzes, and end-of-chapter problems, the book provides robust support to students, particularly in bridging the gap between theoretical probability and its practical applications in engineering.
Table of Contents
Preface vii
1 Random Experiments, Models, and Probabilities 1
Getting Started with Probability 1
1.1 Applying Set Theory to Probability 2
1.2 Probability Axioms 7
1.3 Conditional Probability 10
1.4 Partitions and the Law of Total Probability 13
1.5 Bayes’ Theorem 17
1.6 Independence 18
1.7 Matlab 22
Problems 24
2 Sequential Random Experiments 31
2.1 Tree Diagrams 31
2.2 Counting Methods 35
2.3 Independent Trials 43
2.4 Matlab 46
Problems 48
3 Discrete Random Variables 53
3.1 Definitions 53
3.2 Probability Mass Function 56
3.3 Families of Discrete Random Variables 59
3.4 Cumulative Distribution Function (CDF) 65
3.5 Averages and Expected Value 69
3.6 Functions of a Random Variable 74
3.7 Expected Value of a Derived Random Variable 77
3.8 Variance and Standard Deviation 80
3.9 Matlab 86
Problems 93
4 Continuous Random Variables 103
4.1 Continuous Sample Space 103
4.2 The Cumulative Distribution Function 105
4.3 Probability Density Function 108
4.4 Expected Values 113
4.5 Families of Continuous Random Variables 116
4.6 Gaussian Random Variables 122
4.7 Delta Functions, Mixed Random Variables 128
4.8 Matlab 134
Problems 136
5 Multiple Random Variables 145
5.1 Joint Cumulative Distribution Function 146
5.2 Joint Probability Mass Function 149
5.3 Marginal PMF 152
5.4 Joint Probability Density Function 154
5.5 Marginal PDF 159
5.6 Independent Random Variables 161
5.7 Expected Value of a Function of Two Random Variables 164
5.8 Covariance, Correlation and Independence 167
5.9 Bivariate Gaussian Random Variables 174
5.10 Multivariate Probability Models 178
5.11 Matlab 183
Problems 188
6 Probability Models of Derived Random Variables 199
6.1 PMF of a Function of Two Discrete Random Variables 200
6.2 Functions Yielding Continuous Random Variables 201
6.3 Functions Yielding Discrete or Mixed Random Variables 207
6.4 Continuous Functions of Two Continuous Random Variables 211
6.5 PDF of the Sum of Two Random Variables 214
6.6 Matlab 216
Problems 217
7 Conditional Probability Models 225
7.1 Conditioning a Random Variable by an Event 225
7.2 Conditional Expected Value Given an Event 231
7.3 Conditioning Two Random Variables by an Event 233
7.4 Conditioning by a Random Variable 237
7.5 Conditional Expected Value Given a Random Variable 241
7.6 Bivariate Gaussian Random Variables: Conditional PDFs 245
7.7 Matlab 248
Problems 249
8 Random Vectors 257
8.1 Vector Notation 257
8.2 Independent Random Variables and Random Vectors 260
8.3 Functions of Random Vectors 261
8.4 Expected Value Vector and Correlation Matrix 265
8.5 Gaussian Random Vectors 270
8.6 Matlab 277
Problems 279
9 Sums of Random Variables 285
9.1 Expected Values of Sums 285
9.2 Moment Generating Functions 289
9.3 MGF of the Sum of Independent Random Variables 293
9.4 Characteristic Function and Probability Generating Function 297
9.5 Matlab 301
Problems 304
10 Hypothesis Testing 307
10.1 Significance Testing 308
10.2 Binary Hypothesis Testing 311
10.3 Multiple Hypothesis Test 324
10.4 Matlab 327
Problems 329
11 Estimation of a Random Variable 339
11.1 Minimum Mean Square Error Estimation 339
11.2 Linear Estimation of X given Y 344
11.3 MAP and ML Estimation 349
11.4 Linear Estimation of Random Variables from Random Vectors 353
11.5 Matlab 360
Problems 362
12 Some Probabilistic Inequalities and Bounds 369
12.1 Markov Inequality 369
12.2 Chebyshev’s Inequality 373
12.3 Chernoff Bound 374
12.4 Central Limit Theorem 376
12.5 Sample Mean and Variance 380
12.6 Laws of Large Numbers (LLN) 382
Problems 384
13 Stochastic Processes and Markov Chains 391
13.1 Definitions and Examples 391
13.2 Random Variables from Random Processes 397
13.3 Independent, Identically Distributed Random Sequences 399
13.4 The Poisson Process 400
13.5 Properties of the Poisson Process 404
13.6 The Brownian Motion Process 407
13.7 Markov Process 409
13.8 Discrete-Time Markov Chains 410
13.9 Higher Transition Probabilities: Chapman-Kolmogorov Equations 414
13.10 Long-Run Behavior of Markov Chains 419
13.11 Classification of States of Chains 422
13.12 Markov Chains with Countably Infinite States 426
13.13 Ergodic and Reducible Chains 429
13.14 Birth Process and Death Process 433
13.15 Queuing Models - Poisson Queues 435
13.16 Matlab 441
Problems 447
14 Stationary Processes and Random Signal Processing 457
14.1 Expected Value and Correlation 457
14.2 Stationary Processes 460
14.3 Wide Sense Stationary Processes 463
14.4 Cross-Correlation 466
14.5 Gaussian Processes 469
14.6 Linear Filtering of Continuous-Time Stochastic Processes 471
14.7 Linear Filtering of a Random Sequence 475
14.8 Discrete-Time Linear Filtering: Vectors and Matrices 481
14.9 Power Spectral Density of a Continuous-Time Process 485
14.10 Power Spectral Density of a Random Sequence 490
14.11 Cross Power Spectral Density 494
14.12 Frequency Domain Filter Relationships 496
14.13 Matlab 501
Problems 510
Appendix A The Sample Mean 517
A 1 Sample Mean: Expected Value and Variance 517
A 2 Deviation of a Random Variable from the Expected Value 519
A 3 Laws of Large Numbers 523
A 4 Point Estimates of Model Parameters 525
A 5 Confidence Intervals 531
A 6 Matlab 538
Appendix B Families of Random Variables 541
B 1 Discrete Random Variables 541
B 2 Continuous Random Variables 543
Appendix C A Few Math Facts 547
References 553
Index 555
