A comprehensive and timely edition on an emerging new trend in time series
Linear Models and Time-Series Analysis: Regression, ANOVA, ARMA and GARCH sets a strong foundation, in terms of distribution theory, for the linear model (regression and ANOVA), univariate time series analysis (ARMAX and GARCH), and some multivariate models associated primarily with modeling financial asset returns (copula-based structures and the discrete mixed normal and Laplace). It builds on the author's previous book, Fundamental Statistical Inference: A Computational Approach, which introduced the major concepts of statistical inference. Attention is explicitly paid to application and numeric computation, with examples of Matlab code throughout. The code offers a framework for discussion and illustration of numerics, and shows the mapping from theory to computation.
The topic of time series analysis is on firm footing, with numerous textbooks and research journals dedicated to it. With respect to the subject/technology, many chapters in Linear Models and Time-Series Analysis cover firmly entrenched topics (regression and ARMA). Several others are dedicated to very modern methods, as used in empirical finance, asset pricing, risk management, and portfolio optimization, in order to address the severe change in performance of many pension funds, and changes in how fund managers work.
- Covers traditional time series analysis with new guidelines
- Provides access to cutting edge topics that are at the forefront of financial econometrics and industry
- Includes latest developments and topics such as financial returns data, notably also in a multivariate context
- Written by a leading expert in time series analysis
- Extensively classroom tested
- Includes a tutorial on SAS
- Supplemented with a companion website containing numerous Matlab programs
- Solutions to most exercises are provided in the book
Linear Models and Time-Series Analysis: Regression, ANOVA, ARMA and GARCH is suitable for advanced masters students in statistics and quantitative finance, as well as doctoral students in economics and finance. It is also useful for quantitative financial practitioners in large financial institutions and smaller finance outlets.
Table of Contents
Preface xiii
Part I Linear Models: Regression and ANOVA 1
1 The Linear Model 3
1.1 Regression, Correlation, and Causality 3
1.2 Ordinary and Generalized Least Squares 7
1.2.1 Ordinary Least Squares Estimation 7
1.2.2 Further Aspects of Regression and OLS 8
1.2.3 Generalized Least Squares 12
1.3 The Geometric Approach to Least Squares 17
1.3.1 Projection 17
1.3.2 Implementation 22
1.4 Linear Parameter Restrictions 26
1.4.1 Formulation and Estimation 27
1.4.2 Estimability and Identifiability 30
1.4.3 Moments and the Restricted GLS Estimator 32
1.4.4 Testing With h = 0 34
1.4.5 Testing With Nonzero h 37
1.4.6 Examples 37
1.4.7 Confidence Intervals 42
1.5 Alternative Residual Calculation 47
1.6 Further Topics 51
1.7 Problems 56
1.A Appendix: Derivation of the BLUS Residual Vector 60
1.B Appendix: The Recursive Residuals 64
1.C Appendix: Solutions 66
2 Fixed Effects ANOVA Models 77
2.1 Introduction: Fixed, Random, and Mixed Effects Models 77
2.2 Two Sample t-Tests for Differences in Means 78
2.3 The Two Sample t-Test with Ignored Block Effects 84
2.4 One-Way ANOVA with Fixed Effects 87
2.4.1 The Model 87
2.4.2 Estimation and Testing 88
2.4.3 Determination of Sample Size 91
2.4.4 The ANOVA Table 93
2.4.5 Computing Confidence Intervals 97
2.4.6 A Word on Model Assumptions 103
2.5 Two-Way Balanced Fixed Effects ANOVA 107
2.5.1 The Model and Use of the Interaction Terms 107
2.5.2 Sums of Squares Decomposition without Interaction 108
2.5.3 Sums of Squares Decomposition with Interaction 113
2.5.4 Example and Codes 117
3 Introduction to Random and Mixed Effects Models 127
3.1 One-Factor Balanced Random Effects Model 128
3.1.1 Model and Maximum Likelihood Estimation 128
3.1.2 Distribution Theory and ANOVA Table 131
3.1.3 Point Estimation, Interval Estimation, and Significance Testing 137
3.1.4 Satterthwaite’s Method 139
3.1.5 Use of SAS 142
3.1.6 Approximate Inference in the Unbalanced Case 143
3.1.6.1 Point Estimation in the Unbalanced Case 144
3.1.6.2 Interval Estimation in the Unbalanced Case 150
3.2 Crossed Random Effects Models 152
3.2.1 Two Factors 154
3.2.1.1 With Interaction Term 154
3.2.1.2 Without Interaction Term 157
3.2.2 Three Factors 157
3.3 Nested Random Effects Models 162
3.3.1 Two Factors 162
3.3.1.1 Both Effects Random: Model and Parameter Estimation 162
3.3.1.2 Both Effects Random: Exact and Approximate Confidence Intervals 167
3.3.1.3 Mixed Model Case 170
3.3.2 Three Factors 174
3.3.2.1 All Effects Random 174
3.3.2.2 Mixed: Classes Fixed 176
3.3.2.3 Mixed: Classes and Subclasses Fixed 177
3.4 Problems 177
3.A Appendix: Solutions 178
Part II Time-Series Analysis: ARMAX Processes 185
4 The AR(1) Model 187
4.1 Moments and Stationarity 188
4.2 Order of Integration and Long-Run Variance 195
4.3 Least Squares and ML Estimation 196
4.3.1 OLS Estimator of a 196
4.3.2 Likelihood Derivation I 196
4.3.3 Likelihood Derivation II 198
4.3.4 Likelihood Derivation III 198
4.3.5 Asymptotic Distribution 199
4.4 Forecasting 200
4.5 Small Sample Distribution of the OLS and ML Point Estimators 204
4.6 Alternative Point Estimators of a 208
4.6.1 Use of the Jackknife for Bias Reduction 208
4.6.2 Use of the Bootstrap for Bias Reduction 209
4.6.3 Median-Unbiased Estimator 211
4.6.4 Mean-Bias Adjusted Estimator 211
4.6.5 Mode-Adjusted Estimator 212
4.6.6 Comparison 213
4.7 Confidence Intervals for a 215
4.8 Problems 219
5 Regression Extensions: AR(1) Errors and Time-varying Parameters 223
5.1 The AR(1) Regression Model and the Likelihood 223
5.2 OLS Point and Interval Estimation of a 225
5.3 Testing a = 0 in the ARX(1) Model 229
5.3.1 Use of Confidence Intervals 229
5.3.2 The Durbin-Watson Test 229
5.3.3 Other Tests for First-order Autocorrelation 231
5.3.4 Further Details on the Durbin-Watson Test 236
5.3.4.1 The Bounds Test, and Critique of Use of p-Values 236
5.3.4.2 Limiting Power as a → ±1 239
5.4 Bias-Adjusted Point Estimation 243
5.5 Unit Root Testing in the ARX(1) Model 246
5.5.1 Null is a = 1 248
5.5.2 Null is a < 1 256
5.6 Time-Varying Parameter Regression 259
5.6.1 Motivation and Introductory Remarks 260
5.6.2 The Hildreth-Houck Random Coefficient Model 261
5.6.3 The TVP Random Walk Model 269
5.6.3.1 Covariance Structure and Estimation 271
5.6.3.2 Testing for Parameter Constancy 274
5.6.4 Rosenberg Return to Normalcy Model 277
6 Autoregressive and Moving Average Processes 281
6.1 AR(p) Processes 281
6.1.1 Stationarity and Unit Root Processes 282
6.1.2 Moments 284
6.1.3 Estimation 287
6.1.3.1 Without Mean Term 287
6.1.3.2 Starting Values 290
6.1.3.3 With Mean Term 292
6.1.3.4 Approximate Standard Errors 293
6.2 Moving Average Processes 294
6.2.1 MA(1) Process 294
6.2.2 MA(q) Processes 299
6.3 Problems 301
6.A Appendix: Solutions 302
7 ARMA Processes 311
7.1 Basics of ARMA Models 311
7.1.1 The Model 311
7.1.2 Zero Pole Cancellation 312
7.1.3 Simulation 313
7.1.4 The ARIMA(p, d, q) Model 314
7.2 Infinite AR and MA Representations 315
7.3 Initial Parameter Estimation 317
7.3.1 Via the Infinite AR Representation 318
7.3.2 Via Infinite AR and Ordinary Least Squares 318
7.4 Likelihood-Based Estimation 322
7.4.1 Covariance Structure 322
7.4.2 Point Estimation 324
7.4.3 Interval Estimation 328
7.4.4 Model Mis-specification 330
7.5 Forecasting 331
7.5.1 AR(p) Model 331
7.5.2 MA(q) and ARMA(p, q) Models 335
7.5.3 ARIMA(p, d, q) Models 339
7.6 Bias-Adjusted Point Estimation: Extension to the ARMAX(1, q) model 339
7.7 Some ARIMAX Model Extensions 343
7.7.1 Stochastic Unit Root 344
7.7.2 Threshold Autoregressive Models 346
7.7.3 Fractionally Integrated ARMA (ARFIMA) 347
7.8 Problems 349
7.A Appendix: Generalized Least Squares for ARMA Estimation 351
7.B Appendix: Multivariate AR(p) Processes and Stationarity, and General Block Toeplitz Matrix Inversion 357
8 Correlograms 359
8.1 Theoretical and Sample Autocorrelation Function 359
8.1.1 Definitions 359
8.1.2 Marginal Distributions 365
8.1.3 Joint Distribution 371
8.1.3.1 Support 371
8.1.3.2 Asymptotic Distribution 372
8.1.3.3 Small-Sample Joint Distribution Approximation 375
8.1.4 Conditional Distribution Approximation 381
8.2 Theoretical and Sample Partial Autocorrelation Function 384
8.2.1 Partial Correlation 384
8.2.2 Partial Autocorrelation Function 389
8.2.2.1 TPACF: First Definition 389
8.2.2.2 TPACF: Second Definition 390
8.2.2.3 Sample Partial Autocorrelation Function 392
8.3 Problems 396
8.A Appendix: Solutions 397
9 ARMA Model Identification 405
9.1 Introduction 405
9.2 Visual Correlogram Analysis 407
9.3 Significance Tests 412
9.4 Penalty Criteria 417
9.5 Use of the Conditional SACF for Sequential Testing 421
9.6 Use of the Singular Value Decomposition 436
9.7 Further Methods: Pattern Identification 439
Part III Modeling Financial Asset Returns 443
10 Univariate GARCH Modeling 445
10.1 Introduction 445
10.2 Gaussian GARCH and Estimation 450
10.2.1 Basic Properties 451
10.2.2 Integrated GARCH 452
10.2.3 Maximum Likelihood Estimation 453
10.2.4 Variance Targeting Estimator 459
10.3 Non-Gaussian ARMA-APARCH, QMLE, and Forecasting 459
10.3.1 Extending the Volatility, Distribution, and Mean Equations 459
10.3.2 Model Mis-specification and QMLE 464
10.3.3 Forecasting 467
10.4 Near-Instantaneous Estimation of NCT-APARCH(1,1) 468
10.5 S𝛼,𝛽-APARCH and Testing the IID Stable Hypothesis 473
10.6 Mixed Normal GARCH 477
10.6.1 Introduction 477
10.6.2 The MixN(k)-GARCH(r, s) Model 478
10.6.3 Parameter Estimation and Model Features 479
10.6.4 Time-Varying Weights 482
10.6.5 Markov Switching Extension 484
10.6.6 Multivariate Extensions 484
11 Risk Prediction and Portfolio Optimization 487
11.1 Value at Risk and Expected Shortfall Prediction 487
11.2 MGARCH Constructs Via Univariate GARCH 493
11.2.1 Introduction 493
11.2.2 The Gaussian CCC and DCC Models 494
11.2.3 Morana Semi-Parametric DCC Model 497
11.2.4 The COMFORT Class 499
11.2.5 Copula Constructions 503
11.3 Introducing Portfolio Optimization 504
11.3.1 Some Trivial Accounting 504
11.3.2 Markowitz and DCC 510
11.3.3 Portfolio Optimization Using Simulation 513
11.3.4 The Univariate Collapsing Method 516
11.3.5 The ES Span 521
12 Multivariate t Distributions 525
12.1 Multivariate Student’s t 525
12.2 Multivariate Noncentral Student’s t 530
12.3 Jones Multivariate t Distribution 534
12.4 Shaw and Lee Multivariate t Distributions 538
12.5 The Meta-Elliptical t Distribution 540
12.5.1 The FaK Distribution 541
12.5.2 The AFaK Distribution 542
12.5.3 FaK and AFaK Estimation: Direct Likelihood Optimization 546
12.5.4 FaK and AFaK Estimation: Two-Step Estimation 548
12.5.5 Sums of Margins of the AFaK 555
12.6 MEST: Marginally Endowed Student’s t 556
12.6.1 SMESTI Distribution 557
12.6.2 AMESTI Distribution 558
12.6.3 MESTI Estimation 561
12.6.4 AoNm-MEST 564
12.6.5 MEST Distribution 573
12.7 Some Closing Remarks 574
12.A ES of Convolution of AFaK Margins 575
12.B Covariance Matrix for the FaK 581
13 Weighted Likelihood 587
13.1 Concept 587
13.2 Determination of Optimal Weighting 592
13.3 Density Forecasting and Backtest Overfitting 594
13.4 Portfolio Optimization Using (A)FaK 600
14 Multivariate Mixture Distributions 611
14.1 The Mixk Nd Distribution 611
14.1.1 Density and Simulation 612
14.1.2 Motivation for Use of Mixtures 612
14.1.3 Quasi-Bayesian Estimation and Choice of Prior 614
14.1.4 Portfolio Distribution and Expected Shortfall 620
14.2 Model Diagnostics and Forecasting 623
14.2.1 Assessing Presence of a Mixture 623
14.2.2 Component Separation and Univariate Normality 625
14.2.3 Component Separation and Multivariate Normality 629
14.2.4 Mixed Normal Weighted Likelihood and Density Forecasting 631
14.2.5 Density Forecasting: Optimal Shrinkage 633
14.2.6 Moving Averages of 𝜆 640
14.3 MCD for Robustness and Mix2Nd Estimation 645
14.4 Some Thoughts on Model Assumptions and Estimation 647
14.5 The Multivariate Laplace and Mixk Lapd Distributions 649
14.5.1 The Multivariate Laplace and EM Algorithm 650
14.5.2 The Mixk Lapd and EM Algorithm 654
14.5.3 Estimation via MCD Split and Forecasting 658
14.5.4 Estimation of Parameter b 660
14.5.5 Portfolio Distribution and Expected Shortfall 662
14.5.6 Fast Evaluation of the Bessel Function 663
Part IV Appendices 667
Appendix A Distribution of Quadratic Forms 669
A.1 Distribution and Moments 669
A.1.1 Probability Density and Cumulative Distribution Functions 669
A.1.2 Positive Integer Moments 671
A.1.3 Moment Generating Functions 673
A.2 Basic Distributional Results 677
A.3 Ratios of Quadratic Forms in Normal Variables 679
A.3.1 Calculation of the CDF 680
A.3.2 Calculation of the PDF 681
A.3.2.1 Numeric Differentiation 682
A.3.2.2 Use of Geary’s formula 682
A.3.2.3 Use of Pan’s Formula 683
A.3.2.4 Saddlepoint Approximation 685
A.4 Problems 689
A.A Appendix: Solutions 690
Appendix B Moments of Ratios of Quadratic Forms 695
B.1 For X ∼ Nn(0, 2I) and B = I 695
B.2 For X ∼ N(0, Σ) 708
B.3 For X ∼ N(𝜇, I) 713
B.4 For X ∼ N(𝜇, Σ) 720
B.5 Useful Matrix Algebra Results 725
B.6 Saddlepoint Equivalence Result 729
Appendix C Some Useful Multivariate Distribution Theory 733
C.1 Student’s t Characteristic Function 733
C.2 Sphericity and Ellipticity 739
C.2.1 Introduction 739
C.2.2 Sphericity 740
C.2.3 Ellipticity 748
C.2.4 Testing Ellipticity 768
Appendix D Introducing the SAS Programming Language 773
D.1 Introduction to SAS 774
D.1.1 Background 774
D.1.2 Working with SAS on a PC 775
D.1.3 Introduction to the Data Step and the Program Data Vector 777
D.2 Basic Data Handling 783
D.2.1 Method 1 784
D.2.2 Method 2 785
D.2.3 Method 3 786
D.2.4 Creating Data Sets from Existing Data Sets 787
D.2.5 Creating Data Sets from Procedure Output 788
D.3 Advanced Data Handling 790
D.3.1 String Input and Missing Values 790
D.3.2 Using set with first.var and last.var 791
D.3.3 Reading in Text Files 795
D.3.4 Skipping over Headers 796
D.3.5 Variable and Value Labels 796
D.4 Generating Charts, Tables, and Graphs 797
D.4.1 Simple Charting and Tables 798
D.4.2 Date and Time Formats/Informats 801
D.4.3 High Resolution Graphics 803
D.4.3.1 The GPLOT Procedure 803
D.4.3.2 The GCHART Procedure 805
D.4.4 Linear Regression and Time-Series Analysis 806
D.5 The SAS Macro Processor 809
D.5.1 Introduction 809
D.5.2 Macro Variables 810
D.5.3 Macro Programs 812
D.5.4 A Useful Example 814
D.5.4.1 Method 1 814
D.5.4.2 Method 2 816
D.6 Problems 817
D.7 Appendix: Solutions 819
Bibliography 825
Index 875
