Final Project need help from A+Grade Helper

needhelphw82

2016_Book_TimeSeriesEconometrics.pdf

Home >Mathematics homework help >Statistics homework help >Final Project need help from A+Grade Helper

Springer Texts in Business and Economics

Klaus Neusser

Time Series Econometrics

Springer Texts in Business and Economics

More information about this series at http://www.springer.com/series/10099

http://www.springer.com/series/10099

Klaus Neusser

Time Series Econometrics

123

Klaus Neusser Bern, Switzerland

ISSN 2192-4333 ISSN 2192-4341 (electronic) Springer Texts in Business and Economics ISBN 978-3-319-32861-4 ISBN 978-3-319-32862-1 (eBook) DOI 10.1007/978-3-319-32862-1

Library of Congress Control Number: 2016938514

© Springer International Publishing Switzerland 2016 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made.

Printed on acid-free paper

This Springer imprint is published by Springer Nature The registered company is Springer International Publishing AG Switzerland

Preface

Over the past decades, time series analysis has experienced a proliferous increase of applications in economics, especially in macroeconomics and finance. Today these tools have become indispensable to any empirically working economist. Whereas in the beginning the transfer of knowledge essentially flowed from the natural sciences, especially statistics and engineering, to economics, over the years theoretical and applied techniques specifically designed for the nature of economic time series and models have been developed. Thereby, the estimation and identification of structural vector autoregressive models, the analysis of integrated and cointegrated time series, and models of volatility have been extremely fruitful and far-reaching areas of research. With the award of the Nobel Prizes to Clive W. J. Granger and Robert F. Engle III in 2003 and to Thomas J. Sargent and Christopher A. Sims in 2011, the field has reached a certain degree of maturity. Thus, the idea suggests itself to assemble the vast amount of material scattered over many papers into a comprehensive textbook.

The book is self-contained and addresses economics students who have already some prerequisite knowledge in econometrics. It is thus suited for advanced bachelor, master’s, or beginning PhD students but also for applied researchers. The book tries to bring them in a position to be able to follow the rapidly growing research literature and to implement these techniques on their own. Although the book is trying to be rigorous in terms of concepts, definitions, and statements of theorems, not all proofs are carried out. This is especially true for the more technically and lengthy proofs for which the reader is referred to the pertinent literature.

The book covers approximately a two-semester course in time series analysis and is divided in two parts. The first part treats univariate time series, in particular autoregressive moving-average processes. Most of the topics are standard and can form the basis for a one-semester introductory time series course. This part also contains a chapter on integrated processes and on models of volatility. The latter topics could be included in a more advanced course. The second part is devoted to multivariate time series analysis and in particular to vector autoregressive processes. It can be taught independently of the first part. The identification, modeling, and estimation of these processes form the core of the second part. A special chapter treats the estimation, testing, and interpretation of cointegrated systems. The book also contains a chapter with an introduction to state space models and the Kalman

vi Preface

filter. Whereas the books is almost exclusively concerned with linear systems, the last chapter gives a perspective on some more recent developments in the context of nonlinear models. I have included exercises and worked out examples to deepen the teaching and learning content. Finally, I have produced five appendices which summarize important topics such as complex numbers, linear difference equations, and stochastic convergence.

As time series analysis has become a tremendously growing field with an active research in many directions, it goes without saying that not all topics received the attention they deserved and that there are areas not covered at all. This is especially true for the recent advances made in nonlinear time series analysis and in the application of Bayesian techniques. These two topics alone would justify an extra book.

The data manipulations and computations have been performed using the software packages EVIEWS and MATLAB.1 Of course, there are other excellent packages available. The data for the examples and additional information can be downloaded from my home page www.neusser.ch. To maximize the learning success, it is advised to replicate the examples and to perform similar exercises with alternative data. Interesting macroeconomic time series can, for example, be downloaded from the following home pages:

Germany: www.bundesbank.de Switzerland: www.snb.ch United Kingdom: www.statistics.gov.uk United States: research.stlouisfed.org/fred2

The book grew out of lectures which I had the occasion to give over the years in Bern and other universities. Thus, it is a concern to thank the many students, in particular Philip Letsch, who had to work through the manuscript and who called my attention to obscurities and typos. I also want to thank my colleagues and teaching assistants Andreas Bachmann, Gregor Bäurle, Fabrice Collard, Sarah Fischer, Stephan Leist, Senada Nukic, Kurt Schmidheiny, Reto Tanner, and Martin Wagner for reading the manuscript or part of it and for making many valuable criticisms and comments. Special thanks go to my former colleague and coauthor Robert Kunst who meticulously read and commented on the manuscript. It goes without saying that all errors and shortcomings go to my expense.

Bern, Switzerland/Eggenburg, Austria Klaus Neusser February 2016

1EVIEWS is a product of IHS Global Inc. MATLAB is a matrix-oriented software developed by MathWorks which is ideally suited for econometric and time series applications.

www.neusser.ch

www.bundesbank.de

www.snb.ch

www.statistics.gov.uk

research.stlouisfed.org/fred2

Contents

Part I Univariate Time Series Analysis

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Some Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2 Formal Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3 Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.4 Construction of Stochastic Processes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1.4.1 White Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.4.2 Construction of Stochastic Processes: Some Examples . . 16 1.4.3 Moving-Average Process of Order One . . . . . . . . . . . . . . . . . . . 17 1.4.4 Random Walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.4.5 Changing Mean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1.5 Properties of the Autocovariance Function . . . . . . . . . . . . . . . . . . . . . . . . . 20 1.5.1 Autocovariance Function of MA(1) Processes . . . . . . . . . . . . 21

1.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2 ARMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.1 The Lag Operator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.2 Some Important Special Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.2.1 Moving-Average Process of Order q . . . . . . . . . . . . . . . . . . . . . . 27 2.2.2 First Order Autoregressive Process . . . . . . . . . . . . . . . . . . . . . . . . 29

2.3 Causality and Invertibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.4 Computation of Autocovariance Function . . . . . . . . . . . . . . . . . . . . . . . . . . 38

2.4.1 First Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 2.4.2 Second Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.4.3 Third Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3 Forecasting Stationary Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 3.1 Linear Least-Squares Forecasts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

3.1.1 Forecasting with an AR(p) Process . . . . . . . . . . . . . . . . . . . . . . . . 48 3.1.2 Forecasting with MA(q) Processes . . . . . . . . . . . . . . . . . . . . . . . . 50 3.1.3 Forecasting from the Infinite Past . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.2 The Wold Decomposition Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.3 Exponential Smoothing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

vii

viii Contents

3.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.5 Partial Autocorrelation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

3.5.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 3.5.2 Interpretation of ACF and PACF . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

3.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4 Estimation of Mean and ACF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 4.1 Estimation of the Mean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 4.2 Estimation of ACF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 4.3 Estimation of PACF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 4.4 Estimation of the Long-Run Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

4.4.1 An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 4.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

5 Estimation of ARMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 5.1 The Yule-Walker Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 5.2 OLS Estimation of an AR(p) Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.3 Estimation of an ARMA(p,q) Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.4 Estimation of the Orders p and q . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 5.5 Modeling a Stochastic Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 5.6 Modeling Real GDP of Switzerland . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

6 Spectral Analysis and Linear Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.1 Spectral Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6.2 Spectral Decomposition of a Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6.3 The Periodogram and the Estimation of Spectral Densities . . . . . . . . 117

6.3.1 Non-Parametric Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 6.3.2 Parametric Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

6.4 Linear Time-Invariant Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 6.5 Some Important Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

6.5.1 Construction of Low- and High-Pass Filters . . . . . . . . . . . . . . 127 6.5.2 The Hodrick-Prescott Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 6.5.3 Seasonal Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 6.5.4 Using Filtered Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

6.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

7 Integrated Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 7.1 Definition, Properties and Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

7.1.1 Long-Run Forecast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 7.1.2 Variance of Forecast Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 7.1.3 Impulse Response Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 7.1.4 The Beveridge-Nelson Decomposition . . . . . . . . . . . . . . . . . . . . 138

7.2 Properties of the OLS Estimator in the Case of Integrated Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

7.3 Unit-Root Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 7.3.1 Dickey-Fuller Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 7.3.2 Phillips-Perron Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

Contents ix

7.3.3 Unit-Root Test: Testing Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 7.3.4 Examples of Unit-Root Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

7.4 Generalizations of Unit-Root Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 7.4.1 Structural Breaks in the Trend Function . . . . . . . . . . . . . . . . . . . 153 7.4.2 Testing for Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

7.5 Regression with Integrated Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 7.5.1 The Spurious Regression Problem . . . . . . . . . . . . . . . . . . . . . . . . . 158 7.5.2 Bivariate Cointegration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 7.5.3 Rules to Deal with Integrated Times Series . . . . . . . . . . . . . . . 162

8 Models of Volatility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 8.1 Specification and Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

8.1.1 Forecasting Properties of AR(1)-Models . . . . . . . . . . . . . . . . . . 168 8.1.2 The ARCH(1) Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 8.1.3 General Models of Volatility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 8.1.4 The GARCH(1,1) Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

8.2 Tests for Heteroskedasticity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 8.2.1 Autocorrelation of Quadratic Residuals . . . . . . . . . . . . . . . . . . . 183 8.2.2 Engle’s Lagrange-Multiplier Test . . . . . . . . . . . . . . . . . . . . . . . . . . 184

8.3 Estimation of GARCH(p,q) Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 8.3.1 Maximum-Likelihood Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 184 8.3.2 Method of Moment Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

8.4 Example: Swiss Market Index (SMI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

Part II Multivariate Time Series Analysis

9 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

10 Definitions and Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

11 Estimation of Covariance Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 11.1 Estimators and Asymptotic Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 11.2 Testing Cross-Correlations of Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . 209 11.3 Some Examples for Independence Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

12 VARMA Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 12.1 The VAR(1) Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 12.2 Representation in Companion Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 12.3 Causal Representation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 12.4 Computation of Covariance Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

13 Estimation of VAR Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 13.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 13.2 The Least-Squares Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 13.3 Proofs of Asymptotic Normality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 13.4 The Yule-Walker Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

x Contents

14 Forecasting with VAR Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 14.1 Forecasting with Known Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

14.1.1 Wold Decomposition Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 14.2 Forecasting with Estimated Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 14.3 Modeling of VAR Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 14.4 Example: VAR Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

15 Interpretation of VAR Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 15.1 Wiener-Granger Causality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

15.1.1 VAR Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 15.1.2 Wiener-Granger Causality and Causal Representation . . . 258 15.1.3 Cross-Correlation Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

15.2 Structural and Reduced Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 15.2.1 A Prototypical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 15.2.2 Identification: The General Case . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 15.2.3 Identification: The Case n D 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266

15.3 Identification via Short-Run Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . 268 15.4 Interpretation of VAR Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270

15.4.1 Impulse Response Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 15.4.2 Variance Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 15.4.3 Confidence Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 15.4.4 Example 1: Advertisement and Sales . . . . . . . . . . . . . . . . . . . . . . 274 15.4.5 Example 2: IS-LM Model with Phillips Curve . . . . . . . . . . . . 277

15.5 Identification via Long-Run Restrictions. . . . . . . . . . . . . . . . . . . . . . . . . . . . 282 15.5.1 A Prototypical Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282 15.5.2 The General Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

15.6 Sign Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

16 Cointegration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295 16.1 A Theoretical Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296 16.2 Definition and Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

16.2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 16.2.2 VAR and VEC Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 16.2.3 Beveridge-Nelson Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . 308 16.2.4 Common Trend Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

16.3 Johansen’s Cointegration Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 16.3.1 Specification of the Deterministic Components . . . . . . . . . . . 317 16.3.2 Testing Cointegration Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . 318

16.4 Estimation and Testing of Cointegrating Relationships . . . . . . . . . . . . 319 16.5 An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321

17 Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325 17.1 The State Space Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326

17.1.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 17.2 Filtering and Smoothing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336

Contents xi

17.2.1 The Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339 17.2.2 The Kalman Smoother . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340

17.3 Estimation of State Space Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343 17.3.1 The Likelihood Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344 17.3.2 Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346

17.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346 17.4.1 Estimation of Quarterly GDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346 17.4.2 Structural Time Series Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 349

17.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

18 Generalizations of Linear Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353 18.1 Structural Breaks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

18.1.1 Methodology .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354 18.1.2 An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356

18.2 Time-Varying Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357 18.3 Regime Switching Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364

A Complex Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369

B Linear Difference Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373

C Stochastic Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377

D BN-Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383

E The Delta Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403

List of Figures

Fig. 1.1 Real gross domestic product (GDP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Fig. 1.2 Growth rate of real gross domestic product (GDP) . . . . . . . . . . . . . . . . . 5 Fig. 1.3 Swiss real gross domestic product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Fig. 1.4 Short- and long-term Swiss interest rates . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Fig. 1.5 Swiss Market Index (SMI). (a) Index. (b) Daily return . . . . . . . . . . . . 8 Fig. 1.6 Unemployment rate in Switzerland . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Fig. 1.7 Realization of a random walk. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Fig. 1.8 Realization of a branching process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Fig. 1.9 Processes constructed from a given white noise

process. (a) White noise. (b) Moving-average with � D 0:9. (c) Autoregressive with � D 0:9. (d) Random walk . . . . . 17

Fig. 1.10 Relation between the autocorrelation coefficient of order one, �.1/, and the parameter � of a MA(1) process . . . . . . . . . . 23

Fig. 2.1 Realization and estimated ACF of MA(1) process . . . . . . . . . . . . . . . . . 28 Fig. 2.2 Realization and estimated ACF of an AR(1) process . . . . . . . . . . . . . . . 31 Fig. 2.3 Autocorrelation function of an ARMA(2,1) process . . . . . . . . . . . . . . . 42

Fig. 3.1 Autocorrelation and partial autocorrelation functions. (a) Process 1. (b) Process 2. (c) Process 3. (d) Process 4 . . . . . . . . . . 66

Fig. 4.1 Estimated autocorrelation function of a WN(0,1) process . . . . . . . . . 75 Fig. 4.2 Estimated autocorrelation function of MA(1) process . . . . . . . . . . . . . 76 Fig. 4.3 Estimated autocorrelation function of an AR(1) process . . . . . . . . . . . 77 Fig. 4.4 Estimated PACF of an AR(1) process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Fig. 4.5 Estimated PACF for a MA(1) process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Fig. 4.6 Common kernel functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Fig. 4.7 Estimated autocorrelation function for the growth rate

of GDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Fig. 5.1 Parameter space of causal and invertible ARMA(1,1) process . . . . 100 Fig. 5.2 Real GDP growth rates of Switzerland . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Fig. 5.3 ACF and PACF of GDP growth rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Fig. 5.4 Inverted roots of the ARMA(1,3) model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Fig. 5.5 ACF of the residuals from AR(2) and ARMA(1,3) models . . . . . . . . 107 Fig. 5.6 Impulse responses of the AR(2) and the ARMA(1,3) model . . . . . . 107

xiii

xiv List of Figures

Fig. 5.7 Forecasts of real GDP growth rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

Fig. 6.1 Examples of spectral densities with Zt � WN.0;1/. (a) MA(1) process. (b) AR(1) process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

Fig. 6.2 Raw periodogram of a white noise time series (Xt � WN.0;1/, T D 200) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

Fig. 6.3 Raw periodogram of an AR(2) process (Xt D 0:9Xt�1 � 0:7Xt�2 C Zt with Zt � WN.0;1/, T D 200) . . . . 121

Fig. 6.4 Non-parametric direct estimates of a spectral density . . . . . . . . . . . . . . 121 Fig. 6.5 Nonparametric and parametric estimates of spectral density . . . . . . 123 Fig. 6.6 Transfer function of the Kuznets filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Fig. 6.7 Transfer function of HP-filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Fig. 6.8 HP-filtered US GDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 Fig. 6.9 Transfer function of growth rate of investment in the

construction sector with and without seasonal adjustment . . . . . . . . . 131

Fig. 7.1 Distribution of the OLS estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 Fig. 7.2 Distribution of t-statistic and standard normal distribution . . . . . . . . 144 Fig. 7.3 ACF of a random walk with 100 observations . . . . . . . . . . . . . . . . . . . . . . 145 Fig. 7.4 Three types of structural breaks at TB. (a) Level shift.

(b) Change in slope. (c) Level shift and change in slope . . . . . . . . . . . 154 Fig. 7.5 Distribution of OLS-estimate Ǒ and t-statistic t Ǒ for

two independent random walks and two independent AR(1) processes. (a) Distribution of Ǒ. (b) Distribution of t Ǒ . (c) Distribution of Ǒ and t-statistic t Ǒ . . . . . . . . . . . . . . . . . . . . . . . . . 160

Fig. 7.6 Cointegration of inflation and three-month LIBOR. (a) Inflation and three-month LIBOR. (b) Residuals from cointegrating regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Fig. 8.1 Simulation of two ARCH(1) processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 Fig. 8.2 Parameter region for which a strictly stationary

solution to the GARCH(1,1) process exists assuming �t � IID N.0;1/ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

Fig. 8.3 Daily return of the SMI (Swiss Market Index) . . . . . . . . . . . . . . . . . . . . . 188 Fig. 8.4 Normal-Quantile Plot of SMI returns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 Fig. 8.5 Histogram of SMI returns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Fig. 8.6 ACF of the returns and the squared returns of the SMI . . . . . . . . . . . . 190

Fig. 11.1 Cross-correlations between two independent AR(1) processes . . . . 212 Fig. 11.2 Cross-correlations between consumption and advertisement . . . . . . 213 Fig. 11.3 Cross-correlations between GDP and consumer sentiment . . . . . . . . 214

Fig. 14.1 Forecast comparison of alternative models. (a) log Yt . (b) log Pt. (c) log Mt. (d) Rt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Fig. 14.2 Forecast of VAR(8) model and 80 % confidence intervals . . . . . . . . . 253

Fig. 15.1 Identification in a two-dimensional structural VAR . . . . . . . . . . . . . . . . 267

List of Figures xv

Fig. 15.2 Impulse response functions for advertisement and sales . . . . . . . . . . . 276 Fig. 15.3 Impulse response functions of IS-LM model . . . . . . . . . . . . . . . . . . . . . . . 280 Fig. 15.4 Impulse response functions of the Blanchard-Quah model . . . . . . . . 289

Fig. 16.1 Impulse responses of present discounted value model . . . . . . . . . . . . . 302 Fig. 16.2 Stochastic simulation of present discounted value model . . . . . . . . . . 303

Fig. 17.1 State space model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326 Fig. 17.2 Spectral density of cyclical component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334 Fig. 17.3 Estimates of quarterly GDP growth rates . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 Fig. 17.4 Components of the basic structural model (BSM) for

real GDP of Switzerland. (a) Logged Swiss GDP (demeaned). (b) Local linear trend (LLT). (c) Business cycle component. (d) Seasonal component . . . . . . . . . . . . . . . . . . . . . . . . . 350

Fig. 18.1 Break date UK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357

Fig. A.1 Representation of a complex number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

List of Tables

Table 1.1 Construction of stochastic processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Table 3.1 Forecast function for a MA(1) process with � D �0:9 and �2 D 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Table 3.2 Properties of the ACF and the PACF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

Table 4.1 Common kernel functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

Table 5.1 AIC for alternative ARMA(p,q) models . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Table 5.2 BIC for alternative ARMA(p,q) models . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Table 7.1 The four most important cases for the unit-root test . . . . . . . . . . . . . . 147 Table 7.2 Examples of unit root tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Table 7.3 Dickey-Fuller regression allowing for structural breaks . . . . . . . . . . 155 Table 7.4 Critical values of the KPSS test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 Table 7.5 Rules of thumb in regressions with integrated processes . . . . . . . . . 165

Table 8.1 AIC criterion for variance equation in GARCH(p,q) model . . . . . 191 Table 8.2 BIC criterion for variance equation in GARCH(p,q) model . . . . . . 191 Table 8.3 One percent VaR for the next day of the return on SMI . . . . . . . . . . 193 Table 8.4 One percent VaR for the next 10 days of the return on SMI. . . . . . 193

Table 14.1 Information criteria for the VAR models of different orders . . . . . 249 Table 14.2 Forecast evaluation of alternative VAR models . . . . . . . . . . . . . . . . . . . 252

Table 15.1 Forecast error variance decomposition (FEVD) in terms of demand, supply, price, wage, and money shocks (percentages) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

Table 16.1 Trend specifications in vector error correction models . . . . . . . . . . . 318 Table 16.2 Evaluation of the results of Johansen’s cointegration test . . . . . . . . 322

xvii

List of Definitions

1.3 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.4 Autocovariance Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.5 Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.6 Strict Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.7 Strict Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.8 Gaussian Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.9 White Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.1 ARMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.2 Causality. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.3 Invertibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.1 Deterministic Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.2 Partial Autocorrelation Function I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 3.3 Partial Autocorrelation Function II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

6.1 Spectral Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6.2 Periodogram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

7.2 Cointegration, Bivariate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

8.1 ARCH(1) Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

10.2 Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 10.3 Strict Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

12.1 VARMA process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

15.2 Sign Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291

16.3 Cointegration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

C.1 Almost Sure Convergence .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378 C.2 Convergence in Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378 C.3 Convergence in r-th Mean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378

xix

xx List of Definitions

C.4 Convergence in Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379 C.5 Characteristic Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380 C.6 Asymptotic Normality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 C.7 m-Dependence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

List of Theorems

3.1 Wold Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

4.1 Convergence of Arithmetic Average . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.2 Asymptotic Distribution of Sample Mean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.4 Asymptotic Distribution of Autocorrelations . . . . . . . . . . . . . . . . . . . . . . . . . . 74

5.1 Asymptotic Normality of Yule-Walker Estimator . . . . . . . . . . . . . . . . . . . . . 89 5.2 Asymptotic Normality of the Least-Squares Estimator . . . . . . . . . . . . . . . 92 5.3 Asymptotic Distribution of ML Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

6.1 Properties of a Spectral Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 6.2 Spectral Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 6.3 Spectral Density of ARMA Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 6.4 Autocovariance Function of Filtered Process . . . . . . . . . . . . . . . . . . . . . . . . . . 123

7.1 Beveridge-Nelson Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

13.1 Asymptotic Distribution of OLS Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

16.1 Beveridge-Nelson Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304

18.1 Solution TVC-VAR(1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358 C.1 Cauchy-Bunyakovskii-Schwarz Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 C.2 Minkowski’s Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 C.3 Chebyschev’s Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 C.4 Borel-Cantelli Lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378 C.5 Kolmogorov’s Strong Law of Large Numbers (SLLN) . . . . . . . . . . . . . . . 378 C.6 Riesz-Fisher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379 C.9 Continuous Mapping Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380 C.10 Slutzky’s Lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380 C.11 Convergence of Characteristic Functions, Lévy . . . . . . . . . . . . . . . . . . . . . . . 381 C.12 Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 C.13 CLT for m-Dependent Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 C.14 Basis Approximation Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382

xxi

Notation and Symbols

r number of linearly independent cointegration vectors ˛ n � r loading matrix ˇ n � r matrix of linearly independent cointegration vectors

d��! convergence in distribution m:s:��! convergence in mean square p��! convergence in probability

corr.X; Y/ correlation coefficient beween random variables X and Y �X;� covariance function of process fXtg, covariance function �X;� correlation function of process fXtg, correlation function ACF autocorrelation function J long-run variance ˛X;˛ partial autocorrelation function of process fXtg PACF partial autocorrelation function n dimension of stochastic process, respectively dimension of state

space � is distributed as sgn sign function tr trace of a matrix det determinant of a matrix kk norm of a matrix ˝ Kronecker product ˇ Hadamard product vec.A/ stakes the columns of A into a vector vech.A/ stakes the lower triangular part of a symmetric matrix A into a

vector GL.n/ general linear group of n � n matrices O.n/ group of orthogonal n � n matrices L lag operator

xxiii

xxiv Notation and Symbols

ˆ.L/ autoregressive polynomial ‚.L/ moving-average polynomial ‰.L/ causal representation, MA.1/ polynomial � difference operator, � D 1 � L p order of autoregressive polynomial q order of moving-average polynomial ARMA(p,q) autoregressive moving-average process of order .p; q/ ARIMA(p,d,q) autoregressive integrated moving-average process of order

.p; d; q/ d order of integration I(d) integrated process of order d VAR(p) vector autoregressive process of order p Z integer numbers R real numbers C complex numbers Rn set of n-dimensional vectors { imaginary unit cov.X; Y/ covariance beween random variables X and Y E expectation operator V variance operator ‰.1/ persistence PT XTCh linear least-squares predictor of XTCh given information from

period 1 up to period T ePT XTCh linear least-squares predictor of XTCh using the infinite

remote past up to period T P Probability fXtg stochastic process WN.0;�2/ white noise process with mean zero and variance �2

WN.0;†/ multivariate white noise process with mean zero and covariance matrix †2

IID.0;�2/ identically and independently distributed random variables with mean zero and variance �2

IID N.0;�2/ identically and independently normally distributed random variables with mean zero and variance �2

Xt time indexed random variable xt realization of random variable Xt f . / spectral density F. / spectral distribution function IT periodogram ‰.e�{ / transfer function of filter ‰ VaR value at risk

Part I

Univariate Time Series Analysis

1Introduction and Basic Theoretical Concepts

Time series analysis is an integral part of every empirical investigation which aims at describing and modeling the evolution over time of a variable or a set of variables in a statistically coherent way. The economics of time series analysis is thus very much intermingled with macroeconomics and finance which are concerned with the construction of dynamic models. In principle, one can approach the subject from two complementary perspectives. The first one focuses on descriptive statistics. It characterizes the empirical properties and regularities using basic statistical concepts like mean, variance, and covariance. These properties can be directly measured and estimated from the data using standard statistical tools. Thus, they summarize the external (observable) or outside characteristics of the time series. The second perspective tries to capture the internal data generating mechanism. This mechanism is usually unknown in economics as the models developed in economic theory are mostly of a qualitative nature and are usually not specific enough to single out a particular mechanism.1 Thus, one has to consider some larger class of models. By far most widely used is the class of autoregressive moving-average (ARMA) models which rely on linear stochastic difference equations with constant coefficients. Of course, one wants to know how the two perspectives are related which leads to the important problem of identifying a model from the data.

The observed regularities summarized in the form of descriptive statistics or as a specific model are, of course, of principal interest to economics. They can be used to test particular theories or to uncover new features. One of the main assumptions underlying time series analysis is that the regularities observed in the sample period

1 One prominent exception is the random-walk hypothesis of real private consumption first derived and analyzed by Hall (1978). This hypothesis states that the current level of private consumption should just depend on private consumption one period ago and on no other variable, in particular not on disposable income. The random-walk property of asset prices is another very much discussed hypothesis. See Campbell et al. (1997) for a general exposition and Samuelson (1965) for a first rigorous derivation from market efficiency.

4 1 Introduction and Basic Theoretical Concepts

are not specific to that period, but can be extrapolated into the future. This leads to the issue of forecasting which is another major application of time series analysis.

Although its roots lie in the natural sciences and in engineering, time series analysis, since the early contributions by Frisch (1933) and Slutzky (1937), has become an indispensable tool in empirical economics. Early applications mostly consisted in making the knowledge and methods acquired there available to economics. However, with the progression of econometrics as a separate scientific field, more and more techniques that are specific to the characteristics of economic data have been developed. I just want to mention the analysis of univariate and multivariate integrated, respectively cointegrated time series (see Chaps. 7 and 16), the identification of vector autoregressive (VAR) models (see Chap. 15), and the analysis of volatility of financial market data in Chap. 8. Each of these topics alone would justify the treatment of time series analysis in economics as a separate subfield.

1.1 Some Examples

Before going into more formal analysis, it is useful to examine some prototypical economic time series by plotting them against time. This simple graphical inspection already reveals some of the issues encountered in this book. One of the most popular time series is the real gross domestic product. Figure 1.1 plots the data for the U.S. from 1947 first quarter to 2011 last quarter on logarithmic scale. Several observations are in order. First, the data at hand cover just a part of the time series. There are data available before 1947 and there will be data available after 2011. As there is no natural starting nor end point, we think of a time series as extending back into the infinite past and into the infinite future. Second, the observations are treated as the realizations of a random mechanism. This implies that we observe only one realization. If we could turn back time and let run history again, we would obtain a second realization. This is, of course, impossible, at least in the macroeconomics context. Thus, typically, we are faced with just one realization on which to base our analysis. However, sound statistical analysis needs many realizations. This implies that we have to make some assumption on the constancy of the random mechanism over time. This leads to the concept of stationarity which will be introduced more rigorously in the next section. Third, even a cursory look at the plot reveals that the mean of real GDP is not constant, but is upward trending. As we will see, this feature is typical of many economic time series.2 The investigation into the nature of the trend and the statistical consequences thereof have been the subject of intense research over the last couple of decades. Fourth, a simple way to overcome this

2See footnote 1 for some theories predicting non-stationary behavior.

1.1 Some Examples 5

1950 1960 1970 1980 1990 2000 2010

7.5

8.5

9.5 lo

g a ri th

Fig. 1.1 Real gross domestic product (GDP) of the U.S. (chained 2005 dollars; seasonally adjusted annual rate)

1950 1960 1970 1980 1990 2000 2010 −3

−2

−1

p e rc

e n t

Fig. 1.2 Quarterly growth rate of U.S. real gross domestic product (GDP) (chained 2005 dollars)

problem is to take first differences. As the data have been logged, this amounts to taking growth rates.3 The corresponding plot is given in Fig. 1.2 which shows no trend anymore.

Another feature often encountered in economic time series is seasonality. This issue arises, for example in the case of real GDP, because of a particular regularity within a year: the first quarter being the quarter with the lowest values, the second

3This is obtained by using the approximation ln.1 C "/ � " for small " where " equals the growth rate of GDP.

6 1 Introduction and Basic Theoretical Concepts

1980 1985 1990 1995 2000 2005 2010 2015

11.2

11.3

11.4

11.5

11.6

11.7

11.8

not adjusted seasonally adjusted

Fig. 1.3 Comparison of unadjusted and seasonally adjusted Swiss real gross domestic product (GDP)

and fourth quarter those with the highest values, and the third quarter being in between. These movements are due to climatical and holiday seasonal variations within the year and are viewed to be of minor economic importance. Moreover, these seasonal variations, because of there size, hide the more important business cycle movements. It is therefore customary to work with time series which have been adjusted for seasonality before hand. Figure 1.3 shows the unadjusted and the adjusted real gross domestic product for Switzerland. The adjustment has been achieved by taking a moving-average. This makes the time series much smoother and evens out the seasonal movements.

Other typical economic time series are interest rates plotted in Fig. 1.4. Over the period considered these two variables also seem to trend. However, the nature of this trend must be different because of the theoretically binding zero lower bound. Although the relative level of the two series changes over time—at the beginning of the sample, short-term rates are higher than long-terms ones—they move more or less together. This comovement is true in particular true with respect to the medium- and long-term.

Other prominent time series are stock market indices. In Fig. 1.5 the Swiss Market Index (SMI) in plotted as an example. The first panel displays the raw data on a logarithmic scale. One can clearly discern the different crises: the internet bubble in 2001 and the most recent financial market crisis in 2008. More interesting than the index itself is the return on the index plotted in the second panel. Whereas the mean seems to stay relatively constant over time, the volatility is not: in the periods of crisis volatility is much higher. This clustering of volatility is a typical feature of financial market data and will be analyzed in detail in Chap. 8.

1.2 Formal Definitions 7

1990 1995 2000 2005 2010 0

10 p

e rc

e n

three−month LIBOR 10−year government bond

Fig. 1.4 Short- and long-term Swiss interest rates (three-month LIBOR and 10 year government bond)

Finally, Fig. 1.6 plots the unemployment rate for Switzerland. This is another widely discussed time series. However, the Swiss data have a particular feature in that the behavior of the series changes over time. Whereas unemployment was practically nonexistent in Switzerland up to the end of 1990’s, several policy changes (introduction of unemployment insurance, liberalization of immigration laws) led to drastic shifts. Although such dramatic structural breaks are rare, one has to be always aware of such a possibility. Reasons for breaks are policy changes and simply structural changes in the economy at large.4

1.2 Formal Definitions

The previous section attempted to give an intuitive approach of the subject. The analysis to follow necessitates, however, more precise definitions and concepts. At the heart of the exposition stands the concept of a stochastic process. For this purpose we view the observation at some time t as the realization of random variable Xt. In time series analysis we are, however, in general not interested in a particular point in time, but rather in a whole sequence. This leads to the following definition.

Definition 1.1. A stochastic process fXtg is a family of random variables indexed by t 2 T and defined on some given probability space.

4Burren and Neusser (2013) investigate, for example, how systematic sectoral shifts affect volatility of real GDP growth.

8 1 Introduction and Basic Theoretical Concepts

2/1/1989 30/9/1992 31/7/1996 31/5/2000 31/3/2004 30/1/2008 30/11/2011 7

7.5

8.5

9.5 lo

g a ri tm

2/1/1989 30/9/1992 31/7/1996 31/5/2000 31/3/2004 30/1/2008 30/11/2011

−10

−5

p e rc

e n t

Asian financial crisis

internet bubble

financial market crisis

Fig. 1.5 Swiss Market Index (SMI). (a) Index. (b) Daily return

Thereby T denotes an ordered index set which is typically identified with time. In the literature one can encounter the following index sets:

discrete time: T D f1;2; : : :g D N discrete time: T D f: : : ; �2; �1;0;1;2; : : :g D Z

continuous time: T D Œ0; 1/ D RC or T D .�1; 1/ D R

1.2 Formal Definitions 9

1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 0

p e rc

e n

Fig. 1.6 Unemployment rate in Switzerland

Remark 1.1. Given that T is identified with time and thus has a direction, a characteristic of time series analysis is the distinction between past, present, and future.

For technical reasons which will become clear later, we will work with T D Z, the set of integers. This choice is consistent with the use of time indices in economics as there is, usually, no natural starting point nor a foreseeable endpoint. Although models in continuous time are well established in the theoretical finance literature, we will disregard them because observations are always of a discrete nature and because models in continuous time would need substantially higher mathematical requirements.

Remark 1.2. The random variables fXtg take values in a so-called state space. In the first part of this treatise, we take as the state space the space of real numbers R and thus consider only univariate time series. In part II we extend the state space to Rn and study multivariate times series. Theoretically, it is possible to consider other state spaces (for example, f0;1g, the integers, or the complex numbers), but this will not be pursued here.

Definition 1.2. The function t ! xt which assigns to each point in time t the realization of the random variable Xt , xt, is called a realization or a trajectory of the stochastic process. We denote such a realization by fxtg.

10 1 Introduction and Basic Theoretical Concepts

We denominate by a time series the realization or trajectory (observations or data), or the underlying stochastic process. Usually, there is no room for misun- derstandings. A trajectory therefore represents one observation of the stochastic process. Whereas in standard statistics a sample consists of several, typically, independent draws from the same distribution, a sample in time series analysis is just one trajectory. Thus, we are confronted with a situation where there is in principle just one observation. We cannot turn back the clock and get additional trajectories. The situation is even worse as we typically observe only the realizations in a particular time window. For example, we might have data on US GDP from the first quarter 1960 up to the last quarter in 2011. But it is clear, the United States existed before 1960 and will continue to exist after 2011, so that there are in principle observations before 1960 and after 2011. In order to make a meaningful statistical analysis, it is therefore necessary to assume that the observed part of the trajectory is typical for the time series as a whole. This idea is related to the concept of stationarity which we will introduce more formally below. In addition, we want to require that the observations cover in principle all possible events. This leads to the concept of ergodicity. We avoid a formal definition of ergodicity as this would require a sizeable amount of theoretical probabilistic background material which goes beyond the scope this treatise.5

An important goal of time series analysis is to build a model given the realization (data) at hand. This amounts to specify the joint distribution of some set of Xt’s with corresponding realization fxtg. Definition 1.3 (Model). A time series model or a model for the observations (data) fxtg is a specification of the joint distribution of fXtg for which fxtg is a realization.

The Kolmogorov existence theorem ensures that the specification of all finite dimensional distributions is sufficient to characterize the whole stochastic process (see Billingsley (1986), Brockwell and Davis (1991), or Kallenberg (2002)).

Most of the time it is too involved to specify the complete distribution so that one relies on only the first two moments. These moments are then given by the means EXt , the variances VXt, t 2 Z, and the covariances cov.Xt; Xs/ D E.Xt � EXt/.Xs � EXs/ D E.Xt Xs/ � EXt EXs, respectively the correlations corr.Xt; Xs/ D cov.Xt; Xs/=.

p VXt

p VXs/, t; s 2 Z. If the random variables are jointly normally

distributed then the specification of the first two moments is sufficient to characterize the whole distribution.

5In the theoretical probability theory ergodicity is an important concept which asks the question under which conditions the time average of a property is equal to the corresponding ensemble average, i.e. the average over the entire state space. In particular, ergodicity ensures that the arithmetic averages over time converge to their theoretical counterparts. In Chap. 4 we allude to this principle in the estimation of the mean and the autocovariance function of a time series.

1.3 Stationarity 11

Examples of Stochastic Processes

• fXtg is a sequence of independently distributed random variables with values in f�1;1g such that PŒXt D 1 D PŒXt D �1 D 1=2. Xt represents, for example, the payoff after tossing a coin: if head occurs one gets a Euro whereas if tail occurs one has to pay a Euro.

• The simple random walk fStg is defined by

St D St�1 C Xt D t X

iD1 Xi with t � 0 and S0 D 0;

where fXtg is the process from the example just above. In this case St is the proceeds after t rounds of coin tossing. More generally, fXtg could be any sequence of identically and independently distributed random variables. Figure 1.7 shows a realization of fXtg for t D 1;2; : : : ;100 and the corresponding random walk fStg. For more on random walks see Sect. 1.4.4 and, in particular, Chap. 7.

• The simple branching process is defined through the recursion

XtC1 D Xt X

jD1 Zt;j with starting value: X0 D x0:

In this example Xt represents the size of a population where each member lives just one period and reproduces itself with some probability. Zt;j thereby denotes the number of offsprings of the j-th member of the population in period t. In the simplest case fZt;jg is nonnegative integer valued and identically and independently distributed. A realization with X0 D 100 and with probabilities of one third each that the member has no, one, or two offsprings is shown as an example in Fig. 1.8.

1.3 Stationarity

An important insight in time series analysis is that the realizations in different periods are related with each other. The value of GDP in some year obviously depends on the values from previous years. This temporal dependence can be represented either by an explicit model or, in a descriptive way, by covariances, respectively correlations. Because the realization of Xt in some year t may depend, in principle, on all past realizations Xt�1; Xt�2; : : : , we do not have to specify just a finite number of covariances, but infinitely many covariances. This leads to the concept of the covariance function. The covariance function is not only a tool for summarizing the statistical properties of a time series, but is also instrumental in

12 1 Introduction and Basic Theoretical Concepts

0 10 20 30 40 50 60 70 80 90 100 −6

−4

−2

12 X random walk

Fig. 1.7 Realization of a random walk

0 10 20 30 40 50 60 70 80 90 100 40

100

120

140

160

180

time

p o p u la

ti o n s

iz e

Fig. 1.8 Realization of a branching process

the derivation of forecasts (Chap. 3), in the estimation of ARMA models, the most important class of models (Chap. 5), and in the Wold representation (Sect. 3.2 in Chap. 3). It is therefore of utmost importance to get a thorough understanding of the meaning and properties of the covariance function.

1.3 Stationarity 13

Definition 1.4 (Autocovariance Function). Let fXtg be a stochastic process with VXt < 1 for all t 2 Z then the function which assigns to any two time periods t and s, t; s 2 Z, the covariance between Xt and Xs is called the autocovariance function of fXtg. The autocovariance function is denoted by �X.t; s/. Formally this function is given by

�X.t; s/ D cov.Xt; Xs/ D E Œ.Xt � EXt/.Xs � EXs/ D EXt Xs � EXtEXs:

Remark 1.3. The acronym auto emphasizes that the covariance is computed with respect to the same variable taken at different points in time. Alternatively, one may use the term covariance function for short.

Definition 1.5 (Stationarity). A stochastic process fXtg is called stationary if and only if for all integers r, s and t the following properties hold:

(i) EXt D � constant; (ii) VXt < 1;

(iii) �X.t; s/ D �X.t C r; s C r/.

Remark 1.4. Processes with these properties are often called weakly stationary, wide-sense stationary, covariance stationary, or second order stationary. As we will not deal with other forms of stationarity, we just speak of stationary processes, for short.

Remark 1.5. For t D s, we have �X.t; s/ D �X.t; t/ D VXt which is nothing but the unconditional variance of Xt. Thus, if fXtg is stationary �X.t; t/ D VXt D constant.

Remark 1.6. If fXtg is stationary, by setting r D �s the autocovariance function becomes:

�X.t; s/ D �X.t � s;0/:

Thus the covariance �X.t; s/ does not depend on the points in time t and s, but only on the number of periods t and s are apart from each other, i.e. from t � s. For stationary processes it is therefore possible to view the autocovariance function as a function of just one argument. We denote the autocovariance function in this case by �X.h/, h 2 Z. Because the covariance is symmetric in t and s, i.e. �X.t; s/ D �X.s; t/, we have

�X.h/ D �X.�h/ for all integers h:

It is thus sufficient to look at the autocovariance function for positive integers only, i.e. for h D 0;1;2; : : :. In this case we refer to h as the order of the autocovariance. For h D 0, we get the unconditional variance of Xt, i.e. �X.0/ D VXt.

14 1 Introduction and Basic Theoretical Concepts

In practice it is more convenient to look at the autocorrelation coefficients instead of the autocovariances. The autocorrelation function (ACF) for stationary processes is defined as:

�X.h/ D �X.h/

�X.0/ D corr.XtCh; Xt/ for all integers h

where h is referred to as the order. Note that this definition is equivalent to the ordinary correlation coefficients �.h/ D cov.Xt;Xt�h/p

VXt p VXt�h

because stationarity implies

that VXt D VXt�h so that p VXt

p VXt�h D VXt D �X.0/.

Most of the time it is sufficient to concentrate on the first two moments. However, there are situations where it is necessary to look at the whole distribution. This leads to the concept of strict stationarity.

Definition 1.6 (Strict Stationarity). A stochastic process is called strictly stationary if the joint distributions of .Xt1; : : : ; Xtn/ and .Xt1Ch; : : : ; Xtn Ch/ are the same for all h 2 Z and all .t1; : : : ; tn/ 2 T n, n D 1;2; : : : Definition 1.7 (Strict Stationarity). A stochastic process is called strictly stationary if for all integers h and n � 1 .X1; : : : ; Xn/ and .X1Ch; : : : ; XnCh/ have the same distribution.

Remark 1.7. Both definitions are equivalent.

Remark 1.8. If fXtg is strictly stationary then Xt has the same distribution for all t (n=1). For n D 2 we have that XtCh and Xt have a joint distribution which is independent of t. This implies that the covariance, if it exists, depends only on h. Thus, every strictly stationary process with VXt < 1 is also stationary.6

The converse is, however, not true as shown by the following example:

Xt � (

exponentially distributed with mean 1 (i.e. f .x/ D e�x/; t unevenI N.1;1/; t evenI

whereby the Xt’s are independently distributed. In this example we have:

• EXt D 1 • �X.0/ D 1 and �X.h/ D 0 for h ¤ 0

Thus fXtg is stationary, but not strictly stationary, because the distribution changes depending on whether t is even or uneven.

6An example of a process which is strictly stationary, but not stationary, is given by the IGARCH process (see Sect. 8.1.4). This process is strictly stationary with infinite variance.

1.4 Construction of Stochastic Processes 15

Definition 1.8 (Gaussian Process). A stochastic process fXtg is called a Gaussian process if all finite dimensional distributions .Xt1; : : : ; Xtn/ with .t1; : : : ; tn/ 2 T n, n D 1;2; : : :, are multivariate normally distributed.

Remark 1.9. A Gaussian process is obviously strictly stationary. For all n; h; t1; : : : ; tn, .Xt1; : : : ; Xtn/ and .Xt1Ch; : : : ; XtnCh/ have the same mean and the same covariance matrix.

At this point we will not delve into the relation between stationarity, strict stationarity and Gaussian processes, rather some of these issues will be further discussed in Chap. 8.

1.4 Construction of Stochastic Processes

One important notion in time series analysis is to build up more complicated process from simple ones. The simplest building block is a process with zero autocorrelation called a white noise process which is introduced below. Taking moving-averages from this process or using it in a recursion gives rise to more sophisticated process with more elaborated autocovariance functions. Slutzky (1937) first introduced the idea that moving-averages of simple processes can generate time series whose motion resembles business cycle fluctuations.

1.4.1 White Noise

The simplest building block is a process with zero autocorrelation called a white noise process.

Definition 1.9 (White Noise). A stationary process fZtg is called a white noise process if fZtg satisfies:

• EZt D 0 • �Z.h/ D

(

�2 h D 0I 0 h ¤ 0:

We denote this by Zt � WN.0;�2/. The white noise process is therefore stationary and temporally uncorrelated,

i.e. the ACF is always equal to zero, except for h D 0 where it is equal to one. As the ACF possesses no structure, it is impossible to draw inferences from past observations to its future development, at least in a least square setting with linear forecasting functions (see Chap. 3). Therefore one can say that a white noise process has no memory.

16 1 Introduction and Basic Theoretical Concepts

If fZtg is not only temporally uncorrelated, but also independently and identically distributed, we write Zt � IID.0;�2/. If in addition Zt is normally distributed, we write Zt � IIN.0;�2/. An IID.0;�2/ process is always a white noise process. The converse is, however, not true as will be shown in Chap. 8.

1.4.2 Construction of Stochastic Processes: Some Examples

We will now illustrate how complex stationary processes can be constructed by manipulating of a white noise process. In Table 1.1 we report in column 2 the first 6 realizations of a white noise process fZtg. Figure 1.9a plots the first 100 observations. We can now construct a new process fX.MA/t g by taking moving- averages over adjacent periods. More specifically, we take Xt D Zt C 0:9Zt�1, t D 2;3; : : :. Thus, the realization of fX.MA/t g in period 2 is fx.MA/2 g D �0:8718 C 0:9 � 0:2590 D �0:6387.7 The realization in period 3 is fx.MA/3 g D �0:7879 C 0:9 � �0:8718 D �1:5726, and so on. The resulting realizations of fX.MA/t g for t D 2; : : : ;6 are reported in the third column of Table 1.1 and the plot is shown in Fig. 1.9b. On can see that the averaging makes the series more smooth. In Sect. 1.4.3 we will provide a more detailed analysis of this moving-average process.

Another construction device is a recursion: X .AR/ t D �X.AR/t�1 C Zt, t D 2;3; : : :,

with starting value X .AR/ 1 D Z1. Such a process is called autoregressive because it

refers to its own past. Taking � D 0:9, the realization of fX.AR/t g in period 2 is fx.AR/2 g D �0:6387 D 0:9 � 0:2590 � 0:8718, in period 3 fx.AR/3 g D �1:3627 D 0:9 � �0:6387 � 0:7879, and so on. Again the resulting realizations of fX.AR/t g for t D 2; : : : ;6 are reported in the fourth column of Table 1.1 and the plot is shown in Fig. 1.9c. On can see how the series becomes more persistent. In Sect. 2.2.2 we will provide a more detailed analysis of this autoregressive process.

Finally, we construct a new process by taking cumulative sums: X .RW/ t D

Pt �D1 Z� . This process can also be obtained from the recursion above by taking

� D 1 so that X.RW/t D X.RW/t�1 C Zt . It is called a random walk. Thus, the realization of fX.RW/t g for period 2 is fx.RW/2 g D �0:6128 D 0:2590 � 0:8718, for period 3 fx.RW/3 g D �1:4007 D �0:6128�0:7879, and so on. Again the resulting realizations of fX.RW/t g for t D 2; : : : ;6 are reported in the last column of Table 1.1 and the plot is shown in Fig. 1.9d. On can see how the series moves away from its mean of zero more persistently than all the other three process considered. In Sect. 1.4.4 we will provide a more detailed analysis of this so-called random walk process and show that it is not stationary.

7The following calculations are subject to rounding to four digits.

1.4 Construction of Stochastic Processes 17

Table 1.1 Construction of stochastic processes assuming Z0 D X0 D 0

Time White Moving- Auto- Random noise average regressive walk

1 0:2590 0:2590 0:2590 0:2590

2 �0:8718 �0:6387 �0:6387 �0:6128 3 �0:7879 �1:5726 �1:3627 �1:4007 4 �0:3443 �1:0535 �1:5708 �1:7451 5 0:6476 0:3377 �0:7661 �1:0974 6 2:0541 2:6370 1:3646 0:9567 : : :

: : :

0 20 40 60 80 100 −3

−2

−1

0 20 40 60 80 100 −4

−2

0 20 40 60 80 100 −5

0 20 40 60 80 100 −2

a b

c d

Fig. 1.9 Processes constructed from a given white noise process. (a) White noise. (b) Moving- average with � D 0:9. (c) Autoregressive with � D 0:9. (d) Random walk

1.4.3 Moving-Average Process of Order One

The white noise process can be used as a building block to construct more complex processes with a more involved autocorrelation structure. The simplest procedure is to take moving averages over consecutive periods.8 This leads to the moving- average processes. The moving-average process of order one, MA(1) process, is defined as

8This procedure is an example of a filter. Section 6.4 provides a general introduction to filters.

18 1 Introduction and Basic Theoretical Concepts

Xt D Zt C �Zt�1 with Zt � WN.0;�2/:

Clearly, EXt D EZt C �EZt�1 D 0. The mean is therefore constant and equal to zero.

The autocovariance function can be computed as follows:

�X.t C h; t/ D cov.XtCh; Xt/ D cov.ZtCh C �ZtCh�1; Zt C �Zt�1/ D EZtChZt C �EZtChZt�1 C �EZtCh�1Zt C �2EZtCh�1Zt�1:

Recalling that fZtg is white noise so that EZ2t D �2 and EZt ZtCh D 0 for h ¤ 0, we therefore get the following autocovariance function of fXtg:

�X.h/ D

.1 C �2/�2 h D 0I ��2 h D ˙1I 0 otherwise:

(1.1)

Thus fXtg is stationary irrespective of the value of �. The autocorrelation function is:

�X.h/ D

1 h D 0I �

1C�2 h D ˙1I 0 otherwise:

Note that the newly created process now exhibits a dependence from its past as Xt is correlated with Xt�1. This correlation is restricted to the interval Œ0; 12 , i.e. 0 � j�X.1/j � 12 . As the correlation between Xt and Xs is zero when t and s are more than one period apart, we call a moving-average process a process with finite memory or a process with finite-range dependence.

Remark 1.10. To motivate the name moving-average, we can define the MA(1) process more generally as

Xt D �0Zt C �1Zt�1 with Zt � WN.0;�2/ and �0 ¤ 0:

Thus, Xt is a weighted average of Zt and Zt�1. If �0 D �1 D 1=2, Xt is just the arithmetic mean of Zt and Zt�1. This process is, however, (observationally) equivalent to the process

Xt D QZt C Q� QZt�1 with QZt � WN.0; Q�2/

1.4 Construction of Stochastic Processes 19

where Q� D �1=�0 and Q�2 D �20�2. Both processes would generate the same first two moments and are therefore observationally indistinguishable from each other. Thus, we can set �0 D 1 without loss of generality.

1.4.4 Random Walk

Let Zt � WN.0;�2/ be a white noise process then the new process fXtg defined as

Xt D Z1 C Z2 C : : : C Zt D t X

jD1 Zj; t > 0; (1.2)

is called a random walk. Note that, in contrast to fZtg, fXtg is only defined for t > 0. The random walk may alternatively be defined through the recursion

Xt D Xt�1 C Zt; t > 0 and X0 D 0:

If in each time period a constant ı is added such that

Xt D ı C Xt�1 C Zt;

the process fXtg is called a random walk with drift. Although the random walk has a constant mean of zero, it is a nonstationary

process.

Proposition 1.1. The random walk fXtg as defined in Eq. (1.2) is nonstationary.

Proof. The variance of XtC1 � X1 equals V.XtC1 � X1/ D V �

PtC1 jD2 Zj

�

D PtC1

jD2 VZj D t�2. Assume for the moment that fXtg is stationary then the triangular inequality

implies for t > 0:

0 < p

t�2 D std.XtC1 � X1/ � std.XtC1/ C std.X1/ D 2 std.X1/

where “std” denotes the standard deviation. As the left hand side of the inequality converges to infinity for t going to infinity, also the right hand side must go to infinity. This means that the variance of X1 must be infinite. This, however, contradicts the assumption of stationarity. Thus fXtg cannot be stationary. ut

The random walk represents by far the most widely used nonstationary process in economics. It has proven to be an important ingredient in many economic time series. Typical nonstationary time series which are or are driven by random walks are stock market prices, exchange rates, or the gross domestic product

20 1 Introduction and Basic Theoretical Concepts

(GDP). Usually it is necessary to apply some transformation (filter) first to achieve stationarity. In the example above, one has to replace fXtg by its first difference f�Xtg D fXt � Xt�1g D fZtg which is stationary by construction. Time series which become stationary after differencing are called integrated processes and are the subject of a more in depth analysis in Chap. 7. Besides ordinary differencing, other transformations are often encountered: seasonal differencing, inclusion of a time trend, seasonal dummies, moving averages, etc. Some of them will be discussed as we go along.

1.4.5 Changing Mean

Finally, here is another simple example of a nonstationary process.

Xt D �

Yt; t < tc; Yt C c; t � tc und c ¤ 0

where tc is some specific point in time. fXtg is clearly not stationary because the mean is not constant. In econometrics we refer to such a situation as a structural change which can be accommodated by introducing a so-called dummy variable. Models with more sophisticated forms of structural changes will be discussed in Chap. 18

1.5 Properties of the Autocovariance Function

The autocovariance function represents the directly accessible external properties of the time series. It is therefore important to understand its properties and how it is related to its inner structure. We will deepen the connection between the autocovariance function and a particular class of models in Chap. 2. The estimation of the autocovariance function will be treated in Chap. 4. For the moment we will just give its properties and analyze the case of the MA(1) model as a prototypical example.

Theorem 1.1. The autocovariance function of a stationary process fXtg is charac- terized by the following properties:

(i) �X.0/ � 0; (ii) 0 � j�X.h/j � �X.0/;

(iii) �X.h/ D �X.�h/; (iv)

Pn i;jD1 ai�X.ti � tj/aj � 0 for all n and all vectors .a1; : : : ; an/0 and .t1; : : : ; tn/.

This property is called non-negative definiteness.

Proof. The first property is obvious as the variance is always nonnegative. The second property follows from the Cauchy-Bunyakovskii-Schwarz inequality(see

1.5 Properties of the Autocovariance Function 21

Theorem C.1) applied to Xt and XtCh which yields 0 � j�X.h/j � �X.0/. The third property follows immediately from the definition of the covariance. Define a D .a1; : : : ; an/

0 and X D .Xt1; : : : ; Xtn/0 then the last property follows from the fact that the variance is always nonnegative:0 � V.a0X/ D a0V.X/a D Pni;jD1 ai�X.ti �tj/aj.

Similar properties hold for the correlation function �X , except that we have �X.0/ D 1. Theorem 1.2. The autocorrelation function of a stationary stochastic process fXtg is characterized by the following properties:

(i) �X.0/ D 1; (ii) 0 � j�X.h/j � 1;

(iii) �X.h/ D �X.�h/; (iv)

Pn i;jD1 ai�X.ti � tj/aj � 0 for all n and all vectors .a1; : : : ; an/0 and .t1; : : : ; tn/.

Proof. The proof follows immediately from the properties of the autocovariance function. ut

It can be shown that for any given function with the above properties there exists a stationary process (Gaussian process) which has this function as its autocovariance function, respectively autocorrelation function.

1.5.1 Autocovariance Function of MA(1) Processes

The autocovariance function describes the external observable characteristics of a time series which can be estimated from the data. Usually, we want to understand the internal mechanism which generates the data at hand. For this we need a model. Hence it is important to understand the relation between the autocovariance function and a certain class of models. In this section, by analyzing the MA(1) model, we will show that this relationship is not one-to-one. Thus we are confronted with a fundamental identification problem.

In order to make the point, consider the following given autocovariance function:

�.h/ D 8

�0; h D 0I �1; h D ˙1I 0; jhj > 1:

The problem consists of determining the parameters of the MA(1) model, � and �2, from the values of the autocovariance function. For this purpose we equate �0 D .1 C �2/�2 and �1 D ��2 (see Eq. (1.1)). This leads to an equation system in the two unknowns � und �2. This system can be simplified by dividing the second equation by the first one to obtain: �1=�0 D �=.1C�2/. Because �1=�0 D �.1/ D �1

22 1 Introduction and Basic Theoretical Concepts

one gets a quadratic equation in �:

�1� 2 � � C �1 D 0:

The two solutions of this equation are

�1;2 D 1

2�1

�

1 ˙ q

1 � 4�21 �

The solutions are real if and only if the discriminant 1 � 4�21 is positive. This is the case if and only if �21 � 1=4, respectively j�1j � 1=2. Note that one root is the inverse of the other. The identification problem thus takes the following form:

j�1j < 1=2: there exists two observationally equivalent MA(1) processes corre- sponding to the two solutions �1 and �2.

�1 D ˙1=2: there exists exactly one MA(1) process with � D ˙1. j�1j > 1=2: there exists no MA(1) process with this autocovariance function. The relation between the first order autocorrelation coefficient, �1 D �.1/, and the parameter � of the MA(1) process is represented in Fig. 1.10. As can be seen, there exists for each �.1/ with j�.1/j < 1

2 two solutions. The two solutions are

inverses of each other. Hence one solution is absolutely smaller than one whereas the other is bigger than one. In Sect. 2.3 we will argue in favor of the solution smaller than one. For �.1/ D ˙1=2 there exists exactly one solution, namely � D ˙1. For j�.1/j > 1=2 there is no solution. For j�1j > 1=2, �.h/ actually does not represent a genuine autocorrelation function as the fourth condition in Theorem 1.1, respectively Theorem 1.2 is violated. For �1 >

1 2 , set a D .1; �1;1; �1; : : : ;1; �1/0

to get:

n X

i;jD1 ai�.i � j/aj D n � 2.n � 1/�1 < 0; if n >

2�1

2�1 � 1 :

For �1 D � 12 one sets a D .1;1; : : : ;1/0. Hence the fourth property is violated.

1.6 Exercises

Exercise 1.6.1. Let the process fXtg be generated by a two-sided moving-average process

Xt D 0:5ZtC1 C 0:5Zt�1 with Zt � WN.0;�2/:

Determine the autocovariance and the autocorrelation function of fXtg.

Exercise 1.6.2. Let fXtg be the MA(1) process

1.6 Exercises 23

−3 −2 −1 0 1 2 3

−0.5

−0.4

−0.3

−0.2

−0.1

0.1

0.2

0.3

0.4

0.5

first order ‘‘moving average’’ parameter:θ

fi rs

t o rd

er a

u to

co rr

el at

io n c

o ef

fi ci

en t: ρ

(1 )

θ/(1+θ2)

0 0.5 1 2

Fig. 1.10 Relation between the autocorrelation coefficient of order one, �.1/, and the parameter � of a MA(1) process

Xt D Zt C �Zt�2 with Zt � WN.0;�2/:

(i) Determine the autocovariance and the autocorrelation function of fXtg for � D 0:9.

(ii) Determine the variance of the mean .X1 C X2 C X3 C X4/=4. (iii) How do the previous results change if � D �0:9?

Exercise 1.6.3. Consider the autocovariance function

�.h/ D 8

4; h D 0; �2; h D ˙1; 0; otherwise.

Determine the parameters � and �2, if they exist, of the first order moving-average process Xt D Zt C �Zt�1 with Zt � WN.0;�2/ such that autocovariance function above is the autocovariance function corresponding to fXtg.

24 1 Introduction and Basic Theoretical Concepts

Exercise 1.6.4. Let the stochastic process fXtg be defined as �

Zt; if t is even; .Z2t�1 � 1/=

p 2; if t is uneven,

where fZtg is identically and independently distributed as Zt � N.0;1/. Show that fXtg � WN.0;1/, but not IID.0;1/.

Exercise 1.6.5. Which of the following processes is stationary?

(i) Xt D Zt C �Zt�1 (ii) Xt D ZtZt�1

(iii) Xt D a C �Z0 (iv) Xt D Z0 sin.at/

In all cases we assume that fZtg is identically and independently distributed with Zt � N.0;�2/. � and a are arbitrary parameters.

2Autoregressive Moving-Average Models

A basic idea in time series analysis is to construct more complex processes from simple ones. In the previous chapter we showed how the averaging of a white noise process leads to a process with first order autocorrelation. In this chapter we generalize this idea and consider processes which are solutions of linear stochastic difference equations. These so-called ARMA processes constitute the most widely used class of models for stationary processes.

Definition 2.1 (ARMA Models). A stochastic process fXtg with t 2 Z is called an autoregressive moving-average process (ARMA process) of order .p; q/, denoted by ARMA.p; q/ process, if the process is stationary and satisfies a linear stochastic difference equation of the form

Xt � �1Xt�1 � : : : � �pXt�p D Zt C �1Zt�1 C : : : C �qZt�q (2.1)

with Zt � WN.0;�2/ and �p�q ¤ 0. fXtg is called an ARMA.p; q/ process with mean � if fXt � �g is an ARMA.p; q/ process.

The importance of ARMA processes is due to the fact that every stationary process can be approximated arbitrarily well by an ARMA process. In particular, it can be shown that for any given autocovariance function � with the property limh!1 �.h/ D 0 and any positive integer k there exists an autoregressive moving- average process (ARMA process) fXtg such that �X.h/ D �.h/, h D 0;1; : : : ; k.

For an ARMA process with mean � one often adds a constant c to the right hand side of the difference equation:

Xt � �1Xt�1 � : : : � �pXt�p D c C Zt C �1Zt�1 C : : : C �qZt�q:

The mean of Xt is then: � D c1��1�:::��p . The mean is therefore only well-defined if �1 C : : : C �p ¤ 1. The case �1 C : : : C �p D 1 can, however, be excluded because there exists no stationary solution in this case (see Remark 2.2) and thus no ARMA process.

26 2 ARMA Models

2.1 The Lag Operator

In times series analysis it is customary to rewrite the above difference equation more compactly in terms of the lag operator L. This is, however, not only a compact notation, but will open the way to analyze the inner structure of ARMA processes. The lag or back-shift operator L moves the time index one period back:

LfXtg D fXt�1g:

For ease of notation we write: LXt D Xt�1. The lag operator is a linear operator with the following calculation rules:

(i) L applied to the process fXt D cg where c is an arbitrary constant gives:

Lc D c:

(ii) Applying L n times:

L : : : L „ƒ‚…

n times

Xt D LnXt D Xt�n:

(iii) The inverse of the lag operator is the lead or forward operator. This operator shifts the time index one period into the future.1 We can write L�1:

L�1Xt D XtC1:

(iv) For any integers m and n we have:

LmLnXt D LmCnXt D Xt�m�n:

(v) As L�1LXt D Xt we have that

L0 D 1:

(vi) For any real numbers a and b, any integers m and n, and arbitrary stochastic processes fXtg and fYtg we have:

.aLm C bLn/.Xt C Yt/ D aXt�m C bXt�n C aYt�m C bYt�n:

In this way it is possible to define lag polynomials: A.L/ D a0 C a1L C a2L2 C: : :C apL

p where a0; a1; : : : ; ap are any real numbers. For these polynomials the usual

1One technical advantage of using the double-infinite index set Z is that the lag operators form a group.

2.2 Some Important Special Cases 27

calculation rules apply. Let, for example, A.L/ D 1 � 0:5L and B.L/ D 1 C 4L2 then C.L/ D A.L/B.L/ D 1 � 0:5L C 4L2 � 2L3.

Applied to the stochastic difference equation, we define the autoregressive and the moving-average polynomial as follows:

ˆ.L/ D 1 � �1L � : : : � �pLp; ‚.L/ D 1 C �1L C : : : C �qLq:

The stochastic difference equation defining the ARMA process can then be written compactly as

ˆ.L/Xt D ‚.L/Zt:

Thus, the use of lag polynomials provides a compact notation for ARMA processes. Moreover and most importantly, ˆ.z/ and ‚.z/, viewed as polynomials of the complex number z, also reveal much of their inherent structural properties as will become clear in Sect. 2.3.

2.2 Some Important Special Cases

Before we deal with the general theory of ARMA processes, we will analyze some important special cases first:

q D 0: autoregressive process of order p, AR.p/ process p D 0: moving-average process of order q, MA.q/ process

2.2.1 The Moving-Average Process of Order q (MA.q/ Process)

The MA.q/ process is defined by the following stochastic difference equation:

Xt D ‚.L/Zt D �0Zt C �1Zt�1 C : : : C �qZt�q with �0 D 1 and �q ¤ 0

and Zt � WN.0;�2/. Obviously,

EXt D EZt C �1EZt�1 C : : : C �qEZt�q D 0;

because Zt � WN.0;�2/. As can be easily verified using the properties of fZtg, the autocovariance function of the MA.q/ processes are:

�X.h/ D cov.XtCh; Xt/ D E.XtChXt/ D E.ZtCh C �1ZtCh�1 C : : : C �qZtCh�q/.Zt C �1Zt�1 C : : : C �qZt�q/

28 2 ARMA Models

0 20 40 60 80 100 −4

−2

time

realization

0 5 10 15 20 −1

−0.5

0.5

order

co rr

el at

io n c

o ef

fi ci

en t estimated ACF

upper bound for confidence interval

lower bound for confidence interval

theoretical ACF

Fig. 2.1 Realization and estimated ACF of a MA(1) process: Xt D Zt � 0:8Zt�1 with Zt � IID N.0;1/

D (

�2 Pq�jhj

iD0 �i�iCjhj; jhj � qI 0; jhj > q:

This implies the following autocorrelation function:

�X.h/ D corr.XtCh; Xt/ D (

1 Pq

iD0 � 2 i

Pq�jhj iD0 �i�iCjhj; jhj � qI

0; jhj > q:

Every MA.q/ process is therefore stationary irrespective of its parameters �0;�1; : : : ;�q. Because the correlation between Xt and Xs is equal to zero if the two time points t and s are more than q periods apart, such processes are sometimes called processes with short memory or processes with short range dependence.

Figure 2.1 displays an MA(1) process and its autocorrelation function.

2.2 Some Important Special Cases 29

2.2.2 The First Order Autoregressive Process (AR(1) Process)

The AR.p/ process requires a more thorough analysis as will already become clear from the AR(1) process. This process is defined by the following stochastic difference equation:

Xt D �Xt�1 C Zt; Zt � WN.0;�2/ and � ¤ 0: (2.2)

The above stochastic difference equation has in general several solutions. Given a sequence fZtg and an arbitrary distribution for X0, it determines all random variables Xt, t 2 Z n f0g, by applying the above recursion. The solutions are, however, not necessarily stationary. But, according to the Definition 2.1, only stationary processes qualify for ARMA processes. As we will demonstrate, depending on the value of �, there may exist no or just one solution.

Consider first the case of j�j < 1. Inserting into the difference equation several times leads to:

Xt D �Xt�1 C Zt D �2Xt�2 C �Zt�1 C Zt D : : : D Zt C �Zt�1 C �2Zt�2 C : : : C �kZt�k C �kC1Xt�k�1:

If fXtg is a stationary solution, VXt�k�1 remains constant independently of k. Thus

@Xt � k X

jD0 �jZt�j

A D �2kC2VXt�k�1 ! 0 for k ! 1:

This shows that Pk

jD0 � jZt�j converges in the mean square sense, and thus also in

probability, to Xt for k ! 1 (see Theorem C.8 in Appendix C). This suggests to take

Xt D Zt C �Zt�1 C �2Zt�2 C : : : D 1 X

jD0 �jZt�j (2.3)

as the solution to the stochastic difference equation. As P1

jD0 j�jj D 11�� < 1 this solution is well-defined according to Theorem 6.4 and has the following properties:

EXt D 1 X

jD0 �

j EZt�j D 0;

30 2 ARMA Models

�X.h/ D cov.XtCh; Xt/ D lim k!1

k X

jD0 �jZtCh�j

k X

jD0 �jZt�j

D �2�jhj 1 X

jD0 � 2j D �

jhj

1 � �2 � 2 ; h 2 Z;

�X.h/ D �jhj:

Thus the solution Xt D P1

jD0 � jZt�j is stationary and fulfills the difference equation

as can be easily verified. It is also the only stationary solution which is compatible with the difference equation. Assume that there is second solution f QXtg with these properties. Inserting into the difference equation yields again

@ QXt � k X

jD0 �jZt�j

A D �2kC2V QXt�k�1:

This variance converges to zero for k going to infinity because j�j < 1 and because f QXtg is stationary. The two processes f QXtg and fXtg with Xt D

P1 jD0 �

jZt�j are therefore identical in the mean square sense and thus with probability one.

Finally, note that the recursion (2.2) will only generate a stationary process if it is initialized with X0 having the stationary distribution, i.e. if EX0 D 0 and VX0 D �2=.1 � �2/. If the recursion is initiated with an arbitrary variance of X0, 0 < �20 < 1, Eq. (2.2) implies the following difference equation for the variance of Xt, �2t :

�t D �2�2t�1 C �2:

The solution of this difference equation is

�2t � �2� D .�20 � �2�/.�2/t

where �2� D �2=.1 ��2/ denotes the variance of the stationary distribution. If �20 ¤ �2�, �

2 t is not constant implying that the process fXtg is not stationary. However,

as j�j < 1, the variance of Xt, �2t , will converge to the variance of the stationary distribution.2

Figure 2.2 shows a realization of such a process and its estimated autocorrelation function.

In the case j�j > 1 the solution (2.3) does not converge. It is, however, possible to iterate the difference equation forward in time to obtain:

2Phillips and Sul (2007) provide an application and an in depth discussion of the hypothesis of economic growth convergence.

2.2 Some Important Special Cases 31

0 20 40 60 80 100 −4

−2

time

realization

0 5 10 15 20 −1

−0.5

0.5

order

co rr

el at

io n c

o ef

fi ci

en t estimated ACF

lower bound for confidence interval

upper bound for confidence interval

theoretical ACF

Fig. 2.2 Realization and estimated ACF of an AR(1) process: Xt D 0:8Xt�1 C Zt with Zt � IIN.0;1/

Xt D ��1XtC1 � ��1ZtC1 D ��k�1XtCkC1 � ��1ZtC1 � ��2ZtC2 � : : : � ��k�1ZtCkC1:

This suggests to take

Xt D � 1 X

jD1 �

�jZtCj

as the solution. Going through similar arguments as before it is possible to show that this is indeed the only stationary solution. This solution is, however, viewed to be inadequate because Xt depends on future shocks ZtCj; j D 1;2; : : : Note, however, that there exists an AR(1) process with j�j < 1 which is observationally equivalent, in the sense that it generates the same autocorrelation function, but with a new shock or forcing variable feZt g (see next section).

In the case j�j D 1 there exists no stationary solution (see Sect. 1.4.4) and therefore, according to our definition, no ARMA process. Processes with this property are called random walks, unit root processes or integrated processes. They play an important role in economics and are treated separately in Chap. 7.

32 2 ARMA Models

2.3 Causality and Invertibility

If we interpret fXtg as the state variable and fZtg as an impulse or shock, we can ask whether it is possible to represent today’s state Xt as the outcome of current and past shocks Zt; Zt�1; Zt�2; : : : In this case we can view Xt as being caused by past shocks and call this a causal representation. Thus, shocks to current Zt will not only influence current Xt, but will propagate to affect also future Xt’s. This notion of causality rest on the assumption that the past can cause the future but that the future cannot cause the past. See Sect. 15.1 for an elaboration of the concept of causality and its generalization to the multivariate context.

In the case that fXtg is a moving-average process of order q, Xt is given as a weighted sum of current and past shocks Zt; Zt�1; : : : ; Zt�q. Thus, the moving- average representation is already the causal representation. In the case of an AR(1) process, we have seen that this is not always feasible. For j�j < 1, the solution (2.3) represents Xt as a weighted sum of current and past shocks and is thus the corresponding causal representation. For j�j > 1, no such representation is possible. The following Definition 2.2 makes the notion of a causal representation precise and Theorem 2.1 gives a general condition for its existence.

Definition 2.2 (Causality). An ARMA.p; q/ process fXtg with ˆ.L/Xt D ‚.L/Zt is called causal with respect to fZtg if there exists a sequence f jg with the property P1

jD0 j jj < 1 such that

Xt D Zt C 1Zt�1 C 2Zt�2 C : : : D 1 X

jD0 jZt�j D ‰.L/Zt with 0 D 1:

where ‰.L/ D 1 C 1L C 2L2 C : : : D P1

jD0 jL j. The above equation is referred

to as the causal representation of fXtg with respect to fZtg. The coefficients f jg are of great importance because they determine how an

impulse or a shock in period t propagates to affect current and future XtCj, j D 0;1;2: : : In particular, consider an impulse et0 at time t0, i.e. a time series which is equal to zero except for the time t0 where it takes on the values et0 . Then, f t�t0 et0g traces out the time history of this impulse. For this reason, the coefficients j with j D t � t0, t D t0; t0 C 1; t0 C 2; : : : , are called the impulse response function. If et0 D 1, it is called a unit impulse. Alternatively, et0 is sometimes taken to be equal to �, the standard deviation of Zt. It is customary to plot j as a function of j, j D 0;1;2: : :

Note that the notion of causality is not an attribute of fXtg, but is defined relative to another process fZtg. It is therefore possible that a stationary process is causal with respect to one process, but not with respect to another process. In order to make this point more concrete, consider again the AR(1) process defined by the equation Xt D �Xt�1 CZt with j�j > 1. As we have seen, the only stationary solution is given by Xt D �

P1 jD1 �

�jZtCj which is clearly not causal with respect fZtg. Consider as

2.3 Causality and Invertibility 33

an alternative the process

eZt D Xt � 1

� Xt�1 D ��2Zt C .��2 � 1/

1 X

jD1 �

�jZtCj: (2.4)

This new process is white noise with variance Q�2 D ��2�2.3 Because fXtg fulfills the difference equation

Xt D 1

� Xt�1 C QZt;

fXtg is causal with respect to f QZtg. This remark shows that there is no loss of generality involved if we concentrate on causal ARMA processes.

Theorem 2.1. Let fXtg be an ARMA.p; q/ process with ˆ.L/Xt D ‚.L/Zt and assume that the polynomials ˆ.z/ and ‚.z/ have no common root. fXtg is causal with respect to fZtg if and only if ˆ.z/ ¤ 0 for jzj � 1, i.e. all roots of the equation ˆ.z/ D 0 are outside the unit circle. The coefficients f jg are then uniquely defined by identity :

‰.z/ D 1 X

jD0 jz

j D ‚.z/ ˆ.z/

Proof. Given that ˆ.z/ is a finite order polynomial with ˆ.z/ ¤ 0 for jzj � 1, there exits > 0 such that ˆ.z/ ¤ 0 for jzj � 1 C . This implies that 1=ˆ.z/ is an analytic function on the circle with radius 1 C and therefore possesses a power series expansion:

ˆ.z/ D

1 X

jD0 �jz

j D „.z/; for jzj < 1 C :

This implies that �j.1C =2/j goes to zero for j to infinity. Thus there exists a positive and finite constant C such that

j�jj < C.1 C =2/�j; for all j D 0;1;2; : : :

This in turn implies that P1

jD0 j�jj < 1 and that „.z/ˆ.z/ D 1 for jzj � 1. Applying „.L/ on both sides of ˆ.L/Xt D ‚.L/Zt, gives:

Xt D „.L/ˆ.L/Xt D „.L/‚.L/Zt:

3The reader is invited to verify this.

34 2 ARMA Models

Theorem 6.4 implies that the right hand side is well-defined. Thus ‰.L/ D „.L/‚.L/ is the sought polynomial. Its coefficients are determined by the relation ‰.z/ D ‚.z/=ˆ.z/.

Assume now that there exists a causal representation Xt D P1

jD0 jZt�j with P1

jD0 j jj < 1. Therefore ‚.L/Zt D ˆ.L/Xt D ˆ.L/‰.L/Zt:

Take �.z/ D ˆ.z/‰.z/ D P1jD0 �jzj, jzj � 1. Multiplying the above equation by Zt�k and taking expectations shows that �k D �k, k D 0;1;2; : : : ; q, and that �k D 0 for k > q. Thus we get ‚.z/ D �.z/ D ˆ.z/‰.z/ for jzj � 1. As ‚.z/ and ˆ.z/ have no common roots and because j‰.z/j < 1 for jzj � 1, ˆ.z/ cannot be equal to zero for jzj � 1. ut

Remark 2.1. If the AR and the MA polynomial have common roots, there are two possibilities:

• No common roots lies on the unit circle. In this situation there exists a unique stationary solution which can be obtained by canceling the common factors of the polynomials.

• If at least one common root lies on the unit circle then more than one stationary solution may exist (see the last example below).

Some Examples

We concretize the above Theorem and Remark by investigating some examples starting from the ARMA model ˆ.L/Xt D ‚.L/Zt with Zt � WN.0;�2/. ˆ.L/ D 1 � 0:05L � 0:6L2 and ‚.L/ D 1: The roots of the polynomial ˆ.z/ are

z1 D �4=3 and z2 D 5=4. Because both roots are absolutely greater than one, there exists a causal representation with respect to fZtg.

ˆ.L/ D 1 C 2L C 5=4L2 and ‚.L/ D 1: In this case the roots are conjugate complex and equal to z1 D �4=5 C 2=5{ and z2 D �4=5 � 2=5{. The modulus or absolute value of z1 and z2 equals jz1j D jz2j D

20=25. This number is smaller than one. Therefore there exists a stationary solution, but this solution is not causal with respect to fZtg.

ˆ.L/ D 1 � 0:05L � 0:6L2 and ‚.L/ D 1 C 0:75L: ˆ.z/ and ‚.z/ have the common root z D �4=3 ¤ 1. Thus one can cancel both ˆ.L/ and ‚.L/ by 1 C 3

4 L to obtain the polynomials Q̂ .L/ D 1 � 0:8L and Q‚.L/ D 1. Because the

root of Q̂ .z/ equals 5/4 which is greater than one, there exists a unique stationary and causal representation with respect to fZtg.

ˆ.L/ D 1 C 1:2L � 1:6L2 and ‚.L/ D 1 C 2L: The roots of ˆ.z/ are z1 D 5=4 and z2 D �0:5. Thus one root is outside the unit circle whereas one is inside. This would suggest that there is no causal solution. However, the root �0:5 ¤ 1 is shared by ˆ.z/ and ‚.z/ and can therefore be canceled to obtain Q̂ .L/ D 1�0:8L

2.3 Causality and Invertibility 35

and Q‚.L/ D 1. Because the root of Q̂ .z/ equals 5=4 > 1, there exists a unique stationary and causal solution with respect to fZtg.

ˆ.L/ D 1 C L and ‚.L/ D 1 C L: ˆ.z/ and ‚.z/ have the common root �1 which lies on the unit circle. As before one might cancel both polynomials by 1 C L to obtain the trivial stationary and causal solution fXtg D fZtg. This is, however, not the only solution. Additional solutions are given by fYtg D fZt C A.�1/tg where A is an arbitrary random variable with mean zero and finite variance �2A which is independent from both fXtg and fZtg. The process fYtg has a mean of zero and an autocovariance function �Y.h/ which is equal to

�Y.h/ D (

�2 C �2A; h D 0I .�1/h�2A; h D ˙1; ˙2; : : :

Thus this new process is therefore stationary and fulfills the difference equation.

Remark 2.2. If the AR and the MA polynomial in the stochastic difference equation ˆ.L/Xt D ‚.L/Zt have no common root, but ˆ.z/ D 0 for some z on the unit circle, there exists no stationary solution. In this sense the stochastic difference equation does no longer define an ARMA model. Models with this property are said to have a unit root and are treated in Chap. 7. If ˆ.z/ has no root on the unit circle, there exists a unique stationary solution.

As explained in the previous Theorem, the coefficients f jg of the causal representation are uniquely determined by the relation ‰.z/ˆ.z/ D ‚.z/. If fXtg is a MA process, ˆ.z/ D 1 and the coefficients f jg just correspond to the coefficients of the MA polynomial, i.e. j D �j for 0 � j � q and j D 0 for j > q. Thus in this case no additional computations are necessary. In general this is not the case. In principle there are two ways to find the coefficients f jg. The first one uses polynomial division or partial fractions, the second one uses the method of undetermined coefficients. This book relies on the second method because it is more intuitive and presents some additional insides. For this purpose let us write out the defining relation ‰.z/ˆ.z/ D ‚.z/: �

0 C 1z C 2z2 C : : : ��

1 � �1z � �2z2 � : : :�pzp �

D 1 C �1z C �2z2 C : : : C �qzq

Multiplying out the left hand side one gets:

0 � 0�1z � 0�2z2 � 0�3z3 � � � � � 0�pzp

1z � 1�1z2 � 1�2z3 � � � � � 1�pzpC1

C 2 z2 � 2�1z3 � � � � � 2�pzpC2

36 2 ARMA Models

: : :

D 1 C �1z C �2z2 C �3z3 C � � � C �qzq

Equating the coefficients of the powers of z, zj, j D 0;1;2; : : : , one obtains the following equations:

z0 W 0 D 1; z1 W 1 D �1 C �1 0 D �1 C �1; z2 W 2 D �2 C �2 0 C �1 1 D �2 C �2 C �1�1 C �21;

: : :

As can be seen, it is possible to solve recursively for the unknown coefficients f jg. This is convenient when it comes to numerical computations, but in some cases one wants an analytical solution. Such a solution can be obtained by observing that, for j � maxfp; q C 1g, the recursion leads to the following difference equation of order p:

j D p X

kD1 �k j�k D �1 j�1 C �2 j�2 C : : : C �p j�p; j � maxfp; q C 1g:

This is a linear homogeneous difference equation with constant coefficients. The solution of such an equation is of the form (see Eq. (B.1) in Appendix B):

j D c1z�j1 C : : : C cpz�jp ; j � maxfp; q C 1g � p; (2.5)

where z1; : : : ; zp denote the roots of ˆ.z/ D 1 � �1z � : : : � �pzp D 0.4 Note that the roots are exactly those which have been computed to assess the existence of a causal representation. The coefficients c1; : : : ; cp can be obtained using the p boundary conditions obtained from j D

0<k�j �k j�k D �j, maxfp; q C1g � p � j < maxfp; q C 1g. Finally, the values for j, 0 � j < maxfp; q C 1g � p, must be computed from the first maxfp; q C 1g � p iterations (see the example in Sect. 2.4).

As mentioned previously, the coefficients f jg are of great importance as they quantify the effect of a shock to Zt�j on Xt, respectively of Zt on XtCj. In macroeconomics they are sometimes called dynamic multipliers of a transitory or temporary shock. Because the underlying ARMA process is stationary and causal, the infinite sum

P1 jD0 j jj converges. This implies that the effect j converges to

4In the case of multiple roots one has to modify the formula according to Eq. (B.2).

2.3 Causality and Invertibility 37

zero as j ! 1. Thus the effect of a shock dies out eventually5: @XtCj @Zt

D j ! 0 for j ! 1:

As can be seen from Eq. (2.5), the coefficients f jg even converge to zero exponen- tially fast to zero because each term ciz

�j i , i D 1; : : : ; p, goes to zero exponentially

fast as the roots zi are greater than one in absolute value. Viewing f jg as a function of j one gets the so-called impulse response function which is usually displayed graphically.

The effect of a permanent shock in period t on XtCj is defined as the cumulative effect of a transitory shock. Thus, the effect of a permanent shock to XtCj is given by Pj

iD0 i. Because Pj

iD0 i � Pj

iD0 j ij � P1

iD0 j ij < 1, the cumulative effect remains finite.

In time series analysis we view the observations as realizations of fXtg and treat the realizations of fZtg as unobserved. It is therefore of interest to know whether it is possible to recover the unobserved shocks from the observations on fXtg. This idea leads to the concept of invertibility.

Definition 2.3 (Invertibility). An ARMA(p,q) process for fXtg satisfying ˆ.L/Xt D ‚.L/Zt is called invertible with respect to fZtg if and only if there exists a sequence f�jg with the property

P1 jD0 j�jj < 1 such that

Zt D 1 X

jD0 �jXt�j:

Note that like causality, invertibility is not an attribute of fXtg, but is defined only relative to another process fZtg. In the literature, one often refers to invertibility as the strict miniphase property.6

Theorem 2.2. Let fXtg be an ARMA(p,q) process with ˆ.L/Xt D ‚.L/Zt such that polynomials ˆ.z/ and ‚.z/ have no common roots. Then fXtg is invertible with respect to fZtg if and only if ‚.z/ ¤ 0 for jzj � 1. The coefficients f�jg are then uniquely determined through the relation:

5The use of the partial derivative sign actually represents an abuse of notation. It is inspired by an

alternative definition of the impulse responses: j D @QPt XtCj@xt where QPt denotes the optimal (in the mean squared error sense) linear predictor of XtCj given a realization back to infinite remote past fxt; xt�1; xt�2; : : : g (see Sect. 3.1.3). Thus, j represents the sensitivity of the forecast of XtCj with respect to the observation xt . The equivalence of alternative definitions in the linear and especially nonlinear context is discussed in Potter (2000). 6Without the qualification strict, the miniphase property allows for roots of ‚.z/ on the unit circle. The terminology is, however, not uniform in the literature.

38 2 ARMA Models

….z/ D 1 X

jD0 �jz

j D ˆ.z/ ‚.z/

Proof. The proof follows from Theorem 2.1 with Xt and Zt interchanged. ut

The discussion in Sect. 1.3 showed that there are in general two MA(1) processes compatible with the same autocorrelation function �.h/ given by �.0/ D 1, �.1/ D � with j�j � 1

2 , and �.h/ D 0 for h � 2. However, only one of these solutions

is invertible because the two solutions for � are inverses of each other. As it is important to be able to recover Zt from current and past Xt, one prefers the invertible solution. Section 3.2 further elucidates this issue.

Remark 2.3. If fXtg is a stationary solution to the stochastic difference equation ˆ.L/Xt D ‚.L/Zt with Zt � WN.0;�2/ and if ˆ.z/‚.z/ ¤ 0 for jzj � 1 then

Xt D 1 X

jD0 jZt�j;

Zt D 1 X

jD0 �jXt�j;

where the coefficients f jg and f�jg are determined for jzj � 1 by ‰.z/ D ‚.z/

ˆ.z/ and

….z/ D ˆ.z/ ‚.z/

, respectively. In this case fXtg is causal and invertible with respect to fZtg.

Remark 2.4. If fXtg is an ARMA process with ˆ.L/Xt D ‚.L/Zt such that ˆ.z/ ¤ 0 for jzj D 1 then there exists polynomials Q̂ .z/ and Q‚.z/ and a white noise process f QZtg such that fXtg fulfills the stochastic difference equation Q̂ .L/Xt D Q‚.L/QZt and is causal with respect to f QZtg. If in addition ‚.z/ ¤ 0 for jzj D 1 then Q‚.L/ can be chosen such that fXtg is also invertible with respect to f QZtg (see the discussion of the AR(1) process after the definition of causality and Brockwell and Davis (1991, p. 88)). Thus, without loss of generality, we can restrict the analysis to causal and invertible ARMA processes.

2.4 Computation of the Autocovariance Function of an ARMA Process

Whereas the autocovariance function summarizes the external and directly observ- able properties of a time series, the coefficients of the ARMA process give information of its internal structure. Although there exists for each ARMA model

2.4 Computation of Autocovariance Function 39

a corresponding autocovariance function, the converse is not true as we have seen in Sect. 1.3 where we showed that two MA(1) processes are compatible with the same autocovariance function. This brings up a fundamental identification problem. In order to shed some light on the relation between autocovariance function and ARMA models it is necessary to be able to compute the autocovariance function for a given ARMA model. In the following, we will discuss three such procedures. Each procedure relies on the assumption that the ARMA process ˆ.L/Xt D ‚.L/Zt with Zt � WN.0;�2/ is causal with respect to fZtg. Thus there exists a representation of Xt as a weighted sum of current and past Zt ’s: Xt D

P1 jD0 jZt�j

with P1

jD0 j jj < 1.

2.4.1 First Procedure

Starting from the causal representation of fXtg, it is easy to calculate its autoco- variance function given that fZtg is white noise. The exact formula is proved in Theorem (6.4).

�.h/ D �2 1 X

jD0 j jCjhj;

where

‰.z/ D 1 X

jD0 jz

j D ‚.z/ ˆ.z/

for jzj � 1:

The first step consists in determining the coefficients j by the method of undeter- mined coefficients. This leads to the following system of equations:

j � X

0<k�j �k j�k D �j; 0 � j < maxfp; q C 1g;

j � X

0<k�p �k j�k D 0; j � maxfp; q C 1g:

This equation system can be solved recursively (see Sect. 2.3):

0 D �0 D 1; 1 D �1 C 0�1 D �1 C �1; 2 D �2 C 0�2 C 1�1 D �2 C �2 C �1�1 C �21;

: : :

40 2 ARMA Models

Alternatively one may view the second part of the equation system as a linear homogeneous difference equation with constant coefficients (see Sect. 2.3). Its solution is given by Eq. (2.5). The first part of the equation system delivers the necessary initial conditions to determine the coefficients c1; c2; : : : ; cp. Finally one can insert the ’s in the above formula for the autocovariance function.

A Numerical Example Consider the ARMA(2,1) process with ˆ.L/ D 1 � 1:3L C 0:4L2 and ‚.L/ D 1 C 0:4L. Writing out the defining equation for ‰.z/, ‰.z/ˆ.z/ D ‚.z/, gives:

1 C 1z C 2z2 C 3z3 C : : : � 1:3z � 1:3 1z2 � 1:3 2z3 � : : :

C 0:4z2 C 0:4 1z3 C : : : : : : D 1 C 0:4z:

Equating the coefficients of the powers of z leads to the following equation system:

z0 W 0 D 1; z W 1 � 1:3 D 0:4; z2 W 2 � 1:3 1 C 0:4 D 0; z3 W 3 � 1:3 2 C 0:4 1 D 0;

: : :

j � 1:3 j�1 C 0:4 j�2 D 0; for j � 2:

The last equation represents a linear difference equation of order two. Its solution is given by

j D c1z�j1 C c2z�j2 ; j � maxfp; q C 1g � p D 0;

whereby z1 and z2 are the two distinct roots of the characteristic polynomial ˆ.z/ D 1 � 1:3z C 0:4z2 D 0 (see Eq. (2.5)) and where the coefficients c1 and c2 are determined from the initial conditions. The two roots are

1:3˙p1:69�4�0:4 2�0:4 D

5=4 D 1:25 and 2. The general solution to the homogeneous equation therefore is j D c10:8j C c20:5j. The constants c1 and c2 are determined by the equations:

j D 0 W 0 D 1 D c10:80 C c20:50 D c1 C c2 j D 1 W 1 D 1:7 D c10:81 C c20:51 D 0:8c1 C 0:5c2:

2.4 Computation of Autocovariance Function 41

Solving this equation system in the two unknowns c1 and c2 gives: c1 D 4 and c2 D �3. Thus the solution to the difference equation is given by:

j D 4.0:8/j � 3.0:5/j: Inserting this solution for j into the above formula for �.h/ one obtains after using the formula for the geometric sum:

�.h/ D �2 1 X

jD0

�

4 � 0:8j � 3 � 0:5j��4 � 0:8jCh � 3 � 0:5jCh�

D �2 1 X

jD0 .16 � 0:82jCh � 12 � 0:5j � 0:8jCh

� 12 � 0:8j � 0:5jCh C 9 � 0:52jCh/

D 16�2 0:8 h

1 � 0:64 � 12� 2 0:8

1 � 0:4 � 12� 2 0:5

1 � 0:4 C 9� 2 0:5

1 � 0:25 D 220

9 �2.0:8/h � 8�2.0:5/h:

Dividing �.h/ by �.0/, one gets the autocorrelation function:

�.h/ D �.h/ �.0/

D 55 37

� 0:8j � 18 37

� 0:5j

which is represented in Fig. 2.3.

2.4.2 Second Procedure

Instead of determining the j coefficients first, it is possible to compute the autocovariance function directly from the ARMA model. To see this multiply the ARMA equation successively by Xt�h; h D 0;1; : : : and apply the expectations operator:

EXtXt�h � �1EXt�1Xt�h � � � � � �pEXt�pXt�h D EZt Xt�h C �1EZt�1Xt�h C � � � C �qEZt�qXt�h:

This leads to an equation system for the autocovariances �.h/, h D 0;1;2; : : : :

�.h/ � �1�.h � 1/ � : : : � �p�.h � p/ D �2 X

h�j�q �j j�h; h < maxfp; q C 1g

�.h/ � �1�.h � 1/ � : : : � �p�.h � p/ D 0; h � maxfp; q C 1g:

42 2 ARMA Models

0 2 4 6 8 10 12 14 16 18 20 −1

−0.8

−0.6

−0.4

−0.2

0.2

0.4

0.6

0.8

order

co rr

el at

io n

co ef

fi ci

en t

Fig. 2.3 Autocorrelation function of the ARMA(2,1) process: .1 � 1:3L C 0:4L2/Xt D .1 C 0:4L/Zt

The second part of the equation system consists again of a linear homogeneous difference equation in �.h/ whereas the first part can be used to determine the initial conditions. Note that the initial conditions depend 1;: : : ; q which have to be determined before hand. The general solution of the difference equation is:

�.h/ D c1z�h1 C : : : C cpz�hp (2.6)

where z1; : : : ; zp are the distinct roots of the polynomial ˆ.z/ D 1 � �1z � : : : � �pz

p D 0.7 The constants c1; : : : ; cp can be computed from the first p initial conditions after the 1;: : : q have been calculated like in the first procedure. The form of the solution shows that the autocovariance and hence the autocorrelation function converges to zero exponentially fast.

A Numerical Example We consider the same example as before. The second part of the above equation system delivers a difference equation for �.h/: �.h/ D �1�.h � 1/ C �2�.h � 2/ D 1:3�.h � 1/ � 0:4�.h � 2/, h � 2. The general solution of this difference equation is (see Appendix B):

�.h/ D c1.0:8/h C c2.0:5/h; h � 2

7In case of multiple roots the formula has to be adapted accordingly. See Eq. (B.2) in the Appendix.

2.4 Computation of Autocovariance Function 43

where 0:8 and 0:5 are the inverses of the roots computed from the same polynomial ˆ.z/ D 1 � 1:3z � 0:4z2 D 0.

The first part of the system delivers the initial conditions which determine the constants c1 and c2:

�.0/ � 1:3�.�1/ C 0:4�.�2/ D �2.1 C 0:4 � 1:7/ �.1/ � 1:3�.0/ C 0:4�.�1/ D �20:4

where the numbers on the right hand side are taken from the first procedure. Inserting the general solution in this equation system and bearing in mind that �.h/ D �.�h/ leads to:

0:216c1 C 0:450c2 D 1:68�2

�0:180c1 � 0:600c2 D 0:40�2

Solving this equation system in the unknowns c1 and c2 one gets finally gets: c1 D .220=9/�2 and c2 D �8�2.

2.4.3 Third Procedure

Whereas the first two procedures produce an analytical solution which relies on the solution of a linear difference equation, the third procedure is more suited for numerical computation using a computer. It rests on the same equation system as in the second procedure. The first step determines the values �.0/;�.1/; : : : ;�.p/ from the first part of the equation system. The following �.h/; h > p are then computed recursively using the second part of the equation system.

A Numerical Example Using again the same example as before, the first of the equation delivers �.2/;�.1/ and �.0/ from the equation system:

�.0/ � 1:3�.�1/ C 0:4�.�2/ D �2.1 C 0:4 � 1:7/ �.1/ � 1:3�.0/ C 0:4�.�1/ D �20:4 �.2/ � 1:3�.1/ C 0:4�.0/ D 0

Bearing in mind that �.h/ D �.�h/, this system has three equations in three unknowns �.0/;�.1/ and �.2/. The solution is: �.0/ D .148=9/�2, �.1/ D .140=9/�2, �.2/ D .614=45/�2. This corresponds, of course, to the same numerical values as before. The subsequent values for �.h/; h > 2 are then determined recursively from the difference equation�.h/ D 1:3�.h � 1/ � 0:4�.h � 2/.

44 2 ARMA Models

2.5 Exercises

Exercise 2.5.1. Consider the AR(1) process Xt D 0:8Xt�1 C Zt with Zt � WN.0;�2/. Compute the variance of .X1 C X2 C X3 C X4/=4.

Exercise 2.5.2. Check whether the following stochastic difference equations pos- sess a stationary solution. If yes, is the solution causal and/or invertible with respect to Zt � WN.0;�2/?

(i) Xt D Zt C 2Zt�1 (ii) Xt D 1:3Xt�1 C Zt

(iii) Xt D 1:3Xt�1 � 0:4Xt�2 C Zt (iv) Xt D 1:3Xt�1 � 0:4Xt�2 C Zt � 0:3Zt�1 (v) Xt D 0:2Xt�1 C 0:8Xt�2 C Zt

(vi) Xt D 0:2Xt�1 C 0:8Xt�2 C Zt � 1:5Zt�1 C 0:5Zt�2 Exercise 2.5.3. Compute the causal representation with respect to Zt � WN.0;�2/ for the following ARMA processes:

(i) Xt D 1:3Xt�1 � 0:4Xt�2 C Zt (ii) Xt D 1:3Xt�1 � 0:4Xt�2 C Zt � 0:2Zt�1

(iii) Xt D �Xt�1 C Zt C �Zt�1 with j�j < 1

Exercise 2.5.4. Compute the autocovariance function of the ARMA processes:

(i) Xt D 0:5Xt�1 C 0:36Xt�2 C Zt (ii) Xt D 0:5Xt�1 C 0:36Xt�2 C Zt C 0:5Zt�1 Thereby Zt � WN.0;�2/.

Exercise 2.5.5. Verify that the process feZtg defined in Eq. (2.4) is white noise with eZt � WN.0;��2�2/.

3Forecasting Stationary Processes

An important goal of time series analysis is forecasting. In the following we will consider the problem of forecasting XTCh, h > 0, given fXT; : : : ; X1g where fXtg is a stationary stochastic process with known mean � and known autocovariance function �.h/. In practical applications � and � are unknown so that we must replace these entities by their estimates. These estimates can be obtained directly from the data as explained in Sect. 4.2 or indirectly by first estimating an appropriate ARMA model (see Chap. 5) and then inferring the corresponding autocovariance function using one of the methods explained in Sect. 2.4. Thus the forecasting problem is inherently linked to the problem of identifying an appropriate ARMA model from the data (see Deistler and Neusser 2012).

3.1 The Theory of Linear Least-Squares Forecasts

We restrict our discussion to linear forecast functions, also called linear predictors, PT XTCh. Given observation from period 1 up to period T, these predictors take the form:

PT XTCh D a0 C a1XT C : : : C aT X1 D a0 C T X

iD1 aiXTC1�i

with unknown coefficients a0; a1; a2; : : : ; aT . In principle, we should index these coefficients by T because they may change with every new observations. See the example of the MA(1) process in Sect. 3.1.2. In order not to overload the notation, we will omit this additional index.

In the Hilbert space of random variables with finite second moments the optimal forecast in the mean squared error sense is given by the conditional expectation

46 3 Forecasting Stationary Processes

E.XTChjc; XT; XT�1; : : : ; X1/. However, having practical applications in mind, we restrict ourself to linear predictors for the following reasons1:

(i) The determination of the conditional expectation is usually very difficult because all possible functions must in principle be considered whereas linear predictors are easy to compute.

(ii) The coefficients of the optimal (in the sense of means squared errors) linear forecasting function depend only on the first two moments of the time series, i.e. on EXt and �.j/, j D 0;1; : : : ; h C T � 1.

(iii) In the case of Gaussian processes the conditional expectation coincides with the linear predictor.

(iv) The optimal predictor is linear when the process is a causal and invertible ARMA process even when Zt follows an arbitrary distribution with finite variance (see Rosenblatt 2000, chapter 5).

(v) Practical experience has shown that even non-linear processes can be predicted accurately by linear predictors.

The coefficients a0; : : : ; aT of the forecasting function are determined such that the mean squared errors are minimized. The use of mean squared errors as a criterion leads to a compact representation of the solution to the forecasting problem. It implies that over- and underestimation are treated equally. Thus, we have to solve the following minimization problem:

S D S.a0; : : : ; aT/ D E.XTCh � PT XTCh/2

D E.XTCh � a0 � a1XT � : : : � aT X1/2 �! min a0;a1;:::;aT

As S is a quadratic function, the coefficients, aj; j D 0;1; : : : ; T, are uniquely determined by the so-called normal equations. These are obtained from the first order conditions of the minimization problem, i.e. from @S

@aj D 0; j D 0;1; : : : ; T:

@a0 D E

XTCh � a0 � T X

iD1 aiXTC1�i

D 0; (3.1)

@aj D E

XTCh � a0 � T X

iD1 aiXTC1�i

XTC1�j

D 0; j D 1; : : : ; T: (3.2)

The first equation can be rewritten as a0 D � � PT

iD1 ai� so that the forecasting function becomes:

PT XTCh D � C T X

iD1 ai .XTC1�i � �/:

1Elliott and Timmermann (2008) provide a general overview of forecasting procedures and their evaluations.

3.1 Linear Least-Squares Forecasts 47

The unconditional mean of the forecast error, E.XTCh � PT XTCh/, is therefore equal to zero. This means that there is no bias, neither upward nor downward, in the forecasts. The forecasts correspond on average to the “true” value.

Inserting in the second normal equation the expression for PT XTCh from above, we get:

.XTCh � PT XTCh/ XTC1�j D 0; j D 1;2; : : : ; T:

The forecast error is therefore uncorrelated with the available information repre- sented by past observations. Thus, the forecast errors XTCh �PT XTCh are orthogonal to XT; XT�1; : : : ; X1. Geometrically speaking, the best linear forecast is obtained by finding the point in the linear subspace spanned by fXT; XT�1; : : : ; X1g which is closest to XTCh. This point is found by projecting XTCh on this linear subspace.2

The normal equations (3.1) and (3.2) can be rewritten in matrix notation as follows:

a0 D �

1 � T X

iD1 ai

(3.3)

�.0/ �.1/ : : : �.T � 1/ �.1/ �.0/ : : : �.T � 2/ :::

::: :::

:::

�.T � 1/ �.T � 2/ : : : �.0/

a1 a2 :::

�.h/ �.h C 1/

:::

�.h C T � 1/

: (3.4)

Denoting by �, ˛T and �T.h/ the vectors .1;1; : : : ;1/ 0, .a1; : : : ; aT/0 and

.�.h/; : : : ;�.h C T � 1//0 and by �T D Œ�.i � j/ i;jD1;:::;T the symmetric T � T covariance matrix of .X1; : : : ; XT/0 the normal equations can be written compactly as:

a0 D � �

1 � �0˛T �

(3.5)

�T˛T D �T.h/: (3.6)

Dividing the second equation by �.0/, one obtains an equation in terms autocorre- lations instead of autocovariances:

RT˛T D �T.h/; (3.7)

where RT D �T=�.0/ and �T.h/ D .�.h/; : : : ;�.h C T � 1//0. The coefficients of the forecasting function ˛T are then obtained by inverting �T , respectively RT :

2Note the similarity of the forecast errors with the least-square residuals of a linear regression.

48 3 Forecasting Stationary Processes

˛T D

a1 :::

A D ��1T �T.h/ D R�1T �T.h/:

A sufficient condition which ensures the invertibility of �T , respectively RT , is given by assuming �.0/ > 0 and limh!1 �.h/ D 0.3 The last condition is automatically satisfied for ARMA processes because �.h/ converges even exponentially fast to zero (see Sect. 2.4).

The mean squared error or variance of the forecast error for the forecasting horizon h, vT.h/, is given by:

vT.h/ D E.XTCh � PT XTCh/2

D �.0/ � 2 T X

iD1 ai�.h C i � 1/ C

T X

iD1

T X

jD1 ai�.i � j/aj

D �.0/ � 2˛0T�T.h/ C ˛0T�T˛T D �.0/ � ˛0T�T.h/;

because �T˛T D �T.h/. Bracketing out �.0/, one can write the mean squared forecast error as:

vT.h/ D �.0/ �

1 � ˛0T�T.h/ �

: (3.8)

Because the coefficients of the forecast function have to be recomputed with the arrival of every new observation, it is necessary to have a fast and reliable algorithm at hand. These numerical problems have been solved by the development of appropriate computer algorithms, like the Durbin-Levinson algorithm or the innovation algorithm (see Brockwell and Davis 1991, Chapter 5).

3.1.1 Forecasting with an AR(p) Process

Consider first the case of an AR(1) process:

Xt D �Xt�1 C Zt with j�j < 1 and Zt � WN.0;�2/:

The equation system (3.7) becomes:

3See Brockwell and Davis (1991, p. 167).

3.1 Linear Least-Squares Forecasts 49

1 � �2 : : : �T�1 � 1 � :: : �T�2 �2 � 1 :: : �T�3 :::

::: :::

�T�1 �T�2 �T�3 : : : 1

a1 a2 a3 :::

�h

�hC1 �hC2 :::

�hCT�1

The guess-and-verify method immediately leads to the solution:

˛T D .a1; a2; a3; : : : ; aT/0 D �

�h;0;0; : : : ;0 �0 :

We therefore get the following predictor:

PT XTCh D �hXT:

The forecast therefore just depends on the last observation with the corresponding coefficient a1 D �h being independent of T. All previous observations can be disregarded, they cannot improve the forecast further. To put it otherwise, all the useful information about XTCh in the entire realization previous to XT , i.e. in fXT; XT�1; : : : ; X1g, is contained in XT .

The variance of the prediction error is given by

vT.h/ D 1 � �2h 1 � �2 �

For h D 1, the formula simplifies to �2 and for h ! 1, vT.h/ ! 11��2 �2 the unconditional variance of Xt. Note also that the variance of the forecast error vT.h/ increases with h.

The general case of an AR(p) process, p > 1, can be treated in the same way. The autocovariances follow a p-th order difference equation (see Sect. 2.4):

�.j/ D �1�.j � 1/ C �2�.j � 2/ C : : : C �p�.j � p/:

Applying again the guess-and-verify method for the case h D 1 and assuming that T > p, the solution is given by ˛T D

�

�1;�2; : : : ;�p;0; : : : ;0 �0

. Thus the one-step ahead predictor is

PT XTC1 D �1XT C �2XT�1 C : : : C �pXTC1�p; T > p: (3.9)

The one-step ahead forecast of an AR(p) process therefore depends only on the last p observations.

The above predictor can also be obtained in a different way. View for this purpose PT as an operator with the following meaning: Take the linear least-squares forecast

50 3 Forecasting Stationary Processes

with respect to the information fXT; : : : ; X1g. Apply this operator to the defining stochastic difference equation of the AR(p) process forwarded one period:

PT XTC1 D PT .�1XT/ C PT .�2XT�1/ C : : : C PT �

�pXTC1�p � C PT .ZTC1/ :

In period T observations of XT; XT�1; : : : ; X1 are known so that PT XT�j D XT�j, j D 0;1; : : : ; T � 1. Because fZtg is a white noise process and because fXtg is a causal function with respect to fZtg, ZTC1 is uncorrelated with XT; : : : ; X1. This reasoning leads to the same predictor as in Eq. (3.9).

The forecasting functions for h > 1 can be obtained recursively by successively applying the forecast operator. Take, for example, the case h D 2:

PT XTC2 D PT .�1XTC1/ C PT .�2XT/ C : : : C PT �

�pXTC2�p � C PT .ZTC2/

D �1 �

�1XT C �2XT�1 C : : : C �pXTC1�p �

C �2XT C : : : C �pXTC2�p D ��21 C �2

�

XT C .�1�2 C �3/ XT�1 C : : : C �

�1�p�1 C �p �

XTC2�p

C �1�pXTC1�p:

In this way forecasting functions for h > 2 can be obtained recursively. Note that in the case of AR(p) processes the coefficient of the forecast function

remain constant as long as T > p. Thus with each new observation it is not necessary to recompute the equation system and solve it again. This will be different in the case of MA processes. In practice, the parameters of the AR model are usually unknown and have therefore be replaced by some estimate. Section 14.2 investigates in a more general context how this substitution affects the results.

3.1.2 Forecasting with MA(q) Processes

The forecasting problem becomes more complicated in the case of MA(q) processes. In order to get a better understanding we analyze the case of a MA(1) process:

Xt D Zt C �Zt�1 with j�j < 1 and Zt � WN.0;�2/: Taking a forecast horizon of one period, i.e. h D 1, the equation system (3.7) in the case of a MA(1) process becomes:

1 � 1C�2 0 : : : 0

� 1C�2 1

� 1C�2 : : : 0

0 � 1C�2 1 : : : 0

::: :::

:::

0 0 0 : : : 1

a1 a2 a3 :::

� 1C�2 0

0 :::

: (3.10)

3.1 Linear Least-Squares Forecasts 51

Despite the fact that the equation system has a simple structure, the forecasting function will depend in general on all past observations of XT�j, 0 � j � T. We illustrate this point by a numerical example which will allow us to get a deeper understanding.

Suppose that we know the parameters of the MA(1) process to be � D �0:9 and �2 D 1. We start the forecasting exercise in period T D 0 and assume that, at this point in time, we have no observation at hand. The best forecast is therefore just the unconditional mean which in this example is zero. Thus, P0X1 D 0. The variance of the forecast error then is V.X1 � P0X1/ D v0.1/ D �2 C �2�2 D 1:81. This result is summarized in the first row of Table 3.1. In period 1, the realization of X1 is observed. This information can be used and the forecasting function becomes P1X2 D a1X1. The coefficient a1 is found by solving the equation system (3.10) for T D 1. This gives a1 D �=.1 C �2/ D �0:4972. The corresponding variance of the forecast error according to Eq. (3.8) is V.X2 � P1X2/ D v1.1/ D �.0/.1 � ˛01�1.1// D 1:81.1 � 0:4972 � 0:4972/ D 1:3625. This value is lower compared to the previous forecast because additional information, the observation of the realization of X1, is taken into account. Row 2 in Table 3.1 summarizes these results.

In period 2, not only X1, but also X2 is observed which allows us to base our forecast on both observations: P2X3 D a1X2 C a2X1. The coefficients can be found by solving the equation system (3.10) for T D 2. This amounts to solving the simultaneous equation system

1 � 1C�2

� 1C�2 1

�

a1 a2

�

� 1C�2 0

Inserting � D �0:9, the solution is ˛2 D .a1; a2/0 D .�0:6606; �0:3285/0. The corresponding variance of the forecast error becomes

V.X3 � P2X3/ D v2.1/ D �.0/.1 � ˛02�2.1//

D �.0/

1 � �a1 a2 �

� 1C�2 0

D 1:81 �

1 � ��0:6606 �0:3285 �

��0:4972 0

��

D 1:2155:

These results are summarized in row 3 of Table 3.1. In period 3, the realizations of X1, X2 and X3 are known so that the forecast

function becomes P3X4 D a1X3 C a2X2 C a3X1. The coefficients can again be found by solving the equation system (3.10) for T D 3:

1 � 1C�2 0

� 1C�2 1

� 1C�2

0 � 1C�2 1

a1 a2 a3

A D 0

� 1C�2 0

A :

52 3 Forecasting Stationary Processes

Table 3.1 Forecast function for a MA(1) process with � D �0:9 and �2 D 1 Time Forecasting function ˛T D .a1; a2; : : : ; aT/0 Forecast error variance T D 0 W v0.1/ D 1:8100 T D 1 W ˛1 D .�0:4972/0 v1.1/ D 1:3625 T D 2 W ˛2 D .�0:6606; �0:3285/0 v2.1/ D 1:2155 T D 3 W ˛3 D .�0:7404; �0:4891; �0:2432/0 v3.1/ D 1:1436 T D 4 W ˛4 D .�0:7870; �0:5827; �0:3849; �0:1914/0 v4.1/ D 1:1017 : : : : : : : : :

T D 1 W ˛ 1

D .�0:9000; �0:8100; �0:7290; : : :/0 v 1

.1/ D 1

For � D �0:9, the coefficients of the linear predictor are ˛2 D .a1; a2; a3/0 D .�0:7404; �0:4891; �0:2432/0. The corresponding variance of the forecast error becomes

V.X4 � P3X4/ D v3.1/ D �.0/.1 � ˛03�3.1//

D �.0/ 0

@1 � �a1 a2 a3 �

� 1C�2 0

D 1:81 0

@1 � ��0:7404 �0:4891 �0:2432 �

�0:4972 0

D 1:1436:

These results are summarized in row 4 of Table 3.1. We can, of course, continue in this way and derive successively the forecast functions for T D 4;5; : : :.

From this exercise we can make several observations.

• In contrast to the AR process, every new information is used. The forecast PT XTC1 depends on all available information, in particular on XT; XT�1; : : : ; X1.

• The coefficients of the forecast function are not constant. They change as more and more information comes in.

• The importance of the new information can be “measured” by the last coefficient of ˛T . These coefficients are termed partial autocorrelations (see Definition 3.2) and are of particular relevance as will be explained in Sect. 3.5. In our example they are �0:4972; �0:3285; �0:2432, and �0:1914.

• As more information becomes available, the variance of the forecast error (mean squared error) declines monotonically. It will converge to �2 D 1. The reason for this result can be explained as follows. Applying the forecasting operator to the defining MA(1) stochastic difference equation forwarded by one period gives: PT XTC1 D PT ZTC1 C �PT ZT D �PT ZT with forecast error XTC1 � PT XTC1 D ZTC1. As more and more observation become available, it

3.1 Linear Least-Squares Forecasts 53

becomes better and better possible to recover the “true” value of the unobserved ZT from the observations XT; XT�1; : : : ; X1. As the process is invertible, in the limit it is possible to recover the value of ZT exactly (almost surely). The only uncertainty remaining is with respect to ZTC1 which has a mean of zero and a variance of �2 D 1.

3.1.3 Forecasting from the Infinite Past

The forecasting function based on the infinitely remote past is of particular theoretical interest. Thereby we look at the problem of finding the optimal linear forecast of XTC1 given XT; XT�1; : : : ; X1; X0; X�1; : : : taking the mean squared error again as the criterion function. The corresponding forecasting function (predictor) will be denoted by ePT XTCh, h > 0.

Noting that the MA(1) process with j�j < 1 is invertible, we have

Zt D Xt � �Xt�1 C �2Xt�2 � : : :

We can therefore write XtC1 as

XtC1 D ZtC1 C � �

Xt � �Xt�1 C �2Xt�2 � : : : �

„ ƒ‚ …

The predictor of XTC1 from the infinite past, ePT , is then given by:

ePT XTC1 D � �

XT � �XT�1 C �2XT�2 � : : : �

where the mean squared forecasting error is

v1.1/ D E �

XTC1 � ePT XTC1 �2 D �2:

Applying this result to our example gives:

ePT XTC1 D �0:9XT � 0:81XT�1 � 0:729XT�2 � : : :

with v1.1/ D 1. See last row in Table 3.1.

Example of an ARMA(1,1) Process

Consider now the case of a causal and invertible ARMA(1,1) process fXtg:

Xt D �Xt�1 C Zt C �Zt�1;

54 3 Forecasting Stationary Processes

where j�j < 1, j�j < 1 and Zt � WN.0;�2/. Because fXtg is causal and invertible with respect to fZtg,

XTC1 D ZTC1 C .� C �/ 1 X

jD0 �jZT�j;

ZTC1 D XTC1 � .� C �/ 1 X

jD0 .��/jXT�j:

Applying the forecast operator ePT to the second equation and noting that ePT ZTC1 D 0, one obtains the following one-step ahead predictor

ePT XTC1 D .� C �/ 1 X

jD0 .��/jXT�j:

Applying the forecast operator to the first equation, we obtain

ePT XTC1 D .� C �/ 1 X

jD0 .�/

jZT�j:

This implies that the one-step ahead prediction error is equal to XTC1 � ePT XTC1 D ZTC1 and that the mean squared forecasting error of the one-step ahead predictor given the infinite past is equal to EZ2TC1 D �2.

3.2 The Wold Decomposition Theorem

The Wold Decomposition theorem is essential for the theoretical understanding of stationary stochastic processes. It shows that any stationary process can essentially be represented as a linear combination of current and past forecast errors. Before we can state the theorem precisely, we have to introduce the following definition.

Definition 3.1 (Deterministic Process). A stationary stochastic process fXtg is called (purely) deterministic or (purely) singular if and only if it can be forecasted exactly from the infinite past. More precisely, if and only if

� 2 D E�XtC1 � ePt XtC1

�2 D 0 for all t 2 Z where ePtXtC1 denotes the best linear forecast of XtC1 given its infinite past, i.e. given fXt; Xt�1; : : :g.

The most important class of deterministic processes are the harmonic processes. These processes are characterized by finite or infinite sums of sine and cosine functions with stochastic amplitude.4 A simple example of a harmonic process is given by

4More about harmonic processes can be found in Sect. 6.2.

3.2 The Wold Decomposition Theorem 55

Xt D A cos.!t/ C B sin.!t/ with ! 2 .0;�/:

Thereby, A and B denote two uncorrelated random variables with mean zero and finite variance. One can check that Xt satisfies the deterministic difference equation

Xt D .2 cos !/Xt�1 � Xt�2:

Thus, Xt can be forecasted exactly from its past. In this example the last two obser- vations are sufficient. We are now in a position to state the Wold Decomposition Theorem.

Theorem 3.1 (Wold Decomposition). Every stationary stochastic process fXtg with mean zero and finite positive variance can be represented as

Xt D 1 X

jD0 jZt�j C Vt D ‰.L/Zt C Vt; (3.11)

where

(i) Zt D Xt � ePt�1Xt D ePtZt; (ii) Zt � WN.0;�2/ with �2 D E

�

XtC1 � ePtXtC1 �2 > 0;

(iii) 0 D 1 and P1

jD0 2 j < 1;

(iv) fVtg is deterministic; (v) E.Zt Vs/ D 0 for all t; s 2 Z.

The sequences f jg, fZtg, and fVtg are uniquely determined by (3.11).

Proof. The proof, although insightful, requires some knowledge about Hilbert spaces which is beyond the scope of this book. A rigorous proof can be found in Brockwell and Davis (1991, Section 5.7).

It is nevertheless instructive to give an intuition of the proof. Following the MA(1) example from the previous section, we start in period 0 and assume that no information is available. Thus, the best forecast P0X1 is zero so that trivially

X1 D X1 � P0X1 D Z1:

Starting with X1 D Z1, X2; X3; : : : can then be constructed recursively:

X2 D X2 � P1X2 C P1X2 D Z2 C a.1/1 X1 D Z2 C a.1/1 Z1 X3 D X3 � P2X3 C P2X3 D Z3 C a.2/1 X2 C a.2/2 X1

D Z3 C a.2/1 Z2 C �

a .2/ 1 a

.1/ 1 C a.2/2

�

X4 D X4 � P3X4 C P3X4 D Z4 C a.3/1 X3 C a.3/2 X2 C a.3/3 X1

56 3 Forecasting Stationary Processes

D Z4 C a.3/1 Z3 C �

a .3/ 1 a

.2/ 1 C a.3/2

�

C �

a .3/ 1 a

.2/ 1 a

.1/ 1 C a.3/1 a.2/2 C a.3/2 a.1/1 C a.3/3

�

: : :

where a .t�1/ j , j D 1; : : : ; t � 1, denote the coefficients of the forecast function for Xt

based on Xt�1; : : : ; X1. This shows how Xt unfolds into the sum of forecast errors. The stationarity of fXtg ensures that the coefficients of Zj converge, as t goes to infinity, to j which are independent of t. ut

Every stationary stochastic process is thus representable as the sum of a moving- average of infinite order and a (purely) deterministic process.5 The weights of the infinite moving average are thereby normalized such that 0 D 1. In addition, the coefficients j are square summable. This property is less strong than absolute summability which is required for a causal representation (see Definition 2.2).6 The process fZtg is a white noise process with positive variance �2 > 0. The Zt’s are called innovations as they represent the one-period ahead forecast errors based on the infinite past, i.e Zt D Xt � ePt�1Xt . Zt is the additional information revealed from the t-th observation. Thus, the Wold Decomposition Theorem serves as a justification for the use of causal ARMA models. In this instance, the deterministic component fVtg vanishes.

The second part of Property (i) further means that the innovation process fZtg is fundamental with respect to fXtg, i.e. that Zt lies in the linear space spanned by fXt; Xt�1; Xt�2; : : :g or that Zt D ePtZt . This implies that ‰.L/ must be invertible and that Zt can be perfectly (almost surely) recovered from the observations of Xt; Xt�1; : : : Finally, property (v) says that the two components fZtg and fVtg are uncorrelated with each other at all leads and lags. Thus, in essence, the Wold Decomposition Theorem states that every stationary stochastic process can be uniquely decomposed into a weighted sum of current and past forecast errors plus a deterministic process.

Although the Wold Decomposition is very appealing from a theoretical perspec- tive, it is not directly implementable in practice because it requires the estimation of infinitely many parameters . 1; 2; : : :/. This is impossible with only a finite amount of observations. It is therefore necessary to place some assumptions on . 1; 2; : : :/. One possibility is to assume that fXtg is a causal ARMA process and

5The Wold Decomposition corresponds to the decomposition of the spectral distribution function of F into the sum of FZ and FV (see Sect. 6.2). Thereby the spectral distribution function FZ has spectral density fZ. / D �22� j‰.e�{ /j2. 6The series j D 1=j, for example, is square summable, but not absolutely summable.

3.2 The Wold Decomposition Theorem 57

to recover the j’s from the causal representation. This amounts to say that ‰.L/ is a rational polynomial which means that

‰.L/ D ‚.L/ ˆ.L/

D 1 C �1L C �2L 2 C : : : C �qLq

1 � �1L � �2L2 � : : : � �pLp :

Thus, the process is characterized by only a finite number, p C q, of parameters. Another possibility is to place restrictions on the smoothness of the spectrum (see Chap. 6).

The Wold Decomposition Theorem has several implications which are presented in the following remarks.

Remark 3.1. In the case of ARMA processes, the purely deterministic part fVtg can be disregarded so that the process is represented only by a weighted sum of current and past innovations. Processes with this property are called purely non- deterministic, linearly regular, or regular for short. Moreover, it can be shown that every regular process fXtg can be approximated arbitrarily well by an ARMA process fX.ARMA/t g meaning that

sup t2Z

�

Xt � X.ARMA/t �2

can be made arbitrarily small. The proof of these results can be found in Hannan and Deistler (1988, Chapter 1).

Remark 3.2. The process fZtg is white noise, but not necessarily Gaussian. In particular, fZtg need not be independently and identically distributed (IID). Thus, E.ZtC1jXt; Xt�1; : : :/ need not be equal to zero although ePtZtC1 D 0. The reason is that ePt ZtC1 is only the best linear forecast function, whereas E.ZtC1jXt; Xt�1; : : :/ is the best forecast function among all linear and non-linear functions. Examples of processes which are white noise, but not IID, are GARCH processes discussed in Chap. 8.

Remark 3.3. The innovations fZtg may not correspond to the “true” shocks of the underlying economic system. In this case, the shocks to the economic system cannot be recovered from the Wold Decomposition. Thus, they are not fundamental with respect to fXtg. Suppose, as a simple example, that fXtg is generated by a noninvertible MA(1) process:

Xt D Ut C �Ut�1; Ut � WN.0;�2/ and j�j > 1:

This generates an impulse response function with respect to the true shocks of the system equal to .1;�;0; : : :/. The above mechanism can, however, not be the Wold Decomposition because the noninvertibility implies that Ut cannot be recovered from the observation of fXtg. As shown in the introduction, there is an

58 3 Forecasting Stationary Processes

observationally equivalent MA(1) process, i.e. a process which generates the same ACF. Based on the computation in Sect. 1.5, this MA(1) process is

Xt D Zt C Q�Zt�1; Zt � WN.0; Q�2/;

with Q� D ��1 and Q�2 D 1C�2 1C��2 �

2. This is already the Wold Decomposition. The impulse response function for this process is .1;��1;0; : : :/ which is different from the original system. As j Q�j D j��1j < 1, the innovations fZtg can be recovered from the observations as Zt D

P1 jD0.� Q�/jXt�j, but they do not correspond to the

shocks of the system fUtg. Hansen and Sargent (1991), Quah (1990), and Lippi and Reichlin (1993) among others provide a deeper discussion and present additional more interesting economic examples.

3.3 Exponential Smoothing

Besides the method of least-squares forecasting exponential smoothing can often be seen as a valid alternative. This method views Xt as a function of time:

Xt D f .tI ˇ/ C "t;

whereby f .tI ˇ/ typically represents a polynomial in t with coefficients ˇ. The above equation is similar to a regression model with error term "t. This error term is usually specified as a white noise process f"tg � WN.0;�2/.

Consider first the simplest case where Xt just moves randomly around a fixed mean ˇ. This corresponds to the case where f .tI ˇ/ is a polynomial of degree zero:

Xt D ˇ C "t:

If ˇ is known then PT XTCh, the forecast of XTCh given the observations XT; : : : ; X1, clearly is ˇ. If, however, ˇ is unknown, we can substitute ˇ by XT , the average of the observations:

bPT XTCh D Ǒ D XT D 1

T X

tD1 Xt;

where “ O” means that the model parameter ˇ has been replaced by its estimate. The one-period ahead forecast function can then be rewritten as follows:

bPT XTC1 D T � 1

T bPT�1XT C

T XT

D bPT�1XT C 1

�

XT � bPT�1XT �

3.3 Exponential Smoothing 59

The first equation represents the forecast for T C 1 as a linear combination of the forecast for T and of the last additional information, i.e. the last observation. The weight given to the last observation is equal to 1=T because we assumed that the mean remains constant and because the contribution of one observation to the mean is 1=T. The second equation represents the forecast for T C 1 as the forecast for T plus a correction term which is proportional to the last forecast error. One advantage of this second representation is that the computation of the new forecast, i.e. the forecast for T C1, only depends on the forecast for T and the additional observation. In this way the storage requirements are minimized.

In many applications, the mean does not remain constant, but is a slowly moving function of time. In this case it is no longer meaningful to give each observation the same weight. Instead, it seems plausible to weigh the more recent observation higher than the older ones. A simple idea is to let the weights decline exponentially which leads to the following forecast function:

PT XTC1 D 1 � ! 1 � !T

T�1 X

tD0 !t XT�t with j!j < 1:

! thereby acts like a discount factor which controls the rate at which agents forget information. 1 � ! is often called the smoothing parameter. The value of ! should depend on the speed at which the mean changes. In case when the mean changes only slowly, ! should be large so that all observations are almost equally weighted; in case when the mean changes rapidly, ! should be small so that only the most recent observations are taken into account. The normalizing constant 1�!

1�!T ensures that the weights sum up to one. For large T the term !T can be disregarded so that one obtains the following forecasting function based on simple exponential smoothing:

PT XTC1 D .1 � !/

XT C !XT�1 C !2XT�2 C : : :

D .1 � !/XT C ! PT�1XT D PT�1XT C .1 � !/.XT � PT�1XT/:

In the economics literature this forecasting method is called adaptive expectation. Similar to the model with constant mean, the new forecast is a weighted average between the old forecast and the last (newest) observation, respectively between the previous forecast and a term proportional to the last forecast error.

One important advantage of adaptive forecasting methods is that they can be computed recursively. Starting with value S0, the following values can be computed as follows:

P0X1 D S0 P1X2 D !P0X1 C .1 � !/X1

60 3 Forecasting Stationary Processes

P2X3 D !P1X2 C .1 � !/X2 : : :

PT XTC1 D !PT�1XT C .1 � !/XT:

Thereby S0 has to be determined. Because

PT XTC1 D .1 � !/

XT C !XT�1 C : : : C !T�1; X1 C !T S0;

the effect of the starting value declines exponentially with time. In practice, we can take S0 D X1 or S0 D XT . The discount factor ! is usually set a priori to be a number between 0.7 and 0.95. It is, however, possible to determine ! optimally by choosing a value which minimizes the mean squared one-period forecast error:

T X

tD1 .Xt � Pt�1Xt/2 �! minj!j<1 :

From a theoretical perspective one can ask the question for which class of models exponential smoothing represents the optimal procedure. Muth (1960) showed that this class of models is given by

�Xt D Xt � Xt�1 D Zt � !Zt�1:

Note that the process generated by the above equation is no longer stationary. This has to be expected as the exponential smoothing assumes a non-constant mean. Despite the fact that this class seems rather restrictive at first, practice has shown that it delivers reasonable forecasts, especially in situations when it becomes costly to specify a particular model.7 Additional results and more general exponential smoothing methods can be found in Abraham and Ledolter (1983) and Mertens and Rässler (2005).

3.4 Exercises

Exercise 3.4.1. Compute the linear least-squares predictor PT XTCh, T > 2, and the mean squared error vT.h/, h D 1;2;3, if fXtg is given by the AR(2) process

Xt D 1:3Xt�1 � 0:4Xt�2 C Zt with Zt � WN.0;2/:

To which values do PT XTCh and vT.h/ converge for h going to infinity?

7This happens, for example, when many, perhaps thousands of time series have to be forecasted in a real time situation.

3.5 Partial Autocorrelation 61

Exercise 3.4.2. Compute the linear least-squares predictor PT.XTC1/ and the mean squared error vT.1/, T D 0;1;2;3, if fXtg is given by the MA(1) process

Xt D Zt C 0:8Zt�1 with Zt � WN.0;2/:

To which values do PT XTCh and vT.h/ converge for h going to infinity?

Exercise 3.4.3. Suppose that you observe fXtg for the two periods t D 1 and t D 3, but not for t D 2.

(i) Compute the linear least-squares forecast for X2 if

Xt D �Xt�1 C Zt with j�j < 1 and Zt � WN.0;4/

Compute the mean squared error for this forecast. (ii) Assume now that fXtg is the MA(1) process

Xt D Zt C �Zt�1 with Zt � WN.0;4/:

Compute the mean squared error for the forecast of X2.

Exercise 3.4.4. Let

Xt D A cos.!t/ C B sin.!t/

with A and B being two uncorrelated random variables with mean zero and finite variance. Show that fXtg satisfies the deterministic difference equation:

Xt D .2 cos !/Xt�1 � Xt�2:

3.5 The Partial Autocorrelation Function

Consider again the problem of forecasting XTC1 from observations XT; XT�1; : : : ; X2; X1. Denoting, as before, the best linear predictor by PT XTC1 D a1XT C a2XT�1 C aT�1X2 C aT X1, we can express XTC1 as

XTC1 D PT XTC1 C ZTC1 D a1XT C a2XT�1 C aT�1X2 C aT X1 C ZTC1 where ZTC1 denotes the forecast error which is uncorrelated with XT; : : : ; X1. We can now ask the question whether X1 contributes to the forecast of XTC1 controlling for XT; XT�2; : : : ; X2 or, equivalently, whether aT is equal to zero. Thus, aT can be viewed as a measure of the importance of the additional information provided by X1. It is referred to as the partial autocorrelation. In the case of an AR(p) process, the whole information useful for forecasting XTC1, T > p, is incorporated in the

62 3 Forecasting Stationary Processes

last p observations so that aT D 0. In the case of the MA process, the observations on XT; : : : ; X1 can be used to retrieve the unobserved ZT; ZT�1 : : : ; Zt�qC1. As Zt is an infinite weighted sum of past Xt’s, every new observation contributes to the recovering of the Zt ’s. Thus, the partial autocorrelation aT is not zero. Taking T successively equal to 0, 1, 2, etc. we get the partial autocorrelation function (PACF).

We can, however, interpret the above equation as a regression equation. From the Frisch-Lovell-Waugh Theorem (See Davidson and MacKinnon 1993), we can obtain aT by a two-stage procedure. Project (regress) in a first stage XTC1 on XT; : : : ; X2 and take the residual. Similarly, project (regress) X1 on XT; : : : ; X2 and take the residual. The coefficient aT is then obtained by projecting (regressing) the first residual on the second. Stationarity implies that this is nothing but the correlation coefficient between the two residuals.

3.5.1 Definition

The above intuition suggests two equivalent definitions of the partial autocorrelation function (PACF).

Definition 3.2 (Partial Autocorrelation Function I). The partial autocorrelation function (PACF) ˛.h/; h D 0;1;2; : : :, of a stationary process is defined as follows:

˛.0/ D 1 ˛.h/ D ah; h D 1;2; : : : ;

where ah denotes the last element of the vector ˛h D ��1h �h.1/ D R�1h �h.1/ (see Sect. 3.1 and Eq. (3.7)).

Definition 3.3 (Partial Autocorrelation Function II). The partial autocorrelation function (PACF) ˛.h/; h D 0;1;2; : : :, of a stationary process is defined as follows:

˛.0/ D 1 ˛.1/ D corr .X2; X1/ D �.1/ ˛.h/ D corr ŒXhC1 � P.XhC1j1; X2; : : : ; Xh/ ; X1 � P.X1j1; X2; : : : ; Xh/ ;

where P.XhC1j1; X2; : : : ; Xh/ and P.X1j1; X2; : : : ; Xh/ denote the best, in the sense mean squared forecast errors, linear forecasts of XhC1, respectively X1 given f1; X2; : : : ; Xhg.

Remark 3.4. If fXtg has a mean of zero, then the constant in the projection operator can be omitted.

The first definition implies that the partial autocorrelations are determined from the coefficients of the forecasting function which are themselves functions

3.5 Partial Autocorrelation 63

of the autocorrelation coefficients. It is therefore possible to express the partial autocorrelations as a function of the autocorrelations. More specifically, the partial autocorrelation functions can be computed recursively from the autocorrelation function according to the Durbin-Levinson algorithm (Durbin 1960):

˛.0/ D 1 ˛.1/ D a11 D �.1/

˛.2/ D a22 D �.2/ � �.1/2 1 � �.1/2

: : :

˛.h/ D ahh D �.h/ � Ph�1jD1 ah�1;j�h�j 1 � Ph�1jD1 ah�1;j�j

;

where ah;j D ah�1;j � ahhah�1;h�j for j D 1;2; : : : ; h � 1.

Autoregressive Processes The idea of the PACF can be well illustrated in the case of an AR(1) process

Xt D �Xt�1 C Zt with 0 < j�j < 1 and Zt � WN.0;�2/:

As shown in Chap. 2, Xt and Xt�2 are correlated with each other despite the fact that there is no direct relationship between the two. The correlation is obtained “indirectly” because Xt is correlated with Xt�1 which is itself correlated with Xt�2. Because both correlation are equal to �, the correlation between Xt and Xt�2 is equal to �.2/ D �2. The ACF therefore accounts for all correlation, including the indirect ones. The partial autocorrelation on the other hand only accounts for the direct relationships. In the case of the AR(1) process, there is only an indirect relation between Xt and Xt�h for h � 2, thus the PACF is zero.

Based on the results in Sect. 3.1 for the AR(1) process, the definition 3.2 of the PACF implies:

˛1 D � ) ˛.1/ D �.1/ D �; ˛2 D .�;0/0 ) ˛.2/ D 0; ˛3 D .�;0;0/0 ) ˛.3/ D 0:

The partial autocorrelation function of an AR(1) process is therefore equal to zero for h � 2.

This logic can be easily generalized. The PACF of a causal AR(p) process is equal to zero for h > p, i.e. ˛.h/ D 0 for h > p. This property characterizes an AR(p) process as shown in the next section.

64 3 Forecasting Stationary Processes

Moving-Average Processes Consider now the case of an invertible MA process. For this process we have:

Zt D 1 X

jD0 �jXt�j ) Xt D �

1 X

jD1 �jXt�j C Zt:

Xt is therefore “directly” correlated with each Xt�h, h D 1;2; : : :. Consequently, the PACF is never exactly equal to zero, but converges exponentially to zero. This convergence can be monotonic or oscillating.

Take the MA(1) process as an illustration:

Xt D Zt C �Zt�1 with j�j < 1 and Zt � WN.0;�2/:

The computations in Sect. 3.1.2 showed that

˛1 D �

1 C �2 ) ˛.1/ D �.1/ D �

1 C �2;

˛2 D �

�.1 C �2/ 1 C �2 C �4;

��2 1 C �2 C �4

�0 ) ˛.2/ D ��

1 C �2 C �4:

Thus we get for the MA(1) process:

˛.h/ D � .��/ h

1 C �2 C : : : C �2h D � .��/h.1 � �2/ 1 � �2.hC1/

3.5.2 Interpretation of ACF and PACF

The ACF and the PACF are two important tools to determining the nature of the underlying mechanism of a stochastic process. In particular, they can be used to determine the orders of the underlying AR, respectively MA processes. The analysis of ACF and PACF to identify appropriate models is know as the Box-Jenkins methodology (Box and Jenkins 1976). Table 3.2 summarizes the properties of both tools for the case of a causal AR and an invertible MA process.

If fXtg is a causal and invertible ARMA(p,q) process, we have the following properties. As shown in Sect. 2.4, the ACF is characterized for h > maxfp; qC1g by the homogeneous difference equation �.h/ D �1�.h�1/C: : :C�p�.h�p/. Causality implies that the roots of the characteristic equation are all inside the unit circle. The autocorrelation coefficients therefore decline exponentially to zero. Whether this convergence is monotonic or oscillating depends on the signs of the roots. The PACF starts to decline to zero for h > p. Thereby the coefficients of the PACF exhibit the same behavior as the autocorrelation coefficients of ��1.L/Xt.

3.6 Exercises 65

Table 3.2 Properties of the ACF and the PACF

Processes ACF PACF

AR(p) Declines exponentially (monotonically or oscillating) to zero

˛.h/ D 0 for h > p

MA(q) �.h/ D 0 for h > q Declines exponentially (monotonically or oscillating) to zero

3.6 Exercises

Exercise 3.6.1. Assign the ACF and the PACF from Fig. 3.1 to the following processes:

Xt D Zt; Xt D 0:9Xt�1 C Zt; Xt D Zt C 0:8Zt�1; Xt D 0:9Xt�1 C Zt C 0:8Zt�1

with Zt � WN.0;�2/.

66 3 Forecasting Stationary Processes

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

1 ACF with 95−percent confidence band

order

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

1 PACF with 95−percent confidence band

order

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

1 ACF with 95−percent confidence band

order

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

1 PACF with 95−percent confidence band

order

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

1 ACF with 95−percent confidence band

order

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

1 PACF with 95−procent confidence band

order

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

1 ACF with 95−percent confidence band

order

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

1 PACF with 95−percent confidence band

order

a b

c d

Fig. 3.1 Autocorrelation and partial autocorrelation functions. (a) Process 1. (b) Process 2. (c) Process 3. (d) Process 4

4Estimation of the Meanand the Autocorrelation Function

In the previous chapters we have seen in which way the mean �, and, more importantly, the autocovariance function, �.h/; h D 0; ˙1; ˙2; : : :, of a stationary stochastic process fXtg characterize its dynamic properties, at least if we restrict ourself to the first two moments. In particular, we have investigated how the autocovariance function is related to the coefficients of the corresponding ARMA process. Thus the estimation of the ACF is not only interesting for its own sake, but also for the specification and identification of appropriate ARMA models. It is therefore of outmost importance to have reliable (consistent) estimators for these entities. Moreover, we want to test specific features for a given time series. This means that we have to develop corresponding testing theory. As the small sample distributions are hard to get, we rely for this purpose on asymptotic theory.1

In this section we will assume that the process is stationary and observed for the time periods t D 1;2; : : : ; T. We will refer to T as the sample size. As mentioned previously, the standard sampling theory is not appropriate in the times series context because the Xt’s are not independent draws from some underlying distribution, but are systematically related to each other.

4.1 Estimation of the Mean

The arithmetic average constitutes a “natural” estimator of the mean � of the stochastic process. The arithmetic mean XT is defined as usual by

XT D 1

T .X1 C X2 C : : : C XT/ :

1Recently, bootstrap methods have also been introduced in the time series context.

68 4 Estimation of Mean and ACF

It is immediately clear that the arithmetic average is an unbiased estimator of the mean:

EXT D 1

T .EX1 C EX2 C : : : C EXT/ D �:

Of greater interest are the asymptotic properties of the variance of the arithmetic mean VXT which are summarized in the following theorem:

Theorem 4.1 (Convergence of Arithmetic Average). If fXtg is a stationary stochas- tic process with mean � and ACF �.h/ then the variance of the arithmetic mean VXT has the following asymptotic properties:

VXT D E �

XT � � �2 ! 0; if �.T/ ! 0I

TVXT D TE �

XT � � �2 !

1 X

hD�1 �.h/; if

1 X

hD�1 j�.h/j < 1;

for T going to infinity.

Proof. Immediate algebra establishes:

0 � TVXT D 1

T X

i;jD1 cov.Xi; Xj/ D

jhj<T

�

1 � jhj T

�

�.h/

� X

jhj<T j�.h/j D 2

T X

hD1 j�.h/j C �.0/:

The assumption �.h/ ! 0 for h ! 1 implies that for any given " > 0, we can find T0 such that j�.h/j < "=2 for h � T0. If T > T0 and T > 2T0 �.0/=" then

0 � 1 T

T X

hD1 j�.h/j D 1

T0�1 X

hD1 j�.h/j C 1

T X

hDT0 j�.h/j

� T0 �.0/ T

C 1 T .T � T0/"=2 �

T0 �.0/

T C "=2 � T0 �.0/"

2T0 �.0/ C " 2

D ":

Therefore VXT converges to zero for T ! 1 which establishes the first property. Moreover, we have

lim T!1

TVXT D lim T!1

jhj<T

�

1 � jhj T

�

�.h/ D 1 X

hD�1 �.h/ < 1:

The infinite sum P1

hD�1 �.h/ converges because it converges absolutely by assumption. ut

4.1 Estimation of the Mean 69

This Theorem establishes that the arithmetic average is not only an unbiased estimator of the mean, but also a consistent one. In particular, the arithmetic average converges in the mean-square sense, and therefore also in probability, to the true mean (see appendix C). This result can be interpreted as a reflection of the concept of ergodicity (see Sect. 1.2). The assumptions are relatively mild and are fulfilled for the ARMA processes because for these processes �.h/ converges exponentially fast to zero (see Sect. 2.4.2, in particular Eq. (2.6)). Under little more restrictive assumptions it is even possible to show that the arithmetic mean is asymptotically normally distributed.

Theorem 4.2 (Asymptotic Distribution of Sample Mean). For any stationary process fXtg given by

Xt D � C 1 X

jD�1 jZt�j; Zt � IID.0;�2/;

such that P1

jD�1 j jj < 1 and P1

jD�1 j ¤ 0, the arithmetic average XT is asymptotically normal:

p T.XT � �/

d��! N

1 X

hD�1 �.h/

D N

@ 0;�

1 X

jD�1 j

2 1

A D N.0;�2‰.1/2/

where � is the autocovariance function of fXtg.

Proof. The standard proof invokes the Basic Approximation Theorem C.14 and the Central Limit Theorem for m-dependent processes C.13. To this end we define the 2m-dependent approximate process

X .m/ t D � C

m X

jD�m jZt�j:

For fX.m/t g, we have Vm D Pm

hD�m �.h/ D �2. Pm

jD�m j/ 2. This last assertion can

be verified by noting that

V D 1 X

hD�1 �.h/ D �2

1 X

hD�1

1 X

jD�1 j jCh

D �2 1 X

jD�1 j

1 X

hD�1 jCh D �2

1 X

jD�1 j

70 4 Estimation of Mean and ACF

Note that the assumption P1

jD�1 j jj < 1 guarantees the convergence of the infinite sums. Applying this result to the special case j D 0 for jjj > m, we obtain Vm.

The arithmetic average of the approximating process is

X .m/ T D

T X

tD1 X .m/ t :

The CLT for m-dependent processes C.13 then implies that for T ! 1 p

T �

X .m/ T � �

�

d��! X.m/ D N.0; Vm/:

As m ! 1, �2.PmjD�m j/2 converges to �2. P1

jD�1 j/ 2 and thus

X.m/ d��! X D N.0; V/ D N

@ 0;�2

1 X

jD�1 j

2 1

A :

This assertion can be established by noting that the characteristic functions of X.m/

approaches the characteristic function of X so that by Theorem C.11 X.m/ d��! X.

Finally, we show that the approximation error becomes negligible as T goes to infinity:

p T �

XT � � � �

p T �

X .m/ T � �

�

D T �1=2 T X

tD1 .Xt � X.m/t / D T �1=2

T X

tD1 e .m/ t

where the error e .m/ t is

e .m/ t D

jjj>m jZt�j:

Clearly, fe.m/t g is a stationary process with autocovariance function �e such that P1

hD�1 �e.h/ D �2 �

jjj>m j �2

< 1. We can therefore invoke Theorem 4.1 to show that

�p T �

XT � � � �

p T �

X .m/ T � �

��

D TV

T X

tD1 e .m/ t

4.1 Estimation of the Mean 71

converges to �2 �

jjj>m j �2

as T ! 1. This term converges to zero as m ! 1. The approximation error

p T �

XT � � � �

p T �

X .m/ T � �

�

therefore converges in

mean square to zero and thus, using Chebyschev’s inequality (see Theorem C.3 or C.7), also in probability. We have therefore established the third condition of

Theorem C.14 as well. Thus, we can conclude that p

T �

XT � � � d��! X. ut

Under a more restrictive summability condition which holds, however, within the context of causal ARMA processes, we can provide a less technical proof. This proof follows an idea of Phillips and Solo (1992) and is based on the Beveridge- Nelson decomposition (see Appendix D).2

Theorem 4.3. For any stationary process

Xt D � C 1 X

jD0 jZt�j

with the properties Zt � IID.0;�2/ and P1

jD0 j 2j jj2 < 1, the arithmetic average

XT is asymptotically normal:

p T.XT � �/

d��! N

1 X

hD�1 �.h/

D N

@ 0;�2

1 X

jD0 j

2 1

A D N.0;�2‰.1/2/:

Proof. The application of the Beveridge-Nelson decomposition (see Theorem D.1 in Appendix D) leads to

XT � � D 1

T X

tD1 ‰.L/Zt D

T X

tD1 .‰.1/ � .L � 1//e‰.L//Zt

D ‰.1/

T X

tD1 Zt

C 1 T e‰.L/.Z0 � ZT/

p T.XT � �/ D ‰.1/

p T

PT tD1 Zt T

C 1p T e‰.L/Z0 �

1p T e‰.L/ZT:

2The Beveridge-Nelson decomposition is an indispensable tool for the understanding of integrated and cointegrated processes analyzed in Chaps. 7 and 16.

72 4 Estimation of Mean and ACF

The assumption Zt � IID.0;�2/ allows to invoke the Central Limit Theorem C.12 of Appendix C to the first term. Thus,

p T

PT tD1 Zt T

is asymptotical normal with mean zero and variance �2. Theorem D.1 also implies j‰.1/j < 1. Therefore, the term ‰.1/

p T

PT tD1 Zt T

is asymptotically normal with mean zero and variance �2‰.1/2.

The variances of the second and third term are equal to � 2

PT jD0 Q 2j . The

summability condition then implies according to Theorem D.1 that PT

jD0 Q 2j converges for T ! 1. Thus, the variances of the last two terms converge to zero implying that these terms converge also to zero in probability (see Theorem C.7) and thus also in distribution. We can then invoke Theorem C.10 to establish the Theorem. Finally, the equality of

P1 hD�1 �.h/ and �

2‰.1/2 can be obtained from direct computations or by the application of Theorem 6.4. ut

Remark 4.1. Theorem 4.2 holds with respect to any causal ARMA process because the j’s converge exponentially fast to zero (see the discussion following Eq. (2.5)).

Remark 4.2. If fXtg is a Gaussian process, then for any given fixed T, XT is distributed as

p T �

XT � � � � N

@0; X

jhj<T

�

1 � jhj T

�

�.h/

A :

According to Theorem 4.2, the asymptotic variance of the average depends on the sum of all covariances �.h/. This entity, denoted by J, is called the long-run variance of fXtg:

J D 1 X

hD�1 �.h/ D �.0/

1 C 2 1 X

hD1 �.h/

: (4.1)

Note that the long-run variance equals 2� times the spectral density f . / evaluated at D 0 (see the Definition 6.1 of the spectral density in Sect. 6.1).

As the long-run variance takes into account the serial properties of the time series, it is also called heteroskedastic and autocorrelation consistent variance (HAC variance). If fXtg has some nontrivial autocorrelation (i.e. �.h/ ¤ 0 for h ¤ 0), the long-run variance J is different from �.0/. This implies among other things that the construction of the t-statistic for testing the simple hypothesis H0: � D �0 should be based on J rather than on �.0/.

In case that fXtg is a causal ARMA process with ˆ.L/Xt D ‚.L/Zt, Zt � WN.0;�2/, the long-run variance is given by

J D �

‚.1/

ˆ.1/

�2

�2 D ‰.1/2�2:

4.2 Estimation of ACF 73

If fXtg is a AR(1) process with Xt D �Xt�1 C Zt , Zt � WN.0;�2/ and j�j < 1, �.0/ D �2

1��2 and �.h/ D �jhj. Thus the long-run variance is given by J D � 2

.1��/2 D �.0/ � 1C�

1�� . From this example it is clear that the long-run variance can be smaller or larger than �.0/, depending on the sign of �: for negative values of �, �.0/ overestimates the long-run variance; for positive values, it underestimates J. The estimation of the long-run variance is dealt with in Sect. 4.4.

4.2 Estimation of the Autocovariance and the Autocorrelation Function

With some slight, asymptotically unimportant modifications, we can use the stan- dard estimators for the autocovariances, �.h/, and the autocorrelations, �.h/, of a stationary stochastic process:

O�.h/ D 1 T

T�h X

tD1

�

Xt � XT ��

XtCh � XT �

; (4.2)

O�.h/ D O�.h/O�.0/: (4.3)

These estimators are biased because the sums are normalized (divided) by T rather than T � h. The normalization with T � h delivers an unbiased estimate only if XT is replaced by � which, however is typically unknown in practice. The second modification concerns the use of the complete sample for the estimation of �.3 The main advantage of using the above estimators is that the implied estimator for the covariance matrix, b�T , respectively the autocorrelation matrix, bRT , of .X1; : : : ; XT/

b�T D

O�.0/ O�.1/ : : : O�.T � 1/ O�.1/ O�.0/ : : : O�.T � 2/ :::

::: :::

:::

O�.T � 1/ O�.T � 2/ : : : O�.0/

bRT D b�T

O�.0/ always delivers, independently of the realized observations, non-negative definite and for O�.0/ > 0 non-singular matrices. The resulting estimated autocovariance function will then satisfy the characterization given in Theorem 1.1, in particular property (iv).

3The standard statistical formulas would suggest to estimate the mean appearing in first multipli- cand from X1; : : : ; Xt�h, and the mean appearing in the second multiplicand from XhC1; : : : ; XT .

74 4 Estimation of Mean and ACF

According to Box and Jenkins (1976, p. 33), one can expect reasonable estimates for �.h/ and �.h/ if the sample size is larger than 50 and if the order of the autocorrelation coefficient is smaller than T=4.

The theorem below establishes that these estimators lead under rather mild conditions to consistent and asymptotically normally distributed estimators.

Theorem 4.4 (Asymptotic Distribution of Autocorrelations). Let fXtg be the stationary process

Xt D � C 1 X

jD�1 jZt�j

with Zt � IID.0;�2/, P1

jD�1 j jj < 1 and P1

jD�1 jj jj2 < 1. Then we have for h D 1;2; : : :

O�.1/ :::

O�.h/

d��! N

�.1/ :::

�.h/

A ;

where the elements of W D �wij �

i;j2f1;:::;hg are given by Bartlett’s formula

wij D 1 X

kD1 Œ�.k C i/ C �.k � i/ � 2�.i/�.k/ Œ�.k C j/ C �.k � j/ � 2�.j/�.k/ :

Proof. Brockwell and Davis (1991, section 7.3) ut

Brockwell and Davis (1991) offer a second version of the above theorem where P1

jD�1 jj jj2 < 1 is replaced by the assumption of finite fourth moments, i.e. by assuming EZ4t < 1. As we rely mainly on ARMA processes, we do not pursue this distinction further because this class of process automatically fulfills the above assumptions as soon as fZtg is identically and independently distributed (IID). A proof which relies on the Beveridge-Nelson polynomial decomposition (see Theorem D.1 in Appendix D) can be gathered from Phillips and Solo (1992).

Example: fXtg � IID.0; � 2/ The most important application of Theorem 4.4 is related to the case of a white noise process. For this process �.h/ is equal to zero for jhj > 0. Theorem 4.4 then implies that

wij D �

1; for i D j; 0; otherwise.

4.2 Estimation of ACF 75

0 5 10 15 20 −1

−0.5

0.5

order

co rr

el at

io n c

o ef

fi ci

en t

estimated ACF lower bound for confidence interval upper bound for confidence interval

Fig. 4.1 Estimated autocorrelation function of a WN(0,1) process with 95 % confidence interval for sample size T D 100

The estimated autocorrelation coefficients converge to the true autocorrelation coefficient, in this case zero. The asymptotic distribution of

p T O�.h/ converges to

the standard normal distribution. This implies that for large T we can approximate the distribution of O�.h/ by a normal distribution with mean zero and variance 1=T. This allows the construction of a 95 % confidence interval assuming that the true process is white noise. This confidence interval is therefore given by ˙1:96T � 12 . It can be used to verify if the observed process is indeed white noise.

Figure 4.1 plots the empirical autocorrelation function of a WN(0,1) process with a sample size of T D 100. The implied 95 % confidence interval is therefore equal to ˙0:196. As each estimated autocorrelation coefficient falls within the confidence interval, we can conclude that the observed times series may indeed represent a white noise process.

Instead of examining each correlation coefficient separately, we can test the joint hypothesis that all correlation coefficients up to order N are simultaneously equal to zero, i.e. �.1/ D �.2/ D : : : D �.N/ D 0, N D 1;2; : : : As each

p T O�.h/

has an asymptotic standard normal distribution and is as<mptotically uncorrelated with

p T O�.k/, h ¤ k, the sum of the squared estimated autocorrelation coefficients

is �2 distributed with N degrees of freedom. This test statistic is called Box-Pierce statistic:

Q D T N X

hD1 O�2.h/ � �2N:

A refinement of this test statistic is given by the Ljung-Box statistic:

Q0 D T.T C 2/ N X

hD1

O�2.h/ T � h � �

2 N: (4.4)

76 4 Estimation of Mean and ACF

This test statistic is also asymptotically �2 distributed with the same degree of freedom N. This statistic accounts for the fact that the estimates for high orders h are based on a smaller number of observations and are thus less precise and more noisy. The two test statistics are used in the usual way. The null hypothesis that all correlation coefficients are jointly equal to zero is rejected if Q, respectively Q0 is larger than the critical value corresponding the �2N distribution. The number of summands N is usually taken to be rather large, for a sample size of 150 in the range between 15 and 20. The two test are also referred to as Portmanteau tests.

Example: MA(q) Process: Xt D Zt C �1Zt�1 C : : : C �qZt�q with Zt � IID.0; � 2/ In this case the covariance matrix is determined as

wii D 1 C 2�.1/2 C : : : C 2�.q/2 for i > q:

For i; j > q, wij is equal to zero. The 95 % confidence interval for the MA(1) process

Xt D Zt �0:8Zt�1 is therefore given for a sample size of T D 200 by ˙1:96T � 12 Œ1C 2�.1/2

1 2 D ˙0:1684.

Figure 4.2 shows the estimated autocorrelation function of the above MA(1) pro- cess together with 95 % confidence interval based on a white noise process and a MA(1) process with � D �0:8. As the first order autocorrelation coefficient is

0 5 10 15 20 −1

−0.5

0.5

order

co rr

el at

io n c

o ef

fi ci

en t

estimated ACF

lower bound for WN−confidence interval

upper bound for WN−confidence interval

bound for MA(1)−confidence interval

Fig. 4.2 Estimated autocorrelation function of a MA(1) process with � D �0:8 with correspond- ing 95 % confidence interval for T D 200

4.2 Estimation of ACF 77

clearly outside the confidence interval whereas all other autocorrelation coefficients are inside it, the figure demonstrate that the observations are evidently the realization of MA(1) process.

Example: AR(1) Process Xt � �Xt�1 D Zt with Zt � IID.0; � 2/ In this case the covariance matrix is determined as

wii D i X

kD1 �2i

�

�k � ��k�2 C 1 X

kDiC1 �2k

�

�i � ��i�2

D �

1 � �2i��1 C �2�

1 � �2 � 2i� 2i

� 1 C � 2

1 � �2 for large i:

The formula for wij with i ¤ j are not shown. In any case, this formula is of relatively little importance because the partial autocorrelations are better suited for the identification of AR processes (see Sect. 3.5 and 4.3).

Figure 4.3 shows an estimated autocorrelation function of an AR(1) process. The autocorrelation coefficients decline exponentially which is a characteristic for an

0 5 10 15 20 25 30 35 40 −1

−0.5

0.5

order

co rr

el at

io n

co ef

fi ci

en t

theoretical ACF estimated ACF upper bound for AR(1)−confidence interval lower bound for AR(1)−confidence interval

Fig. 4.3 Estimated autocorrelation function of an AR(1) process with � D 0:8 and corresponding 95 % confidence interval for T D 100

78 4 Estimation of Mean and ACF

AR(1) process.4 Furthermore the coefficients are outside the confidence interval up to order 8 for white noise processes.

4.3 Estimation of the Partial Autocorrelation Function

According to its definition (see Definition 3.2), the partial autocorrelation of order h, ˛.h/, is equal to ah, the last element of the vector ˛h D ��1h �h.1/ D R�1h �h.1/. Thus, ˛h and consequently ah can be estimated by Ǫ h D b��1h O�h.1/ D OR�1h O�h.1/. As �.h/ can be consistently estimated and is asymptotically normally distributed (see Sect. 4.2), the continuous mapping theorem (see Appendix C) ensures that the above estimator for ˛.h/ is also consistent and asymptotically normal. In particular we have for an AR(p) process (Brockwell and Davis 1991)

p T Ǫ.h/ d��! N.0;1/ for T ! 1 and h > p:

This result allows to construct, as in the case of the autocorrelation coefficients, confidence intervals for the partial autocorrelations coefficients. The 95 % confi- dence interval is given by ˙1:96p

T . The AR(p) process is characterized by the fact that

the partial autocorrelation coefficients are zero for h > p. Ǫ.h/ should therefore be inside the confidence interval for h > p and outside for h � p. Figure 4.4 confirms this for an AR(1) process with � D 0:8.

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

order

p ar

ti al

a u to

co rr

el at

io n c

o ef

fi ci

en t

estimated PACF lower bound for confidence interval upper bound for confidence interval

Fig. 4.4 Estimated PACF for an AR(1) process with � D 0:8 and corresponding 95 % confidence interval for T D 200

4As a reminder: the theoretical autocorrelation coefficients are �.h/ D �jhj.

4.4 Estimation of the Long-Run Variance 79

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

order

p ar

ti al

a u to

co re

ll at

io n c

o ef

fi ci

en t

estimated PACF lower bound for confidence interval upper bound for confidence interval

Fig. 4.5 Estimated PACF for a MA(1) process with � D 0:8 and corresponding 95 % confidence interval for T D 200

Figure 4.5 shows the estimated PACF for an MA(1) process with � D 0:8. In conformity with the theory, the partial autocorrelation coefficients converge to zero. They do so in an oscillating manner because � is positive (see formula in Sect. 3.5).

4.4 Estimation of the Long-Run Variance

For many applications5 it is necessary to estimate the long-run variance J which is defined according to Eq. (4.1) as follows6

J D 1 X

hD�1 �.h/ D �.0/ C 2

1 X

hD1 �.h/ D �.0/

1 C 2 1 X

hD1 �.h/

: (4.5)

This can, in principle, be done in two different ways. The first one consists in the estimation of an ARMA model which is then used to derive the implied covariances as explained in Sect. 2.4. These covariances are then inserted into Eq. (4.5). The second method is a nonparametric one and is the subject for the rest of this Section. It has the advantage that it is not necessary to identify and estimate an appropriate ARMA model, a step which can be cumbersome in practice. Additional and more

5For example, when testing the null hypothesis H0: � D �0 in the case of serially correlated observations (see Sect. 4.1); for the Phillips-Perron unit-root test explained in Sect. 7.3.2. 6See Theorem 4.2 and the comments following it.

80 4 Estimation of Mean and ACF

advanced material on this topic can be found in Andrews (1991), Andrews and Monahan (1992), or among others in Haan and Levin (1997).7

If the sample size is T > 1, only the covariances �.0/; : : : ;�.T � 1/ can, in principle, be estimated. Thus, a first naive estimator of J is given by OJT defined as

OJT D T�1 X

hD�TC1 O�.h/ D O�.0/ C 2

T�1 X

hD1 O�.h/ D O�.0/

1 C 2 T�1 X

hD1 O�.h/

;

where O�.h/ and O�.h/ are the estimators for �.h/ and �.h/, respectively, given in Sect. 4.2. As the estimators of the higher order autocovariances are based on smaller samples, their estimates become more erratic. At the same time, their weight in the above sum is the same as the lower order and more precisely estimated autocovariances. Thus, the higher order autocovariances have a disproportionate hazardous influence on the above estimator.

A remedy for this problem is to use only a certain number `T of autocovariances and/or to use a weighted sum instead of an unweighted one. This idea leads to the following class estimators:

OJT D OJT.`T/ D T

T � r T�1 X

hD�TC1 k

�

O�.h/;

where k is a weighting or kernel function.8 The kernel functions are required to have the following properties:

(i) k W R ! Œ�1;1 is, with the exception of a finite number of points a continuous function. In particular, k is continuous at x D 0.

(ii) k is quadratically integrable, i.e. R

R k.x/2dx < 1;

(iii) k.0/ D 1; (iv) k is symmetric, i.e. k.x/ D k.�x/ for all x 2 R.

The basic idea of the kernel function is to give relatively little weight to the higher order autocovariances and relatively more weight to the smaller order ones. As k.0/ equals one, the variance O�.0/ receives weight one by construction. The continuity assumption implies that also the covariances of smaller order, i.e. for h small, receive a weight close to one. Table 4.1 lists some of the most popular kernel functions used in practice.

Figure 4.6 shows a plot of these functions. The first three functions are nonzero only for jxj < 1. This implies that only the orders h for which jhj � `T are taken into account. `T is called the lag truncation parameter or the bandwidth. The quadratic spectral kernel function is an example of a kernel function which takes all

7Note the connection between the long-run variance and the spectral density at frequency zero: J D 2�f .0/ where f is the spectral density function (see Sect. 6.3). 8Kernel functions are also relevant for spectral estimators. See in particular Sect. 6.3.

4.4 Estimation of the Long-Run Variance 81

Table 4.1 Common kernel functions

Name k.x/ D Boxcar (“truncated”) 1

Bartlett 1 � jxj Daniell sin.� x/

� x

Tukey-Hanning .1 C cos.�x//=2 Quadratic Spectral 25

12�2x2

�

sin.6�x=5/ 6�x=5

� cos.6�x=5/ �

The function are, with the exception of the quadratic spectral function, only defined for jxj � 1. Outside this interval they are set to zero

-1.5 -1 -0.5 0 0.5 1 1.5 -0.2

0.2

0.4

0.6

0.8

1.2

Tukey- Hanning

quadratic spectral

Daniell

Boxcar (truncated)

Bartlett

Fig. 4.6 Common kernel functions

covariances into account. Note that some weights are negative in this case as shown in Fig. 4.6.9

The estimator for the long-run variance is subject to the correction term T T�r .

This factor depends on the number of parameters estimated in a first step and is only relevant when the sample size is relatively small. In the case of the estimation of the mean r would be equal to one and the correction term is negligible. If on the other hand Xt, t D 1; : : : ; T, are the residuals from multivariate regression, r designates the number of regressors. In many applications the correction term is omitted.

The lag truncation parameter or bandwidth, `T , depends on the number of observations. It is intuitive that the number of autocovariances accounted for in

9Phillips (2004) has proposed a nonparametric regression-based method which does not require a kernel function.

82 4 Estimation of Mean and ACF

the computation of the long-run variance should increase with the sample size, i.e. we should have `T ! 1 for T ! 1.10 The relevant issue is, at which rate the lag truncation parameter should go to infinity. The literature made several suggestions.11 In the following we concentrate on the Bartlett and the quadratic spectral kernel because these function always deliver a positive long-run variance in small samples. Andrews (1991) proposes the following formula to determine the optimal bandwidth:

Bartlett W `T D 1:1447Œ˛Bartlett T 1 3

Quadratic Spectral W `T D 1:3221

˛QuadraticSpectral T 1 5

where Œ: rounds to the nearest integer. The two coefficients ˛Bartlett and ˛QuadraticSpectral are data dependent constants which have to be determined in a first step from the data (see (Andrews 1991, 832–839), (Andrews and Monahan 1992, 958) and (Haan and Levin 1997)). If the underlying process is approximated by an AR(1) model, we get:

˛Bartlett D 4 O�2

.1 � O�2/.1 C O�2/

˛QuadraticSpectral D 4 O�2

.1 � O�/4;

where O� is the first order empirical autocorrelation coefficient. In order to avoid the cumbersome determination of the ˛’s Newey and West

(1994) suggest the following rules of thumb:

Bartlett W `T D ˇBartlett �

100

� 2 9

Quadratic Spectral W `T D ˇQuadraticSpectral �

100

� 2 25

It has been shown that values of 4 for ˇBartlett as well as for ˇQuadraticSpectral lead to acceptable results. A comparison of these formulas with the ones provided by Andrews shows that the latter imply larger values for `T when the sample sizes gets

10This is true even when the underlying process is known to be a MA(q) process. Even in this case it is advantageous to include also the autocovariances for h > q. The reason is twofold. First, only when `T ! 1 for T ! 1, do we get a consistent estimator, i.e. OJT ! JT , respectively J. Second, the restriction to O�.h/; jhj � q, does not necessarily lead to positive value for the estimated long-run variance OJT , even when the Bartlett kernel is used. See Ogaki (1992) for details. 11See Haan and Levin (1997) for an overview.

4.4 Estimation of the Long-Run Variance 83

larger. Both approaches lead to consistent estimates, i.e. OJT.`T/ � JT p��! 0 for

T ! 1. In practice, a combination of both parametric and nonparametric methods proved

to deliver the best results. This combined method consists of five steps:

(i) The first step is called prewhitening and consists in the estimation of a simple ARMA model for the process fXtg to remove the most obvious serial correlations. The idea, which goes back to Press and Tukey (1956) (see also Priestley (1981)), is to get a process for the residuals OZt which is close to a white noise process. Usually, an AR(1) model is sufficient.12

(ii) Choose a kernel function and, if the method of Andrews has been chosen, the corresponding data dependent constants, i.e. ˛Bartlett or ˛QuadraticSpectral for the Bartlett, respectively the quadratic spectral kernel function.

(iii) Compute the lag truncation parameter for the residuals using the above formulas.

(iv) Estimate the long-run variance for the residuals OZt . (v) Compute the long-run variance for the original time series fXtg.

If in the first step an AR(1) model, Xt D �Xt�1 C Zt , was used, the last step is given by:

OJXT .`T/ D OJZT.`T/ .1 � O�/2

;

where OJZT.`T/ and OJXT .`T/ denote the estimated long-run variances of fXtg and f OZtg. In the general case, of an arbitrary ARMA model, ˆ.L/Xt D ‚.L/Zt, we get:

OJXT .`T/ D �

‚.1/

ˆ.1/

�2

OJZT.`T/:

4.4.1 An Example

Suppose we want to test whether the yearly growth rate of Switzerland’s real GDP in the last 25 years was higher than 1 %. For this purpose we compute the percentage change against the corresponding quarter of the last year over the period 1982:1 to 2006:1 (97 observations in total), i.e. we compute Xt D .1 � L4/ log.GDPt/. The arithmetic average of these growth rates is 1.4960 with a variance of 3.0608 .

12If in this step an AR(1) model is used and if a first order correlation O� larger in absolute terms than 0:97 is obtained, Andrews and Monahan (1992, 457) suggest to replace O� by �0:97, respectively 0.97. Instead of using a arbitrary fixed value, it turns out that a data driven value is superior. Sul et al. (2005) suggest to replace 0.97 by 1 � 1=pT and �0:97 by �1 C 1=pT.

84 4 Estimation of Mean and ACF

0 2 4 6 8 10 12 14 16 18 20 −1

−0.8

−0.6

−0.4

−0.2

0.2

0.4

0.6

0.8

Ordnung

K o rr

el at

io n sk

o ef

fi zi

en t

Fig. 4.7 Estimated autocorrelation function for Switzerland’s real GDP growth (percentage change against corresponding last year’s quarter)

We test the null hypothesis that the growth rate is smaller than one against the alternative that it is greater than one. The corresponding value of the t-statistic is .1:4960 � 1/=

3:0608=97 D 2:7922. Taking a 5 % significance level, the critical value for this one-sided test is 1.661. Thus the null hypothesis is clearly rejected.

The above computation is, however, not valid because the serial correlation of the time series was not taken into account. Indeed the estimated autocorrelation function shown in Fig. 4.7 clearly shows that the growth rate is subject to high and statistically significant autocorrelations.

Taking the Bartlett function as the kernel function, the rule of thumb formula for the lag truncation parameter suggest `T D 4. The weights in the computation of the long-run variance are therefore

k.h=`T/ D

1; h D 0I 3=4; h D ˙1I 2=4; h D ˙2I 1=4; h D ˙3I 0; jhj � 4:

4.5 Exercises 85

The corresponding estimate for the long-run variance is therefore given by:

OJT D 3:0608 �

1 C 23 4 0:8287 C 22

4 0:6019 C 21

4 0:3727

�

D 9:2783:

Using the long-run variance instead of the simple variance leads to a quite different value of the t-statistic: .1:4960 � 1/=

9:2783=97 D 1:6037. The null hypothesis is thus not rejected at the 5 % significance level when the serial correlation of the process is taken into account.

4.5 Exercises

Exercise 4.5.1. You regress 100 realizations of a stationary stochastic process fXtg against a constant c. The least-squares estimate of c equals Oc D 004 with an estimated standard deviation of O�c D 0:15. In addition, you have estimated the autocorrelation function up to order h D 5 and obtained the following values:

O�.1/ D �0:43; O�.2/ D 0:13; O�.3/ D �0:12; O�.4/ D 0:18; O�.5/ D �0:23:

(i) How do you interpret the estimated parameter value of 0.4? (ii) Examine the autocorrelation function. Do you think that fXtg is white noise?

(iii) Why is the estimated standard deviation O�c D 0:15 incorrect? (iv) Estimate the long-run variance using the Bartlett kernel. (v) Test the null hypothesis that fXtg is a mean-zero process.

5Estimation of ARMA Models

The specification and estimation of an ARMA(p,q) model for a given realization involves several intermingled steps. First one must determine the orders p and q. Given the orders one can then estimate the parameters �j, �j and �

2. Finally, the model has to pass several robustness checks in order to be accepted as a valid model. These checks may involve tests of parameter constancy, forecasting performance or tests for the inclusion of additional exogenous variables. This is usually an iterative process in which several models are examined. It is rarely the case that one model imposes itself. All too often, one is confronted in the modeling process with several trade-offs, like simple versus complex models or data fit versus forecasting performance. Finding the right balance among the different dimensions therefore requires some judgement based on experience.

We start the discussion by assuming that the orders of the ARMA process is known and the problem just consists in the estimation of the corresponding parameters from a realization of length T. For simplicity, we assume that the data are mean adjusted. We will introduce three estimation methods. The first one is a method of moments procedure where the theoretical moments are equated to the empirical ones. This procedure is known under the name of Yule-Walker estimator. The second procedure interprets the stochastic difference as a regression model and estimates the parameters by ordinary least-squares (OLS). These two methods work well if the underlying model is just an AR model and thus involves no MA terms. If the model comprises MA terms, a maximum likelihood (ML) approach must be pursued.

5.1 The Yule-Walker Estimator

We assume that the stochastic process has mean zero and is governed by a causal purely autoregressive model of order p:

ˆ.L/Xt D Zt with Zt � WN.0;�2/

88 5 Estimation of ARMA Models

where ˆ.L/ D 1��1L��2L2�: : :��pLp. Causality with respect to fZt g implies that there exists a sequence f jg with

P1 jD0 j jj < 1 such that Xt D

P1 jD0 jZt�j D

‰.L/Zt. Multiplying the above difference equation by Xt�j; j D 0;1; : : : ; p and taking expectations leads to the following equation system for the parameters ˆ D .�1; : : : ;�p/0 and �2:

�.0/ � �1�.1/ � : : : � �p�.p/ D �2 �.1/ � �1�.0/ � : : : � �p�.p � 1/ D 0

:: :

�.p/ � �1�.p � 1/ � : : : � �p�.0/ D 0

This equation system is known as the Yule-Walker equations. It can be written compactly in matrix algebra as:

�.0/ � ˆ0�p.1/ D �2;

�.0/ �.1/ : : : �.p � 1/ �.1/ �.0/ : : : �.p � 2/ :::

::: :::

:::

�.p � 1/ �.p � 2/ : : : �.0/

�1

�2 :::

�p

�.1/

�.2/ :::

�.p/

;

respectively

�.0/ � ˆ0�p.1/ D �2; �pˆ D �p.1/:

The Yule-Walker estimator is obtained by replacing the theoretical moments by the empirical ones and solving the resulting equation system for the unknown parameters:

bˆ D b��1p O�p.1/ D bR�1p O�p.1/ O�2 D O�.0/ � bˆ0 O�p.1/ D O�.0/

�

1 � O�p.1/ 0 bR�1p O�p.1/ �

Note the recursiveness of the equation system: the estimate bˆ is obtained without knowledge of O�2 as the estimator bR�1p O�p.1/ involves only autocorrelations. The estimates b�p, bRp, O�p.1/, O�p.1/, and O�.0/ are obtained in the usual way as explained in Chap. 4.1

1Note that the application of the estimator introduced in Sect. 4.2 guarantees that b�p is always invertible.

5.1 The Yule-Walker Estimator 89

The construction of the Yule-Walker estimator implies that the first p values of the autocovariance, respectively the autocorrelation function, implied by the estimated model exactly correspond to their estimated counterparts. It can be shown that this moment estimator always delivers coefficients bˆ which imply that fXtg is causal with respect to fZtg. In addition, the following Theorem establishes that the estimated coefficients are asymptotically normal.

Theorem 5.1 (Asymptotic Normality of Yule-Walker Estimator). Let fXtg be an AR(p) process which is causal with respect to fZtg whereby fZtg � IID.0;�2/. Then the Yule-Walker estimator is consistent and bˆ is asymptotically normal with distribution given by:

p T �

bˆ � ˆ �

d��! N �0;�2��1p �

In addition we have that

O�2 p��! �2:

Proof. See Brockwell and Davis (1991, 233–234). ut

Noteworthy, the asymptotic covariance matrix of the Yule-Walker estimate is independent of �2. In practice, the unknown parameters �2��1p are replaced by their empirical counterparts.

Example: AR(1) Process

In the case of an AR(1) process, the Yule-Walker equation is b�1ˆ D O�1.0/ which simplifies to O�.0/� D O�.1/. The Yule-Walker estimator thus becomes:

bˆ D O� D O�.1/O�.0/ D O�.1/:

The asymptotic distribution then is

p T �

O� � � �

d��! N �

0; �2

�.0/

�

D N �0;1 � �2� :

This shows that the assumption of causality, i.e. j�j < 1, is crucial. Otherwise no strictly positive value for the variance would exist. For the case � D 1 which corresponds to the random walk, the asymptotic distribution of

p T. O� � 1/

becomes degenerate as the variance is equal to zero. This case is, however, of prime importance in economics and is treated detail in Chap. 7.

In practice the order of the model is usually unknown. However, one can expect when estimating an AR(m) model whereby the true order p is strictly smaller than m

90 5 Estimation of ARMA Models

that the estimated coefficients O�pC1; : : : ; O�m should be close to zero. This is indeed the case as shown in Brockwell and Davis (1991, 241). In particular, under the assumptions of Theorem 5.1 it holds that

p T O�m

d��! N.0;1/ for m > p: (5.1)

This result justifies the following strategy to identify the order of an AR-model. Estimate in a first step a highly parameterized model (overfitted model), i.e. a model with a large value of m, and test via a t-test whether �m is zero. If the hypothesis cannot be rejected, reduce the order of the model from m to m � 1 and repeat the same procedure now with respect to �m�1. This is done until the hypothesis can no longer be rejected.

If the order of the initial model is too low (underfitted model) so that the true order is higher than m, one incurs an “omitted variable bias”. The corresponding estimates are no longer consistent. In Sect. 5.4, we take closer look at the problem of determining the order of a model.

Example: MA(q) Process

The Yule-Walker estimator can, in principle, also be applied to MA(q) or ARMA(p,q) processes with q > 0. However, the analysis of the simple MA(1) process in Sect. 1.5.1 showed that the relation between the autocorrelations and the model parameters is nonlinear and may have two, one, or no solution. Consider again the MA(1) process as an example. It is given by the stochastic difference equation Xt D Zt C �Zt�1 with Zt � IID.0;�2/. The Yule-Walker equations are then as follows:

O�.0/ D O�2.1 C O�2/ O�.1/ D O�2 O�

As shown in Sect. 1.5.1, this system of equations has for the case j O�.1/j D j O�.1/= O�.0/j < 1=2 two solutions; for j O�.1/j D j O�.1/= O�.0/j D 1=2 one solution; and for j O�.1/j D j O�.1/= O�.0/j > 1=2 no real solution. In the case of several solutions, we usually take the invertible one which leads to j�j < 1. Invertibility is, however, a restriction which is hard to implement in the case of higher order MA processes. Moreover, it can be shown that Yule-Walker estimator is no longer consistent in general (see Brockwell and Davis (1991, 246) for details). For these reasons, it is not advisable to use the Yule-Walker estimator in the case of MA processes, especially when there exist consistent and efficient alternatives.

5.2 OLS Estimation of an AR(p) Model 91

5.2 Ordinary Least-Squares (OLS) Estimation of an AR(p) Model

An alternative approach is to view the AR model as a regression model for Xt with regressors Xt�1; : : : ; Xt�p and error term Zt :

Xt D �1Xt�1 C : : : C �pXt�p C Zt; Zt � WN.0;�2/:

Given observation for X1; : : : ; XT , the regression model can be compactly written in matrix algebra as follows:

XpC1 XpC2 :::

Xp Xp�1 : : : X1 XpC1 Xp : : : X2 :::

::: :::

:::

XT�1 XT�2 : : : XT�p

�1

�2 :::

�p

ZpC1 ZpC2 :::

;

Y D Xˆ C Z: (5.2)

Note that the first p observations are lost and that the effective sample size is thus reduced to T � p. The least-squares estimator (OLS estimator) is obtained as the minimizer of the sum of squares S.ˆ/:

S.ˆ/ D Z0Z D .Y � Xˆ/0.Y � Xˆ/

D T X

tDpC1

�

Xt � �1Xt�1 � : : : � �pXt�p �2

D T X

tDpC1 .Xt � Pt�1Xt/2 �! min

ˆ : (5.3)

Note that the optimization problem involves no constraints, in particular causality is not imposed as a restriction. The solution of this minimization problem is given by usual formula:

bˆ D �X0X��1 �X0Y� :

Though Eq. (5.2) resembles very much an ordinary regression model, there are some important differences. First, the standard orthogonality assumption between regressors and error is violated. The regressors Xt�j, j D 1; : : : ; p, are correlated with the error terms Zt�j; j D 1;2; : : :. Second, there is a dependency on the starting values Xp; : : : ; X1. The assumption of causality, however, insures that these features do not play a role asymptotically. It can be shown that .X0X/=T converges in probability to b�p and .X0Y/=T to O�p. In addition, under quite general conditions,

92 5 Estimation of ARMA Models

T �1=2X0Z is asymptotically normally distributed with mean 0 and variance �2�p.

Then by Slutzky’s Lemma C.10, p

T.bˆ � ˆ/ D �

X0X T

��1 � X0 Zp

�

converges in

distribution to N.0;�2��1p / Thus, the OLS estimator is asymptotically equivalent to the Yule-Walker estimator.

Theorem 5.2 (Asymptotic Normality of the Least-Squares Estimator). Under the same assumptions as in Theorem 5.1, the ordinary least-squares estimator (OLS estimator) bˆ D .X0X/�1 .X0Y/ is asymptotically distributed as

p T �

bˆ � ˆ �

d��! N �0;�2��1p �

;

plim s2T D �2

where s2T D bZ0bZ=T and bZt are the OLS residuals.

Proof. See Chap. 13 and in particular Sect. 13.3 for a proof in the multivariate case. Additional details may be gathered from Brockwell and Davis (1991, chapter 8).

Remark 5.1. In practice �2��1p is approximated by s 2 T.X

0X=T/�1. Thus, for large T, bˆ can be viewed as being normally distributed as N.ˆ; s2T.X

0X/�1/. This result allows the application of the usual t- and F-tests.

Because the regressors Xt�j, j D 1; : : : ; p are correlated with the errors terms Zt�j, j D 1;2; : : :, the Gauss-Markov theorem cannot be applied. This implies that the least-squares estimator is no longer unbiased in finite samples. It can be shown that the estimates of an AR(1) model are downward biased when the true value of � is between zero and one. MacKinnon and Smith (1998, figure 1) plots the bias as a function of the sample size and the true parameter (see also Fig. 7.1). As the bias function is almost linear in the range �0:85 < � < 0:85, an approximately unbiased estimator for the AR(1) model has been proposed by Marriott and Pope (1954), Kendall (1954), and Orcutt and Winokur (1969) (for further details see MacKinnon and Smith 1998):

O�corrected D 1

T � 3.T O�OLS C 1/:

Remark 5.2. The OLS estimator does in general not deliver coefficients Ô for which fXtg is causal with respect fZtg. In particular, in the case of an AR(1) model, it can happen that, in contrast to the Yule-Walker estimator, j O�j is larger than one despite the fact that the true parameter is absolutely smaller than one. Nevertheless, the least-squares estimator is to be preferred in practice because it delivers small- sample biases of the coefficients which are smaller than those of Yule-Walker estimator, especially for roots of ˆ.z/ close to the unit circle (Tjøstheim and Paulsen 1983; Shaman and Stine 1988; Reinsel 1993).

5.2 OLS Estimation of an AR(p) Model 93

Appendix: Proof of the Asymptotic Normality of the OLS Estimator

The proofs of Theorems 5.1 and 5.2 are rather involved and will therefore not be pursued here. A proof for the more general multivariate case will be given in Chap. 13. It is, however, instructive to look at a simple case, namely the AR(1) model with j�j < 1, Zt � IIN.0;�2/ and X0 D 0. Denoting by O�T the OLS estimator of �, we have:

p T �

O�T � � �

D 1p T

PT tD1 Xt�1Zt

1 T

PT tD1 X

2 t�1

: (5.4)

Moreover, Xt can be written as follows:

Xt D Zt C �Zt�1 C : : : C �t�1Z1: By assumption each Zj, j D 1; : : : ; t, is normally distributed so that Xt as a sum normally distributed random variables is also normally distributed. Because the Zj’s

are independent we have: Xt � N �

0;�2 1��2t 1��2

�

The expected value of 1p T

PT tD1 Xt�1Zt is zero because Zt � IIN.0;�2/. The

variance of this expression is given by

1p T

T X

tD1 Xt�1Zt

D 1 T

T X

tD1 EX2t�1Z

2 t C

T X

tD1

t�1 X

jD1 EZtEXt�1XjZj

D � 2

T X

tD1 EX2t�1:

Moreover, PT

tD1 X 2 t D

PT tD1 X

2 t�1 � .X20 � X2T/ D �2

PT tD1 X

2 t�1 C

PT tD1 Z

2 t C

2� PT

tD1 Xt�1Zt so that

T X

tD1 X2t�1 D

1 � �2 .X 2 0 � X2T/ C

1 � �2 T X

tD1 Z2t C

2�

1 � �2 T X

tD1 Xt�1Zt:

The expected value multiplied by �2=T thus is equal to

�2

T X

tD1 EX2t�1 D

�2

1 � �2 EX20 � EX2T

C � 2

1 � �2 PT

tD1 EZ 2 t

T C 2� 1 � �2

PT tD1 Xt�1Zt

D �� 4.1 � �2T/

T.1 � �2/2 C �4

1 � �2:

94 5 Estimation of ARMA Models

For T going to infinity, we finally get:

lim T!1

1p T

T X

tD1 Xt�1Zt

D � 4

1 � �2:

The numerator in Eq. (5.4) therefore converges to a normal random variable with mean zero and variance �

1��2 . The denominator in Eq. (5.4) can be rewritten as

T X

tD1 X2t�1 D

X20 � X2T .1 � �2/T C

.1 � �2/T T X

tD1 Z2t C

2�

.1 � �2/T T X

tD1 Xt�1Zt:

The expected value and the variance of X2T=T converge to zero. Chebyschev’s inequality (see Theorem C.3 in Appendix C) then implies that the first term converges also in probability to zero. X0 is equal to zero by assumption. The second term has a constant mean equal to �2=.1 � �2/ and a variance which converges to zero. Theorem C.8 in Appendix C then implies that the second term converges in probability to �2=.1 � �2/. The third term has a mean of zero and a variance which converges to zero. Thus the third term converges to zero in probability. This implies:

T X

tD1 X2t�1

p��! � 2

1 � �2:

Putting the results for the numerator and the denominator together and applying Theorem C.10 and the continuous mapping theorem for the convergence in distri- bution one finally obtains:

p T �

O�T � � �

d��! N.0;1 � �2/: (5.5) Thereby the value for the variance is derived from

�4

1 � �2 � 1

�

�2

1��2 �2

D 1 � �2:

5.3 Estimation of an ARMA(p,q) Model

While the estimation of AR models by OLS is rather straightforward and leads to consistent and asymptotically efficient estimates, the estimation of ARMA models is more complex. The reason is that, in contrast to past Xt’s Zt; Zt�1; : : : ; Zt�q are not directly observable from the data. They must be inferred from the observations of Xt. The standard method for the estimation of ARMA models is the method of maximum likelihood which will be explained in this section.

5.3 Estimation of an ARMA(p,q) Model 95

We assume that the process fXtg is a causal and invertible ARMA(p,q) process following the difference equation

Xt � �1Xt�1 � : : : � �pXt�p D Zt C �1Zt�1 C : : : C �qZt�q with Zt � IID.0;�2/. We also assume that ˆ.z/ and ‚.z/ have no roots in common. We then stack the parameters of the model into a vector ˇ and a scalar �2:

ˇ D ��1; : : : ;�p;�1; : : : ;�q �0

and �2:

Given the assumption above the admissible parameter space for ˇ, C, is described by the following set:

C D fˇ 2 RpCq W ˆ.z/‚.z/ ¤ 0 for jzj � 1;�p�q ¤ 0; ˆ.z/ and ‚.z/ have no roots in commong

The estimation by the method of maximum likelihood (ML method) is based on some assumption about the joint distribution of XT D .X1; : : : ; XT/0 given the parameters ˇ and �2. This joint distribution function is called the likelihood function. The method of maximum likelihood then determines the parameters such that the probability of observing a given sample xT D .x1; : : : ; xT/ is maximized. This is achieved by maximizing the likelihood function with respect to the parameters.

By far the most important case is given by assuming that fXtg is a Gaussian process with mean zero and autocovariance function �. This implies that XT D .X1; : : : ; XT/

0 is distributed as a multivariate normal with mean zero and variance �T .

2 The Gaussian likelihood function given the observations xT , LT.ˇ;�2jxT/, is then given by

LT.ˇ;� 2jxT/ D .2�/�T=2.det �T/�1=2 exp

�

�1 2

x0T� �1 T xT

�

D .2��2/�T=2.det GT/�1=2 exp �

� 1 2�2

x0T G �1 T xT

�

where GT D ��2�T . Note that, in contrast to �T , GT does only depend on ˇ and not on �2.3 If one wants to point out the dependence of GT from ˇ we write GT.ˇ/. The method of maximum likelihood then consists in the maximization of the likelihood function with respect to ˇ and �2 taking the data xT as given.

2If the process does not have a mean of zero, we can demean the data in a preliminary step. 3In Sect. 2.4 we showed how the autocovariance function � and as a consequence �T , respectively GT can be inferred from a given ARMA model, i.e from a given ˇ.

96 5 Estimation of ARMA Models

The first order condition of this maximization problem with respect to �2

is obtained by taking the logarithm of the likelihood function LT.ˇ;� 2jxT/ and

differentiating with respect �2 and setting the resulting equation equal to zero:

@ ln LT.ˇ;� 2jxT/

@�2 D � T

�2 C X

0 T G

�1 T XT 2�4

D 0:

Solving this equation with respect to �2 we get as the solution: �2 D T �1x0T G�1T xT . Inserting this value into the original likelihood function and taking the logarithm, one gets the concentrated log-likelihood function:

ln LT.ˇjxT/ D � ln.2�/ � T

2 ln �

T �1x0T GT.ˇ/ �1xT

� � 1 2

ln det GT.ˇ/ � T

2 :

This function is then maximized with respect to ˇ 2 C. This is, however, equivalent to minimizing the function

`T.ˇjxT/ D ln �

T �1x0T GT.ˇ/ �1xT

� C T �1 ln det GT.ˇ/ �! min ˇ2C

The value of ˇ which minimizes the above function is called maximum-likelihood estimator of ˇ. It will be denoted by ǑML. The maximum-likelihood estimator for �2, O�2ML, is then given by

O�2ML D T �1x0T GT. ǑML/�1xT: The actual computation of det GT.ˇ/ and GT.ˇ/

�1 is numerically involved, especially when T is large, and should therefore be avoided. It is therefore convenient to rewrite the likelihood function in a different, but equivalent form:

LT.ˇ;� 2jxT/ D.2��2/�T=2.r0r1 : : : rT�1/�1=2

exp

� 1 2�2

T X

tD1

.Xt � Pt�1Xt/2 rt�1

Thereby Pt�1Xt denotes least-squares predictor of Xt given Xt�1; : : : ; X1 and rt D vt=�

2 where vt is the mean squared forecast error as defined in Sect. 3.1. Several numerical algorithms have been developed to compute these forecast in a numeri- cally efficient and stable way.4

Pt�1Xt and rt do not depend on �2 so that the partial differentiation of the log-likelihood function ln L.ˇ;�2jxT/ with respect to the parameters leads to the maximum likelihood estimator. This estimator fulfills the following equations:

4One such algorithm is the innovation algorithm. See Brockwell and Davis (1991, section 5) for details.

5.3 Estimation of an ARMA(p,q) Model 97

O�2ML D 1

T S. ǑML/;

where

S. ǑML/ D T X

tD1

.Xt � Pt�1Xt/2 rt�1

and where ǑML denote the value of ˇ which minimizes the function

`T.ˇjxT/ D ln �

T S.ˇ/

�

C 1 T

T X

tD1 ln rt�1

subject to ˇ 2 C. This optimization problem must be solved numerically. In practice, one chooses as a starting value ˇ0 for the iteration an initial estimate such that ˇ0 2 C. In the following iterations this restriction is no longer imposed to enhance speed and reduce the complexity of the optimization problem. This implies that one must check whether the so obtained final estimates are indeed in C.

If instead of `T.ˇjxT/, the function

S.ˇ/ D T X

tD1

.Xt � Pt�1Xt/2 rt�1

is minimized subject to constraint ˇ 2 C, we obtain the least-squares estimator of ˇ denoted by ǑLS. The least-squares estimator of �2, O�2LS, is then

O�2LS D S. ǑLS/

T � p � q:

The term 1 T

PT tD1 ln rt�1 disappears asymptotically because, given the restriction

ˇ 2 C, the mean-squared forecast error vT converges to �2 and thus rT goes to one as T goes to infinity. This implies that for T going to infinity the maximization of the likelihood function becomes equivalent to the minimization of the least-squares criterion. Thus the maximum-likelihood estimator and the least-squares estimator share the same asymptotic normal distribution.

Note also that in the case of autoregressive models rt is constant and equal to one. In this case, the least-squares criterion S.ˇ/ reduces to the criterion (5.3) discussed in the previous Sect. 5.2.

98 5 Estimation of ARMA Models

Theorem 5.3 (Asymptotic Distribution of ML Estimator). If fXtg is an ARMA process with true parameters ˇ 2 C and Zt � IID.0;�2/ with �2 > 0 then the maximum-likelihood estimator and the least-squares estimator have asymptotically the same normal distribution:

p T �

Ǒ ML � ˇ

�

d��! N .0; V.ˇ//; p

T �

Ǒ LS � ˇ

�

d��! N .0; V.ˇ// :

The asymptotic covariance matrix V.ˇ/ is thereby given by

V.ˇ/ D �

EUt U 0 t EUtV

0 t

EVtU 0 t EVt V

0 t

��1

Ut D �

ut; ut�1; : : : ; ut�pC1 �0

Vt D �

vt;vt�1; : : : ;vt�qC1 �0

where futg and fvtg denote autoregressive processes defined as ˆ.L/ut D wt and ‚.L/vt D wt with wt � WN.0;1/.

Proof. See Brockwell and Davis (1991, Section 8.8). ut

It can be shown that both estimators are asymptotically efficient.5 Note that the asymptotic covariance matrix V.ˇ/ is independent of �2.

The use of the Gaussian likelihood function makes sense even when the process is not Gaussian. First, the Gaussian likelihood can still be interpreted as a measure of fit of the ARMA model to the data. Second, the asymptotic distribution is still Gaussian even when the process is not Gaussian as long as Zt � IID.0;�2/. The Gaussian likelihood is then called the quasi Gaussian likelihood. The use of the Gaussian likelihood under this circumstance is, however, in general no longer efficient.

Example: AR(p) Process

In this case ˇ D .�1; : : : ;�p/ and V.ˇ/ D �

EUtU 0 t

��1 D �2��1p . This is, however, the same asymptotic distribution as the Yule-Walker estimator. The Yule-Walker, the least-squares, and the maximum likelihood estimator are therefore asymptotically equivalent in the case of an AR(p) process. The main difference lies in the treatment of the first p observations.

5See Brockwell and Davis (1991) and Fan and Yao (2003) for details.

5.4 Estimation of the Orders p and q 99

In particular, we have:

AR.1/ W O� � N ��;.1 � �2/=T� ;

AR.2/ W � O�1

O�2 �

� N ��

�1 �2

�

; 1

�

1 � �22 ��1.1 C �2/ ��1.1 C �2/ 1 � �22

��

Example: MA(q) Process

Similarly, one can compute the asymptotic distribution for an MA(q) process. In particular, we have:

MA.1/ W O� � N ��;.1 � �2/=T� ;

MA.2/ W

O�1 O�2

� N ��

�1

�2

�

; 1

�

1 � �22 �1.1 � �2/ �1.1 � �2/ 1 � �22

��

Example: ARMA(1,1) Process

For an ARMA(1,1) process the asymptotic covariance matrix is given by

V.�;�/ D

�

1 � �2��1 .1 C ��/�1 .1 C ��/�1 �1 � �2��1

!�1 :

Therefore we have:

O� O�

� N ��

�

; 1

1 C �� .� C �/2

�

.1 � �2/.1 C ��/ �.1 � �2/.1 � �2/ �.1 � �2/.1 � �2/ .1 � �2/.1 C ��/

��

5.4 Estimation of the Orders p and q

Up to now we have always assumed that the true orders of the ARMA model p and q are known. This is, however, seldom the case in practice. As economic theory does usually not provide an indication, it is all too often the case that the orders of the ARMA model must be identified from the data. In such a situation one can make two type of errors: p and q are too large in which case we speak of overfitting; p and q are too low in which case we speak of underfitting.

In the case of overfitting, the maximum likelihood estimator is no longer consistent for the true parameter, but still consistent for the coefficients of the causal representation j, j D 0;1;2; : : :, where .z/ D �.z/�.z/ . This can be illustrated by the

100 5 Estimation of ARMA Models

-1.5 -1 -0.5 0 0.5 1 1.5 -1.5

-1

-0.5

0.5

1.5

0-1 1

-1

ML-estimator

Fig. 5.1 Parameter space of a causal and invertible ARMA(1,1) process

following example. Suppose that fXtg is a white noise process, i.e. Xt D Zt � WN.0;�2/, but we fit an ARMA(1,1) model given by Xt � �Xt�1 D Zt C �Zt�1. Then, the maximum likelihood estimator does not converge to � D � D 0, but only to the line-segment � D �� with j�j < 1 and j�j < 1. For values of � and � on this line-segment we have .z/ D �.z/=�.z/ D 1. The maximum likelihood estimator converges to the true values of j, i.e. to the values 0 D 1 and j D 0 for j > 0. The situation is depicted in Fig. 5.1. There it is shown that the estimator has a tendency to converge to the points .�1;1/ and .1; �1/, depending on the starting values. This indeterminacy of the estimator manifest itself as a numerical problem in the optimization of the likelihood function. Thus models with similar roots for the AR and MA polynomials which are close in absolute value to the unit circle are probably overparametrized. The problem can be overcome by reducing the orders of the AR and MA polynomial by one.

This problem does not appear in a purely autoregressive models. As explained in section “Example: AR(1) Process”, the estimator for the redundant coefficients converges to zero with asymptotic distribution N.0;1=T/ (see the result in Eq. (5.1)). This is one reason why purely autoregressive models are often preferred. In addition the estimator is easily implemented and every stationary stochastic process can be arbitrarily well approximated by an AR process. This approximation may, however, necessitate high order models when the true process encompasses a MA component.

In the case of underfitting the maximum likelihood estimator converges to those values which are closest to the true parameters given the restricted parameter space. The estimates are, however, inconsistent due to the “omitted variable bias”.

5.4 Estimation of the Orders p and q 101

For these reasons the identification of the orders is an important step. One method which goes back to Box and Jenkins (1976) consists in the analysis of the autocorrelation function (ACF) and the partial autocorrelation function (PACF) (see Sect. 3.5). Although this method requires some experience, especially when the process is not a purely AR or MA process, the analysis of the ACF und PACF remains an important first step in every practical investigation of a time series.

An alternative procedure relies on the automatic order selection. The objective is to minimize a so-called information criterion over different values of p and q. These criteria are based on the following consideration. Given a fixed number of observations, the successive increase of the orders p and q increases the fit of the model so that variance of the residuals O�2p;q steadily decreases. In order to compensate for this tendency to overfitting a penalty is introduced. This penalty term depends on the number of free parameters and on the number of observations at hand.6 The most important information criteria have the following additive form:

ln O�2p;q C (# free parameters) C.T/

T D ln O�2p;q C .p C q/

C.T/

T �! min

p;q ;

where ln O�2p;q measures the goodness of fit of the ARMA(p,q) model and .p C q/C.T/T denotes the penalty term. Thereby C.T/ represents a nondecreasing function of T which governs the trade-off between goodness of fit and complexity (dimension) of the model. Thus, the information criteria chooses higher order models for larger sample sizes T. If the model includes a constant term or other exogenous variables, the criterion must be adjusted accordingly. However, this will introduce, for a given sample size, just a constant term in the objective function and will therefore not influence the choice of p and q.

The most common criteria are the Akaike information criterion (AIC), the Schwarz or Bayesian information criterion (BIC), and the Hannan-Quinn informa- tion criterion (HQ criterion):

AIC.p; q/ D ln O�2p;q C .p C q/ 2

BIC.p; q/ D ln O�2p;q C .p C q/ ln T

HQC.p; q/ D ln O�2p;q C .p C q/ 2 ln.ln T/

Because AIC < HQC < BIC for a given sample size T � 16, Akaike’s criterion delivers the largest models, i.e. the highest order p C q; the Bayesian criterion is more restrictive and delivers therefore the smallest models, i.e. the lowest p C q. Although Akaike’s criterion is not consistent with respect to p and q and has a

6See Brockwell and Davis (1991) for details and a deeper appreciation.

102 5 Estimation of ARMA Models

tendency to deliver overfitted models, it is still widely used in practice. This feature is sometimes desired as overfitting is seen as less damaging than underfitting.7 Only the BIC and HQC lead to consistent estimates of the orders p and q.

5.5 Modeling a Stochastic Process

The identification of a satisfactory ARMA model typically involves in practice several steps.

Step 1: Transformations to Achieve Stationary Time Series

Economic time series are often of a non-stationary nature. It is therefore necessary to transform the time series in a first step to achieve stationarity. Time series which exhibit a pronounced trend (GDP, stock market indices, etc.) should not be modeled in levels, but in differences. If the variable under consideration is already in logs, as is often case, then taking first differences effectively amounts to working with growth rates. Sometimes first differences are not enough and further differences have to be taken. Price indices or monetary aggregates are typical examples where first differences may not be sufficient to achieve stationarity. Thus instead of Xt one works with the series Yt D .1 � L/dXt with d D 1;2; : : :. A non-stationary process fXtg which needs to be differentiated d-times to arrive at a stationary time series is called integrated of order d, Xt � I.d/.8 If Yt D .1 � L/d Xt is generated by an ARMA(p,q) process, fXtg is said to be an ARIMA(p,d,q) process.

An alternative method to eliminate the trend is to regress the time series against a polynomial in t of degree s, i.e. against .1; t; : : : ; ts/, and to proceed with the residuals. These residuals can then be modeled as an ARMA(p,q) process. Chapter 7 discusses in detail which of the two detrending methods is to be preferred under which circumstances.

Often the data are subject to seasonal fluctuations. As with the trend there are several alternative available. The first possibility is to pass the time series through a seasonal filter and work with the seasonally adjusted data. The construction of seasonal filters is discussed in Chap. 6. A second alternative is to include seasonal dummies in the ARMA model. A third alternative is to take seasonal differences. In the case of quarterly observations, this amounts to work with Yt D .1 � L4/Xt. As 1 � L4 D .1 � L/.1 C L C L2 C L3/, this transformation involves a first difference and will therefore also account for the trend.

7See, for example, the section on the unit root tests 7.3. 8An exact definition will be provided in Chap. 7. In this chapter we will analyze the consequences of non-stationarity and discuss tests for specific forms of non-stationarity.

5.6 Modeling Real GDP of Switzerland 103

Step 2: Finding the Orders p and q

Having achieved stationarity, one has to find the appropriate orders p and q of the ARMA model. Thereby one can rely either on the analysis of the ACF and the PACF, or on the information criteria outlined in the previous Sect. 5.4.

Step 3: Checking the Plausibility

After having identified a particular model or a set of models, one has to inspect its adequacy. There are several dimensions along which the model(s) can be checked.

(i) Are the residuals white noise? This can be checked by investigating at the ACF of the residuals and by applying the Ljung-Box test (4.4). If they are not this means that the model failed to capture all the dynamics inherent in the data.

(ii) Are the parameters plausible? (iii) Are the parameters constant over time? Are there structural breaks? This can

be done by looking at the residuals or by comparing parameter estimates across subsamples. More systematic approaches are discussed in Perron (2006). These involve the revolving estimation of parameters by allowing the break point to vary over the sample. Thereby different type of structural breaks can be distinguished. A more in depth analysis of structural breaks is presented in Sect. 18.1.

(iv) Does the model deliver sensible forecasts? It is particularly useful to investi- gate the out-of-sample forecasting performance. If one has several candidate models, one can perform a horse-race among them.

In case the model turns out to be unsatisfactory, one has to go back to steps 1 and 2.

5.6 An example: Modeling Real GDP in the Case of Switzerland

This section illustrates the concepts and ideas just presented by working out a specific example. We take the seasonally unadjusted Swiss real GDP as an example. The data are plotted in Fig. 1.3. To take the seasonality into account we transform the logged time series by taking first seasonal differences, i.e. Xt D .1�L4/ ln GDPt . Thus, the variable corresponds to the growth rate with respect to quarter of the previous year. The data are plotted in Fig. 5.2. A cursory inspection of the plot reveals that this transformation eliminated the trend as well as the seasonality.

First we analyze the ACF and the PACF. They are plotted together with corresponding confidence intervals in Fig. 5.3. The slowly monotonically declining ACF suggests an AR process. As only the first two orders of the PACF are significantly different from zero, it seems that an AR(2) model is appropriate. The least-squares estimate of this model are:

104 5 Estimation of ARMA Models

1985 1990 1995 2000 2005 2010 −4

−2

time

p er

ce n t

Fig. 5.2 Real GDP growth rates of Switzerland

Xt � 1.134 (0,103)

Xt�1 C 0.310 (0.104)

Xt�2 D 0:218 C Zt with O�2 D 0:728

The numbers in parenthesis are the estimated standard errors of the corresponding parameter above. The roots of the AR-polynomial are 1.484 and 2.174. They are clearly outside the unit circle so that there exists a stationary and causal representation.

Next, we investigate the information criteria AIC and BIC to identify the orders of the ARMA(p,q) model. We examine all models with 0 � p; q � 4. The AIC and the BIC values, are reported in Tables 5.1 and 5.2. Both criteria reach a minimum at .p; q/ D .1;3/ (bold numbers) so that both criteria prefer an ARMA(1,3) model. The parameters of this models are as follows:

Xt � 0.527 (0.134)

Xt�1 D 0:6354 C Zt C 0.5106 (0.1395)

Zt�1

C 0.5611 (0.1233)

Zt�2 C 0.4635 (0.1238)

Zt�3 with O�2 D 0:648:

The estimated standard errors of the estimated parameters are again reported in parenthesis below. The AR(2) model is not considerably worse than the ARMA(1,3) model, according to the BIC criterion it is even the second best model.

5.6 Modeling Real GDP of Switzerland 105

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

1 partial autocorrelation function (PACF)

order

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

1 autocorrelation function (ACF)

order

Fig. 5.3 Autocorrelation (ACF) and partial autocorrelation (PACF) function (PACF) of real GDP growth rates of Switzerland with 95 % confidence interval

Table 5.1 Values of Akaike’s information criterium (AIC) for alternative ARMA(p,q) models

p 0 1 2 3 4

0 0.3021 0.0188 �0.2788 �0.3067 1 �0.2174 �0.2425 �0.2433 �0.3446 �0.2991 2 �0.2721 �0.2639 �0.2613 �0.3144 �0.2832 3 �0.2616 �0.2276 �0.2780 �0.2663 �0.2469 4 �0.2186 �0.1990 �0.2291 �0.2574 �0.2099

Minimum in bold

The inverted roots of the AR- and the MA-polynomial are plotted together with their corresponding 95 % confidence regions in Fig. 5.4.9 As the confidence regions are all inside the unit circle, also the ARMA(1,3) has a stationary and causal representation. Moreover, the estimated process is also invertible. In addition, the roots of the AR- and the MA-polynomial are distinct.

9The confidence regions are determined by the delta-method (see Appendix E).

106 5 Estimation of ARMA Models

Table 5.2 Values of Bayes’ information criterium (BIC) for alternative ARMA(p,q) models

p 0 1 2 3 4

0 0.3297 0.0740 �0.1961 �0.1963 1 �0.1896 �0.1869 �0.1600 �0.2335 �0.1603 2 �0.2162 �0.1801 �0.1495 �0.1746 �0.1154 3 �0.1772 �0.1150 �0.1373 �0.0974 �0.0499 4 �0.1052 �0.0573 �0.0591 �0.0590 0.0169 Minimum in bold

Fig. 5.4 Inverted roots of the AR- and the MA-polynomial of the ARMA(1,3) model together with the corresponding 95 % confidence regions

−1 −0.5 0 0.5 1 −1

−0.8

−0.6

−0.4

−0.2

0.2

0.4

0.6

0.8

real part

im ag

in ar

y p ar

t roots of Θ(z)

unit circle

root of Φ(z)

The autocorrelation functions of the AR(2) and the ARMA(1,3) model are plotted in Fig. 5.5. They show no sign of significant autocorrelations so that both residual series are practically white noise. We can examine this hypothesis formally by the Ljung-Box test (see Sect. 4.2 Eq. (4.4)). Taking N D 20 the values of the test statistics are Q0AR.2/ D 33:80 and Q0ARMA.1;3/ D 21:70, respectively. The 5 % critical value according to the �2

20 distribution is 31:41. Thus the null hypothesis

�.1/ D : : : D �.20/ D 0 is rejected for the AR(2) model, but not for the ARMA(1,3) model. This implies that the AR(2) model does not capture the full dynamics of the data.

Although the AR(2) and the ARMA(1,3) model seem to be quite different at first glance, they deliver similar impulse response functions as can be gathered from Fig. 5.6. In both models, the impact of the initial shock is first built up to values higher than 1:1 in quarters one and two, respectively. Then the effect monotonically declines to zero. After 10 to 12 quarters the effect of the shock has practically dissipated.

As a final exercise, we use both models to forecast real GDP growth over the next nine quarters, i.e. for the period fourth quarter 2003 to fourth quarter 2005. As can

5.6 Modeling Real GDP of Switzerland 107

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

1 ACF of the residuals from the ARMA(1,3) model

order

0 2 4 6 8 10 12 14 16 18 20 −1

−0.5

0.5

1 ACF of the residuals from AR(2) model

order

Fig. 5.5 Autocorrelation function (ACF) of the residuals from the AR(2) and the ARMA(1,3) model

0 2 4 6 8 10 12 14 16 18 20 0

0.2

0.4

0.6

0.8

1.2

1.4

order j

ψ j

ARMA(1,3)

AR(2)

Fig. 5.6 Impulse responses of the AR(2) and the ARMA(1,3) model

108 5 Estimation of ARMA Models

2001Q1 2001Q3 2002Q1 2002Q3 2003Q1 2003Q3 2004Q1 2004Q3 2005Q1 2005Q3 2006Q1 −1

−0.5

0.5

1.5

2 p er

ce n t

observations

forecast ARMA(1,3)

forecast AR(2)

Fig. 5.7 Forecasts of real GDP growth rates for Switzerland

be seen from Fig. 5.7, both models predict that the Swiss economy should move out of recession in the coming quarters. However, the ARMA(1,3) model indicates that the recovery is taking place more quickly and the growth overshooting its long-run mean of 1.3 % in about a year. The forecast of the AR(2) predicts a more steady approach to the long-run mean.

6Spectral Analysis and Linear Filters

Up to now we have viewed a time series as a time indexed sequence of random variables. The class of ARMA process was seen as an adequate class of models for the analysis of stationary time series. This approach is usually termed as time series analysis in the time domain. There is, however, an equivalent perspective which views a time series as overlayed waves of different frequencies. This view point is termed in time series analysis as the analysis in the frequency domain. The decomposition of a time series into sinusoids of different frequencies is called the spectral representation. The estimation of the importance of the waves at particular frequencies is referred to as spectral or spectrum estimation. Priestley (1981) provides an excellent account of these methods. The use of frequency domain methods, in particular spectrum estimation, which originated in the natural sciences was introduced to economics by Granger (1964).1 Notably, he showed that most of the fluctuations in economic time series can be attributed low frequencies cycles (Granger 1966).

Although both approaches are equivalent, the analysis in the frequency domain is more convenient when it comes to the analysis and construction of linear filters. The application of a filter to a time series amounts to take some moving-average of the time series. These moving-average may extend, at least in theory, into the infinite past, but also into the infinite future. A causal ARMA process fXtg may be regarded as filtered white-noise process with filter weights given by j, j D 1;2; : : : In economics, filters are usually applied to remove cycles of a particular frequency, like seasonal cycles (for example Christmas sales in a store), or to highlight particular cycles, like business cycles.

1The use of spectral methods in the natural sciences can be traced many centuries back. The modern statistical approach builds on to the work of N. Wiener, G. U. Yule, J. W. Tukey, and many others. See the interesting survey by Robinson (1982).

109

110 6 Spectral Analysis and Linear Filters

From a mathematical point of view, the equivalence between time and frequency domain analysis rest on the theory of Fourier series. An adequate representation of this theory is beyond the scope of this book. The interested reader may consult Brockwell and Davis (1991, chapter 4). An introduction to the underlying mathematical theory can be found in standard textbooks like Rudin (1987).

6.1 The Spectral Density

In the following, we assume that fXtg is a mean-zero (centered) stationary stochastic process with autocovariance function �.h/, h D 0; ˙1; ˙2; : : : Mathematically, �.h/ represents an double-infinite sequence which can be mapped into a real valued function f . /, 2 R, by the Fourier transform. This function is called the spectral density function or spectral density. Conversely, we retrieve from the spectral density each covariance. Thus, we have a one-to-one relation between autocovariance functions and spectral densities: both objects summarize the same properties of the time series, but represent them differently.

Definition 6.1 (Spectral Density). Let fXtg be a mean-zero stationary stochastic process absolutely summable autocovariance function � then the function

f . / D 1 2�

1 X

hD�1 �.h/e�{h ; �1 < < 1; (6.1)

is called the spectral density function or spectral density of fXtg. Thereby { denotes the imaginary unit (see Appendix A).

The sine is an odd function whereas the cosine and the autocovariance function are even functions.2 This implies that the spectral density can be rewritten as:

f . / D 1 2�

1 X

hD�1 �.h/.cos.h / � { sin.h //

D 1 2�

1 X

hD�1 �.h/ cos.h / C 0 D 1

2�

1 X

hD�1 �.h/ cos.�h /

D �.0/ 2�

C 1 �

1 X

hD1 �.h/ cos.h /: (6.2)

2A function f is called even if f .�x/ D f .x/; the function is called odd if f .�x/ D �f .x/. Thus, we have sin.��/ D � sin.�/ and cos.��/ D cos.�/.

6.1 Spectral Density 111

Because of the periodicity of the cosine function, i.e. because

f . C 2k�/ D f . /; for all k 2 Z;

it is sufficient to consider the spectral density only in the interval .��;� . As the cosine is an even function so is f . Thus, we restrict the analysis of the spectral density f . / further to the domain 2 Œ0;� .

In practice, we often use the period or oscillation length instead of the radiant . They are related by the formula:

period length D 2� : (6.3)

If, for example, the data are quarterly observations, a value of 0.3 for corresponds to a period length of approximately 21 quarters.

Remark 6.1. We gather some properties of the spectral density function f :

• Because f .0/ D 1 2�

P1 hD�1 �.h/, the long-run variance of fXtg J (see Sect. 4.4)

equals 2�f .0/, i.e. 2�f .0/ D J. • f is an even function so that f . / D f .� /. • f . / � 0 for all 2 .��;� . The proof of this proposition can be found

in Brockwell and Davis (1996, chapter 4). This property corresponds to the non-negative definiteness of the autocovariance function (see property 4 in Theorem 1.1 of Sect. 1.3).

• The single autocovariances are the Fourier-coefficients of the spectral density f :

�.h/ D Z �

�� e{h f . /d D

Z �

�� cos.h /f . /d :

For h D 0, we therefore get �.0/ D R �� f . /d .

The last property allows us to compute the autocovariances from a given spectral density. It shows how time and frequency domain analysis are related to each other and how a property in one domain is reflected as a property in the other.

These properties of a non-negative definite function can be used to characterize the spectral density of a stationary process fXtg with autocovariance function �. Theorem 6.1 (Properties of a Spectral Density). A function f defined on .��;� is the spectral density of a stationary process if and only if the following properties hold:

• f . / D f .� /; • f . / � 0; • R �

�� f . /d < 1.

112 6 Spectral Analysis and Linear Filters

Corollary 6.1. An absolutely summable function � is the autocovariance function of a stationary process if and only if

f . / D 1 2�

1 X

hD�1 �.h/e�{h � 0; for all 2 .��;� :

In this case f is called the spectral density of �.

The function f . /=�.0/ can be considered as a density function of some probability distribution defined on Œ��;� because f. /

�.0/ � 0 and

Z �

�� f . /

�.0/ d D 1:

The corresponding cumulative distribution function G is then defined as:

G. / D Z

�� f .!/

�.0/ d!; �� :

It satisfies: G.��/ D 0, G.�/ D 1, 1 � G. / D G. /, und G.0/ D 1=2. The autocorrelation function � is then given by

�.h/ D �.h/ �.0/

D Z �

�� e{ h dG. /:

Some Examples

Some relevant examples illustrating the above are:

white noise: Let fXtg be a white noise process with Xt � WN.0;�2/. For this process all autocovariances, except �.0/, are equal to zero. The spectral density therefore is equal to

f . / D 1 2�

1 X

hD�1 �.h/e�{h D �.0/

2� D �

2� :

Thus, the spectral density is equal to a constant which is proportional to the variance. This means that no particular frequency dominates the spectral density. This is the reason why such a process is called white noise.

MA(1): Let fXtg be a MA(1) process with autocovariance function

�.h/ D 8

1; h D 0; �; h D ˙1; 0; otherwise.

6.2 Spectral Decomposition of a Time Series 113

The spectral density therefore is:

f . / D 1 2�

1 X

hD�1 �.h/e�{h D �e

{ C 1 C �e�{ 2�

D 1 C 2� cos 2�

Thus, f . / � 0 if and only j�j � 1=2. According to Corollary 6.1 above, � is the autocovariance function of a stationary stochastic process if and only if j�j � 1=2. This condition corresponds exactly to the condition derived in the time domain (see Sect. 1.3). The spectral density for � D 0:4 or equivalently � D 0:5, respectively for � D �0:4 or equivalently � D �0:5, and �2 D 1 is plotted in Fig. 6.1a. As the process is rather smooth when the first order autocorrelation is positive, the spectral density is large in the neighborhood of zero and small in the neighborhood of �. For a negative autocorrelation the picture is just reversed.

AR(1): The spectral density of an AR(1) process Xt D �Xt�1 C Zt with Zt � WN.0;�2/ is:

f . / D �.0/ 2�

1 C 1 X

hD1 �h �

e�{h C e{h � !

D � 2

2�.1 � �2/ �

1 C �e {

1 � �e{ C �e�{

1 � �e�{ �

D � 2

2�

1 � 2� cos C �2

The spectral density for � D 0:6 and � D �0:6 and �2 D 1 are plotted in Fig. 6.1b. As the process with � D 0:6 exhibits a relatively large positive autocorrelation so that it is rather smooth, the spectral density takes large values for low frequencies. In contrast, the process with � D �0:6 is rather volatile due to the negative first order autocorrelation. Thus, high frequencies are more important than low frequencies as reflected in the corresponding figure. Note that, as � approaches one, the spectral density evaluated at zero tends to infinity, i.e. lim #0 f . / D 1. This can be interpreted in the following way. As the process gets closer to a random walk more and more weight is given to long-run fluctuations (cycles with very low frequency or very high periodicity) (Granger 1966).

6.2 Spectral Decomposition of a Time Series

Consider the simple harmonic process fXtg which just consists of a cosine and a sine wave:

Xt D A cos.! t/ C B sin.! t/: (6.4)

114 6 Spectral Analysis and Linear Filters

0 1 2 3

0.05

0.1

0.15

0.2

0.25

0.3

θ = 0.5 ρ = 0.4

θ = −0.5 ρ = −0.4

0 1 2 3 0

0.5

φ = 0.6 φ = −0.6

a b

Fig. 6.1 Examples of spectral densities with Zt � WN.0;1/. (a) MA(1) process. (b) AR(1) process

Thereby A and B are two uncorrelated random variables with EA D EB D 0 and VA D VB D 1. The autocovariance function of this process is �.h/ D cos.! h/. This autocovariance function cannot be represented as

R �

�� e {h f . /d . However, it

can be regarded as the Fourier transform of a discrete distribution function F:

�.h/ D cos.! h/ D Z

.��;� e{h dF. /;

where

F. / D 8

0; for < �!; 1=2; for �! � < !; 1; for � !.

(6.5)

The integral with respect to the discrete distribution function is a so-called Riemann- Stieltjes integral.3 F is a step function with jumps at �! and ! and step size of 1=2 so that the above integral equals 1

2 e�{ h! C 1

2 e�{ h! D cos.h!/.

These considerations lead to a representation, called the spectral representation, of the autocovariance function as the Fourier transform a distribution function over Œ��;� .

3The Riemann-Stieltjes integral is a generalization of the Riemann integral. Let f and g be two bounded functions defined on the interval Œa; b then the Riemann-Stieltjes integral

R b a f .x/dg.x/ is

defined as limn!1 Pn

iD1 f .�i/Œg.xi/ � g.xi�1/ where a D x1 < x2 < � � � < xn�1 < xn D b. For g.x/ D x we obtain the standard Riemann integral. If g is a step function with a countable number of steps xi of height hi then

R b a f .x/dg.x/ D

i f .xi/hi.

6.2 Spectral Decomposition of a Time Series 115

Theorem 6.2 (Spectral Representation). � is the autocovariance function of a stationary process fXtg if and only if there exists a right-continuous, nondecreasing, bounded function F on .��;� with the properties F.��/ D 0 and

�.h/ D Z

.��;� e{h dF. /: (6.6)

F is called the spectral distribution function of �.

Remark 6.2. If the spectral distribution function F has a density f such that F. / D R

�� f .!/d! then f is called the spectral density and the time series is said to have a continuous spectrum.

Remark 6.3. According to the Lebesgue-Radon-Nikodym Theorem (see, for exam- ple, Rudin (1987)), the spectral distribution function F can be represented uniquely as the sum of a distribution function FZ which is absolutely continuous with respect to the Lebesgue measure and a discrete distribution function FV . The distribution function FZ corresponds to the regular part of the Wold Decomposition (see Theorem 3.1 in Sect. 3.2) and has spectral density

fZ. / D �2

2�

ˇ‰.e�{ / ˇ

2 D � 2

2�

1 X

jD0 je

�{j ˇ

The discrete distribution FV corresponds to the deterministic part fVtg.

The process (6.4) considers just a single frequency !. We may, however, generalize this process by superposing several sinusoids. This leads to the class of harmonic processes:

Xt D k X

jD1 Aj cos.!j t/ C Bj sin.!j t/; 0 < !1 < � � � < !k < � (6.7)

where A1; B1; : : : ; Ak; Bk are random variables which are uncorrelated with each other and which have means EAj D EBj D 0 and variances VAj D VBj D �2j , j D 1; : : : ; k. The autocovariance function of such a process is given by �.h/ D PkjD1 �2j cos.!j h/. According to the spectral representation theorem the corresponding distribution function F can be represented as a weighted sum of distribution functions like those in Eq. (6.5):

F. / D k X

jD1 �2j Fj. /

116 6 Spectral Analysis and Linear Filters

with

Fj. / D 8

0; for < �!j; 1=2; for �!j � < !j; 1; for � !j.

This generalization points to the following properties:

• Each of the k components Aj cos.!j t/ C Bj sin.!j t/, j D 1; : : : ; k, is completely associated to a specific frequency !j.

• The k components are uncorrelated with each other. • The variance of each component is �2j . The contribution of each component to

the variance of Xt given by Pk

jD1 � 2 j therefore is �

2 j .

• F is a nondecreasing step-function with jumps at frequencies ! D ˙!j and step sizes 1

2 �2j .

• The corresponding probability distribution is discrete with values 1 2 �2j at the

frequencies ! D ˙!j and zero otherwise.

The interesting feature of harmonic processes as represented in Eq. (6.7) is that every stationary process can be represented as the superposition of uncorrelated sinusoids. However, in general infinitely many (even uncountably many) of these processes have to be superimposed. The generalization of (6.7) then leads to the spectral representation of a stationary stochastic process:

Xt D Z

.��;� e{t dZ. /: (6.8)

Thereby fZ. /g is a complex-valued stochastic process with uncorrelated incre- ments defined on the interval .��;� . The above representation is known as the spectral representation of the process fXtg.4 Note the analogy to the spectral representation of the autocovariance function in Eq. (6.6).

For the harmonic processes in Eq. (6.7), we have:

dZ. / D

Aj C{ Bj 2

; for D �!j and j D 1;2; : : : ; k; Aj �{ Bj 2 ; for D !j and j D 1;2; : : : ; k;

0; otherwise.

In this case the variance of dZ is given by:

EdZ. /dZ. / D (

�2j 2 ; if D ˙!j;

0; otherwise.

4A mathematically precise statement is given in Brockwell and Davis (1991, chapter 4) where also the notion of stochastic integration is explained.

6.3 The Periodogram and the Estimation of Spectral Densities 117

In general, we have:

D �

F. / � F. �/; discrete spectrum; f . /d ; continuous spectrum.

Thus, a large jump of the spectrum at frequency is associated with a large sinusoidal component with frequency .5

6.3 The Periodogram and the Estimation of Spectral Densities

Although the spectral distribution function is uniquely determined, its estimation from a finite sample with realizations fx1; x2; : : : ; xT g is not easy. This has to do with the problem of estimating a function from a finite number of points. We will present two-approaches: a non-parametric and a parametric one.

6.3.1 Non-Parametric Estimation

A simple estimator of the spectral density, OfT. /, can be obtained by replacing in the defining equation (6.1) the theoretical autocovariances � by their estimates O�. However, instead of a simple sum, we consider a weighted sum:

OfT. / D 1

2�

jhj�`T k

�

O�.h/e�{h : (6.9)

The weighting function k, also known as the lag window, is assumed to have exactly the same properties as the kernel function introduced in Sect. 4.4. This correspondence is not accidental, indeed the long-run variance defined in Eq. (4.1) is just 2� times the spectral density evaluated at D 0. Thus, one might choose a weighting, kernel or lag window from Table 4.1, like the Bartlett-window, and use it to estimate the spectral density. The lag truncation parameter is chosen in such a way that `T ! 1 as T ! 1. The rate of divergence should, however, be smaller than T so that `T

T approaches zero as T goes to infinity. As an estimator of

the autocovariances one uses the estimator given in Eq. (4.2) of Sect. 4.2. The above estimator is called an indirect spectral estimator because it requires

the estimation of the autocovariances in the first step. The periodogram provides an alternative direct spectral estimator. For this purpose, we represent the observations as linear combinations of sinusoids of specific frequencies. These so-called Fourier frequencies are defined as !k D 2�kT , k D �b T�12 c; : : : ; b T2 c. Thereby bxc denotes

5Thereby F. �/ denotes the left-sided limit, i.e. F. �/ D lim!" F.!/.

118 6 Spectral Analysis and Linear Filters

the largest integer smaller or equal to x. With this notation, the observations xt, t D 1; : : : ; T, can be represented as a sum of sinusoids:

xt D b T 2

c X

kD�b T�1 2

c ake

{!k t D b T 2

c X

kD�b T�1 2

c ak.cos.!kt/ C { sin.!k t//:

The coefficients fakg are the discrete Fourier-transform of the observations fx1; x2; : : : ; xT g. The periodogram IT is then defined as follows. Definition 6.2 (Periodogram). Given observations fx1; x2; : : : ; xT g, the peri- odogram is defined as the function

IT. / D 1

T X

tD1 xte

�{t ˇ

For each Fourier-frequency !k, the periodogram IT.!k/ equals jakj2. This implies that

T X

tD1 jxtj2 D

b T 2

c X

kD�b T�1 2

c jakj2 D

b T 2

c X

kD�b T�1 2

c IT.!k/:

The value of the periodogram evaluated at the Fourier-frequency !k is therefore nothing but the contribution of the sinusoid with frequency !k to the variation of fxtg as measured by sum of squares. In particular, for any Fourier-frequency different from zero we have that

IT.!k/ D T�1 X

hD�TC1 O�.h/e�{h!k:

Thus the periodogram represents, disregarding the proportionality factor 2�, the sample analogue of the spectral density and therefore carries the same information.

Unfortunately, it turns out that the periodogram is not a consistent estimator of the spectral density. In particular, the covariance between IT. 1/ and IT. 2/, 1 ¤ 2, goes to zero for T going to infinity. The periodogram thus has a tendency to get very jagged for large T leading to the detection of spurious sinusoids. A way out of this problem is to average the periodogram over neighboring frequencies, thereby reducing its variance. This makes sense because the variance is relatively constant within a small frequency band. The averaging (smoothing) of the periodogram over neighboring frequencies leads to the class of discrete spectral average estimators which turn out to be consistent:

OfT. / D 1

2�

jhj�`T KT.h/IT

�

Q!T; C 2�h

�

(6.10)

6.3 The Periodogram and the Estimation of Spectral Densities 119

where Q!T; denotes the multiple of 2�T which is closest to . `T is the bandwidth of the estimator, i.e. the number of ordinates over which the average is taken. `T satisfies the same properties as in the case of the indirect spectral estimator (6.9): `T ! 1 and `T=T ! 0 for T ! 1. Thus, as T goes to infinity, on the one hand the average is taken over more and more values, but on the other hand the frequency band over which the average is taken is getting smaller and smaller. The spectral weighting function or spectral window KT is a positive even function satisfying

jhj�`T KT.h/ D 1 and P

jhj�`T K 2 T.h/ ! 0 for T ! 1. It can be

shown that under these conditions the discrete spectral average estimator is mean- square consistent. Moreover, the estimator in Eq. (6.9) can be approximated by a corresponding discrete spectral average estimator by defining the spectral window as

KT.!/ D 1

2�

jhj�`T k

�

e�{h!

or vice versa

k.h/ D Z �

�� KT.!/e

�{h!d!:

Thus, the lag and the spectral window are related via the Fourier transform. For details and the asymptotic distribution the interested reader is referred to Brockwell and Davis (1991, Chapter10). Although the indirect and the direct estimator give approximately the same result when the kernels used are related as in the equation above, the direct estimator (6.10) is usually preferred in practice because it is, especially for long times series, computationally more efficient, in particular in connection with the fast fourier transformation (FFT).6

A simple spectral weighting function, known as the Daniell spectral window, is given by KT.h/ D .2`T C1/�1 when jhj � `T and 0 otherwise and where `T D

p T .

It averages over 2`T C 1 values within a frequency band of approximate width 4�p T

. This function corresponds to the Daniell kernel function or Daniell lag window k.x/ D sin.�x/=.�x/ for jxj � 1 and zero otherwise (see Sect. 4.4). In practice, the sample size is fixed and the researcher is faced with a trade-off between variance and bias. On the one hand, a weighting function which averages over a wide frequency band produces a smooth spectral density, but has probably a large bias because the estimate of f . / depends on frequencies which are rather far away from . On the other hand, a weighting function which averages only over a small frequency band produces a small bias, but probably a large variance. It is thus advisable in practice

6The FFT is seen as one of the most important numerical algorithms ever as it allows a rapid computation of Fourier transformations and its inverse. The FFT is widely in digital signal processing.

120 6 Spectral Analysis and Linear Filters

0 0.5 1 1.5 2 2.5 3 0

frequency in radians

true spectrum

Fig. 6.2 Raw periodogram of a white noise time series (Xt � WN.0;1/, T D 200)

to work with alternative weighting functions and to choose the one which delivers a satisfying balance between bias and variance.

The following two examples demonstrate the large variance of the periodogram. The first example consists of 200 observations from a simulated white noise time series with variance equal. Whereas the true spectrum is constant equal to one, the raw periodogram, i.e. the periodogram without smoothing, plotted in Fig. 6.2 is quite erratic. However, it is obvious that by taking averages of adjacent frequencies the periodogram becomes smoother and more in line with the theoretical spectrum. The second example consists of 200 observations of a simulated AR(2) process. Figure 6.3 demonstrates again the jaggedness of the raw periodogram. However, these erratic movements are distributed around the true spectrum. Thus, by smoothing one can hope to get closer to the true spectrum and even detect the dominant cycle with radian equal to one. It is also clear that by smoothing over a too large range, in the extreme over all frequencies, no cycle could be detected.

Figure 6.4 illustrates these considerations with real life data by estimating the spectral density of quarterly growth rates of real investment in constructions for the Swiss economy using alternative weighting functions. To obtain a better graphical resolution we have plotted the estimates on a logarithmic scale. All three estimates show a peak (local maximum) at the frequency D �

2 . This corresponds to a wave

with a period of one year. The estimator with a comparably wide frequency band (dotted line) smoothes the minimum D 1 away. The estimator with a comparable small frequency band (dashed line), on the contrary, reveals additional waves with frequencies D 0:75 and 0:3 which correspond to periods of approximately two, respectively five years. Whether these waves are just artifacts of the weighting function or whether there really exist cycles of that periodicity remains open.

6.3 The Periodogram and the Estimation of Spectral Densities 121

0 0.5 1 1.5 2 2.5 3 10−2

10−1

100

101

102

frequency in radians

true spectrum

Fig. 6.3 Raw periodogram of an AR(2) process (Xt D 0:9Xt�1 � 0:7Xt�2 C Zt with Zt � WN.0;1/, T D 200)

10−1 100 10−4

10−3

10−2

λ 0,3 π/20,75

optimal choice

too jagged too

smooth

Fig. 6.4 Non-parametric direct estimates of a spectral density with alternative weighting functions

6.3.2 Parametric Estimation

An alternative to the nonparametric approaches just outlined consists in the estima- tion of an ARMA model and followed by deducing the spectral density from it. This approach was essentially first proposed by Yule (1927).

Theorem 6.3 (Spectral Density of ARMA Processes). Let fXtg be a causal ARMA(p,q) process given by ˆ.L/Xt D ‚.L/Zt and Zt � WN.0;�2/. Then the spectral density fX is given by

122 6 Spectral Analysis and Linear Filters

fX. / D �2

2�

ˇ‚.e�{ / ˇ

ˇˆ.e�{ / ˇ

2 ; �� : (6.11)

Proof. fXtg is generated by applying the linear filter ‰.L/ with transfer function ‰.e�{ / D ‚.e�{ /

ˆ.e�{ / to fZtg (see Sect. 6.4). Formula (6.11) is then an immediate

consequence of Theorem 6.5 because the spectral density of fZtg is equal to �22� . ut

Remark 6.4. As the spectral density of an ARMA process fXtg is given by a quotient of trigonometric functions, the process is said to have a rational spectral density.

The spectral density of the AR(2) process Xt D �1Xt�1 C �2Xt�2 C Zt with Zt � WN.0;�2/, for example, is then given by

fX. / D �2

2�.1 C �21 C 2�2 C �22 C 2.�1�2 � �1/ cos � 4�2 cos2 / :

The spectral density of an ARMA(1,1) process Xt D �Xt�1 C Zt C �Zt�1 with Zt � WN.0;�2/ is

fX. / D �2.1 C �2 C 2� cos / 2�.1 C �2 C 2� cos /:

An estimate of the spectral density is then obtained by replacing the unknown coefficients of the ARMA model by their corresponding estimates and by applying Formula (6.11) to the estimated model. Figure 6.5 compares the nonparametric to the parametric method based on an AR(4) model using the same data as in Fig. 6.4. Both methods produce similar estimates. They clearly show waves of periodicity of half a year and a year, corresponding to frequencies �

2 and �. The nonparametric

estimate is, however, more volatile in the frequency band Œ0:6;1 and around 2:5.

6.4 Linear Time-Invariant Filters

Time-invariant linear filters are an indispensable tool in time series analysis. Their objective is to eliminate or amplify waves of a particular periodicity. For example, they may be used to purge a series from seasonal movements. The seasonally adjusted time series should then reflect more strongly the business cyclical movements which are viewed to have period length between two and eight years. The spectral analysis provides just the right tools to construct and analyze such filters.

Definition 6.3. fYtg is the output of the linear time-invariant filter (LTF) ‰ D f j; j D 0; ˙1; ˙2; : : : g applied to the input fXtg if

6.4 Linear Time-Invariant Filters 123

10−1 100 10−4

10−3

10−2

nonparametric estimate parametric estimate using an AR(4) model

π/2

Fig. 6.5 Comparison of nonparametric and parametric estimates of the spectral density of the growth rate of investment in the construction sector

Yt D ‰.L/Xt D 1 X

jD�1 jXt�j with

1 X

jD�1 j jj < 1:

The filter is called causal or one-sided if j D 0 for j < 0; otherwise it is called two-sided.

Remark 6.5. Time-invariance in this context means that the lagged process fYt�sg is obtained for all s 2 Z from fXt�sg by applying the same filter ‰.

Remark 6.6. MA processes, causal AR processes and causal ARMA processes can be viewed as filtered white noise processes.

It is important to recognize that the application of a filter systematically changes the autocorrelation properties of the original time series. This may be warranted in some cases, but may lead to the “discovery” of spurious regularities which just reflect the properties of the filter. See the example of the Kuznets filter below.

Theorem 6.4 (Autocovariance Function of Filtered Process). Let fXtg be a mean- zero stationary process with autocovariance function �X . Then the filtered process fYtg defined as

Yt D 1 X

jD�1 jXt�j D ‰.L/Xt

124 6 Spectral Analysis and Linear Filters

with P1

jD�1 j jj < 1 is also a mean-zero stationary process with autocovariance function �Y . Thereby the two autocovariance functions are related as follows:

�Y.h/ D 1 X

jD�1

1 X

kD�1 j k�X.h C k � j/; h D 0; ˙1; ˙2; : : :

Proof. We first show the existence of the output process fYtg. For this end, consider the sequence of random variables fY.m/t gmD1;2;::: defined as

Y .m/ t D

m X

jD�m jXt�j:

To show that the limit for m ! 1 exists in the mean square sense, it is, according to Theorem C.6, enough to verify the Cauchy criterion

ˇY .m/ t � Y.n/t

�! 0; for m; n ! 1: Taking without loss of generality m > n, Minkowski’s inequality (see Theorem C.2 or triangular inequality) leads to

@ E

m X

jD�m jXt�j �

n X

jD�n jXt�j

2 1

1=2

�

@ E

m X

jDnC1 jXt�j

2 1

1=2

@ E

�m X

jD�n�1 jXt�j

2 1

1=2

Using the Cauchy-Bunyakovskii-Schwarz inequality and the stationarity of fXtg, the first term on the right hand side is bounded by

m X

j;kDnC1 j jXt�j k Xt�kj

1=2

� 0

m X

j;kDnC1 j jjj kjE.jXt�jjjXt�kj/

1=2

� 0

m X

j;kDnC1 j jjj kj.EX2t�j/1=2

.EX2t�k/ 1=2

1=2

D �X.0/1=2 m X

jDnC1 j jj:

6.4 Linear Time-Invariant Filters 125

As P1

jD�1 j jj < 1 by assumption, the last term converges to zero. Thus, the limit of fY.m/t g, m ! 1, denoted by St, exists in the mean square sense with ES2t < 1.

In remains to show that St and P1

jD�1 jXt�j are actually equal with probability one. This is established by noting that

E jSt � ‰.L/Xtj2 D E lim inf m!1

St � m X

jD�m Xt�j

� lim inf m!1 E

St � m X

jD�m Xt�j

D 0

where use has been of Fatou’s lemma. The stationarity of fYtg can be checked as follows:

EYt D lim m!1

m X

jD�m jEXt�j D 0;

EYt Yt�h D lim m!1 E

m X

jD�m jXt�j

m X

kD�m k Xt�h�k

D 1 X

jD�1

1 X

kD�1 j k�X.h C k � j/:

Thus, EYt and EYt Yt�h are finite and independent of t. fYt g is therefore stationary. ut

Corollary 6.2. If Xt � WN.0;�2/ and Yt D P1

jD0 jXt�j with P1

jD0 j jj < 1 then the above expression for �Y.h/ simplifies to

�Y.h/ D �2 1 X

jD0 j jCjhj:

Remark 6.1. In the proof of the existence of fYtg, the assumption of the stationarity of fXtg can be weakened by assuming only supt EX2t < 1.

Theorem 6.5. Under the conditions of Theorem 6.4, the spectral densities of fXtg and fYtg are related as

fY. / D ˇ

ˇ‰.e�{ / ˇ

2 fX. / D ‰.e{ /‰.e�{ /fX. /

where ‰.e�{ / D P1jD�1 je�{ j . ‰.e�{ / is called the transfer function of the filter.

126 6 Spectral Analysis and Linear Filters

To understand the effect of the filter ‰, consider the simple harmonic process Xt D 2 cos. t/ D e{ t C e�{ t . Passing fXtg through the filter ‰ leads to a transformed time series fYtg defined as

Yt D 2 ˇ

ˇ‰.e�{ / ˇ

ˇ cos

�

t � �. /

��

where �. / D arg ‰.e�{ /. The filter therefore amplifies some frequencies by the factor g. / D

ˇ‰.e�{ / ˇ

ˇ and delays Xt by �. /

periods. Thus, we have a change in

amplitude given by the amplitude gain function g. / and a phase shift given by the phase gain function �. /. If the gain function is bigger than one the corresponding frequency is amplified. On the other hand, if the value is smaller than one the corresponding frequency is dampened.

Examples of Filters

• First differences (changes with respect to previous period):

‰.L/ D � D 1 � L:

The transfer function of this filter is .1 � e�{ / and the gain function is 2.1 � cos /. These functions take the value zero for D 0. Thus, the filter eliminates the trend which can be considered as a wave with an infinite period length.

• Change with respect to same quarter last year, assuming that the data are quarterly observations:

‰.L/ D 1 � L4:

The transfer function and the gain function are 1 � e�4{ and 2.1 � cos.4 //, respectively. Thus, the filter eliminates all frequencies which are multiples of �

including the zero frequency. In particular, it eliminates the trend and waves with periodicity of four quarters.

• A famous example of a filter which led to wrong conclusions is the Kuznets filter (see Sargent 1987, 273–276). Assuming yearly data, this filter is obtained from two transformations carried out in a row. The first transformation which should eliminate cyclical movements takes centered five year moving averages. The second one take centered non-overlapping first differences. Thus, the filter can be written as:

‰.L/ D 1 5

�

L�2 C L�1 C 1 C L C L2� „ ƒ‚ …

first transformation

�

L�5 � L5� „ ƒ‚ …

second transformation

6.5 Some Important Filters 127

0 0.5 1 1.5 2 2.5 3 0

0.5

1.5

0.2886

Fig. 6.6 Transfer function of the Kuznets filters

Figure 6.6 gives a plot of the transfer function of the Kuznets filter. Thereby it can be seen that all frequencies are dampened, except those around D 0:2886. The value D 0:2886 corresponds to a wave with periodicity of approximately 2�=0:2886 D 21:77 � 22 years. Thus, as first claimed by Howrey (1968), even a filtered white noise time series would exhibit a 22 year cycle. This demonstrates that cycles of this length, related by Kuznets (1930) to demographic processes and infrastructure investment swings, may just be an artefact produced by the filter and are therefore not endorsed by the data.

6.5 Some Important Filters

6.5.1 Construction of Low- and High-Pass Filters

For some purposes it is desirable to eliminate specific frequencies. Suppose, we want to purge a time series from all movements with frequencies above c, but leave those below this value unchanged. The transfer function of such an ideal low-pass filter would be:

‰.e�{ / D �

1; for � c; 0; for > c.

By expanding ‰.e�{ / into a Fourier-series ‰.e�{ / D P1jD�1 je�{ j , it is possible to determine the appropriate filter coefficients f jg. In the case of a low- pass filter they are given by:

j D 1

2�

Z c

� c e�{j!d! D

(

c � ; j D 0;

sin.j c/ j�

; j ¤ 0.

128 6 Spectral Analysis and Linear Filters

The implementation of the filter in practice is not straightforward because only a finite number of coefficients can be used. Depending on the number of observations, the filter must be truncated such that only those j with jjj � q are actually employed. The problem becomes more severe as one gets to the more recent observations because less future observations are available. For the most recent period even no future observation is available. This problem is usually overcome by replacing the missing future values by their corresponding forecast. Despite this remedy, the filter works best in the middle of the sample and is more and more distorted as one approaches the beginning or the end of the sample.

Analogously for low-pass filters, it is possible to construct high-pass filters. Figure 6.7 compares the transfer function of an ideal high-pass filter with two filters truncated at q D 8 and q D 32, respectively. Obviously, the transfer function with the higher q approximates the ideal filter better. In the neighborhood of the critical frequency, in our case �=16, however, the approximation remains inaccurate. This is known as the Gibbs phenomenon.

6.5.2 The Hodrick-Prescott Filter

The Hodrick-Prescott filter (HP-Filter) has gained great popularity in the macroeco- nomic literature, particularly in the context of the real business cycles theory. This high-pass filter is designed to eliminate the trend and cycles of high periodicity and to emphasize movements at business cycles frequencies (see Hodrick and Prescott 1980; King and Rebelo 1993; Brandner and Neusser 1992).

One way to introduce the HP-filter is to examine the problem of decompos- ing a time series fXtg additively into a growth component fGtg and a cyclical component fCtg:

Xt D Gt C Ct:

This decomposition is, without further information, not unique. Following the suggestion of Whittaker (1923), the growth component should be approximated by a smooth curve. Based on this recommendation Hodrick and Prescott suggest to solve the following restricted least-squares problem given a sample fXtgtD1;:::;T :

T X

tD1 .Xt � Gt/2 C

T�1 X

tD2 Œ.GtC1 � Gt/ � .Gt � Gt�1/ 2 �! minfGt g

The above objective function has two terms. The first one measures the fit of fGtg to the data. The closer fGtg is to fXtg the smaller this term becomes. In the limit when Gt D Xt for all t, the term is minimized and equal to zero. The second term measures the smoothness of the growth component by looking at the discrete analogue to the second derivative. This term is minimized if the changes of the growth component from one period to the next are constant. This, however, implies that Gt is a linear

6.5 Some Important Filters 129

10−2 10−1 100 −0.2

0.2

0.4

0.6

0.8

1.2

π/16

ideal high−pass filter

truncated high−pass filter

with q = 8

truncated hochpass filter

with q = 32

Hodrick−Prescott filter with λ = 1600

Fig. 6.7 Transfer function of HP-filter in comparison to high-pass filters

function. Thus the above objective function represents a trade-off between fitting the data and smoothness of the approximating function. This trade-off is governed by the meta-parameter which must be fixed a priori.

The value of depends on the critical frequency and on the periodicity of the data (see Uhlig and Ravn 2002, for the latter). Following the proposal by Hodrick and Prescott (1980) the following values for are common in the literature:

D 8

6:25; yearly observations; 1600; quarterly observations; 14400; monthly observations.

It can be shown that these choices for practically eliminate waves of periodicity longer than eight years. The cyclical or business cycle component is therefore composed of waves with periodicity less than eight years. Thus, the choice of implicitly defines the business cycle. Figure 6.7 compares the transfer function of the HP-filter to the ideal high-pass filter and two approximate high-pass filters.7

As an example, Fig. 6.8 displays the HP-filtered US logged GDP together with the original series in the upper panel and the implied business cycle component in the lower panel.

7As all filters, the HP-filter systematically distorts the properties of the time series. Harvey and Jaeger (1993) show how the blind application of the HP-filter can lead to the detection of spurious cyclical behavior.

130 6 Spectral Analysis and Linear Filters

1950 1960 1970 1980 1990 2000 2010 7.5

8.5

9.5

1950 1960 1970 1980 1990 2000 2010 −6

−4

−2

p e

rc e

n t

Fig. 6.8 HP-filtered US logged GDP (upper panel) and cyclical component (lower panel)

6.5.3 Seasonal Filters

Besides the elimination of trends, the removal of seasonal movements represents another important application in practice. Seasonal movements typically arise when a time series is observed several times within a year giving rise to the possibility of waves with periodicity less than twelve month in the case of monthly observations, respectively of four quarters in the case of quarterly observations. These cycles are usually considered to be of minor economic interest because they are due to systematic seasonal variations in weather conditions or holidays (Easter, for example).8 Such variations can, for example, reduce construction activity during the winter season or production in general during holiday time. These cycles have usually quite large amplitude so that they obstruct the view to the economically and politically more important business cycles. In practice, one may therefore prefer to work with seasonally adjusted series. This means that one must remove the seasonal components from the time series in a preliminary stage of the analysis. In section, we will confine ourself to only few remarks. Comprehensive treatment of seasonality can be found in Hylleberg (1986) and Ghysels and Osborn (2001).

Two simple filters for the elimination of seasonal cycles in the case of quarterly data are given by the one-sided filter

‰.L/ D .1 C L C L2 C L3/=4

8An exception to this view is provided by Miron (1996).

6.5 Some Important Filters 131

10−1 100 10−1

100

101

102

103

λ π/2 π

seasonally adjusted

not seasonally adjusted

0.72

Fig. 6.9 Transfer function of growth rate of investment in the construction sector with and without seasonal adjustment

or the two-sided filter

‰.L/ D 1 8

L2 C 1 4

L C 1 4

C 1 4

L�1 C 1 8

L�2:

In practice, the so-called X–11-Filter or its enhanced versions X–12 and X–13 filter developed by the United States Census Bureau are often applied. This filter is a two-sided filter which makes, in contrast to two examples above, use of all sample observations. As this filter not only adjusts for seasonality, but also corrects for outliers, a blind mechanical use is not recommended. Gómez and Maravall (1996) developed an alternative method known under the name TRAMO-SEATS. More details on the implementation of both methods can be found in Eurostat (2009).

Figure 6.9 shows the effect of seasonal adjustment using TRAMO-SEATS by looking at the corresponding transfer functions of the growth rate of construction investment. One can clearly discern how the yearly and the half-yearly waves corresponding to the frequencies �=2 and � are dampened. On the other hand, the seasonal filter weakly amplifies a cycle of frequency 0:72 corresponding to a cycle of periodicity of two years.

6.5.4 Using Filtered Data

Whether or not to use filtered, especially seasonally adjusted, data is still an ongoing debate. Although the use of unadjusted data together with a correctly specified model is clearly the best choice, there is a nonnegligible uncertainty in modeling

132 6 Spectral Analysis and Linear Filters

economic time series. Thus, in practice one faces several trade-offs which must be taken into account and which may depend on the particular context (Sims 1974, 1993; Hansen and Sargent 1993). One the one hand, the use of adjusted data may disregard important information on the dynamics of the time series and introduce some biases. On the other hand, the use of unadjusted data encounters the risk of misspecification, especially because usual measures of fit may put too large emphasis on fitting the seasonal frequencies thereby neglecting other frequencies.

6.6 Exercises

Exercise 6.6.1.

(i) Show that the process defined in Eq. (6.4) has an autocovariance function equal to �.h/ D cos.!h/.

(ii) Show that the process defined in Eq. (6.7) has autocovariance function

�.h/ D k X

jD1 � 2 j cos.!jh/

Exercise 6.6.2. Compute the transfer and the gain function for the following filters:

(i) ‰.L/ D 1 � L (ii) ‰.L/ D 1 � L4

7Integrated Processes

7.1 Definition, Properties and Interpretation

Up to now the discussion concentrated on stationary processes and in particular ARMA processes. According to the Wold decomposition theorem (see Theorem 3.1) every purely non-deterministic processes possesses the following representation:

Xt D � C ‰.L/Zt;

where fZtg � WN.0;�2/ and P1

jD0 2 j < 1. Typically, we model Xt as an ARMA

process so that ‰.L/ D ‚.L/ ˆ.L/ . This representation implies:

• EXt D �, • limh!1 PtXtCh D �.

The above property is often referred to as mean reverting because the process moves around a constant mean. Deviations from this mean are only temporary or transitory. Thus, the best long-run forecast is just the mean of the process.

This property is often violated by economic time series which typically show a tendency to growth. Classic examples are time series for GDP (see Fig. 1.3) or some stock market index (see Fig. 1.5). This trending property is not compatible with stationarity as the mean is no longer constant. In order to cope with this characteristic of economic time series, two very different alternatives have been proposed. The first one consists in letting the mean � be a function of time �.t/. The most popular specification for �.t/ is a linear function, i.e. �.t/ D ˛ C ıt. In this case we get:

Xt D ˛ C ıt „ƒ‚…

linear trend

C‰.L/Zt

133

134 7 Integrated Processes

The process fXtg is then referred to as a trend-stationary process. In practice one also encounters quadratic polynomials of t or piecewise linear functions. For example, �.t/ D ˛1 C ı1t for t � t0 and �.t/ D ˛2 C ı2t for t > t0. In the following, we restrict ourself to linear trend functions.

The second alternative assumes that the time series becomes stationary after differentiation. The number of times one has to differentiate the process to achieve stationarity is called the order of integration. If d times differentiation is necessary, the process is called integrated of order d and is denoted by Xt � I.d/. If the resulting time series, �d Xt D .1 � L/d Xt, is an ARMA(p,q) process, the original process is called an ARIMA(p,d,q) process. Usually it is sufficient to differentiate the time series only once, i.e. d D 1. For expositional purposes we will stick to this case.

The formal definition of an I(1) process is given as follows.

Definition 7.1. The stochastic process fXtg is called integrated of order one or difference-stationary, denoted as Xt � I.1/, if and only if �Xt D Xt � Xt�1 can be represented as

�Xt D .1 � L/Xt D ı C ‰.L/Zt; ‰.1/ ¤ 0; with fZtg � WN.0;�2/ and

P1 jD0 jj jj < 1.

The qualification ‰.1/ ¤ 0 is necessary to avoid trivial and uninteresting cases. Suppose for the moment that ‰.1/ D 0, then it would be possible to write ‰.L/ as .1 � L/e‰.L/ for some lag polynomial e‰.L/. This would, however, imply that the factor 1 � L could be canceled in the above definition so that fXtg is already stationary and that the differentiation would be unnecessary. The assumption ‰.1/ ¤ 0 thus excludes the case where a trend-stationary process could be regarded as an integrated process. For each trend-stationary process Xt D ˛Cıt Ce‰.L/Zt we have �Xt D ıC‰.L/Zt with ‰.L/ D .1�L/e‰.L/. This would violate the condition ‰.1/ ¤ 0. Thus a trend-stationary process cannot be a difference-stationary process.

The condition P1

jD0 jj jj < 1 implies P1

jD0 2 j < 1 and is therefore

stronger than necessary for the Wold representation to hold. In particular, it implies the Beveridge-Nelson decomposition of integrated processes into a linear trend, a random walk, and a stationary component (see Sect. 7.1.4). The condition is automatically fulfilled for all ARMA processes because f jg decays exponentially to zero.

Integrated processes with d > 0 are also called unit-root processes. This designation results from the fact that ARIMA processes with d > 0 can be viewed as ARMA processes, whereby the AR polynomial has a d-fold root of one.1 An important prototype of an integrated process is the random walk with drift ı:

Xt D ı C Xt�1 C Zt; Zt � WN.0;�2/:

1Strictly speaking this does not conform to the definitions used in this book because our definition of ARMA processes assumes stationarity.

7.1 Definition, Properties and Interpretation 135

Trend-stationary and difference-stationary processes have quite different characteristics. In particular, they imply different behavior with respect to the long-run forecast, the variance of the forecast error, and the impulse response function. In the next section, we will explore these properties in detail.

7.1.1 Long-Run Forecast

The optimal forecast in the least-squares sense given the infinite past of a trend- stationary process is given by

ePtXtCh D ˛ C ı.t C h/ C hZt C hC1Zt�1 C : : :

Thus we have

lim h!1

E �

ePt XtCh � ˛ � ı.t C h/ �2 D �2 lim

h!1

1 X

jD0 2 hCj D 0

because P1

jD0 2 j < 1. Thus the long-run forecast is given by the linear trend.

Even if Xt deviates temporarily from the trend line, it is assumed to return to it. A trend-stationary process therefore behaves in the long-run like �.t/ D ˛ C ıt.

The forecast of the differentiated series is

ePt�XtCh D ı C hZt C hC1Zt�1 C hC2Zt�2 C : : :

The level of XtCh is by definition

XtCh D .XtCh � XtCh�1/ C .XtCh�1 � XtCh�2/ C : : : C .XtC1 � Xt/ C Xt so that

ePtXtCh D ePt�XtCh C ePt�XtCh�1 C : : : C ePt�XtC1 C Xt D ı C hZt C hC1Zt�1 C hC2Zt�2 C : : :

C ı C h�1Zt C hZt�1 C hC1Zt�2 C : : : C ı C h�2Zt C h�1Zt�1 C hZt�2 C : : : C : : : C Xt

D Xt C ıh C . h C h�1 C : : : C 1/ Zt C . hC1 C h C : : : C 2/ Zt�1 : : :

136 7 Integrated Processes

This shows that also for the integrated process the long-run forecast depends on a linear trend with slope ı. However, the intercept is no longer a fixed number, but given by Xt which is stochastic. With each new realization of Xt the intercept changes so that the trend line moves in parallel up and down. This issue can be well illustrated by the following two examples.

Example 1. Let fXtg be a random walk with drift ı. Then best forecast of XtCh, PtXtCh, is

PtXtCh D ıh C Xt:

The forecast thus increases at rate ı starting from the initial value of Xt. ı is therefore the slope of a linear trend. The intercept of this trend is stochastic and equal to Xt. Thus the trend line moves in parallel up or down depending on the realization of Xt.

Example 2. Let fXtg be an ARIMA(0,1,1) process given by �Xt D ı C Zt C �Zt�1 with j�j < 1. The best forecast of XtCh is then given by

PtXtCh D ıh C Xt C �Zt:

As before the intercept changes in a stochastic way, but in contrary to the previous example it is now given by Xt C �Zt. If we consider the forecast given the infinite past, the invertibility of the process implies that Zt can be expressed as a weighted sum of current and past realizations of �Xt (see Sects. 2.3 and 3.1).

7.1.2 Variance of Forecast Error

In the case of a trend-stationary process the forecast error is

XtCh � ePtXtCh D ZtCh C 1ZtCh�1 C : : : C h�1ZtC1:

As the mean of the forecast error is zero, the variance is

E �

XtCh � ePtXtCh �2 D �1 C 21 C 22 C : : : C 2h�1

�

� 2 :

For h going to infinity this expression converges to �2 P1

jD0 2 j < 1. This is

nothing but the unconditional variance of Xt. Thus the variance of the forecast error increases with the length of the forecasting horizon, but remains bounded.

For the integrated process the forecast error can be written as

XtCh � ePt XtCh D ZtCh C .1 C 1/ ZtCh�1C : : : C .1 C 1 C 2 C : : : C h�1/ ZtC1:

7.1 Definition, Properties and Interpretation 137

The forecast error variance therefore is

E �

XtCh � ePt XtCh �2 D

1 C .1 C 1/2 C : : : C .1 C 1 C : : : C h�1/2 i

� 2 :

This expression increases with the length of the forecast horizon h, but is no longer bounded. It increases linearly in h to infinity.2 The precision of the forecast therefore not only decreases with the forecasting horizon h as in the case of the trend- stationary model, but converges to zero. In the example above of the ARIMA(0,1,1) process the forecasting error variance is

E.XtCh � PtXtCh/2 D h

1 C .h � 1/.1 C �/2 i

�2:

This expression clearly increases linearly with h.

7.1.3 Impulse Response Function

The impulse response function (dynamic multiplier) is an important analytical tool as it gives the response of the variable Xt to the underlying shocks. In the case of the trend-stationary process the impulse response function is

@ePtXtCh @Zt

D h �! 0 for h ! 1:

The effect of a shock thus declines with time and dies out. Shocks have therefore only transitory or temporary effects. In the case of an ARMA process the effect even declines exponentially (see the considerations in Sect. 2.3).3

In the case of integrated processes the impulse response function for �Xt implies:

@ePtXtCh @Zt

D 1 C 1 C 2 C : : : C h:

For h going to infinity, this expression converges P1

jD0 j D ‰.1/ ¤ 0. This implies that a shock experienced in period t will have a long-run or permanent effect. This long-run effect is called persistence. If f�Xtg is an ARMA process then the persistence is given by the expression

2Proof : By assumption f jg is absolutely summable so that ‰.1/ converges. Moreover, as ‰.1/ ¤ 0, there exists " > 0 and an integer m such that

Ph jD0 j

ˇ > " for all h > m. The

squares are therefore bounded from below by "2 > 0 so that their infinite sum diverges to infinity. 3The use of the partial derivative is just for convenience. It does not mean that XtCh is differentiated in the literal sense.

138 7 Integrated Processes

‰.1/ D ‚.1/ ˆ.1/

Thus, for an ARIMA(0,1,1) the persistence is ‰.1/ D ‚.1/ D 1 C �. In the next section we will discuss some examples.

7.1.4 The Beveridge-Nelson Decomposition

The Beveridge-Nelson decomposition represents an important tool for the under- standing of integrated processes.4 It shows how an integrated time series of order one can be represented as the sum of a linear trend, a random walk, and a stationary series. It may therefore be used to extract the cyclical component (business cycle component) of a time series and can thus be viewed as an alternative to the HP-filter (see Sect. 6.5.2) or to more elaborated so-called structural time series models (see Sects. 17.1 and 17.4.2).

Assuming that fXtg is an integrated process of order one, there exists, according to Definition 7.1, a causal representation for f�Xtg:

�Xt D ı C ‰.L/Zt with Zt � WN �

0;�2 �

with the property ‰.1/ ¤ 0 and P1jD0 jj jj < 1. Before proceeding to the main theorem, we notice the following simple, but extremely useful polynomial decomposition of ‰.L/:

‰.L/ � ‰.1/ D 1 C 1L C 2L2 C 3L3 C 4L4 C : : : � 1 � 1 � 2 � 3 � 4 � : : :

D 1.L � 1/ C 2.L2 � 1/ C 3.L3 � 1/ C 4.L4 � 1/ C : : : D .L � 1/ 1 C 2.L C 1/ C 3.L2 C L C 1/ C : : :

D .L � 1/ . 1 C 2 C 3 C : : :/ C . 2 C 3 C 4 C : : :/L C . 3 C 4 C 5 C : : :/L2 C : : :

We state this results in the following Lemma:

Lemma 7.1. Let ‰.L/ D PjD0 jLj, then ‰.L/ D ‰.1/ C .L � 1/e‰.L/

where e‰.L/ D P1jD0 e jLj with Q j D P1

iDjC1 i.

4Neusser (2000) shows how a Beveridge-Nelson decomposition can also be derived for higher order integrated processes.

7.1 Definition, Properties and Interpretation 139

As fXtg is integrated and because ‰.1/ ¤ 0 we can express Xt as follows:

Xt D X0 C t X

jD1 �Xj

D X0 C t X

jD1

ı C ‰.1/ C .L � 1/e‰.L/ Zj

D X0 C ıt C ‰.1/ t X

jD1 Zj C

t X

jD1 .L � 1/e‰.L/Zj

D X0 C ıt „ ƒ‚ …

linear trend

C ‰.1/ t X

jD1 Zj

„ƒ‚…

random walk

C e‰.L/Z0 � e‰.L/Zt „ ƒ‚ …

stationary component

This leads to the following theorem.

Theorem 7.1 (Beveridge-Nelson Decomposition). Every integrated process fXtg has a decomposition of the following form:

Xt D X0 C ıt „ ƒ‚ …

linear trend

C ‰.1/ t X

jD1 Zj

„ƒ‚…

random walk

C e‰.L/Z0 � e‰.L/Zt „ ƒ‚ …

stationary component

The above representation is referred to as the Beveridge-Nelson decomposition.

Proof. The only substantial issue is to show that e‰.L/Z0 � e‰.L/Zt defines a stationary process. According to Theorem 6.4 it is sufficient to show that the coefficients of e‰.L/ are absolutely summable. We have that:

1 X

jD0 j Q jj D

1 X

jD0

1 X

iDjC1 i

� 1 X

jD0

1 X

iDjC1 j ij D

1 X

jD1 jj jj < 1;

where the first inequality is a consequence of the triangular inequality and the second inequality follows from the Definition 7.1 of an integrated process. ut

Shocks of a random walk component have a permanent effect. This effect is measured by the persistence ‰.1/, the coefficient of the random walk component. In macroeconomics aggregate supply shocks are ascribed to have a long-run effect as they affect productivity. In contrast monetary or demand shocks are viewed to have temporary effects only. Thus the persistence ‰.1/ can be interpreted as a measure for the importance of supply shocks (see Campbell and Mankiw (1987),

140 7 Integrated Processes

Cochrane (1988) or Christiano and Eichenbaum (1990)). For a critical view from an econometric standpoint see Hauser et al. (1999). A more sophisticated multivariate approach to identify supply and demand shocks and to disentangle their relative importance is provided in Sect. 15.5.

In business cycle analysis it is often useful to decompose fXtg into a sum of a trend component �t and a cyclical component "t:

Xt D �t C "t:

In the case of a difference-stationary series, the cyclical component can be identified with the stationary component in the Beveridge-Nelson decomposition and the trend component with the random walk plus the linear trend. Suppose that f�Xtg follows an ARMA process ˆ.L/�Xt D cC‚.L/Zt then ��t D ıC‰.1/Zt can be identified as the trend component. This means that the trend component can be recursively determined from the observations by applying the formula

�t D ˆ.L/

‚.L/ ‰.1/Xt:

The cyclical component is then simply the residual: "t D Xt � �t . In the above decomposition both the permanent (trend) component as well as

the stationary (cyclical) component are driven by the same shock Zt . A more sophisticated model would, however, allow that the two components are driven by different shocks. This idea is exploited in the so-called structural time series analysis where the different components (trend, cycle, season, and irregular) are modeled as being driven by separated shocks. As only the series fXtg is observed, not its components, this approach leads to serious identification problems. See the discussion in Harvey (1989), Hannan and Deistler (1988), or Mills (2003). In Sects. 17.1 and 17.4.2 we will provide an overall framework to deal with these issues.

Examples Let f�Xtg be a MA(q) process with �Xt D ıCZt C: : :C�qZt�q then the persistence is given simply by the sum of the MA-coefficients: ‰.1/ D 1 C �1 C : : : C �q. Depending on the value of these coefficients. The persistence can be smaller or greater than one.

If f�Xtg is an AR(1) process with �Xt D ı C��Xt�1 C Zt and assuming j�j < 1 then we get: �Xt D ı1�� C

P1 jD0 �

jZt�j. The persistence is then given as ‰.1/ D P1

jD0 � j D 1

1�� . For positive values of �, the persistence is greater than one. Thus, a shock of one is amplified to have an effect larger than one in the long-run.

If f�Xtg is assumed to be an ARMA(1,1) process with �Xt D ıC��Xt�1 CZt C �Zt�1 and j�j < 1 then �Xt D ı1�� C Zt C .� C �/

P1 jD0 �

jZt�j�1. The persistence is therefore given by ‰.1/ D 1 C .� C �/P1jD0 �j D 1C�1�� .

7.2 Properties of the OLS Estimator in the Case of Integrated Variables 141

The computation of the persistence for the model estimated for Swiss GDP in Sect. 5.6 is more complicated because a fourth order difference 1 � L4 has been used instead of a first order one. As 1� L4 D .1� L/.1C L C L2 C L3/, it is possible to extend the above computations also to this case. For this purpose we compute the persistence for .1 C L C L2 C L3/ ln BIPt in the usual way. The long-run effect on ln BIPt is therefore given by ‰.1/=4 because .1 C L C L2 C L3/ ln BIPt is nothing but four times the moving-average of the last four values. For the AR(2) model we get a persistence of 1.42 whereas for the ARMA(1,3) model the persistence is 1.34. Both values are definitely above one so that the permanent effect of a one-percent shock to Swiss GDP is amplified to be larger than one in the long-run. Campbell and Mankiw (1987) and Cochrane (1988) report similar values for the US.

7.2 Properties of the OLS Estimator in the Case of Integrated Variables

The estimation and testing of coefficients of models involving integrated variables is not without complications and traps because the usual asymptotic theory may become invalid. The reason being that the asymptotic distributions are in general no longer normal so that the usual critical values for the test statistics are no longer valid. A general treatment of these issues is beyond this text, but can be found in Banerjee et al. (1993) and Stock (1994). We may, however, illustrate the kind of problems encountered by looking at the Gaussian AR(1) case5:

Xt D �Xt�1 C Zt; t D 1;;2; : : : ;

where Zt � IID N.0;�2/ and X0 D 0. For observations on X1; X2; : : : ; XT the OLS- estimator of � is given by the usual expression:

O�T D PT

tD1 Xt�1Xt PT

tD1 X 2 t�1

D � C PT

tD1 Xt�1Zt PT

tD1 X 2 t�1

For j�j < 1, the OLS estimator of �, O�T , converges in distribution to a normal random variable (see Chap. 5 and in particular Sect. 5.2):

p T �

O�T � � �

d��! N �0;1 � �2� :

The estimated density of the OLS estimator of � for different values of � is represented in Fig. 7.1. This figure was constructed using a Monte-Carlo simulation of the above model for a sample size of T D 100 using 10;000 replications for

5We will treat more general cases in Sect. 7.5 and Chap. 16.

142 7 Integrated Processes

0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 0

true value of φ = 0.8

true value of φ = 0.9

true value of φ = 0.5

true value of φ = 0.95

true value of φ = 1.0

Fig. 7.1 Distribution of the OLS estimator of � for T D 100 and 10,000 replications

each value of �.6 The figure shows that the distribution of O�T becomes more and more concentrated if the true value of � gets closer and closer to one. Moreover the distribution gets also more and more skewed to the left. This implies that the OLS estimator is downward biased and that this bias gets relatively more and more pronounced in small samples as � approaches one.

The asymptotic distribution would be degenerated for � D 1 because the variance approaches zero as � goes to one. Thus the asymptotic distribution becomes useless for statistical inferences under this circumstance. In order to obtain a non-degenerate distribution the estimator must be scaled by T instead by

p T. It

can be shown that

T �

O�T � � �

d��! �:

This result was first established by Dickey and Fuller (1976) and Dickey and Fuller (1981). However, the asymptotic distribution � need no longer be normal. It was first tabulated in Fuller (1976). The scaling with T instead of

p T means that the OLS-

estimator converges, if the true value of � equals one, at a higher rate to � D 1. This property is known as superconsistency.

6The densities were estimated using an adaptive kernel density estimator with Epanechnikov window (see Silverman (1986)).

7.2 Properties of the OLS Estimator in the Case of Integrated Variables 143

In order to understand this result better, in particular in the light of the derivation in the Appendix of Sect. 5.2, we take a closer look at the asymptotic distribution of

T �

O�T � � �

T �

O�T � � �

D 1 �2T

PT tD1 Xt�1Zt

1 �2T2

PT tD1 X

2 t�1

Under the assumption � D 1, Xt becomes a random walk so that Xt can be written as Xt D Zt C : : : C Z1. Moreover, as a sum of normally distributed random variables Xt becomes itself normally distributed as Xt � N.0;�2t/. In addition, we get

X2t D .Xt�1 C Zt/2 D X2t�1 C 2Xt�1Zt C Z2t ) Xt�1Zt D

�

X2t � X2t�1 � Z2t �

) T X

tD1 Xt�1Zt D

X2T � X20 2

� PT

tD1 Z 2 t

) 1 T

T X

tD1 Xt�1Zt D

X2T T

� PT

tD1 Z 2 t

) 1 �2T

T X

tD1 Xt�1Zt D

�

� p

�2

� 1 2�2

PT tD1 Z

2 t

d��! 1 2

�

� 2 1 � 1

�

The numerator therefore converges to a �21 distribution. The distribution of the denominator is more involved, but its expected value is given by:

T X

tD1 X2t�1 D �2

T X

tD1 .t � 1/ D �

2T.T � 1/ 2

;

because Xt�1 � N �

0;�2.t � 1/�. To obtain a nondegenerate random variable one must scale by T2. Thus, intuitively, T. O�T ��/ will no longer converge to a degenerate distribution.

Using similar arguments it can be shown that the t-statistic

tT D O�T � 1

O� O� D

O�T � 1 r

s2T PT

tD1 X 2 t�1

with s2T D 1T�2 PT

tD2 �

Xt � O�T Xt�1 �2

is not asymptotically normal. Its distribution

was also first tabulated by Fuller (1976). Figure 7.2 compares its density with the standard normal distribution in a Monte-Carlo experiment using again a sample of

144 7 Integrated Processes

−4 −3 −2 −1 0 1 2 3 4 0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

t−statistic

Normal distribution

Dickey−Fuller distribution

Fig. 7.2 Distribution of t-statistic for T D 100 and 10,000 replications and standard normal distribution

T D 100 and 10;000 replications. It is obvious that the t-distribution is shifted to the left. This implies that the critical values will be absolutely higher than for the standard case. In addition, one may observe a slight skewness.

Finally, we also want to investigate the autocovariance function of a random walk. Using similar arguments as in Sect. 1.3 we get:

�.h/ D E.XT XT�h/ D E Œ.ZT C ZT�1 C : : : C Z1/.ZT�h C ZT�h�1 C : : : C Z1/ D E�Z2T�h C Z2T�h�1 C : : : C Z21

� D .T � h/�2:

Thus the correlation coefficient between XT and XT�h is:

�.h/ D �.h/p VXT

p VXT�h

D T � hp T.T � h/

D r

T � h T

; h � T:

The autocorrelation coefficient �.h/ therefore monotonically decreases with h, holding the sample size T constant. The rate at which �.h/ falls is, however, smaller than for ARMA processes for which �.h/ declines exponentially fast to zero. Given h, the autocorrelation coefficient converges to one for T ! 1. Figure 7.3 compares the theoretical and the estimated ACF of a simulated random walk with T D 100. Typically, the estimated coefficients lie below the theoretical ones. In addition, we

7.3 Unit-Root Tests 145

0 5 10 15 20 25 −1

−0.5

0.5

order

theoretical ACF of a random walkestimated ACF

of a random walk

estimated ACF of an AR(1) process with φ = 0.9

generated with the same innovations as the random walk

Fig. 7.3 ACF of a random walk with 100 observations

show the estimated ACF for an AR(1) process with � D 0:9 and using the same realizations of the white noise process as in the construction of the random walk. Despite the large differences between the ACF of an AR(1) process and a random walk, the ACF is only of limited use to discriminate between an (stationary) ARMA process and a random walk.

The above calculation also shows that �.1/ < 1 so that the expected value of the OLS estimator is downward biased in finite samples: E O�T < 1.

7.3 Unit-Root Tests

The previous Sects. 7.1 and 7.2 have shown that, depending on the nature of the non-stationarity (trend versus difference stationarity), the stochastic process has quite different algebraic (forecast, forecast error variance, persistence) and statistical (asymptotic distribution of OLS-estimator) properties. It is therefore important to be able to discriminate among these two different types of processes. This also pertains to standard regression models for which the presence of integrated variables can lead to non-normal asymptotic distributions.

The ability to differentiate between trend- and difference-stationary processes is not only important from a statistical point of view, but can be given an economic interpretation. In macroeconomic theory, monetary and demand disturbances are alleged to have only temporary effects whereas supply disturbances, in particular technology shocks, are supposed to have permanent effects. To put it in the language

146 7 Integrated Processes

of time series analysis: monetary and demand shocks have a persistence of zero whereas supply shocks have nonzero (positive) persistence. Nelson and Plosser (1982) were the first to investigate the trend properties of economic time series from this angle. In their influential study they reached the conclusion that, with the important exception of the unemployment rate, most economic time series in the US are better characterized as being difference stationary. Although this conclusion came under severe scrutiny (see Cochrane (1988) and Campbell and Perron (1991)), this issue resurfaces in many economic debates. The latest discussion relates to the nature and effect of technology shocks (see Galí (1999) or Christiano et al. (2003)).

The following exposition focuses on the Dickey-Fuller test (DF-test) and the Phillips-Perron test(PP-test). Although other test procedures and variants thereof have been developed in the meantime, these two remain the most widely applied in practice. These types of tests are also called unit-root tests.

Both the DF- as well as the PP-test rely on a regression of Xt on Xt�1 which may include further deterministic regressors like a constant or a linear time trend. We call this regression the Dickey-Fuller regression:

Xt D deterministic

variables C �Xt�1 C Zt: (7.1)

Alternatively and numerically equivalent, one may run the Dickey-Fuller regression in difference form:

�Xt D deterministic

variables C ˇXt�1 C Zt

with ˇ D � � 1. For both tests, the null hypothesis is that the process is integrated of order one, difference stationary, or has a unit-root. Thus we have

H0 W � D 1 or ˇ D 0:

The alternative hypothesis H1 is that the process is trend-stationary or stationary with constant mean and is given by:

H1 W �1 < � < 1 or � 2 < ˇ D � � 1 < 0:

Thus the unit root test is a one-sided test. The advantage of the second formulation of the Dickey-Fuller regression is that the corresponding t-statistic can be readily read off from standard outputs of many computer packages which makes additional computations unnecessary.

7.3 Unit-Root Tests 147

7.3.1 The Dickey-Fuller Test (DF-Test)

The Dickey-Fuller test comes in two forms. The first one, sometimes called the �- test, takes T. O� � 1/ as the test statistic. As shown previously, this statistic is no longer asymptotically normally distributed. However, it was first tabulated by Fuller and can be found in textbooks like Fuller (1976) or Hamilton (1994b). The second and much more common one relies on the usual t-statistic for the hypothesis � D 1:

t O� D . O�T � 1/=O� O�:

This test-statistic is also not asymptotically normally distributed. It was for the first time tabulated by Fuller (1976) and can be found, for example, in Hamilton (1994b). Later MacKinnon (1991) presented much more detailed tables where the critical values can be approximated for any sample size T by using interpolation formulas (see also Banerjee et al. (1993)).7

The application of the Dickey-Fuller test as well as the Phillips-Perron test is obfuscated by the fact that the asymptotic distribution of the test statistic (�- or t- test) depends on the specification of the deterministic components and on the true data generating process. This implies that depending on whether the Dickey-Fuller regression includes, for example, a constant and/or a time trend and on the nature of the true data generating process one has to use different tables and thus different critical values. In the following we will focus on the most common cases listed in Table 7.1.

In case 1 the Dickey-Fuller regression includes no deterministic component. Thus, a rejection of the null hypothesis implies that fXtg has to be a mean zero stationary process. This specification is, therefore, only warranted if one can make sure that the data have indeed mean zero. As this is rarely the case, except, for example, when the data are the residuals from a previous regression,8 case 1 is

Table 7.1 The four most important cases for the unit-root test

Data generating process Estimated regression �-test:

(null hypothesis) T. O� � 1/ (Dickey-Fuller regression) t-test Xt D Xt�1 C Zt Xt D �Xt�1 C Zt Case 1 Case 1 Xt D Xt�1 C Zt Xt D ˛ C �Xt�1 C Zt Case 2 Case 2 Xt D ˛ C Xt�1 C Zt; Xt D ˛ C �Xt�1 C Zt N(0,1) ˛ ¤ 0 Xt D ˛ C Xt�1 C Zt Xt D ˛ C ıt

C�Xt�1 C Zt Case 4 Case 4

7These interpolation formula are now implemented in many software packages, like EVIEWS, to compute the appropriate critical values. 8This fact may pose a problem by itself.

148 7 Integrated Processes

very uncommon in practice. Thus, if the data do not display a trend, which can be checked by a simple time plot, the Dickey-Fuller regression should include a constant. A rejection of the null hypothesis then implies that fXtg is a stationary process with mean � D c

1�� . If the data display a time trend, the Dickey-Fuller regression should also include a linear time trend as in case 4. A rejection of the null hypothesis then implies that the process is trend-stationary. In the case that the Dickey-Fuller regression contains no time trend and there is no time trend under the alternative hypothesis, asymptotic normality holds. This case is only of theoretical interest as it should a priori be clear whether the data are trending or not. In the instance where one is not confident about the trending nature of the time series see the procedure outlined in Sect. 7.3.3.

In the cases 2 and 4 it is of interest to investigate the joint hypothesis H0 W ˛ D 0 and � D 1, and H0 W ı D 0 and � D 1 respectively. Again the corresponding F-statistic is no longer F-distributed, but has been tabulated (see Hamilton (1994b, Table B7)). The trade-off between t- and F-test is discussed in Sect. 7.3.3.

Most economic time series display a significant amount of autocorrelation. To take this feature into account it is necessary to include lagged differences �Xt�1; : : : ;�Xt�pC1 as additional regressors. The so modified Dickey-Fuller regression then becomes:

Xt D deterministic

variables C �Xt�1 C �1�Xt�1 C : : : C �p�1�Xt�pC1 C Zt:

This modified test is called the augmented Dickey-Fuller test (ADF-test). This autoregressive correction does not change the asymptotic distribution of the test statistics. Thus the same tables can be used as before. For the coefficients of the autoregressive terms asymptotic normality holds. This implies that the standard testing procedures (t-test, F-test) can be applied in the usual way. This is true if instead of autoregressive correction terms moving-average terms are used instead (see Said and Dickey (1984)).

For the ADF-test the order p of the model should be chosen such that the residuals are close to being white noise. This can be checked, for example, by looking at the ACF of the residuals or by carrying out a Ljung-Box test (see Sect. 4.2). In case of doubt, it is better to choose a higher order. A consistent procedure to find the right order is to use the Akaike’s criterion (AIC). Another alternative strategy advocated by Ng and Perron (1995) is an iterative testing procedure which makes use of the asymptotic normality of the autoregressive correction terms. Starting from a maximal order p � 1 D pmax, the method amounts to the test whether the coefficient corresponding to the highest order is significantly different from zero. If the null hypothesis that the coefficient is zero is not rejected, the order of the model is reduced by one and the test is repeated. This is done as long as the null hypothesis is not rejected. If the null hypothesis is finally rejected, one sticks with the model and performs the ADF-test. The successive test are standard t-tests. It is advisable to use a rather high significance level, for example a 10 % level. The simulation results by Ng and Perron (1995) show that this procedure leads to a smaller bias compared to using the AIC criterion and that the reduction in power remains negligible.

7.3 Unit-Root Tests 149

7.3.2 The Phillips-Perron Test (PP-Test)

The Phillips-Perron test represents a valid alternative to the ADF-test. It is based on the simple Dickey-Fuller regression (without autoregressive correction terms) and corrects for autocorrelation by modifying the OLS-estimate or the corresponding value of the t-statistic. The simple Dickey-Fuller regression with either constant and/or trend is:

Xt D deterministic

variables C �Xt�1 C Zt;

where fZtg need no longer be a white noise process, but can be any mean zero stationary process. fZtg may, for example, be an ARMA process. In principle, the approach also allows for heteroskedasticity.9

The first step in the Phillips-Perron unit-root test estimates the above appropri- ately specified Dickey-Fuller regression. The second step consists in the estimation of the unconditional variance �Z.0/ and the long-run variance J of the residuals OZt. This can be done using one of the methods prescribed in Sect. 4.4. These two estimates are then used in a third step to correct the �- and the t-test statistics. This correction would then take care of the autocorrelation present in the data. Finally, one can use the so modified test statistics to carry out the unit-root test applying the same tables for the critical values as before.

In case 1 where no deterministic components are taken into account (see case 1 in Table 7.1) the modified test statistics according to Phillips (1987) are:

�-Test W T �

O� � 1 �

� 1 2

�

bJT � O�Z.0/ �

T X

tD1 X2t�1

!�1

t-Test W s

O�Z.0/ bJT

t O� � 1

�

bJT � O�Z.0/ �

bJT T2

T X

tD1 X2t�1

!�1=2 :

If fZt g would be white noise so that J D �.0/, respectively bJT � O�Z.0/ one gets the ordinary Dickey-Fuller test statistic. Similar formulas can be derived for the cases 2 and 4. As already mentioned these modifications will not alter the asymptotic distributions so the same critical values as for the ADF-test can be used.

The main advantage of the Phillips-Perron test is that the non-parametric correc- tion allows for very general fZtg processes. The PP-test is particularly appropriate if fZtg has some MA-components which can be only poorly approximated by low order autoregressive terms. Another advantage is that one can avoid the exact modeling of the process. It has been shown by Monte-Carlo studies that the PP-test has more power compared to the DF-test, i.e. the PP-test rejects the null hypothesis more often when it is false, but that, on the other hand, it has also a higher size distortion, i.e. that it rejects the null hypothesis too often.

9The exact assumptions can be read in Phillips (1987) and Phillips and Perron (1988).

150 7 Integrated Processes

7.3.3 Unit-Root Test: Testing Strategy

Independently whether the Dickey-Fuller or the Phillips-Perron test is used, the specification of the deterministic component is important and can pose a problem in practice. On the one hand, if the deterministic part is underrepresented, for example when only a constant, but no time trend is used, the test results are biased in favor of the null hypothesis, if the data do indeed have a trend. On the other hand, if too many deterministic components are used, the power of the test is reduced. It is therefore advisable to examine a plot of the series in order to check whether a long run trend is visible or not. In some circumstances economic reasoning may help in this regard.

Sometimes, however, it is difficult to make an appropriate choice a priori. We therefore propose the following testing strategy based on Elder and Kennedy (2001).

Xt has a long-run trend: As Xt grows in the long-run, the Dickey-Fuller regres- sion

Xt D ˛ C ıt C �Xt�1 C Zt should contain a linear trend.10 In this case either � D 1, ı D 0 and ˛ ¤ 0 (unit root case) or � < 1 with ı ¤ 0 (trend stationary case). We can then test the joint null hypothesis

H0 W � D 1 and ı D 0

by a corresponding F-test. Note that the F-statistic, like the t-test, is not distributed according to the F-distribution. If the test does not reject the null, we conclude that fXtg is a unit root process with drift or equivalently a difference- stationary (integrated) process. If the F-test rejects the null hypothesis, there are three possible situations:

(i) The possibility � < 1 and ı D 0 contradicts the primary observation that fXtg has a trend and can therefore be eliminated.

(ii) The possibility � D 1 and ı ¤ 0 can also be excluded because this would imply that fXtg has a quadratic trend, which is unrealistic.

(iii) The possibility � < 1 and ı ¤ 0 represents the only valid alternative. It implies that fXtg is stationary around a linear trend, i.e. that fXtg is trend- stationary.

Similar conclusions can be reached if, instead of the F-test, a t-test is used to test the null hypothesis H0 W � D 1 against the alternative H1 W � < 1. Thereby a non-rejection of H0 is interpreted that ı D 0. If, however, the null hypothesis H0 is rejected, this implies that ı ¤ 0, because fXtg exhibits a long-run trend.

10In case of the ADF-test additional regressors, �Xt�j; j > 0, might be necessary.

7.3 Unit-Root Tests 151

The F-test is more powerful than the t-test. The t-test, however, is a one-sided test, which has the advantage that it actually corresponds to the primary objective of the test. In Monte-Carlo simulations the t-test has proven to be marginally superior to the F-test.

Xt has no long-run trend: In this case ı D 0 and the Dickey-Fuller regression should be run without a trend11:

Xt D ˛ C �Xt�1 C Zt:

Thus we have either � D 1 and ˛ D 0 or � < 1 and ˛ ¤ 0. The null hypothesis in this case therefore is

H0 W � D 1 and ˛ D 0:

A rejection of the null hypothesis can be interpreted in three alternative ways:

(i) The case � < 1 and ˛ D 0 can be eliminated because it implies that fXtg would have a mean of zero which is unrealistic for most economic time series.

(ii) The case � D 1 and ˛ ¤ 0 can equally be eliminated because it implies that fXtg has a long-run trend which contradicts our primary assumption.

(iii) The case � < 1 and ˛ ¤ 0 is the only realistic alternative. It implies that the time series is stationary around a constant mean given by ˛

1�� .

As before one can use, instead of a F-test, a t-test of the null hypothesis H0 W � D 1 against the alternative hypothesis H1 W � < 1. If the null hypothesis is not rejected, we interpret this to imply that ˛ D 0. If, however, the null hypothesis H0 is rejected, we conclude that ˛ ¤ 0. Similarly, Monte-Carlo simulations have proven that the t-test is superior to the F-test.

The trend behavior of Xt is uncertain: This situation poses the following prob- lem. Should the data exhibit a trend, but the Dickey-Fuller regression contains no trend, then the test is biased in favor of the null hypothesis. If the data have no trend, but the Dickey-Fuller regression contains a trend, the power of the test is reduced. In such a situation one can adapt a two-stage strategy. Estimate the Dickey-Fuller regression with a linear trend:

Xt D ˛ C ıt C �Xt�1 C Zt:

Use the t-test to test the null hypothesis H0 W � D 1 against the alternative hypothesis H1 W � < 1. If H0 is not rejected, we conclude the process has a unit root with or without drift. The presence of a drift can then be investigated by a simple regression of �Xt against a constant followed by a simple t-test of the

11In case of the ADF-test additional regressors, �Xt�j; j > 0, might be necessary.

152 7 Integrated Processes

null hypothesis that the constant is zero against the alternative hypothesis that the constant is nonzero. As �Xt is stationary, the usual critical values can be used.

If the t-test rejects the null hypothesis H0, we conclude that there is no unit root. The trend behavior can then be investigated by a simple t-test of the hypothesis H0 W ı D 0. In this test the usual critical values can be used as fXtg is already viewed as being stationary.

7.3.4 Examples of Unit-Root Tests

As our first example, we examine the logged real GDP for Switzerland, ln.BIPt/, where we have adjusted the series for seasonality by taking a moving-average. The corresponding data are plotted in Fig. 1.3. As is evident from this plot, this variable exhibits a clear trend so that the Dickey-Fuller regression should include a constant and a linear time trend. Moreover, f� ln.BIPt/g is typically highly autocorrelated which makes an autoregressive correction necessary. One way to make this correction is by augmenting the Dickey-Fuller regression by lagged f� ln.BIPt/g as additional regressors. Thereby the number of lags is determined by AIC. The corresponding result is reported in the first column of Table 7.2. It shows that AIC chooses only one autoregressive correction term. The value of t- test statistic is �3:110 which is just above the 5-% critical value. Thus, the null hypothesis is not rejected. If the autoregressive correction is chosen according to the method proposed by Ng and Perron five autoregressive lags have to be included. With this specification, the value of the t-test statistic is clearly above the critical value, implying that the null hypothesis of the presence of a unit root cannot be rejected (see second column in Table 7.2).13 The results of the ADF-tests is confirmed by the PP-test (column 3 in Table 7.2) with quadratic spectral kernel function and band width 20:3 chosen according to Andrews’ formula (see Sect. 4.4).

The second example, examines the three-month LIBOR, fR3Mt g. The series is plotted in Fig. 1.4. The issue whether this series has a linear trend or not is not easy to decide. On the one hand, the series clearly has a negative trend over the sample period considered. On the other hand, a negative time trend does not make sense from an economic point of view because interest rates are bounded from below by zero. Because of this uncertainty, it is advisable to include in the Dickey-Fuller regression both a constant and a trend to be on the safe side. Column 5 in Table 7.2 reports the corresponding results. The value of the t-statistic of the PP-test with Bartlett kernel function and band width of 5 according to the Newey-West rule of thumb is �2:142 and thus higher than the corresponding 5-% critical of �3:435.

12Eventually, one must correct the corresponding standard deviation by taking the autocorrelation in the residual into account. This can be done by using the long-run variance. In the literature this correction is known as the Newey-West correction. 13The critical value changes slightly because the inclusion of additional autoregressive terms changes the sample size.

7.4 Generalizations of Unit-Root Tests 153

Table 7.2 Examples of unit root tests

ln.BIPt/ ln.BIPt/ ln.BIPt/ R3Mt R3Mt Test ADF ADF PP PP PP

Autoregressive Ng and Quadratic

correction AIC Perron spectral Bartlett Bartlett

Band width 20.3 5 5

˛ 0.337 0.275 0.121 0.595 �0.014 ı 0.0001 0.0001 0.0002 �0.0021 � 0.970 0.975 0.989 0.963 �0.996 �1 0.885 1.047

�2 �0.060 �3 �0.085 �4 �0.254 �5 0.231

t O� �3.110 �2.243 �1.543 �2.142 �0.568

Critical value �3.460 �3.463 �3.460 �3.435 �2.878 (5 %)

Critical values from MacKinnon (1996)

Thus, we cannot reject the null hypothesis of the presence of a unit root. We therefore conclude that the process fR3Mt g is integrated of order one, respectively difference-stationary. Based on this conclusion, the issue of the trend can now be decided by running a simple regression of �R3Mt against a constant. This leads to the following results:

�R3Mt D �0.0315 (0.0281)

C et:

where et denotes the least-squares residual. The mean of �R3Mt is therefore �0:0315. This value is, however, statistically not significantly different from zero as indicated by the estimated standard error in parenthesis. Note that this estimate of the standard error has been corrected for autocorrelation (Newey-West correction). Thus, fR3Mtg is not subject to a linear trend. One could have therefore run the Dickey-Fuller regression without the trend term. The result of corresponding to this specification is reported in the last column of Table 7.2. It confirms the presence of a unit root.

7.4 Generalizations of Unit-Root Tests

7.4.1 Structural Breaks in the Trend Function

As we have seen, the unit-root test depends heavily on the correct specification of the deterministic part. Most of the time this amounts to decide whether a linear trend is present in the data or not. In the previous section we presented a rule how

154 7 Integrated Processes

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.5

1.5

2.5

3.5

4.5

5.5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.5

1.5

2.5

3.5

4.5

5.5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0

a b

Fig. 7.4 Three types of structural breaks at TB. (a) Level shift. (b) Change in slope. (c) Level shift and change in slope

to proceed in case of uncertainty about the trend. Sometimes, however, the data exhibit a structural break in their deterministic component. If this structural break is ignored, the unit-root test is biased in favor of the null hypothesis (i. e. in favor of a unit root) as demonstrated by Perron (1989). Unfortunately, the distribution of the test statistic under the null hypothesis, in our case the t-statistic, depends on the exact nature of the structural break and on its date of occurrence in the data. Following Perron (1989) we concentrate on three exemplary cases: a level shift, a change in the slope (change in the growth rate), and a combination of both possibilities. Figure 7.4 shows the three possibilities assuming that a break occurred in period TB. Thereby an AR(1) process with � D 0:8 was superimposed on the deterministic part.

The unit-root test with the possibility of a structural break in period TB is carried out using the Dickey-Fuller test. Thereby the date of the structural break is assumed to be known. This assumption, although restrictive, is justifiable in many applications. The first oil price shock in 1973 or the German reunification in 1989 are examples of structural breaks which can be dated exactly. Other examples would

7.4 Generalizations of Unit-Root Tests 155

Table 7.3 Dickey-Fuller regression allowing for structural breaks

Model A: Level Shift

H0 W Xt D ˛ C 1ftDTB C1gıB C Xt�1 C Zt H1 W Xt D ˛ C ıt C 1ft>TB g.˛B � ˛/ C �Xt�1 C Zt , � < 1

Model B: Change in Slope (Change in Growth Rate)

H0 W Xt D ˛ C 1ft>TB g.˛B � ˛/ C Xt�1 C Zt H1 W Xt D ˛ C ıt C 1ft>TB g.ıB � ı/.t � TB/ C �Xt�1 C Zt , � < 1

Model C: Level Shift and Change in Slope

H0 W Xt D ˛ C 1ftDTB C1gıB C 1ft>TB g.˛B � ˛/ C Xt�1 C Zt H1 W Xt D ˛ C ıt C 1ft>TB g.˛B � ˛/ C 1ft>TB g.ıB � ı/.t � TB/ C�Xt�1 C Zt; � < 1

1 ftDTB C1g and 1ft>TB g denotes the indicator function which takes the value one if the

condition is satisfied and the value zero otherwise

include changes in the way the data are constructed. These changes are usually documented by the data collecting agencies. Table 7.3 summarizes the three variants of Dickey-Fuller regression allowing for structural breaks.14

Model A allows only for a level shift. Under the null hypothesis the series undergoes a one-time shift at time TB. This level shift is maintained under the null hypothesis which posits a random walk. Under the alternative, the process is viewed as being trend-stationary whereby the trend line shifts parallel by ˛B � ˛ at time TB. Model B considers a change in the mean growth rate from ˛ to ˛B at time TB. Under the alternative, the slope of time trend changes from ı to ıB. Model C allows for both types of break to occur at the same time.

The unit-root test with possible structural break for a time series Xt, t D 0;1; : : : ; T, is implemented in two stages as follows. In the first stage, we regress Xt on the corresponding deterministic component using OLS. The residuals eX0;eX1; : : : ;eXT from this regression are then used to carry out a Dickey-Fuller test:

eXt D �eXt�1 C Zt; t D 1; : : : ; T:

The distribution of the corresponding t-statistic under the null hypothesis depends not only on the type of the structural break, but also on the relative date of the break in the sample. Let this relative date be parameterized by D TB=T. The asymptotic distribution of the t-statistic has been tabulated by Perron (1989). This table can be used to determine the critical values for the test. These critical values are smaller than those from the normal Dickey-Fuller table. Using a 5 % significance level, the critical values range between �3:80 and �3:68 for model A, between �3:96 and �3:65 for model B, and between �4:24 and �3:75 for model C, depending on the value of . These values also show that the dependence on is only weak.

In the practical application of the test one has to control for the autocorrelation in the data. This can be done by using the Augmented Dickey-Fuller (ADF) test. This

14See Eq. (7.1) and Table 7.1 for comparison.

156 7 Integrated Processes

amounts to the introduction of �eXt�j, t D 1;2; : : : ; p � 1, as additional regressors in the above Dickey-Fuller regression. Thereby the order p can be determined by Akaike’s information criterion (AIC) or by the iterative testing procedure of Ng and Perron (1995). Alternatively, one may use, instead of the ADF test, the Phillips- Perron test. In this case one computes the usual t-statistic for the null hypothesis � D 1 and corrects it using the formulas in Phillips and Perron (1988) as explained in Sect. 7.3.2. Which of the two methods is used, is irrelevant for the determination of the critical values which can be extracted from Perron (1989).

Although it may be legitimate in some cases to assume that the time of the structural break is known, we cannot take this for granted. It is therefore important to generalize the test allowing for an unknown date for the occurrence of a structural break. The work of Zivot and Andrews (1992) has shown that the procedure proposed by Perron can be easily expanded in this direction. We keep the three alternative models presented in Table 7.3, but change the null hypothesis to a random walk with drift with no exogenous structural break. Under the null hypothesis, fXtg is therefore assumed to be generated by

Xt D ˛ C Xt�1 C Zt; Zt � WN.0;�2/: The time of the structural TB, respectively D TB=T, is estimated in such a way that fXtg comes as close as possible to a trend-stationary process. Under the alternative hypothesis fXtg is viewed as a trend-stationary process with unknown break point. The goal of the estimation strategy is to chose TB, respectively , in such a way that the trend-stationary alternative receives the highest weight. Zivot and Andrews (1992) propose to estimate by minimizing the value of the t-statistic t O�. / under the hypothesis � D 1:

t O�.O inf/ D inf 2ƒ

t O�. / (7.2)

where ƒ is a closed subinterval of .0;1/.15 The distribution of the test statistic under the null hypothesis for the three cases is tabulated in Zivot and Andrews (1992). This table then allows to determine the appropriate critical values for the test. In practice, one has to take the autocorrelation of the time series into account by one of the methods discussed previously.

This testing strategy can be adapted to determine the time of a structural break in the linear trend irrespective of whether the process is trend-stationary or integrated of order one. The distributions of the corresponding test statistics have been tabulated by Vogelsang (1997).16

15Taking the infimum over ƒ instead over .0;1/ is for theoretical reasons only. In practice, the choice of ƒ plays no important role. For example, one may take ƒ D Œ0:01; 0:99 . 16See also the survey by Perron (2006).

7.4 Generalizations of Unit-Root Tests 157

7.4.2 Testing for Stationarity (KPSS Test)

The unit-root tests we discussed so far tested the null hypothesis that the process is integrated of order one against the alternative hypothesis that the process is integrated of order zero (i.e. is stationary). However, one may be interested in reversing the null and the alternative hypothesis and test the hypothesis of stationarity against the alternative that the process is integrated of order one. Such a test has been proposed by Kwiatkowski et al. (1992), called the KPSS-Test. This test rests on the idea that according to the Beveridge-Nelson decomposition (see Sect. 7.1.4) each integrated process of order one can be seen as the sum of a linear time trend, a random walk and a stationary process:

Xt D ˛ C ıt C d t X

jD1 Zj C Ut;

where fUtg denotes a stationary process. If d D 0 then the process becomes trend- stationary, otherwise it is integrated of order one.17 Thus, one can state the null and the alternative hypothesis as follows:

H0 W d D 0 against H1 W d ¤ 0:

Denote by fStg the process of partial sums obtained from the residuals fetg of a regression of Xt against a constant and a linear time trend, i.e. St D

Pt jD1 ej.

Under the null hypothesis d D 0, fStg is integrated of order one whereas under the alternative fStg is integrated of order two. Based on this consideration Kwiatkowski et al. propose the following test statistic for a time series consisting of T observations:

KPSS test statistic: WT D PT

tD1 S 2 t

T2bJT (7.3)

wherebJT is an estimate of the long-run variance of fUtg (see Sect. 4.4). As fStg is an integrated process under the null hypothesis, the variance of fStg grows linearly in t (see Sect. 1.4.4 or 7.2) so that the sum of squared St diverges at rate T

2. Thus, the test statistic remains bounded and can be shown to converge. Note that the test statistic is independent from further nuisance parameters. Under the alternative hypothesis, however, fStg is integrated of order two. Thus, the null hypothesis will be rejected for large values of WT . The corresponding asymptotic critical values of the test statistic are reported in Table 7.4.

17If the data exhibit no trend, one can set ı equal to zero. 18This auxiliary regression may include additional exogenous variables.

158 7 Integrated Processes

Table 7.4 Critical values of the KPSS test

Regression without time trend

Significance level 0.1 0.05 0.01

Critical value 0.347 0.463 0.739

Regression with time trend

Significance level 0.1 0.05 0.01

Critical value 0.119 0.146 0.216

See Kwiatkowski et al. (1992)

7.5 Regression with Integrated Variables

7.5.1 The Spurious Regression Problem

The discussion on the Dickey-Fuller and Phillips-Perron tests showed that in a regression of the integrated variables Xt on its past Xt�1 the standard

p T -

asymptotics no longer apply. A similar conclusion also holds if we regress an integrated variable Xt against another integrated variable Yt. Suppose that both processes fXtg and fYtg are generated as a random walk:

Xt D Xt�1 C Ut; Ut � IID.0;�2U/ Yt D Yt�1 C Vt; Vt � IID.0;�2V/

where the processes fUtg and fVtg are uncorrelated with each other at all leads and lags. Thus,

E.UtVs/ D 0; for all t; s 2 Z:

Consider now the regression of Yt on Xt and a constant:

Yt D ˛ C ˇXt C "t:

As fXtg and fYtg are two random walks which are uncorrelated with each other by construction, one would expect that the OLS-estimate of the coefficient of Xt, Ǒ, should tend to zero as the sample size T goes to infinity. The same is expected for the coefficient of determination R2. This is, however, not true as has already been remarked by Yule (1926) and, more recently, by Granger and Newbold (1974). The above regression will have a tendency to “discover” a relationship between Yt and Xt despite the fact that there is none. This phenomenon is called spurious correlation or spurious regression. Similarly, unreliable results would be obtained by using a simple t-test for the null hypothesis ˇ D 0 against the alternative hypothesis ˇ ¤ 0. The reason for these treacherous findings is that the model is incorrect under the null as well as under the alternative hypothesis. Under the null hypothesis f"tg is an integrated process which violates the standard assumption for OLS. The alternative hypothesis is not true by construction. Thus, OLS-estimates should be

7.5 Regression with Integrated Variables 159

interpreted with caution when a highly autocorrelated process fYtg is regressed on another highly correlated process fXtg. A detailed analysis of the spurious regression problem is provided by Phillips (1986).

The spurious regression problem can be illustrated by a simple Monte Carlo study. Specifying Ut � IIDN.0;1/ and Vt � IIDN.0;1/, we constructed N D 1000 samples for fYtg and fXtg of size T D 1000 according to the specification above. The sample size was chosen especially large to demonstrate that this is not a small sample issue. As a contrast, we constructed two independent AR(1) processes with AR-coefficients �X D 0:8 and �Y D �0:5.

Figures 7.5a,b show the drastic difference between a regression with stationary variables and integrated variables. Whereas the distribution of the OLS-estimates of ˇ is highly concentrated around the true value ˇ D 0 in the stationary case, the distribution is very flat in the case of integrated variables. A similar conclusion holds for the corresponding t-value. The probability of obtaining a t-value greater than 1:96 is bigger than 0:9. This means that in more than 90 % of the time the t-statistic leads to a rejection of the null hypothesis and therefore suggests a relationship between Yt and Xt despite their independence. In the stationary case, this probability turns out to be smaller than 0:05. These results are also reflected in the coefficient of determination R2. The median R2 is approximately 0:17 in the case of the random walks, but only 0:0002 in the case of AR(1) processes.

The problem remains the same if fXtg and fYtg are specified as random walks with drift:

Xt D ıX C Xt�1 C Ut; Ut � IID.0;�2U/ Yt D ıY C Yt�1 C Vt; Vt � IID.0;�2V/

where fUtg and fVtg are again independent from each other at all leads and lags. The regression would be same as above:

Yt D ˛ C ˇXt C "t:

7.5.2 Bivariate Cointegration

The spurious regression problem cannot be circumvented by first testing for a unit root in Yt and Xt and then running the regression in first differences in case of no rejection of the null hypothesis. The reason being that a regression in the levels of Yt and Xt may be sensible even when both variables are integrated. This is the case when both variables are cointegrated. The concept of cointegration goes back to Engle and Granger (1987) and initiated a literal research boom. We will give a more general definition in Chap. 16 when we deal with multivariate time series. Here we stick to the case of two variables and present the following definition.

Definition 7.2 (Cointegration, Bivariate). Two stochastic processes fXtg and fYtg are called cointegrated if the following two conditions are fulfilled:

160 7 Integrated Processes

−2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 0

25 stationary

AR(1) processes

random walks

−100 −80 −60 −40 −20 0 20 40 60 80 100 0

0.1

0.2

0.3

0.4

0.5

0.6

random walks

random walk

stationary AR(1) processes

Fig. 7.5 Distribution of OLS-estimate Ǒ and t-statistic t Oˇ

for two independent random walks and

two independent AR(1) processes. (a) Distribution of Ǒ. (b) Distribution of t Oˇ . (c) Distribution of

Ǒ and t-statistic t Oˇ

(i) fXtg and fYtg are both integrated processes of order one, i.e. Xt � I.1/ and Yt � I.1/;

(ii) there exists a constant ˇ ¤ 0 such that fYt � ˇXtg is a stationary process, i.e. fYt � ˇXtg � I.0/.

The issue whether two integrated processes are cointegrated can be decided on the basis of a unit root test. Two cases can be distinguished. In the first one, ˇ is

7.5 Regression with Integrated Variables 161

assumed to be known. Thus, one can immediately apply the augmented Dickey- Fuller (ADF) or the Phillips-Perron (PP) test to the process fYt � ˇXt g. Thereby the same issue regarding the specification of the deterministic part arises. The critical values can be retrieved from the usual tables (for example from MacKinnon 1991). In the second case, ˇ is not known and must be estimated from the data. This can be done running, as a first step, a simple (cointegrating) regression of Yt on Xt including a constant and/or a time trend.19 Thereby the specification of the deterministic part follows the same rules as before. The unit root test is then applied, in the second step, to the residuals from this regression. As the residuals have been obtained from a preceding regression, we are faced with the so-called “generated regressor problem”.20 This implies that the usual Dickey-Fuller tables can no longer be used, instead the tables provided by Phillips and Ouliaris (1990) become the relevant ones. As before, the corresponding asymptotic distribution depends on the specification of the deterministic part in the cointegrating regression. If this regression included a constant, the residuals have necessary a mean of zero so that the Dickey-Fuller regression should include no constant (case 1 in Table 7.1):

et D �et�1 C �t where et and �t denote the residuals from the cointegrating and the residuals of the Dickey-Fuller regression, respectively. In most applications it is necessary to correct for autocorrelation which can be done by including additional lagged differences �O"t�1; : : : ;�O"t�pC1 as in the ADF-test or by adjusting the t-statistic as in the PP- test. The test where ˇ is estimated from a regression is called the regression test for cointegration. Note that if the two series are cointegrated then the OLS estimate of ˇ is (super) consistent.

In principle it is possible the generalize this single equation approach to more than two variables. This encounters, however, some conceptual problems. First, there is the possibility of more than one linearly independent cointegrating rela- tionships which cannot be detected by a single regression. Second, the dependent variable in the regression may not be part of the cointegrating relation which might involves only the other variables. In such a situation the cointegrating regression is again subject to the spurious regression problem. These issues turned the interest of the profession towards multivariate approaches. Chapter 16 presents alternative procedures and discusses the testing, estimation, and interpretation of cointegrating relationships in detail.

19Thereby, in contrast to ordinary OLS regressions, it is irrelevant which variable is treated as the left hand, respectively right hand variable. 20This problem was first analyzed by Nicholls and Pagan (1984) and Pagan (1984) in a stationary context.

162 7 Integrated Processes

An Example for Bivariate Cointegration As an example, we consider the relation between the short-term interest rate, fR3Mtg, and inflation, fINFLt g, in Switzerland over the period January 1989 to February 2012. As the short-term interest rate we take the three month LIBOR. Both time series are plotted in Fig. 7.6a. As they are integrated according to the unit root tests (not shown here), we can look for cointegration. The cointegrating regression delivers:

INFLt D �0:0088 C 0:5535 R3Mt C et; R2 D 0:7798: (7.4) The residuals from this regression, denoted by et, are represented in Fig. 7.6b. The ADF unit root test of these residuals leads to a value of �3:617 for the t-statistic. Thereby an autoregressive correction of 13 lags was necessary according to the AIC criterion. The corresponding value of the t-statistic resulting from the PP unit root test using a Bartlett window with band width 7 is �4:294. Taking a significance level of 5 %, the critical value according to Phillips and Ouliaris (1990, Table IIb) is �3:365.21 Thus, the ADF as well as the PP test reject the null hypothesis of a unit root in the residuals. This implies that inflation and the short-term interest rate are cointegrated.

7.5.3 Rules to Deal with Integrated Times Series

The previous sections demonstrated that the handling of integrated variables has to be done with care. We will therefore in this section examine some rules of thumb which should serve as a guideline in practical empirical work. These rules are summarized in Table 7.5. In that this section follows very closely the paper by Stock and Watson (1988b) (see also Campbell and Perron 1991).22 Consider the linear regression model:

Yt D ˇ0 C ˇ1X1;t C : : : C ˇK XK;t C "t: (7.5) This model is usually based on two assumptions:

(1) The disturbance term "t is white noise and is uncorrelated with any regressor. This is, for example, the case if the regressors are deterministic or exogenous.

(2) All regressors are either deterministic or stationary processes.

If Eq. (7.5) represents the true data generating process, fYtg must be a stationary process. Under the above assumptions, the OLS-estimator is consistent and the OLS-estimates are asymptotically normally distributed so that the corresponding t- and F-statistics will be approximately distributed as t- and F- distributions.

21For comparison, the corresponding critical value according to MacKinnon (1991) is �2:872. 22For a thorough analysis the interested reader is referred to Sims et al. (1990).

7.5 Regression with Integrated Variables 163

1990 1995 2000 2005 2010 −2

10 p er

ce n t

inflation three−month LIBOR

1990 1995 2000 2005 2010 −2

−1.5

−1

−0.5

0.5

1.5

2.5

Fig. 7.6 Cointegration of inflation and three-month LIBOR. (a) Inflation and three-month LIBOR. (b) Residuals from cointegrating regression

164 7 Integrated Processes

Consider now the case that assumption 2 is violated and that some or all regressors are integrated, but that instead one of the two following assumptions holds:

(2.a) The relevant coefficients are coefficients of mean-zero stationary variables. (2.b) Although the relevant coefficients are those of integrated variables, the

regression can be rearranged in such a way that the relevant coefficients become coefficients of mean-zero stationary variables.

Under assumptions 1 and 2.a or 2.b the OLS-estimator remains consistent. Also the corresponding t- and F-statistics remain valid so that the appropriate critical values can be retrieved from the t-, respectively F-distribution. If neither assumption 2.a nor 2.b holds, but the following assumption:

(2.c) The relevant coefficients are coefficients of integrated variables and the regression cannot be rewritten in a way that they become coefficients of stationary variables.

If assumption 1 remains valid, but assumption 2.c holds instead of 2.a and 2.b, the OLS-estimator is still consistent. However, the standard asymptotic theory for the t- and the F-statistic fails so that they become useless for normal statistical inferences.

If we simply regress one variable on another in levels, the error term "t is likely not to follow a white noise process. In addition, it may even be correlated with some regressors. Suppose that we replace assumption 1 by:

(1.a) The integrated dependent variable is cointegrated with at least one integrated regressor such that the error term is stationary, but may remain autocorrelated or correlated with the regressors.

Under assumptions 1.a and 2.a, respectively 2.b, the regressors are stationary, but correlated with the disturbance term, in this case the OLS-estimator becomes incon- sistent. This situation is known as the classic omitted variable bias, simultaneous equation bias or errors-in-variable bias. However, under assumptions 1.a and 2.c, the OLS-estimator is consistent for the coefficients of interest. However, the standard asymptotic theory fails. Finally, if both the dependent variable and the regressors are integrated without being cointegrated, then the disturbance term is integrated and the OLS-estimator becomes inconsistent. This is the spurious regression problem treated in Sect. 7.5.1.

Example: Term Structure of Interest We illustrate the above rules of thumb by investigating again the relation between inflation (fINFLtg) and the short-term interest rate (fR3Mt g). In Sect. 7.5.2 we found that the two variables are cointegrated with coefficient Ǒ D 0:5535 (see Eq. (7.4)). In a further step we want to investigate a dynamic relation between the two variables and estimate the following equation:

7.5 Regression with Integrated Variables 165

Table 7.5 Rules of thumb in regressions with integrated processes

Remarks

Assumptions OLS-estimator Consistency Standard asymptotics

(1) (2) Yes Yes Classic results for OLS

(1) (2.a) Yes Yes

(1) (2.b) Yes Yes

(1) (2.c) Yes No

(1.a) (2.a) No No Omitted variable bias

(1.a) (2.b) No No Omitted variable bias

(1.a) (2.c) Yes No

Neither (1) nor (1.a) (2.c) No No Spurious regression

Source: Stock and Watson (1988b); results for the coefficients of interest

R3Mt D c C �1R3Mt�1 C �2R3Mt�2 C �3R3Mt�3 C ı1INFLt�1 C ı2INFLt�2 C "t where "t � WN.0;�2/. In this regression we want to test the hypotheses �3 D 0 against �3 ¤ 0 and ı1 D 0 against ı1 ¤ 0 by examining the corresponding simple t-statistics. Note that we are in the context of integrated variables so that the rules of thumb summarized in Table 7.5 apply. We can rearrange the above equation as

�R3Mt D c C .�1 C �2 C �3 � 1/R3Mt�1 � .�2 C �3/�R3Mt�1 � �3�R3Mt�2

C ı1INFLt�1 C ı2INFLt�2 C "t:

�3 is now a coefficient of a stationary variable in a regression with a stationary dependent variables. In addition "t � WN.0;�2/ so that assumptions (1) und (2.b) are satisfied. We can therefore use the ordinary t-statistic to test the hypothesis �3 D 0 against �3 ¤ 0. Note that it is not necessary to actually carry out the rearrangement of the equation. All relevant item can be retrieved from the original equation.

To test the hypothesis ı1 D 0, we rearrange the equation to yield:

�R3Mt D c C .�1 C ı1 Ǒ � 1/R3Mt�1 C �2R3Mt�2 C �3R3Mt�3

C ı1.INFLt�1 � ǑR3Mt�1/ C ı2INFLt�2 C "t:

As fR3Mtg and fINFLt g are cointegrated, INFLt�1 � ǑR3Mt�1 is stationary. Thus assumptions (1) and (2.b) hold again and we use once more the simple t-test. As before it is not necessary to actually carry out the transformation.

8Models of Volatility

The prices of financial market securities are often shaken by large and time-varying shocks. The amplitudes of these price movements are not constant. There are periods of high volatility and periods of low volatility. Within these periods volatility seems to be positively autocorrelated: high amplitudes are likely to be followed by high amplitudes and low amplitudes by low amplitudes. This observation which is particularly relevant for high frequency data such as, for example, daily stock market returns implies that the conditional variance of the one-period forecast error is no longer constant (homoskedastic), but time-varying (heteroskedastic). This insight motivated Engle (1982) and Bollerslev (1986) to model the time-varying variance thereby triggering a huge and still growing literature.1 The importance of volatility models stems from the fact that the price of an option crucially depends on the variance of the underlying security price. Thus with the surge of derivative markets in the last decades the application of such models has seen a tremendous rise. Another use of volatility models is to assess the risk of an investment. In the computation of the so-called value at risk (VaR), these models have become an indispensable tool. In the banking industry, due to the regulations of the Basel accords, such assessments are in particular relevant for the computation of the required equity capital backing-up assets of different risk categories.

The following exposition focuses on the class of autoregressive conditional heteroskedasticity models (ARCH models) and their generalization the generalized autoregressive conditional heteroskedasticity models (GARCH models). These

1Robert F. Engle III was awarded the Nobel prize in 2003 for his work on time-varying volatility. His Nobel lecture (Engle 2004) is a nice and readable introduction to this literature.

167

168 8 Models of Volatility

models form the basis for even more generalized models (see Bollerslev et al. (1994) or Gouriéroux (1997)). Campbell et al. (1997) provide a broader economically motivated approach to the econometric analysis of financial market data.

8.1 Specification and Interpretation

8.1.1 Forecasting Properties of AR(1)-Models

Models of volatility play an important role in explaining the behavior of financial market data. They start from the observation that periods of high (low) volatility are clustered in specific time intervals. In these intervals high (low) volatility periods are typically followed by high (low) volatility periods. Thus volatility is usually positively autocorrelated as can be observed in Fig. 8.3. In order to understand this phenomenon we recapitulate the forecasting properties of the AR(1) model.2

Starting from the model

Xt D c C �Xt�1 C Zt; Zt � IID.0;�2/ and j�j < 1; the best linear forecast in the mean-squared-error sense of XtC1 conditional on fXt; Xt�1; : : :g, denoted by PtXtC1, is given by (see Chap. 3)

PtXtC1 D c C �Xt: In practice the parameters c and � are replaced by an estimate.

The conditional variance of the forecast error then becomes:

Et .XtC1 � PtXtC1/2 D Et Z2tC1 D �2;

where Et denotes the conditional expectation operator based on information Xt; Xt�1; : : :. The conditional variance of the forecast error is therefore constant, irrespective of the current state.

The unconditional forecast is simply the expected value of EXtC1 D � D c1�� with forecast error variance:

�

XtC1 � c

1 � � �2

D E�ZtC1 C �Zt C �2Zt�1 C : : : �2 D �

1 � �2 > � 2:

Thus the conditional as well as the unconditional variance of the forecast error are constant. In addition, the conditional variance is smaller and thus more precise because it uses more information. Similar arguments can be made for ARMA models in general.

2Instead of assuming Zt � WN.0;�2/, we make for convenience the stronger assumption that Zt � IID.0;�2/.

8.1 Specification and Interpretation 169

8.1.2 The ARCH(1) Model

The volatility of financial market prices exhibit a systematic behavior so that the conditional forecast error variance is no longer constant. This observation led Engle (1982) to consider the following simple model for heteroskedasticity (non-constant variance).

Definition 8.1 (ARCH(1) Model). A stochastic process fZtg, t 2 Z, is called an autoregressive conditional heteroskedastic process of order one, ARCH(1) process, if it is the solution of the following stochastic difference equation:

Zt D �t q

˛0 C ˛1Z2t�1 with ˛0 > 0 and 0 < ˛1 < 1; (8.1)

where �t � IID N.0;1/ and where �t and Zt�1 are independent from each other for all t 2 Z.

We will discuss the implications of this simple model below and consider generalizations in the next sections. First we prove the following theorem.

Theorem 8.1. Under conditions stated in the definition of the ARCH(1) process, the difference equation (8.1) possesses a unique and strictly stationary solution with EZ2t < 1. This solution is given by

Zt D �t

t˛0

@1 C 1 X

jD1 ˛

j 1� 2 t�1�

2 t�2 : : :�

2 t�j

A: (8.2)

Proof. Define the process

Yt D Z2t D �2t .˛0 C ˛1Yt�1/ (8.3)

Iterating backwards k times we get:

Yt D ˛0�2t C ˛1�2t Yt�1 D ˛0�2t C ˛1�2t �2t�1.˛0 C ˛1Yt�2/ D ˛0�2t C ˛0˛1�2t �2t�1 C ˛21�2t �2t�1Yt�2

: : :

D ˛0�2t C ˛0˛1�2t �2t�1 C : : : C ˛0˛k1�2t �2t�1 : : :�2t�k C ˛kC11 �2t �2t�1 : : :�2t�kYt�k�1:

170 8 Models of Volatility

Define the process fY 0t g as

Y 0t D ˛0�2t C ˛0 1 X

jD1 ˛

j 1� 2 t �

2 t�1 : : :�

2 t�j:

The right-hand side of the above expression just contains nonnegative terms. Moreover, making use of the IID N.0;1/ assumption of f�tg,

EY 0t D E.˛0�2t / C ˛0E 0

1 X

jD1 ˛

j 1 �2t �

2 t�1 : : :�

2 t�j

D ˛0 1 X

jD0 ˛

j 1 D

˛0

1 � ˛1 :

Thus, 0 � Y 0t < 1 a.s. Therefore, fY 0t g is strictly stationary and satisfies the difference equation (8.3). This implies that Zt D

Y 0t is also strictly stationary and satisfies the difference equation (8.1).

To prove uniqueness, we follow Giraitis et al. (2000). For any fixed t, it follows from the definitions of Yt and Y

0 t that for any k � 1

jYt � Y 0t j � ˛kC11 �2t �2t�1 : : :�2t�kjYt�k�1j C ˛0 1 X

jDkC1 ˛

j 1� 2 t �

2 t�1 : : :�

2 t�j:

The expectation of the right-hand side is bounded by

�

EjY1j C ˛0

1 � ˛1 �

˛kC11 :

Define the event Ak by Ak D fjYt � Y 0t j > 1=kg. Then,

P.Ak/ � kEjYt � Y 0t j � k �

EjY1j C ˛0

1 � ˛1 �

˛ kC1 1

where the first inequality follows from Chebyschev’s inequality setting r D 1 (see Theorem C.3). Thus,

P1 kD1 P.Ak/ < 1. The Borel-Cantelli lemma (see

Theorem C.4) then implies that PfAk i:o:g D 0. However, as Ak AkC1, P.Ak/ D 0 for any k. Thus, Yt D Y 0t a.s. ut

Remark 8.1. Note that the normality assumption is not necessary for the proof. The assumption �t � IID.0;1/ would be sufficient. Indeed, in practice it has been proven useful to adopt distributions with fatter tail than the normal, like the t-distribution (see the discussion in Sect. 8.1.3).

8.1 Specification and Interpretation 171

Given the assumptions made above fZtg has the following properties:

(i) The expected value of Zt is:

EZt D E�tE q

˛0 C ˛1Z2t�1 D 0: This follows from the assumption that �t and Zt�1 are independent.

(ii) The covariances between Zt and Zt�h, EZt Zt�h, for h ¤ 0 are given by:

EZt Zt�h D E �

�t

˛0 C ˛1Z2t�1 �t�h q

˛0 C ˛1Z2t�h�1 �

D E�t�t�h E q

˛0 C ˛1Z2t�1 q

˛0 C ˛1Z2t�h�1 D 0:

This is also a consequence of the independence assumption between �t and Zt�1, respectively between �t�h and Zt�h�1.

(iii) The variance of Zt is:

VZt D EZ2t D E�2t �

˛0 C ˛1Z2t�1 �

D E�2t E �

˛0 C ˛1Z2t�1 � D ˛0

1 � ˛1 < 1:

This follows from the independence assumption between �t and Zt�1 and from the stationarity of fZtg. Because ˛0 > 0 and 0 < ˛1 < 1, the variance is always strictly positive and finite.

(iv) As �t is normally distributed, its skewness, E� 3 t , equals zero. The independence

assumption between �t and Z 2 t�1 then implies that the skewness of Zt is also

zero, i.e.

EZ3t D 0: Zt therefore has a symmetric distribution.

The properties (i), (ii) and (iii) show that fZt g is a white noise process. According to Theorem 8.1 it is not only stationary but even strictly stationary. Thus fZtg is uncorrelated with Zt�1; Zt�2; : : :, but not independent from its past! In particular we have:

E.Zt jZt�1; Zt�2; : : :/ D Et�t q

˛0 C ˛1Z2t�1 D 0 V.Zt jZt�1; Zt�2; : : :/ D E

�

Z2t jZt�1; Zt�2; : : : �

D Et�2t �

˛0 C ˛1Z2t�1 � D ˛0 C ˛1Z2t�1:

The conditional variance of Zt therefore depends on Zt�1. Note that this dependence is positive because ˛1 > 0.

172 8 Models of Volatility

In order to guarantee that this conditional variance is always positive, we must postulate that ˛0 > 0 and ˛1 > 0. The stability of the difference equation requires in addition that ˛1 < 1.

3 Thus high volatility in the past, a large realization of Zt�1, is followed by high volatility in the future. The precision of the forecast, measured by the conditional variance of the forecast error, thus depends on the history of the process. This feature is not compatible with linear models and thus underlines the non-linear character of the ARCH model and its generalizations.

Despite the fact that �t was assumed to be normally distributed, Zt is not normally distributed. Its distribution deviates from the normal distribution in that extreme realizations are more probable. This property is called the heavy-tail property. In particular we have4:

EZ4t D E�4t �

˛0 C ˛1Z2t�1 �2 D E�4t

�

˛20 C 2˛0˛1Z2t�1 C ˛21Z4t�1 �

D 3˛20 C 6˛20˛1

1 � ˛1 C 3˛21EZ4t�1:

The strict stationarity of fZtg implies EZ4t D EZ4t�1 so that

.1 � 3˛21/EZ4t D 3˛20.1 C ˛1/ 1 � ˛1

EZ4t D 1

1 � 3˛21 � 3˛

2 0.1 C ˛1/ 1 � ˛1

EZ4t is therefore positive and finite if and only if 3˛ 2 1 < 1, respectively if 0 < ˛1 <

1= p 3 D 0:5774. For high correlation of the conditional variance, i.e. high ˛1 >

1= p 3, the fourth moment and therefore also all higher even moments will no longer

exist. The kurtosis � is

� D EZ 4 t

ŒEZ2t 2

D 3 � 1 � ˛ 2 1

1 � 3˛21 > 3;

if EZ4t exists. The heavy-tail property manifests itself by a kurtosis greater than 3 which is the kurtosis of the normal distribution. The distribution of Zt is therefore leptokurtic and thus more vaulted than the normal distribution.

Finally, we want to examine the autocorrelation function of Z2t . This will lead to a test for ARCH effects, i.e. for time varying volatility (see Sect. 8.2 below).

3The case ˛1 D 1 is treated in Sect. 8.1.4. 4As �t � N.0;1/ its even moments, m2k D E�2kt , k D 1;2; : : :, are given by m2k D

Qk jD1.2j � 1/.

Thus we get m4 D 3, m6 D 15, etc. As the normal distribution is symmetric, all odd moments are equal to zero.

8.1 Specification and Interpretation 173

Theorem 8.2. Assuming that EZ4t exists, Yt D Z 2 t ˛0

has the same autocorrelation function as the AR(1) process Wt D ˛1Wt�1 C Ut with Ut � WN.0;1/. In addition, under the assumption 0 < ˛1 < 1, the process fWtg is also causal with respect to fUtg.

Proof. From Yt D �2t .1 C ˛1Yt�1/ we get:

�Y.h/ D EYt Yt�h � EYtEYt�h D EYt Yt�h � 1

.1 � ˛1/2

D E�2t .1 C ˛1Yt�1/ Yt�h � 1

.1 � ˛1/2

D EYt�h C ˛1EYt�1Yt�h � 1

.1 � ˛1/2

D 1 1 � ˛1

C ˛1 �

�Y.h � 1/ C 1

.1 � ˛1/2 �

� 1 .1 � ˛1/2

D ˛1�Y.h � 1/ C 1 � ˛1 C ˛1 � 1 .1 � ˛1/2

D ˛1�Y.h � 1/:

Therefore, �Y.h/ D ˛h1�Y.0/ ) �.h/ D ˛h1. ut

The unconditional variance of Xt is:

VXt D V 0

1 � � C 1 X

jD0 �jZt�j

A D 1 1 � �2 VZt D

˛0

1 � ˛1 � 1 1 � �2:

The unconditional variance of Xt involves all parameters of the model. Thus modeling the variance of Xt induces a trade-off between �, ˛0 and ˛1.

Figure 8.1 plots the realizations of two AR(1)-ARCH(1) processes.Both pro- cesses have been generated with the same realization of f�tg and the same parameters � D 0:9 and ˛0 D 1. Whereas the first process (shown on the left panel of the figure) was generated with a value of ˛1 D 0:9 , the second one had a value of ˛1 D 0:5. In both cases the stability condition, ˛1 < 1, is fulfilled, but for the first process 3˛21 > 1, so that the fourth moment does not exist. One can clearly discern the large fluctuations, in particular for the first process.

8.1.3 General Models of Volatility

The simple ARCH(1) model can be and has been generalized in several directions. A straightforward generalization proposed by Engle (1982) consists by allowing further lags to enter the ARCH equation (8.1). This leads to the ARCH(p) model:

174 8 Models of Volatility

0 20 40 60 80 100

−5

Zt with α1 = 0.9

0 20 40 60 80 100

−5

Zt with α1 = 0.5

0 20 40 60 80 100

−5

Xt = 0.9 Xt−1 + Zt

0 20 40 60 80 100

−5

Xt = 0.9 Xt−1 + Zt

Fig. 8.1 Simulation of two ARCH(1) processes (˛1 D 0:9 and ˛1 D 0:5)

ARCH.p/ W Zt D �t�t with �2t D ˛0 C p X

jD1 ˛jZ

2 t�j (8.4)

where ˛0 � 0, ˛j � 0 and �t � IID N.0;1/ with �t independent from Zt�j, j � 1. A further popular generalization was proposed by Bollerslev (1986):

GARCH.p; q/ W Zt D �t�t with �2t D ˛0 C p X

jD1 ˛jZ

2 t�j C

q X

jD1 ˇj�

2 t�j (8.5)

where we assume ˛0 � 0, ˛j � 0, ˇj � 0 and �t � IID N.0;1/ with �t independent from Zt�j, j � 1, as before. This model is analogous the ordinary ARMA model and allows for a parsimonious specification of the volatility process. All coefficients should be positive to guarantee that the variance is always positive. In addition it can be shown (see for example Fan and Yao (2003, 150) and the literature cited therein) that fZtg is (strictly) stationary with finite variance if and only if

Pp jD1 ˛j C

Pq jD1 ˇj < 1.

5 Under this condition fZtg � WN.0;�2Z/ with

�2Z D V.Zt/ D ˛0

1 � PpjD1 ˛j � Pq

jD1 ˇj :

5A detailed exposition of the GARCH(1,1) model is given in Sect. 8.1.4.

8.1 Specification and Interpretation 175

As �t is still normally distributed the uneven moments of the distribution of Zt are zero and the distribution is thus symmetric. The fourth moment of Zt, EZ

4 t , exists if

p 3

Pp jD1 ˛j

1 � PqjD1 ˇj < 1:

This condition is sufficient, but not necessary.6 Furthermore, fZtg is a white noise process with heavy-tail property if fZtg is strictly stationary with finite fourth moment.

In addition, fZ2t g is a causal and invertible ARMA.max fp; qg; q/ process satisfy- ing the following difference equation:

Z2t D ˛0 C p X

jD1 ˛jZ

2 t�j C

q X

jD1 ˇj�

2 t�j C et

D ˛0 C maxfp;qg X

jD1 .˛j C ˇj/Z2t�j C et �

q X

jD1 ˇjet�j;

where ˛pCj D ˇqCj D 0 for j � 1 and “error term”

et D Z2t � �2t D .�2t � 1/ 0

@˛0 C p X

jD1 ˛jZ

2 t�j C

q X

jD1 ˇj�

2 t�j

A :

Note, however, there is a circularity here because the noise process fetg is defined in terms of Z2t and is therefore not an exogenous process driving Z

2 t . Thus, one has

to be precautious in the interpretation of fZ2t g as an ARMA process. Further generalizations of the GARCH(p,q) model can be obtained by allowing

deviations from the normal distribution for �t. In particular, distributions such as the t-distribution which put more weight on extreme values have become popular. This seems warranted as prices on financial markets exhibit large and sudden fluctuations.7

The Threshold GARCH Model Assuming a symmetric distribution for �t and specifying a linear relationship between �2t and Z

2 t�j bzw. �

2 t�j, j > 0, leads to a symmetric distribution for Zt .

It has, however, been observed that downward movements seem to be different from

6Zadrozny (2005) derives a necessary and sufficient condition for the existence of the fourth moment. 7A thorough treatment of the probabilistic properties of GARCH processes can be found in Nelson (1990), Bougerol and Picard (1992a), Giraitis et al. (2000), Klüppelberg et al. (2004, theorem 2.1), and Lindner (2009).

176 8 Models of Volatility

upward movements. This asymmetric behavior is accounted for by the asymmetric GARCH(1,1) model or threshold GARCH(1,1) model (TGARCH(1,1) model). This model was proposed by Glosten et al. (1993) and Zakoïan (1994):

asymmetric GARCH.1; 1/ W Zt D �t�t with �2t D ˛0 C ˛1Z2t�1 C ˇ�2t�1

C �1fZt�1<0gZ2t�1: 1fZt�1<0g denotes the indicator function which takes on the value one if Zt�1 is negative and the value zero otherwise. Assuming, as before, that all parameters ˛0, ˛1, ˇ and � are greater than zero, this specification postulates a leverage effect because negative realizations have a greater impact than positive ones. In order to obtain a stationary process the condition ˛1 C ˇ C �=2 < 1 must hold. This model can be generalized in an obvious way by allowing additional lags Z2t�j and �

2 t�j, j > 1

to enter the above specification.

The Exponential GARCH Model Another interesting and popular class of volatility models was introduced by Nelson (1991). The so-called exponential GARCH models or EGARCH models are defined as follows:

log �2t D ˛0 C ˇ log �2t�1 C � ˇ

Zt�1 �t�1

C ı Zt�1 �t�1

D ˛0 C ˇ log �2t�1 C �j�t�1j C ı�t�1: Note that, in contrast to the previous specifications, the dependent variable is the logarithm of �2t and not �

2 t itself. This has the advantage that the variance is always

positive irrespective of the values of the coefficients. Furthermore, the leverage effect is exponential rather than quadratic because Zt D �t exp.�t=2/. The EGARCH model is also less recursive than the GARCH model as the volatility is specified directly in terms of the noise process f�tg. Thus, the above EGARCH model can be treated as an AR(1) model of log �2t with noise process �j�t�1j Cı�t�1. It is obvious that the model can be generalized to allow for additional lags both in �2t and �t. This results in an ARMA process for flog �2t g for which the usual conditions for the existence of a causal and invertible solution can be applied (see Sect. 2.3). A detailed analysis and further properties of this model class can be found in Bollerslev et al. (1994), Gouriéroux (1997) and Fan and Yao (2003, 143–180).

The ARCH-in-Mean Model The ARCH-in-mean model or ARCH-M model was introduced by Engle et al. (1987) to allow for a feedback of volatility into the mean equation. More specifi- cally, assume for the sake of simplicity that the variance equation is just represented the ARCH(1) model

Zt D �t�t with �2t D ˛0 C ˛1Z2t�1:

8.1 Specification and Interpretation 177

Then, the ARCH-M model is given

Xt D Mtˇ C g.�2t / C Zt (8.6) where g is a function of the volatility �2t and where M

0 t consists of a vector of

regressors, including lagged values of Xt. If Mt D .1; Xt�1/ then we get the AR(1)- ARCH-M model. The most commonly used specification for g is a linear function: g.�2t / D ı0 C ı1�2t . In the asset pricing literature, higher volatility would require a higher return to compensate the investor for the additional risk. Thus, if Xt denotes the return on some asset, we expect ı1 to be positive. Note that any time variation in �2t translates into a serial correlation of fXtg (see Hong 1991, for details). Of course, one could easily generalize the model to allow for more sophisticated mean and variance equations.

8.1.4 The GARCH(1,1) Model

The Generalized Autoregressive Conditional Heteroskedasticity model of order (1,1), GARCH(1,1) model for short, is considered as a benchmark for more general specifications and often serves as a starting point for further empirical investigations. We therefore want to explore its properties in more detail. Many of its properties generalize in a straightforward way to the GARCH(p,q) process. According to Eq. (8.5) the GARCH(1,1) model is defined as:

GARCH.1; 1/ W Zt D �t�t with �2t D ˛0 C ˛1Z2t�1 C ˇ�2t�1 (8.7) where ˛0;˛1; and ˇ � 0. We assume ˛1 C ˇ > 0 to avoid the degenerate case ˛1 D ˇ D 0 which implies that fZtg is just a sequence of IID random variables. Moreover, �t � IID.0;1/ with �t being independent of Zt�j, j � 1. Note that we do not make further distributional assumption. In particular, �t need not required to be normally distributed. For this model, we can formulate a similar theorem as for the ARCH(1) model (see Theorem: 8.1):

Theorem 8.3. Let fZtg be a GARCH(1,1) process as defined above. Under the assumption

E log.˛1� 2 t C ˇ/ < 0;

the difference equation (8.7) possess strictly stationary solution:

Zt D �t v

t˛0

1 X

jD0

j Y

iD1 .˛1�

2 t�i C ˇ/

where Qj

i D 1 whenever i > j. The solution is also unique given the sequence f�tg. The solution is unique and (weakly) stationary with variance EZ2t D ˛01�˛1�ˇ < 1 if ˛1 C ˇ < 1.

178 8 Models of Volatility

Proof. The proof proceeds similarly to Theorem 8.1. For this purpose, we define Yt D �2t and rewrite the GARCH(1,1) model as

Yt D ˛0 C ˛1�2t�1Yt�1 C ˇYt�1 D ˛0 C .˛1�2t�1 C ˇ/Yt�1:

This defines an AR(1) process with time-varying coefficients �t D ˛1�2t C ˇ � 0. Iterate this equation backwards k times to obtain:

Yt D ˛0 C ˛0�t�1 C : : : C ˛0�t�1 : : :�t�k C �t�1 : : :�t�k�t�k�1Yt�k�1

D ˛0 k X

jD0

j Y

iD1 �t�i C

kC1 Y

iD1 �t�iYt�k�1:

Taking the limit k ! 1, we define the process fY 0t g

Y 0t D ˛0 1 X

jD0

j Y

iD1 �t�i: (8.8)

The right-hand side of this expression converges almost surely as can be seen from the following argument. Given that �t � IID and given the assumption E log.˛1�2t C ˇ/ < 0, the strong law of large numbers (Theorem C.5) implies that

lim sup j!1

j X

iD1 log.�t�i/

< 0 a.s.,

or equivalently,

lim sup j!1

log

j Y

iD1 �t�i

!1=j

< 0 a.s.

Thus,

lim sup j!1

j Y

iD1 �t�i

!1=j

< 1 a.s.

The application of the root test then shows that the infinite series (8.8) converges almost surely. Thus, fY 0t g is well-defined. It is easy to see that fY 0t g is strictly stationary and satisfies the difference equation. Moreover, if ˛1 C ˇ < 1, we get

EY 0t D ˛0 1 X

jD0 E

j Y

iD1 �t�i D ˛0

1 X

jD0 .˛1 C ˇ/j D

˛0

1 � ˛1 � ˇ :

8.1 Specification and Interpretation 179

Thus, EZ2t D ˛01�˛1�ˇ < 1. To show uniqueness, we assume that there exists another strictly stationary

process fYtg which also satisfies the difference equation. This implies that

jYt � Y 0t j D j�t�1j jYt�1 � Y 0t�1j D

k Y

iD1 �t�i

jYt�k � Y 0t�k j

�

k Y

iD1 �t�i

jYt�kj C

k Y

iD1 �t�i

jY 0t�k j

The assumption E log �t D E log.˛1�2t C ˇ/ < 0 together with the strong law of large numbers (Theorem C.5) imply

k Y

iD1 �t�i D

exp

k X

iD1 log �t�i

!!k

�! 0 a.s.

As both solutions are strictly stationary so that the distribution of jYt�kj and jY 0t�kj do not depend on t, this implies that both

�

Qk iD1 �t�i

�

jYt�kj and �

Qk iD1 �t�i

�

jY 0t�kj converge in probability to zero. Thus, Yt D Y 0t a.s. once the sequence �t , respectively �t, is given. Because Zt D �t

Y 0t this completes the proof. ut

Remark 8.2. Using Jensen’s inequality, we see that

E log.˛1� 2 t C ˇ/ � log E.˛1�2t C ˇ/ D log.˛1 C ˇ/:

Thus, the condition ˛1 C ˇ < 1 is sufficient, but not necessary, to ensure the existence of a strictly stationary solution. Thus even when ˛1 C ˇ D 1, a strictly stationary solution exists, albeit one with infinite variance. This case is known as the IGARCH model and is discussed below. In the case ˛1 C ˇ < 1, the Borel-Cantelli lemma can be used as Theorem 8.1 to establish the uniqueness of the solution. Further details can be found in the references listed in footnote 7.

Assume that ˛1 Cˇ < 1, then a unique strictly stationary process fZtg with finite variance which satisfies the above difference equation exists. In particular Zt � WN.0;�2Z/ such that

V.Zt/ D ˛0

1 � ˛1 � ˇ :

180 8 Models of Volatility

0 0.5 1 1.5 2 2.5 3 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1 β

α1

α1 + β = 1

E log(α1νt 2 + β) = 0

sqrt(3) α1 / (1−β) = 1

region I

region II

region III

region IV

Fig. 8.2 Parameter region for which a strictly stationary solution to the GARCH(1,1) process exists assuming �t � IID N.0;1/

The assumption 1�˛1�ˇ > 0 guarantees that the variance exists. The third moment of Zt is zero due to the assumption of a symmetric distribution for �t . The condition for the existence of the fourth moment is:

p 3 ˛1 1�ˇ < 1.

8 The kurtosis is then

� D EZ 4 t

ŒEZ2t 2

D 3 � 1 � .˛1 C ˇ/ 2

1 � .˛1 C ˇ/2 � 2˛21 > 3

if EZ4t exists. 9 Therefore the GARCH(1,1) model also possesses the heavy-tail

property because Zt is more peaked than the normal distribution. Figure 8.2 shows how the different assumptions and conditions divide up the

parameter space. In region I, all conditions are fulfilled. The process has a strictly stationary solution with finite variance and kurtosis. In region II, the kurtosis does no longer exist, but the variance does as ˛1 C ˇ < 1 still holds. In region III, the process has infinite variance, but a strictly stationary solution yet exists. In region IV, no such solution exists.

Viewing the equation for �2t as a stochastic difference equation, its solution is given by

�2t D ˛0

1 � ˇ C ˛1 1 X

jD0 ˇjZ2t�1�j: (8.9)

8A necessary and sufficient condition is .˛1 C ˇ/2 C 2˛21 < 1 (see Zadrozny (2005)). 9The condition for the existence of the fourth moment implies 3˛21 < .1 � ˇ/2 so that the denominator 1 � ˇ2 � 2˛1ˇ � 3˛21 > 1 � ˇ2 � 2˛1ˇ � 1 � ˇ2 C 2ˇ D 2ˇ.1 � ˛1 � ˇ/ > 0.

8.1 Specification and Interpretation 181

This expression is well-defined because 0 < ˇ < 1 so that the infinite sum converges. The conditional variance given the infinite past is therefore equal to

V.Zt jZt�1; Zt�2; : : :/ D E �

Z2t jZt�1; Zt�2; : : : � D ˛0

1 � ˇ C ˛1 1 X

jD0 ˇjZ2t�1�j:

Thus, the conditional variance depends on the entire history of the time series and not just on Zt�1 as in the case of the ARCH(1) model. As all coefficients are assumed to be positive, the clustering of volatility is more persistent than for the ARCH(1) model.

Defining a new time series fetg by et D Z2t ��2t D .�2t �1/.˛0 C˛1Z2t�1 Cˇ�2t�1/, one can verify that Z2t obeys the stochastic difference equation

Z2t D ˛0 C ˛1Z2t�1 C ˇ�2t�1 C et D ˛0 C ˛1Z2t�1 C ˇ.Z2t�1 � et�1/ C et D ˛0 C .˛1 C ˇ/Z2t�1 C et � ˇet�1: (8.10)

This difference equation defines an ARMA(1,1) process if et has finite variance which is the case if the fourth moment of Zt exists. In this case, it is easy to verify that fetg is white noise. The so-defined ARMA(1,1) process is causal and invertible with respect to fetg because 0 < ˛1 C ˇ < 1 and 0 < ˇ < 1. The autocorrelation function (ACF), �Z2.h/, can be computed using the methods laid out Sect. 2.4. This gives

�Z2.1/ D .1 � ˇ2 � ˛1ˇ/˛1 1 � ˇ2 � 2˛1ˇ

D .1 � ˇ'/.' � ˇ/ 1 C ˇ2 � 2'ˇ ;

�Z2.h/ D .˛1 C ˇ/�Z2.h � 1/ D '�Z2.h � 1/; h D 2;3; : : : (8.11)

with ' D ˛1 C ˇ (see also Bollerslev (1988)).

The IGARCH Model Practice has shown that the sum ˛1 C ˇ is often close to one. Thus, it seems interesting to examine the limiting case where ˛1 C ˇ D 1. This model was proposed by Engle and Bollerslev (1986) and was termed the integrated GARCH (IGARCH) model in analogy to the notion of integrated processes (see Chap. 7). From Eq. (8.10) we get

Z2t D ˛0 C Z2t�1 C et � ˇet�1 with et D Z2t � �2t D .�2t � 1/.˛0 C .1 � ˇ/Z2t�1 C ˇ�2t�1/. As fetg is white noise, the squared innovations Z2t behave like a random walk with a MA(1) error term. Although the variance of Zt becomes infinite, the difference equation still allows for a strictly stationary solution provided that E log.˛1�

2 t C ˇ/ < 0 (see Theorem 8.3

182 8 Models of Volatility

and the citations in footnote 7 for further details).10 It has been shown by Lumsdaine (1986) and Lee and Hansen (1994) that standard inferences can still be applied although ˛1 C ˇ D 1. The model may easily generalized to higher lag orders.

Forecasting On many occasions it is necessary to obtain forecasts of the conditional variance �2t . An example is given in Sect. 8.4 where the value at risk (VaR) of a portfolio several periods ahead must be evaluated. Denote by Pt�

2 tCh the h period ahead forecast based

on information available in period t. We assume that predictions are based on the infinite past. Then the one-period ahead forecast based on Zt and �

2 t , respectively �t

and �2t , is:

Pt� 2 tC1 D ˛0 C ˛1Z2t C ˇ�2t D ˛0 C .˛1�2t C ˇ/�2t : (8.12)

As �t � IID.0;1/ and independent of Zt�j, j � 1,

Pt� 2 tC2 D ˛0 C .˛1 C ˇ/Pt�2tC1:

Thus, forecast for h � 2 can be obtained recursively as follows:

Pt� 2 tCh D ˛0 C .˛1 C ˇ/Pt�2tCh�1

D ˛0 h�2 X

jD0 .˛1 C ˇ/j C .˛1 C ˇ/h�1Pt�2tC1: (8.13)

Assuming ˛1 C ˇ < 1, the second term in the above expression vanishes as h goes to infinity. Thus, the contribution of the current conditional variance vanishes when we look further and further into the future. The forecast of the conditional variance then approaches the unconditional one: limh!1 Pt�2tCh D ˛01�˛1�ˇ . If ˛1 C ˇ D 1 as in the IGARCH model, the contribution of the current conditional variance is constant, but diminishes to zero relative to the first term. Finally, if ˛1 C ˇ > 1, the two terms are of the same order and we have a particularly persistent situation.

In practice, the parameters of the model are unknown and have therefore be replaced by an estimate. The method can be easily adapted for higher order models. Instead of using the recursive approach outlined above, it is possible to use simulation methods by drawing repeatedly from the actual empirical distribution of the O�t D OZt=O�t. This has the advantage to capture deviations from the underlying distributional assumptions (see Sect. 8.4 for a comparison of both methods). Such methods must be applied if nonlinear models for the conditional variance, like the TARCH model, are employed.

10As the variance becomes infinite, the IGARCH process is an example of a stochastic process which is strictly stationary, but not stationary.

8.2 Tests for Heteroskedasticity 183

8.2 Tests for Heteroskedasticity

Before modeling the volatility of a time series it is advisable to test whether heteroskedasticity is actually present in the data. For this purpose the literature proposed several tests of which we are going to examine two. For both tests the null hypothesis is that there is no heretoskedasticity i.e. that there are no ARCH effects. These tests can also be useful in a conventional regression setting.

8.2.1 Autocorrelation of Quadratic Residuals

The first test is based on the autocorrelation function of squared residuals from a preliminary regression. This preliminary regression or mean regression produces a seriesbZt which should be approximately white noise if the equation is well specified. Then we can look at the ACF of the squared residuals fbZ2t g and apply the Ljung-Box test (see Eq. (4.4)). Thus the test can be broken down into three steps.

(i) Estimate an ARMA model for fXtg and retrieve the residuals bZt from this model. Compute bZ2t . These data can be used to estimate �

2 as

O�2 D 1 T

T X

tD1 bZ2t

Note that the ARMA model should be specified such that the residuals are approximately white noise.

[(ii) ] Estimate the ACF for the squared residuals in the usual way:

O�Z2.h/ D PT

tDhC1 �

bZ2t � O�2 ��

OZ2t�h � O�2 �

PT tD1

�

bZ2t � O�2 �2

(iii) It is now possible to use one of the methods laid out in Chap. 4 to test the null hypothesis that fZ2t g is white noise. It can be shown that under the null hypothesis

p T O�Z2.h/

d��! N.0;1/. One can therefore construct confidence intervals for the ACF in the usual way. Alternatively, one may use the Ljung- Box test statistic (see Eq. (4.4)) to test the hypothesis that all correlation coefficients up to order N are simultaneously equal to zero.

Q0 D T.T C 2/ N X

hD1

O�2 Z2 .h/

T � h :

Under the null hypothesis this statistic is distributed as �2N . To carry out the test, N should be chosen rather high, for example equal to T=4.

184 8 Models of Volatility

8.2.2 Engle’s Lagrange-Multiplier Test

Engle (1982) proposed a Lagrange-Multiplier test. This test rests on an ancil- lary regression of the squared residuals against a constant and lagged values of bZ2t�1;bZ

2 t�2; : : : ;bZ

2 t�p where the fbZtg is again obtained from a preliminary regression.

The auxiliary regression thus is

bZ2t D ˛0 C ˛1bZ2t�1 C ˛2bZ2t�2 C : : : C ˛pbZ2t�p C "t;

where "t denotes the error term. Then the null hypothesis H0 W ˛1 D ˛2 D : : : D ˛p D 0 is tested against the alternative hypothesis H1 W ˛j ¤ 0 for at least one j. As a test statistic one can use the coefficient of determination times T, i.e. TR2. This test statistic is distributed as a �2 with p degrees of freedom. Alternatively, one may use the conventional F-test.

8.3 Estimation of GARCH(p,q) Models

8.3.1 Maximum-Likelihood Estimation

The literature has proposed several approaches to estimate models of volatility (see Fan and Yao (2003, 156–162)). The most popular one, however, rest on the method of maximum-likelihood. We will describe this method using the GARCH(p,q) model. Related and more detailed accounts can be found in Weiss (1986), Bollerslev et al. (1994) and Hall and Yao (2003).

In particular we consider the following model:

mean equation: Xt D c C �1Xt�1 C : : : C �r Xt�r C Zt;

where

Zt D �t�t with �t � IIDN.0;1/ and

variance equation: �2t D ˛0 C p X

jD1 ˛jZ

2 t�j C

q X

jD1 ˇj�

2 t�j:

The mean equation represents a simple AR(r) process for which we assume that it is causal with respect to fZtg, i.e. that all roots of ˆ.z/ are outside the unit circle. The method demonstrated here can be easily generalized to ARMA processes or even ARMA process with additional exogenous variables (so-called ARMAX processes) as noted by Weiss (1986). The method also incorporates the ARCH-in-mean model (see equation (8.6)) which allows for an effect of the conditional variance �t on Xt.

8.3 Estimation of GARCH(p,q) Models 185

In addition, we assume that the coefficients of the variance equation are all positive, that

Pp jD1 ˛j C

Pq jD1 ˇj < 1 and that EZ

4 t < 1 exists.11

As �t is identically and independently standard normally distributed, the dis- tribution of Xt conditional on Xt�1 D fXt�1; Xt�2; : : :g is normal with mean c C �1Xt�1 C : : : C �r Xt�r and variance �2t . The conditional density, f .XtjXt�1/, therefore is:

f .XtjXt�1/ D 1

2��2t exp

�

� Z 2 t

2�2t

�

where Zt equals Xt � c � �1Xt�1 � : : : � �r Xt�r and �2t is given by the variance equation.12 The joint density f .X1; X2; : : : ; XT/ of a random sample .X1; X2; : : : ; XT/ can therefore be factorized as

f .X1; X2; : : : ; XT/ D f .X1; X2; : : : ; Xs�1/ T Y

tDs f .XtjXt�1/

where s is an integer greater than p. The necessity, not to factorize the first s � 1 observations, relates to the fact that �2t can only be evaluated for s > p in the ARCH(p) model. For the ARCH(p) model s can be set to p C 1. In the case of a GARCH model �2t is given by weighted infinite sum of the Z

2 t�1; Z

2 t�2; : : : (see the

expression (8.9) for �2t in the GARCH(1,1) model). For finite samples, this infinite sum must be approximated by a finite sum of s summands such that the numbers of summands s is increasing with the sample size.(see Hall and Yao (2003)).

We then merge all parameters of the model as follows: � D .c;�1; : : : ;�r/0, ˛ D .˛0;˛1; : : : ;˛p/

0 and ˇ D .ˇ1; : : : ;ˇq/0. For a given realization x D .x1; x2; : : : ; xT/ the likelihood function conditional on x, L.�;˛;ˇjx/, is defined as

L.�;˛;ˇjx/ D f .x1; x2; : : : ; xs�1/ T Y

tDs f .xtjXt�1/

where in Xt�1 the random variables are replaced by their realizations. The likelihood function can be seen as the probability of observing the data at hand given the values for the parameters. The method of maximum likelihood then consist in choosing the parameters .�;˛;ˇ/ such that the likelihood function is maximized. Thus we chose the parameter so that the probability of observing the data is maximized. In this way

11The existence of the fourth moment is necessary for the asymptotic normality of the maximum- likelihood estimator, but not for the consistence. It is possible to relax this assumption somewhat (see Hall and Yao (2003)). 12If �t is assumed to follow another distribution than the normal, one may use this distribution instead.

186 8 Models of Volatility

we obtain the maximum likelihood estimator. Taking the first s realizations as given deterministic starting values, we then get the conditional likelihood function.

In practice we do not maximize the likelihood function but the logarithm of it where we take f .x1; : : : ; xs�1/ as a fixed constant which can be neglected in the optimization:

log L.�;˛;ˇjx/ D T X

tDs log f .xtjXt/

D � T 2

log 2� � 1 2

T X

tDs log �2t �

T X

tDs

z2t �2t

where zt D xt � c � �1xt�1 � : : : � �r xt�r denotes the realization of Zt . The maximum likelihood estimator is obtained by maximizing the likelihood function over the admissible parameter space. Usually, the implementation of the stationarity condition and the condition for the existence of the fourth moment turns out to be difficult and cumbersome so that often these conditions are neglected and only checked in retrospect or some ad hoc solutions are envisaged. It can be shown that the (conditional) maximum likelihood estimator leads to asymptotically normally distributed estimates.13 The maximum likelihood estimator remains meaningful even when f�tg is not normally distributed. In this case the quasi maximum likelihood estimator is obtained (see Hall and Yao (2003) and Fan and Yao (2003)).

For numerical reasons it is often convenient to treat the mean equation and the variance equation separately. As the mean equation is a simple AR(r) model, it can be estimated by ordinary least-squares (OLS) in a first step. This leads to consistent parameter estimates. However, due to the heteroskedasticity, this no longer true for the covariance matrix of the coefficients so that the usual t- and F-tests are nor reliable. This problem can be circumvented by the use of the White correction (see White (1980)). In this way it is possible to find an appropriate specification for the mean equation without having to estimate the complete model. In the second step, one can then work with the residuals to find an appropriate ARMA model for the squared residuals. This leads to consistent estimates of the parameters of the variance equation. These estimates are under additional weakly assumptions asymptotically normally distributed (see Weiss (1986)). It should, however, be noted that this way of proceeding is, in contrast to the maximum likelihood estimator, not efficient because it neglect the nonlinear character of the GARCH model. The parameters found in this way can, however, serve as meaningful starting values for the numerical maximization procedure which underlies the maximum likelihood estimation.

13Jensen and Rahbek (2004) showed that, at least for the GARCH(1,1) case, the stationarity condition is not necessary.

8.3 Estimation of GARCH(p,q) Models 187

A final remark concerns the choice of the parameter r, p and q. Similarly to the ordinary ARMA models, one can use information criteria such as the Akaike or the Bayes criterion, to determine the order of the model (see Sect. 5.4).

8.3.2 Method of Moment Estimation

The maximization of the likelihood function requires the use of numerical opti- mization routines. Depending on the routine actually used and on the starting value, different results may be obtained if the likelihood function is not well-behaved. It is therefore of interest to have alternative estimation methods at hand. The method of moments is such an alternative. It is similar to the Yule-Walker estimator (see Sect. 5.1) applied to the autocorrelation function of fZ2t g. This method not only leads to an analytic solution, but can also be easily implemented. Following Kristensen and Linton (2006), we will illustrate the method for the GARCH(1,1) model.

Equation (8.11) applied to �Z2.1/ and �Z2.2/ constitutes a nonlinear equation system in the unknown parameters ˇ and ˛1. This system can be reparameterized to yield an equation system in ' D ˛1 C ˇ and ˇ which can be reduced to a single quadratic equation in ˇ:

ˇ2 � bˇ � 1 D 0 where b D ' 2 C 1 � 2�Z2.1/' ' � �Z2.1/

The parameter b is well-defined because ' D ˛1 C ˇ � �Z2.1/ with equality only if ˇ D 0. In the following we will assume that ˇ > 0. Under this assumption b > 2 so that the only solution with the property 0 < ˇ < 1 is given by

ˇ D b � p

b2 � 4 2

The moment estimator can therefore be constructed as follows:

(i) Estimate the correlations �Z2.1/ and �Z2.2/ and � 2 based on the formulas (8.11)

in Sect. 8.2. (ii) An estimate for ' D ˛1 C ˇ is then given by

O' D 3.˛1 C ˇ/ D O�Z2.2/ O�Z2.1/

(iii) use the estimate O' to compute an estimate for b:

Ob D O' 2 C 1 � 2 O�Z2.1/ O'

O' � O�Z2.1/ :

188 8 Models of Volatility

The estimate Ǒ for ˇ is then

Ǒ D Ob �

Ob2 � 4 2

(iv) The estimate for ˛1 is Ǫ1 D O'� Ǒ. Because ˛0 D �2.1�.˛1 Cˇ//, the estimate for ˛0 is equal to Ǫ0 D O�2.1 � O'/.

Kristensen and Linton (2006) show that, given the existence of the fourth moment of Zt, this method of moment leads to consistent and asymptotically normal distributed estimates. These estimates may then serve as starting values for the maximization of the likelihood function to improve efficiency.

8.4 Example: Swiss Market Index (SMI)

In this section, we will illustrate the methods discussed previously to analyze the volatility of the Swiss Market Index (SMI). The SMI is the most important stock market index for Swiss blue chip companies. It is constructed solely from stock market prices, dividends are not accounted for. The data are the daily values of the index between the 3rd of January 1989 and the 13th of February 2004. Figure 1.5 shows a plot of the data. Instead of analyzing the level of the SMI, we will investigate the daily return computed as the logged difference. This time series is denoted by Xt and plotted in Fig. 8.3. One can clearly discern phases of high (observations around t D 2500 and t D 3500) and low (t D 1000 and t D 2000) volatility. This represents a first sign of heteroskedasticity and positively correlated volatility.

0 500 1000 1500 2000 2500 3000 3500 4000

−10

−5

p e

rc e

n t

Fig. 8.3 Daily return of the SMI (Swiss Market Index) computed as � log.SMIt/ between January 3rd 1989 and February 13th 2004

8.4 Example: Swiss Market Index (SMI) 189

−10 −8 −6 −4 −2 0 2 4 6

0.001 0.003 0.01 0.02 0.05 0.10

0.25

0.50

0.75

0.90 0.95 0.98 0.99

0.997 0.999

daily return of SMI

P ro

b a b ili

Fig. 8.4 Normal-Quantile Plot of the daily returns of the SMI (Swiss Market Index)

Figure 8.4 shows a normal-quantile plot to compare the empirical distribution of the returns with those from a normal distribution. This plot clearly demonstrates that the probability of observing large returns is bigger than warranted from a normal distribution. Thus the distribution of returns exhibits the heavy-tail property. A similar argument can be made by comparing the histogram of the returns and the density of a normal distribution with the same mean and the same variance. shown in Fig. 8.5. Again one can seen that absolutely large returns are more probable than expected from a normal distribution. Moreover, the histogram shows no obvious sign for an asymmetric distribution, but a higher peakedness.

After the examination of some preliminary graphical devices, we are going to analyze the autocorrelation functions of fXtg and fX2t g. Figure 8.6 shows the estimated ACFs. The estimated ACF of fXtg shows practically no significant autocorrelation so that we can consider fXtg be approximately white noise. The corresponding Ljung-Box statistic with L D 100, however, has a value of 129.62 which is just above the 5 % critical value of 124.34. Thus there is some sign of weak autocorrelation. This feature is not in line with efficiency of the Swiss stock market (see Campbell et al. (1997)). The estimated ACF of X2t is clearly outside the 95 % confidence interval for at least up to order 20. Thus we can reject the hypothesis of homoskedasticity in favor of heteroskedasticity. This is confirmed by the Ljung-Box statistic with L D 100 with a value of 2000.93 which is much higher than the critical value of 124.34.

After these first investigations, we want to find an appropriate model for the mean equation. We will use OLS with the White-correction. It turns out that a MA(1) model fits the data best although an AR(1) model leads to almost the same results. In the next step we will estimate a GARCH(p,q) model with the method of maximum

190 8 Models of Volatility

−12 −10 −8 −6 −4 −2 0 2 4 6 8 0

100

150

200

250

300

350

400

450

500

550

Fig. 8.5 Histogram of the daily returns of the SMI (Swiss Market Index) and the density of a fitted normal distribution (red line)

0 5 10 15 20 25 30 35 40 45 50 −0.5

0.5

order

co rr

e la

tio n

c o

e ff

ic ie

n t

0 5 10 15 20 25 30 35 40 45 50 −0.5

0.5

1 (Δ log(SMIt))

order

co rr

e la

tio n

c o

e ff

ic ie

n t

Δ log(SMIt)

Fig. 8.6 ACF of the daily returns and the squared daily returns of the SMI

8.4 Example: Swiss Market Index (SMI) 191

Table 8.1 AIC criterion for the variance equation in the GARCH(p,q) model

p 0 1 2 3

1 3.0886 2.9491 2.9491 2.9482

2 3.0349 2.9496 2.9491 2.9486

3 2.9842 2.9477 2.9472 2.9460

Minimum value in bold

Table 8.2 BIC criterion for the variance equation in the GARCH(p,q) model

p 0 1 2 3

1 3.0952 2.9573 2.9590 2.9597 2 3.0431 2.9595 2.9606 2.9617

3 2.9941 2.9592 2.9604 2.9607

Minimum value in bold

likelihood where p is varied between p 1 and 3 and q between 0 and 3. The values of the AIC, respectively BIC criterion corresponding to the variance equation are listed in Tables 8.1 and 8.2.

The results reported in these tables show that the AIC criterion favors a GARCH(3,3) model corresponding to the bold number in Table 8.1 whereas the BIC criterion opts for a GARCH(1,1) model corresponding to the bold number in Table 8.2. It also turns out that high dimensional models, in particular those for which q > 0, the maximization algorithm has problems to find an optimum. Furthermore, the roots of the implicit AR and the MA polynomial corresponding to the variance equation of the GARCH(3,3) model are very similar. These two arguments lead us to prefer the GARCH(1,1) over the GARCH(3,3) model. This model was estimated to have the following mean equation:

Xt D 0.0755 (0.0174)

C Zt C 0.0484 (0.0184)

Zt�1

with the corresponding variance equation

�2t D 0.0765 (0.0046)

C 0.1388 (0.0095)

Z2t�1 C 0.8081 (0.0099)

�2t�1;

where the estimated standard deviations are reported below the corresponding coefficient estimate. The small, but significant value of 0.0484 for the MA(1) coefficient shows that there is a small but systematic correlation of the returns from one day to the next. The coefficients of the GARCH model are all positive and their sum ˛1 C ˇ D 0:1388 C 0:8081 D 0:9469 is statistically below one so that all conditions for a stationary process are fulfilled.14 Because

p 3 ˛1 1�ˇ D

p 3 0:1388 1�0:8081 D

1:2528 > 1, the condition for the existence of the fourth moment of Zt is violated.

14The corresponding Wald test clearly rejects the null hypothesis ˛1 C ˇ D 1 at a significance level of 1 %.

192 8 Models of Volatility

As a comparison we also estimate the GARCH(1,1) model using the methods of moments. First we estimate a MA(1) model for � log SMI. This results in an estimate O� D 0:034 (compare this with the ML estimate). The squared residuals have correlation coefficients

O�Z2.1/ D 0:228 and O�Z2.2/ D 0:181:

The estimate of b therefore is Ob D 2:241 which leads to an estimate of ˇ equal to Ǒ D 0:615. This finally results in the estimates of ˛1 and ˛0 equal to Ǫ1 D 0:179

and Ǫ0 D 0:287 with an estimate for �2 equal to O�2 D 1:391. Thus these estimates are quite different from those obtained by the ML method.

Value at Risk

We are now in a position to use our ML estimates to compute the Value-at-risk (VaR). The VaR is a very popular measure to estimate the risk of an investment. In our case we consider the market portfolio represented by the stocks in the SMI. The VaR is defined as the maximal loss (in absolute value) of an investment which occurs with probability ˛ over a time horizon h. Thus a 1 % VaR for the return on the SMI for the next day is the threshold value of the return such that one can be confident with 99 % that the loss will not exceed this value. Thus the ˛ VaR at time t for h periods, VaR˛t;tCh, is nothing but the ˛-quantile of the distribution of the forecast of the return in h periods given information Xt�k, k D 0;1;2; : : :. Formally, we have:

VaR˛t;tCh D inf ˚

x W P eXtCh � xjXt; Xt�1; : : : � ˛ ;

where eXtCh is the return of the portfolio over an investment horizon of h periods. This return is approximately equal to the sum of the daily returns: eXtCh D Ph

jD1 XtCj. The one period forecast error is given by XtC1 �ePtXtC1 which is equal to ZtC1 D

�tC1�tC1. Thus the VaR for the next day is

VaR˛t;tC1 D inf (

x W P "

�tC1 � x � ePtXtC1 �tC1

� ˛ )

This entity can be computed by replacing the forecast given the infinite past, ePtXtC1, by a forecast given the finite sample information Xt�k, k D 0;1;2; : : : ; t �1, PtXtC1, and by substituting �tC1 by the corresponding forecast from variance equation, O�t;tC1. Thus we get:

2VaR˛t;tC1 D inf �

x W P �

�tC1 � x � PtXtC1

O�t;tC1 �

� ˛ �

8.4 Example: Swiss Market Index (SMI) 193

Table 8.3 One percent VaR for the next day of the return to the SMI according to the ARMA(0,1)-GARCH(1,1) model

VaR (2VaR0;01t;tC1)

Date Pt XtC1 O�2t;tC1 Parametric Non-parametric 31.12.2001 0.28 6.61 5.71 6.30

5.2.2002 �0.109 6.80 6.19 6.79 24.7.2003 0.0754 0.625 1.77 1.95

Table 8.4 One percent VaR for the next 10 days of the return to the SMI according to the ARMA(0,1) -GARCH(1,1) model

VaR ( 2VaR0;01t;tC10)

Date Pt QXtC1 Parametric Non-parametric 31.12.2001 0.84 18.39 22.28

5.2.2002 0.65 19.41 21.53

24.7.2003 0.78 6.53 7.70

The computation of 2VaR˛t;tC1 requires to determine the ˛-quantile of the distribution of �t. This can be done in two ways. The first one uses the assumption about the distribution of �t explicitly. In the simplest case, �t is distributed as a standard normal so that the appropriate quantile can be easily retrieved. The 1 % quantile for the standard normal distribution is �2.33. The second approach is a non-parametric one and uses the empirical distribution function of O�t D OZt=O�t to determine the required quantile. This approach has the advantage that deviations from the standard normal distribution are accounted for. In our case, the 1 % quantile is �2:56 and thus considerably lower than the �2:33 obtained from the normal distribution. Thus the VaR is under estimated by using the assumption of the normal distribution.

The corresponding computations for the SMI based on the estimated ARMA(0,1)-GARCH(1,1) model are reported in Table 8.3. A value of 5.71 for 31st of December 2001 means that one can be 99 % sure that the return of an investment in the stocks of the SMI will not be lower than �5.71 %. The values for the non-parametric approach are typically higher. The comparison of the VaR for different dates clearly shows how the risk evolves over time.

Due to the nonlinear character of the model, the VaR for more than one day can only be gathered from simulating the one period returns over the corresponding horizon. Starting from a given date 10’000 realizations of the returns over the next 10 days have been simulated whereby the corresponding values for �t are either drawn from a standard normal distribution (parametric case) or from the empirical distribution function of O�t (non-parametric case). The results from this exercise are reported in Table 8.4. Obviously, the risk is much higher for a 10 day than for a one day investment. Alternatively, one may use the forecasting equation (8.12) and the corresponding recursion formula (8.13).

Part II

Multivariate Time Series Analysis

9Introduction

The Keynesian macroeconomic theory developed in the 1930s and 1940s, in particular its representation in terms of IS- and LM-diagram, opened a new area in the application of statistical methods to economics. Based on the path breaking work by Tinbergen (1939) and Klein (1950) this research gave rise to simultaneous equation systems which should capture all relevant aspect of an economy. The goal was to establish an empirically grounded tool which would enable the politicians to analyze the consequences of their policies and thereby fine tune the economy to overcome or at least mitigate major business cycle fluctuations. This development was enhanced by the systematic compilation of national accounting data and by the advances in computer sciences.1 These systems were typically built by putting together single equations such as consumption, investment, money demand, export- and import, Phillips-curve equations to an overall model. The Klein–Goldberger model for the United States was a first successful attempt in this direction (Klein and Goldberger 1955). Shocked by disturbances, Adelman and Adelman (1959) showed that these type of models exhibited cycles with properties similar to those found for the United States Economy.

As the model became more and detailed over time, they could, in the end, well account for several hundreds or even thousands of equations. The climax of this development was the project LINK which linked the different national models to a world model by accounting for their interrelation through trade flows (Klein 1985). Although this research program brought many insights and spurred the development of econometrics as a separate field, by the mid 1970s one had to admit that the idea to use large and very detailed models for forecasting and policy analysis was overly optimistic. In particular, the inability to forecast and cope with the oil crisis of the beginning 1970s raised doubts about the viability of this research strategy. In addition, more and more economist had concerns about the theoretical foundations of these models.

1See Epstein (1987) for an historical overview.

197

198 9 Introduction

The critique had several facets. First, it was argued that the bottom-up strategy of building a system from single equations is not compatible with general equilibrium theory which stresses the interdependence of economic activities. This insight was even reinforced by the advent of the theory of rational expectations. This theory postulated that expectations should be formed on the basis of all available infor- mation and not just by mechanically extrapolating from the past. This implies that developments in every part of the economy, in particular in the realm of economic policy making, should in principle be taken into account and shape the expectation formation. As expectations are omnipresent in almost every economic decision, all aspects of economic activities (consumption capital accumulation, investment, etc.) are inherently linked. Thus the strategy of using zero restrictions—which meant that certain variables were omitted from a particular equation—to identify the parameters in a simultaneous equation system was considered to be flawed. Second, the theory of rational expectations implied that the typical behavioral equations underlying these models are not invariant to changes in policies because economic agents would take into account systematic changes in the economic environment in their decision making. This so-called Lucas-critique (Lucas 1976) undermined the basis for the existence of large simultaneous equation models. Third, simple univariate ARMA models proved to be as good in forecasting as the sophisticated large simultaneous models. Thus it was argued that the effort or at least part of the effort devoted to these models was wasted.

In 1980 Sims (1980b) and proposed an alternative modeling strategy. This strategy concentrates the modeling activity to only a few core variables, but places no restrictions what so ever on the dynamic interrelation among them. Thus every variable is considered to be endogenous and, in principle, dependent on all other variables of the model. In the linear context, the class of vector autoregressive (VAR) models has proven to be most convenient to capture this modeling strategy. They are easy to implement and to analyze. In contrast to the simultaneous equation approach, however, it is no longer possible to perform comparative static exercises and to analyze the effect of one variable on another one because every variable is endogenous a priori. Instead, one tries to identify and quantify the effect of shocks over time. These shocks are usually given some economic content, like demand or supply disturbances. However, these shocks are not directly observed, but are disguised behind the residuals from the VAR. Thus, the VAR approach also faces a fundamental identification problem. Since the seminal contribution by Sims, the literature has proposed several alternative identification schemes which will be discussed in Chap. 15 under the header of structural vector autoregressive (SVAR) models. The effects of these shocks are then further analyzed by computing impulse responses and forecast error variance decompositions.2

The reliance on shocks can be seen as a substitute for the lack of experiments in macroeconomics. The approach can be interpreted as a statistical analogue to the identification of specific episodes where some unforseen event (shock)

2Watson (1994) and Kilian (2013) provide a general introduction to this topic.

9 Introduction 199

impinges on and propagates throughout the economy. Singling-out these episodes of quasi “natural experiments” as in Friedman and Schwartz (1963) convinced many economist of the role and effects of monetary policy.

Many concepts which have been introduced in the univariate context carry over in a straightforward manner to the multivariate context. However there are some new aspects. First, we will analyze the interaction among several variables. This can be done in a nonparametric way by examining the cross-correlations between time series or by building an explicit model. We will restrict ourself to the class of VAR models as they are easy to handle and are overwhelmingly used in practice. Second, we will discuss several alternative approaches to identify the structural shocks from VAR models. After analyzing the identification problem in general, we describe short-run, long-run, and sign restrictions as possible remedies. Third, we will discuss the modeling of integrated variables in a more systematic way. In particular, we will extend the concept of cointegration to more than two variables. Finally, we will provide an introduction to the state space models as a general modeling approach. State space models are becoming increasingly popular in economics as they can be more directly linked to theoretical economic models.

10Definitions and Stationarity

Similarly to the univariate case, we start our exposition with the concept of stationarity which is also crucial in the multivariate setting. Before doing so let us define the multivariate stochastic process.

Definition 10.1. A multivariate stochastic Process, fXtg, is a family of random variables indexed by t, t 2 Z, which take values in Rn, n � 1. n is called the dimension of the process.

Setting n D 1, the above definition includes as a special case univariate stochastic processes. This implies that the statements for multivariate processes carry over analogously to the univariate case. We view Xt as a column vector:

Xt D

X1t :::

Xnt

A :

Each element fXitg thereby represents a particular variable which may be treated as a univariate process. As in the example of Sect. 15.4.5, fXtg represents the multivariate process consisting of the growth rate of GDP Yt, the unemployment rate Ut , the inflation rate Pt , the wage inflation rate Wt, and the growth rate of money Mt. Thus, Xt D .Yt; Ut; Pt; Wt; Mt/0.

As in the univariate case, we characterize the joint distribution of the elements Xit and Xjt by the first two moments (if they exist), i.e. by the mean and the variance, respectively covariance:

�it D EXit; i D 1; : : : ; n �ij.t; s/ D E.Xit � �it/.Xjs � �js/; i; j D 1; : : : ; nI t; s 2 Z (10.1)

It is convenient to write these entities compactly as vectors and matrices:

201

202 10 Definitions and Stationarity

�t D

�1t :::

�nt

A D EXt D

EX1t :::

EXnt

�.t; s/ D

�11.t; s/ : : : �1n.t; s/ :::

::: :::

�n1.t; s/ : : : �nn.t; s/

A D E.Xt � �t/.Xs � �s/0

Thus, we apply the expectations operator element-wise to vectors and matrices. The matrix-valued function �.t; s/ is called the covariance function of fXtg.

In analogy to the univariate case, we define stationarity as the invariance of the first two moments to time shifts:

Definition 10.2 (Stationarity). A multivariate stochastic process fXtg is stationary if and only if for all integers r, s and t we have:

(i) � D �t D EXt is constant (independent of t); (ii) EX0t Xt < 1;

(iii) �.t; s/ D �.t C r; s C r/.

In the literature these properties are often called weak stationarity, covariance stationarity, or stationarity of second order. If fXtg is stationary, the covariance function only depends on the number of periods between t and s (i.e. on t � s) and not on t or s themselves. This implies that by setting r D �s and h D t � s the covariance function simplifies to

�.h/ D �.t � s/ D �.t C r; s C r/ D �.t C h; t/ D E.XtCh � �/.Xt � �/0 D E.Xt � �/.Xt�h � �/0:

For h D 0, �.0/ is the unconditional covariance matrix of Xt. Using the definition of the covariances in Eq. (10.1) we get:

�.h/ D �.�h/0:

Note that �.h/ is in general not symmetric for h ¤ 0 because �ij.h/ ¤ �ji.h/ for h ¤ 0.

Based on the covariance function of a stationary process, we can define the correlation function R.h/ where R.h/ D .�ij.h//i;j with

�ij.h/ D �ij.h/

�ii.0/�jj.0/ :

10 Definitions and Stationarity 203

In the case i ¤ j we refer to the cross-correlations between two variables fXitg and fXjtg. The correlation function can be written in matrix notation as

R.h/ D V �1=2�.h/V �1=2

where V represents a diagonal matrix with diagonal elements equal to �ii.0/. Clearly, �ii.0/ D 1. As for the covariance matrix we have that in general �ij.h/ ¤ �ji.h/ for h ¤ 0. It is possible that �ij.h/ > �ij.0/. We can summarize the properties of the covariance function by the following theorem.1

Theorem 10.1. The covariance function of a stationary process fXtg has the following properties:

(i) For all h 2 Z, �.h/ D �.�h/0; (ii) for all h 2 Z, j�ij.h/j �

�ii.0/ � �jj.0/; (iii) for each i D 1; : : : ; n, �ii.h/ is a univariate autocovariance function; (iv)

Pm r;kD1 a

0 r�.r � k/ak � 0 for all m 2 N and all a1; : : : ; am 2 Rn. This property

is called non-negative definiteness (see Property 4 in Theorem 1.1 of Sect. 1.3 in the univariate case).

Proof. Property (i) follows immediately from the definition. Property (ii) follows from the fact that the correlation coefficient is always smaller or equal to one. �ii.h/ is the autocovariance function of fXitg which delivers property (iii). Property (iv) follows from E

�Pm kD1 a

0 k.Xt�k � �/

�2 � 0. ut

If not only the first two moments are invariant to time shifts, but the distribution as a whole we arrive at the concept of strict stationarity.

Definition 10.3 (Strict Stationarity). A process multivariate fXtg is called strictly stationary if and only if, for all n 2 N, t1; : : : ; tn, h 2 Z, the joint distributions of .Xt1; : : : ; Xtn/ and .Xt1Ch; : : : ; Xtn Ch/ are the same.

An Example

Consider the following example for n D 2:

X1t D Zt X2t D Zt C 0:75Zt�2

where Zt � WN.0;1/. We then have � D EXt D 0. The covariance function is given by

1We leave it to the reader to derive an analogous theorem for the correlation function.

204 10 Definitions and Stationarity

�.h/ D

�

1 1

1 1:5625

�

; h D 0I �

0 0

�

; h D 1I �

0 0

0:75 0:75

�

; h D 2:

The covariance function is zero for h > 2. The values for h < 0 are obtained from property (i) in Theorem 10.1. The correlation function is:

R.h/ D

�

1 0:8

0:8 1

�

; h D 0I �

0 0

�

; h D 1I �

0 0

0:60 0:48

�

; h D 2:

The correlation function is zero for h > 2. The values for h < 0 are obtained from property (i) in Theorem 10.1.

One idea in time series analysis is to construct more complicated process from simple ones, for example by taking moving-averages. The simplest process is the white noise process which is uncorrelated with its own past. In the multivariate context the white noise process is defined as follows.

Definition 10.4. A stochastic process fZtg is called (multivariate) white noise process with mean zero and covariance matrix † > 0, denoted by Zt � WN.0;†/, if it is stationary fZtg and

EZt D 0;

�.h/ D �

†; h D 0I 0; h ¤ 0:

If fZtg is not only white noise, but independently and identically distributed we write Zt � IID.0;†/.

Remark 10.1. Even if each component of fZitg is univariate white noise, this does not imply that fZtg D f.Z1t; : : : ; Znt/0g is multivariate white noise. Take, for example the process Zt D .ut; ut�1/0 where ut � WN.0;�2u /. Then �.1/ D

�

0 0

�2u 0

�

¤ 0.

Taking moving averages of a white noise process it is possible to generate new stationary processes. This leads to the definition of a linear process.

Definition 10.5. A stochastic process fXtg is called linear if there exists a represen- tation

10 Definitions and Stationarity 205

Xt D 1 X

jD�1 ‰jZt�j

where Zt � IID.0;†/ and where the sequence f‰jg of the n�n matrices is absolutely summable, i.e.

P1 jD�1 k‰jk < 1. If for all j < 0 ‰j D 0, the linear process is also

called an MA.1/ process. Theorem 10.2. A linear process is stationary with a mean of zero and with covariance function

�.h/ D 1 X

jD�1 ‰jCh†‰0j D

1 X

jD�1 ‰j†‰

0 j�h; h D 0; ˙1; ˙2; : : :

Proof. The required result is obtained by applying the properties of fZtg to �.h/ D EXtChX0t D limm!1 E

�

Pm jD�m ‰jZtCh�j

�

�Pm kD�m ‰kZt�k

�0 . ut

Remark 10.1. The same conclusion is reached if fZtg is a white noise process and not an IID process.

Appendix: Norm and Summability of Matrices

As in the definition of a linear process it is often necessary to analyze the convergence of a sequence of matrices f‰jg, j D 0;1;2; : : :. For this we need to define a norm for matrices. The literature considers different alternative approaches. For our purposes, the choice is not relevant as all norms are equivalent in the finite dimensional vector space. We therefore choose the Frobenius, Hilbert-Schmidt or Schur norm which is easy to compute.2 This norm treats the elements of a m � n matrix A D .aij/ as an element of the Rm�n Euclidean space and defines the length of A, denoted by kAk, as kAk D

i;j a 2 ij. This leads to the formal definition below.

Definition 10.6. The Frobenius, Hilbert-Schmidt or Schur norm of a m � n matrix A, denoted by kAk, is defined as:

kAk2 D X

i;j

a2ij D tr.A0A/ D n X

iD1 i

where tr.A0A/ denotes the trace of A0A, i.e. the sum of the diagonal elements of A0A, and where i are the n eigenvalues of A

0A.

2For details see Meyer (2000, 279ff).

206 10 Definitions and Stationarity

The matrix norm has the following properties:

kAk � 0 and kAk D 0 is equivalent to A D 0; k˛Ak D j˛jkAk for all ˛ 2 R; kAk D kA0k; kA C Bk � kAk C kBk for all matrices A and B of the same dimension; kABk � kAkkBk for all conformable matrices A and B:

The last property is called submultiplicativity. A sequence of matrices f‰jg, j D 0;1; : : : , is called absolutely summable if and

only if P1

jD0 k‰jk < 1; the sequence is said to be quadratic summable if and only if P1

jD0 k‰jk2 < 1. The absolute summability implies the quadratic summability, but not vice versa.

Corollary 10.1. Absolute summability of f‰jg is equivalent to the absolute summa- bility of each sequence fŒ‰j klg, k; l D 1; : : : ; n, i.e. to limj!1 jŒ‰j klj exists and is finite.

Proof. In particular, he have:

jŒ‰j klj � k‰jk D v

n X

kD1

n X

lD1 Œ‰j

2 kl �

n X

kD1

n X

lD1 jŒ‰j klj:

Summation over j gives

1 X

jD0 jŒ‰j klj �

1 X

jD0 k‰jk �

1 X

jD0

n X

kD1

n X

lD1 jŒ‰j klj D

1 X

kD1

n X

lD1

n X

jD0 jŒ‰j klj

The absolute convergence of each sequence fŒ‰j klg, k; l D 1; : : : ; n, follows from the absolute convergence of f‰jg by the first inequality. Conversely, the absolute convergence of f‰jg is implied by the absolute convergence of each sequence fŒ‰j klg, k; l D 1; : : : ; n, from the second inequality. ut

11Estimation of Mean and Covariance Function

11.1 Estimators and Their Asymptotic Distributions

We characterize the stationary process fXtg by its mean and its (matrix) covariance function. In the Gaussian case, this already characterizes the whole distribution. The estimation of these entities becomes crucial in the empirical analysis. As it turns out, the results from the univariate process carry over analogously to the multivariate case. If the process is observed over the periods t D 1;2; : : : ; T, then a natural estimator for the mean � is the arithmetic mean or sample average:

O� D XT D 1

T .X1 C : : : C XT/ D

X1 :::

A :

We get a theorem analogously to Theorem 4.1 in Sect. 4.1.

Theorem 11.1. Let fXtg be stationary process with mean � and covariance function �.h/ then asymptotically, for T ! 1, we get

E �

XT � � �0 �

XT � � � ! 0; if �ii.T/ ! 0 for all 1 � i � nI

TE �

XT � � �0 �

XT � � � !

n X

iD1

1 X

hD�1 �ii.h/;

if 1 X

hD�1 j�ii.h/j < 1 for all 1 � i � n:

Proof. The Theorem can be established by applying Theorem 4.1 individually to each time series fXitg, i D 1;2; : : : ; n. ut

207

208 11 Estimation of Covariance Function

Thus, the sample average converges in mean square and therefore also in probability to the true mean. Thereby the second condition is more restrictive than the first one. They are, in particular, fulfilled for all VARMA processes (see Chap. 12). As in the univariate case analyzed in Sect. 4.1, it can be shown with some mild additional assumptions that XT is also asymptotically normally distributed.

Theorem 11.2. For any stationary process fXtg

Xt D � C 1 X

jD�1 ‰jZt�j

with Zt � IID.0;†/ and P1

jD�1 k‰jk < 1, the arithmetic average XT is asymptotically normal:

p T �

XT � � � d��! N

1 X

hD�1 �.h/

D N 0

@0;

1 X

jD�1 ‰j

A†

1 X

jD�1 ‰0j

D N �0;‰.1/†‰.1/0� :

Proof. The proof is a straightforward extension to the multivariate case of the one given for Theorem 4.2 of Sect. 4.1. ut

The summability condition is quite general. It is, in particular, fulfilled by causal VARMA processes (see Chap. 12) as their coefficients matrices ‰j go exponentially fast to zero. Remarks similar to those following Theorem 4.2 apply also in the multivariate case.

The above formula can be used to construct confidence regions for �. This turns out, however, to be relatively complicated in practice so that often univariate approximations are used instead (Brockwell and Davis 1996, 228–229).

As in the univariate case, a natural estimator for the covariance matrix function �.h/ is given by the corresponding empirical moments b�.h/:

b�.h/ D 8

1 T

PT�h tD1

�

XtCh � XT ��

Xt � XT �0 ; 0 � h � T � 1I

b�0.�h/; �T C 1 � h < 0:

The estimator of the covariance function can then be applied to derive an estimator for the correlation function:

bR.h/ D OV �1=2b�.h/ OV �1=2

11.2 Testing Cross-Correlations of Time Series 209

where OV1=2 D diag �

O�11.0/; : : : ; p

O�nn.0/ �

. Under the conditions given in

Theorem 11.2 the estimator of the covariance matrix of order h, b�.h/, converges

to the true covariance matrix �.h/. Moreover, p

T �

b�.h/ � �.h/ �

is asymptotically

normally distributed. In particular, we can state the following Theorem:

Theorem 11.3. Let fXtg be a stationary process with

Xt D � C 1 X

jD�1 ‰jZt�j

where Zt � IID.0;†/, P1

jD�1 k‰jk < 1, and P1

jD�1 ‰j ¤ 0. Then, for each fixed h, b�.h/ converges in probability as T ! 1 to �.h/:

b�.h/ p��! �.h/

Proof. A proof can be given along the lines given in Proposition 13.1. ut

As for the univariate case, we can define the long-run covariance matrix J as

J D 1 X

hD�1 �.h/: (11.1)

As a non-parametric estimator we can again consider the following class of estimators:

OJT D T�1 X

hD�TC1 k

�

b�.h/

where k.x/ is a kernel function and where b�.h/ is the corresponding estimate of the covariance matrix at lag h. For the choice of the kernel function and the lag truncation parameter the same principles apply as in the univariate case (see Sect. 4.4 and Haan and Levin (1997)).

11.2 Testing Cross-Correlations of Bivariate Time Series

The determination of the asymptotic distribution of b�.h/ is complicated. We therefore restrict ourselves to the case of two time series.

Theorem 11.4. Let fXtg be a bivariate stochastic process whose components can be described by

210 11 Estimation of Covariance Function

X1t D 1 X

jD�1 ˛jZ1;t�j with Z1t � IID.0;�21/

X2t D 1 X

jD�1 ˇjZ2;t�j with Z2t � IID.0;�22/

where fZ1tg and fZ2tg are independent from each other at all leads and lags and where

j j˛jj < 1 and P

j jˇjj < 1. Under these conditions the asymptotic distribution of the estimator of the cross-correlation function �12.h/ between fX1tg and fX2tg is

p T O�12.h/

d�! N 0

@0;

1 X

jD�1 �11.j/�22.j/

A ; h � 0: (11.2)

For all h and k with h ¤ k, . p

T O�12.h/; p

T O�12.k//0 converges in distribution to a bivariate normal distribution with mean zero and variances and covariances given by P1

jD�1 �11.j/�22.j/ and P1

jD�1 �11.j/�22.j C k � h/, respectively. This result can be used to construct a test of independence, respectively uncor-

relatedness, between two time series. The above theorem, however, shows that the asymptotic distribution of

p T O�12.h/ depends on �11.h/ and �22.h/ and is therefore

unknown. Thus, the test cannot be based on the cross-correlation alone.1

This problem can, however, be overcome by implementing the following two- step procedure suggested by Haugh (1976).

First step: Estimate for each times series separately a univariate invertible ARMA model and compute the resulting residuals OZit as OZit D

P1 jD0 O�.i/j Xi;t�j,

i D 1;2. If the ARMA models correspond to the true ones, these residuals should approximately be white noise. This first step is called pre-whitening.

Second step: Under the null hypothesis the two time series fX1tg and fX2tg are uncorrelated with each other. This implies that the residuals fZ1tg and fZ2t g should also be uncorrelated with each other. The variance of the cross- correlations between fZ1tg and fZ2tg are therefore asymptotically equal to 1=T under the null hypothesis. Thus, one can apply the result of Theorem 11.4 to construct confidence intervals based on formula (11.2). A 95-% confidence interval is therefore given by ˙1:96T �1=2. The Theorem may also be used to construct a test whether the two series are uncorrelated.

If one is not interested in modeling the two time series explicitly, the simplest way is to estimate a high order AR model in the first step. Thereby, the order should be chosen high enough to obtain white noise residuals in the first step. Instead

1The theorem may also be used to conduct a causality test between two times series (see Sect. 15.1).

11.3 Some Examples for Independence Tests 211

of looking at each cross-correlation separately, one may also test the joint null hypothesis that all cross-correlations are simultaneously equal to zero. Such a test can be based on T times the sum of the squared cross-correlation coefficients. This statistic is distributed as a �2 with L degrees of freedom where L is the number of summands (see the Haugh-Pierce statistic (15.1) in Sect. 15.1).

11.3 Some Examples for Independence Tests

Two Independent AR Processes

Consider two AR(1) process fX1tg and fX2tg governed by the following stochastic difference equation Xit D 0:8Xi;t�1 C Zit, i D 1;2. The two white noise processes fZ1tg and fZ2t g are such that they are independent from each other. fX1tg and fX2tg are therefore independent from each other too. We simulate realizations of these two processes over 400 periods. The estimated cross-correlation function of these so-generated processes are plotted in the upper panel of Fig. 11.1. From there one can see that many values are outside the 95 % confidence interval given by ˙1:96T �1=2 D 0:098, despite the fact that by construction both series are independent of each other. The reason is that the so computed confidence interval is not correct because it does not take the autocorrelation of each series into account. The application of Theorem 11.4 leads to the much larger 95-% confidence interval of

˙1:96p T

1 X

jD�1 �11.j/�22.j/ D

˙1:96 20

1 X

jD�1 0:8j0:8j

D ˙1:96 20

1 C 2 � 0:64 1 � 0:64 D ˙0:209

which is more than twice as large. This confidence interval then encompasses most the cross-correlations computed with respect to the original series.

If one follows the testing procedure outline above instead and fits an AR(10) model for each process and then estimates the cross-correlation function of the corresponding residual series (filtered or pre-whitened time series), the plot in the lower panel of Fig. 11.1 is obtained.2 This figure shows no significant cross- correlation anymore so that one cannot reject the null hypothesis that both time series are independent from each other.

2The order of the AR processes are set arbitrarily equal to 10 which is more than enough to obtain white noise residuals.

212 11 Estimation of Covariance Function

−20 −15 −10 −5 0 5 10 15 20

−0.2

−0.1

0.1

0.2

order

cr os

s− co

rr el

at io

n original series

−20 −15 −10 −5 0 5 10 15 20

−0.2

−0.1

0.1

0.2

order

cr os

s− co

rr el

at io

filtered series

Fig. 11.1 Cross-correlations between two independent AR(1) processes with � D 0:8

Consumption Expenditure and Advertisement Expenses

This application focuses on the interaction between nominal aggregate private consumption expenditure and nominal aggregate advertisement expenditures. Such an investigation was first conducted by Ashley et al. (1980) for the United States.3

The upper panel of Fig. 11.2 shows the raw cross-correlations between the two time series where the order h runs from �20 to C20. Although almost all cross- correlations are positive and outside the conventional confidence interval, it would be misleading to infer a statistically significant positive cross-correlation. In order to test for independence, we filter both time series by an AR(10) model and estimate the cross-correlations for the residuals.4 These are displayed in the lower panel of Fig. 11.2. In this figure, only the correlations of order 0 and 16 fall outside the confidence interval and can thus be considered as statistically significant. Thus, we

3The quarterly data are taken from Berndt (1991). They cover the period from the first quarter 1956 to the fourth quarter 1975. In order to achieve stationarity, we work with first differences. 4The order of the AR processes are set arbitrarily equal to 10 which is more than enough to obtain white noise residuals.

11.3 Some Examples for Independence Tests 213

−20 −15 −10 −5 0 5 10 15 20

−0.2

0.2

0.4

order

cr o ss

− co

rr el

at io

n original series

−20 −15 −10 −5 0 5 10 15 20

−0.2

0.2

0.4

0.6

order

cr o ss

− co

rr el

at io

filtered series

Fig. 11.2 Cross-correlations between aggregate nominal private consumption expenditures and aggregate nominal advertisement expenditures

can reject the null hypothesis of independence between the two series. However, most of the interdependence seems to come from the correlation within the same quarter. This is confirmed by a more detailed investigation in Berndt (1991) where no significant lead and/or lag relations are found.

Real Gross Domestic Product and Consumer Sentiment The procedure outlined above can be used to examine whether one of the two time series is systematically leading the other one. This is, for example, important in the judgment of the current state of the economy because first provisional national accounting data are usually published with a lag of at least one quarter. However, in the conduct of monetary policy more up-to-date knowledge is necessary. Such a knowledge can be retrieved from leading indicator variables. These variables should be available more quickly and should be highly correlated with the variable of interest at a lead.

We investigate whether the Consumer Sentiment Index is a leading indicator for the percentage changes in real Gross Domestic Product (GDP).5 The raw

5We use data for Switzerland as published by the State Secretariat for Economic Affairs SECO.

214 11 Estimation of Covariance Function

−20 −15 −10 −5 0 5 10 15 20

0.5

order

cr o ss

− co

rr el

at io

n original series

−20 −15 −10 −5 0 5 10 15 20

−0.2

0.2

0.4

0.6

order

cr o ss

− co

rr el

at io

filtered series

consumer sentiment is leading by one quarter

Fig. 11.3 Cross-correlations between real growth of GDP and the consumer sentiment index

cross-correlations are plotted in the upper panel of Fig. 11.3. It shows several correlations outside the conventional confidence interval. The use of this confidence interval is, however, misleading as the distribution of the raw cross-correlations depends on the autocorrelations of each series. Thus, instead we filter both time series by an AR(8) model and investigate the cross-correlations of the residuals.6

The order of the AR model was chosen deliberately high to account for all autocorrelations. The cross-correlations of the filtered data are displayed in the lower panel of Fig. 11.3. As it turns out, only the cross-correlation which is significantly different from zero is for h D 1. Thus the Consumer Sentiment Index is leading the growth rate in GDP. In other words, an unexpected higher consumer sentiment is reflected in a positive change in the GDP growth rate of next quarter.7

6With quarterly data it is wise to set to order as a multiple of four to account for possible seasonal movements. As it turns out p D 8 is more than enough to obtain white noise residuals. 7During the interpretation of the cross-correlations be aware of the ordering of the variables because �12.1/ D �21.�1/ ¤ �21.1/.

12Stationary Time Series Models: VectorAutoregressive Moving-Average Processes (VARMA Processes)

The most important class of models is obtained by requiring fXtg to be the solution of a linear stochastic difference equation with constant coefficients. In analogy to the univariate case, this leads to the theory of vector autoregressive moving-average processes (VARMA processes or just ARMA processes).

Definition 12.1 (VARMA process). A multivariate stochastic process fXtg is a vec- tor autoregressive moving-average process of order .p; q/, denoted as VARMA.p; q/ process, if it is stationary and fulfills the stochastic difference equation

Xt � ˆ1Xt�1 � : : : � ˆpXt�p D Zt C ‚1Zt�1 C : : : C ‚qZt�q (12.1)

where ˆp ¤ 0, ‚q ¤ 0 and Zt � WN.0;†/. fXtg is called a VARMA.p; q/ process with mean � if fXt � �g is a VARMA.p; q/ process.

With the aid of the lag operator we can write the difference equation more compactly as

ˆ.L/Xt D ‚.L/Zt where ˆ.L/ D In � ˆ1L � : : : � ˆpLp and ‚.L/ D In C ‚1L C : : : C ‚qLq. ˆ.L/ and ‚.L/ are n � n matrices whose elements are lag polynomials of order smaller or equal to p, respectively q. If q D 0, ‚.L/ D In so that there is no moving-average part. The process is then a purely autoregressive one which is simply called a VAR(p) process. Similarly if p D 0, ˆ.L/ D In and there is no autoregressive part. The process is then a purely moving-average one and simply called a VMA(q) process. The importance of VARMA processes stems from the fact that every stationary process can be arbitrarily well approximated by a VARMA process, VAR process, or VMA process.

215

216 12 VARMA Processes

12.1 The VAR(1) Process

We start our discussion by analyzing the properties of the VAR(1) process which is defined as the solution the following stochastic difference equation:

Xt D ˆXt�1 C Zt with Zt � WN.0;†/:

We assume that all eigenvalues of ˆ are absolutely strictly smaller than one. As the eigenvalues correspond to the inverses of the roots of the matrix polynomial det.ˆ.z// D det.In � ˆz/, this assumption implies that all roots must lie outside the unit circle:

det.In � ˆz/ ¤ 0 for all z 2 C with jzj � 1:

For the sake of exposition, we will further assume that ˆ is diagonalizable, i.e. there exists an invertible matrix P such that J D P�1ˆP is a diagonal matrix with the eigenvalues of ˆ on the diagonal.1

Consider now the stochastic process

Xt D Zt C ˆZt�1 C ˆ2Zt�2 C : : : D 1 X

jD0 ˆjZt�j:

We will show that this process is stationary and fulfills the first order difference equation above. For fXtg to be well-defined, we must show that

P1 jD0 kˆjk < 1.

Using the properties of the matrix norm we get:

1 X

jD0 kˆjk D

1 X

jD0 kPJjP�1k �

1 X

jD0 kPkkJj kP�1k

� 1 X

jD0 kPk kP�1k

n X

iD1 j ij2j

� kPk kP�1kpn 1 X

jD0 j maxj2j < 1;

where max denotes the maximal eigenvalue of ˆ in absolute terms. As all eigenvalues are required to be strictly smaller than one, this clearly also holds for max so that infinite matrix sum converges. This implies that the process fXtg is stationary. In addition, we have that

1The following exposition remains valid even if ˆ is not diagonalizable. In this case one has to rely on the Jordan form which complicates the computations (Meyer 2000).

12.1 The VAR(1) Process 217

Xt D 1 X

jD0 ˆ

jZt�j D Zt C ˆ 1 X

jD0 ˆ

jZt�1�j D ˆXt�1 C Zt:

Thus, the process fXtg also fulfills the difference equation. Next we demonstrate that this process is also the unique stationary solution to

the difference equation. Suppose that there exists another stationary process fYtg which also fulfills the difference equation. By successively iterating the difference equation one obtains:

Yt D Zt C ˆZt�1 C ˆ2Yt�2 : : :

D Zt C ˆZt�1 C ˆ2Zt�2 C : : : C ˆkZt�k C ˆkC1Yt�k�1:

Because fYtg is assumed to be stationary, VYt D VYt�k�1 D �.0/ so that

@Yt � k X

jD0 ˆjZt�j

A D ˆkC1V.Yt�k�1/ˆ0kC1 D ˆkC1�.0/ˆ0kC1:

The submultiplicativity of the norm then implies:

�

�ˆ kC1 �.0/ˆ

0kC1� � �

�

�ˆ kC1� �

2 k�.0/k D kPk2kP�1k2k�.0/k

n X

iD1 j ij2.kC1/

As all eigenvalues of ˆ are absolutely strictly smaller than one, the right hand side of the above expression converges to zero for k going to infinity. This implies that Yt and Xt D

P1 jD0 ˆ

jZt�j are equal in the mean square sense and thus also in probability.

Based on Theorem 10.2, the mean and the covariance function of the VAR(1) process is:

EXt D 1 X

jD0 ˆ

j EZt�j D 0;

�.h/ D 1 X

jD0 ˆjCh†ˆ0j D ˆh

1 X

jD0 ˆj†ˆ0j D ˆh�.0/:

Analogously to the univariate case, it can be shown that there still exists a unique stationary solution if all eigenvalues are absolutely strictly greater than one. This solution is, however, no longer causal with respect to fZtg. If some of the eigenvalues of ˆ are on the unit circle, there exists no stationary solution.

218 12 VARMA Processes

12.2 Representation in Companion Form

A VAR(p) process of dimension n can be represented as a VAR(1) process of dimen- sion p � n. For this purpose we define the pn vector Yt D .X0t; X0t�1; : : : ; X0t�pC1/0. This new process fYt g is characterized by the following first order stochastic difference equation:

Yt D

Xt Xt�1 Xt�2 :::

Xt�pC1

ˆ1 ˆ2 :: : ˆp�1 ˆp In 0 :: : 0 0 0 In : : : 0 0 :::

::: :::

0 0 : : : In 0

Xt�1 Xt�2 Xt�3 :::

Xt�p

Zt 0

0 :::

D ˆYt�1 C Ut

where Ut D .Zt;0;0; : : : ;0/0 with Ut � WN �

�

† 0

0 0

��

. This representation is

also known as the companion form or state space representation (see also Chap. 17). In this representation the last p.n �1/ equations are simply identities so that there is no error term attached. The latter name stems from the fact that Yt encompasses all the information necessary to describe the state of the system. The matrix ˆ is called the companion matrix of the VAR(p) process.2

The main advantage of the companion form is that by studying the properties of the VAR(1) model, one implicitly encompasses VAR models of higher order and also univariate AR(p) models which can be considered as special cases. The relation between the eigenvalues of the companion matrix and the roots of the polynomial matrix ˆ.z/ is given by the formula (Gohberg et al. 1982):

det �

Inp � ˆz � D det �In � ˆ1z � : : : � ˆpzp

�

: (12.2)

In the case of the AR(p) process the eigenvalues of ˆ are just the inverses of the roots of the polynomial ˆ.z/. Further elaboration of state space models is given in Chap. 17.

12.3 Causal Representation

As will become clear in Chap. 15 and particularly in Sect. 15.2, the issue of the existence of a causal representation is even more important than in the univariate

2The representation of a VAR(p) process in companion form is not uniquely defined. Permutations of the elements in Yt will lead to changes in the companion matrix.

12.3 Causal Representation 219

case. Before stating the main theorem let us generalize the definition of a causal representation from the univariate case (see Definition 2.2 in Sect. 2.3) to the multivariate one.

Definition 12.2. A VARMA((p,q) process fXtg with ˆ.L/Xt D ‚.L/Zt is called causal with respect to fZtg if and only if there exists a sequence of absolutely summable matrices f‰jg, j D 0;1;2; : : :, i.e.

P1 jD0 k‰jk < 1, such that

Xt D 1 X

jD0 ‰jZt�j:

Theorem 12.1. Let fXtg be a VARMA(p,q) process with ˆ.L/Xt D ‚.L/Zt and assume that

det ˆ.z/ ¤ 0 for all z 2 C with jzj � 1;

then the stochastic difference equation ˆ.L/Xt D ‚.L/Zt has exactly one stationary solution with causal representation

Xt D 1 X

jD0 ‰jZt�j;

whereby the sequence of matrices f‰jg is absolutely summable and where the matrices are uniquely determined by the identity

ˆ.z/‰.z/ D ‚.z/:

Proof. The proof is a straightforward extension of the univariate case. ut

As in the univariate case, the coefficient matrices which make up the causal representation can be found by the method of undetermined coefficients, i.e. by equating ˆ.z/‰.z/ D ‚.z/. In the case of the VAR(1) process, the f‰jg have to obey the following recursion:

0 W ‰0 D In z W ‰1 D ˆ‰0 D ˆ z2 W ‰2 D ˆ‰1 D ˆ2

:: :

zj W ‰j D ˆ‰j�1 D ˆj

The recursion in the VAR(2) case is:

220 12 VARMA Processes

0 W ‰0 D In z W �ˆ1 C ‰1 D 0 ) ‰1 D ˆ1 z2 W �ˆ2 � ˆ1‰1 C ‰2 D 0 ) ‰2 D ˆ2 C ˆ21 z3 W �ˆ1‰2 � ˆ2‰1 C ‰3 D 0 ) ‰3 D ˆ31 C ˆ1ˆ2 C ˆ2ˆ1

:: :

Remark 12.1. Consider a VAR(1) process with ˆ D �

0 �

0 0

�

with � ¤ 0 then the matrices in the causal representation are ‰j D ˆj D 0 for j > 1. This means that fXtg has an alternative representation as a VMA(1) process because Xt D Zt C ˆZt�1. This simple example demonstrates that the representation of fXtg as a VARMA process is not unique. It is therefore impossible to always distinguish between VAR and VMA process of higher orders without imposing additional assumptions. These additional assumptions are much more complex in the multivariate case and are known as identifying assumptions. Thus, a general treatment of this identification problem is outside the scope of this book. See Hannan and Deistler (1988) for a general treatment of this issue. For this reason we will concentrate exclusively on VAR processes where these identification issues do not arise.

Example

We illustrate the above concept by the following VAR(2) model:

Xt D �

0:8 �0:5 0:1 �0:5

�

Xt�1 C ��0:3 �0:3

�0:2 0:3 �

Xt�2 C Zt

with Zt � WN ��

�

;

�

1:0 0:4

0:4 2:0

��

In a first step, we check whether the VAR model admits a causal representation with respect to fZtg. For this purpose we have to compute the roots of the equation det.I2 � ˆ1z � ˆ2z2/ D 0:

det

�

1 � 0:8z C 0:3z2 0:5z C 0:3z2 �0:1z C 0:2z2 1 C 0:5z � 0:3z2

�

D 1 � 0:3z � 0:35z2 C 0:32z3 � 0:15z4 D 0:

12.4 Computation of Covariance Function 221

The four roots are: �1:1973;0:8828 ˙ 1:6669�;1:5650. As they are all outside the unit circle, there exists a causal representation which can be found from the equation ˆ.z/‰.z/ D I2 by the method of undetermined coefficients. Multiplying the equation system out, we get:

I2 � ˆ1z �ˆ2z2

C ‰1z�ˆ1‰1z2 � ˆ2‰1z3

C‰2z2 � ˆ1‰2z3�ˆ2‰2z4 : : : D I2:

Equating the coefficients corresponding to zj, j D 1;2; : : :: z W ‰1 D ˆ1 z2 W ‰2 D ˆ1‰1 C ˆ2 z3 W ‰3 D ˆ1‰2 C ˆ2‰1 :: : : : :

zj W ‰j D ˆ1‰j�1 C ˆ2‰j�2: The last equation shows how to compute the sequence f‰jg recursively:

‰1 D �

0:8 �0:5 0:1 �0:5

�

‰2 D �

0:29 �0:45 �0:17 0:50

�

‰3 D �

0:047 �0:310 �0:016 �0:345

�

: : :

12.4 Computation of the Covariance Function of a Causal VAR Process

As in the univariate case, it is important to be able to compute the covariance and the correlation function of VARMA process (see Sect. 2.4). As explained in Remark 12.1 we will concentrate on VAR processes. Consider first the case of a causal VAR(1) process:

Xt D ˆXt�1 C Zt Zt � WN.0;†/: Multiplying the above equation first by X0t and then successively by X

0 t�h from the

left, h D 1;2; : : :, and taking expectations, we obtain the Yule-Walker equations: E �

XtX 0 t

� D �.0/ D ˆE�Xt�1X0t � C E�ZtX0t

� D ˆ�.�1/ C †; E �

XtX 0 t�h � D �.h/ D ˆE�Xt�1X0t�h

� C E�Zt X0t�h � D ˆ�.h � 1/:

222 12 VARMA Processes

Knowing �.0/ and ˆ, �.h/, h > 0, can be computed recursively from the second equation as

�.h/ D ˆh�.0/; h D 1;2; : : : (12.3)

Given ˆ and †, we can compute �.0/. For h D 1, the second equation above implies �.1/ D ˆ�.0/. Inserting this expression in the first equation and using the fact that �.�1/ D �.1/0, we get an equation in �.0/:

�.0/ D ˆ�.0/ˆ0 C †:

This equation can be solved for �.0/:

vec�.0/ D vec.ˆ�.0/ˆ0/ C vec† D .ˆ ˝ ˆ/vec�.0/ C vec†;

where ˝ and “vec” denote the Kronecker-product and the vec-operator, respec- tively.3 Thus,

vec�.0/ D .In2 � ˆ ˝ ˆ/�1vec†: (12.4)

The assumption that fXtg is causal with respect to fZtg guarantees that the eigenvalues of ˆ ˝ ˆ are strictly smaller than one in absolute value, implying that In2 � ˆ ˝ ˆ is invertible.4

If the process is a causal VAR(p) process the covariance function can be found in two ways. The first one rewrites the process in companion form as a VAR(1) process and applies the procedure just outlined. The second way relies on the Yule-Walker equation. This equation is obtained by multiplying the stochastic difference equation from the left by X0t and then successively by X

0 t�h, h > 0, and taking expectations:

�.0/ D ˆ1�.�1/ C : : : C ˆp�.�p/ C †; D ˆ1�.1/0 C : : : C ˆp�.p/0 C †;

�.h/ D ˆ1�.h � 1/ C : : : C ˆp�.h � p/: (12.5)

The second equation can be used to compute �.h/, h � p, recursively taking ˆ1;: : : ;ˆp and the starting values �.p � 1/; : : : ;�.0/ as given. The starting value can be retrieved by transforming the VAR(p) model into the companion form and proceeding as explained above.

3The vec-operator stacks the column of a n � m matrix to a column vector of dimension nm. The properties of ˝ and vec can be found, e.g. in Magnus and Neudecker (1988). 4If the eigenvalues of ˆ are i, i D 1; : : : ; n, then the eigenvalues of ˆ˝ˆ are i j, i; j D 1; : : : ; n (see Magnus and Neudecker (1988)).

12.4 Computation of Covariance Function 223

Example

We illustrate the computation of the covariance function using the same example as in Sect. 12.3. First, we transform the model into the companion form:

Yt D

X1;t X2;t

X1;t�1 X2;t�1

0:8 �0:5 �0:3 �0:3 0:1 �0:5 �0:2 0:3 1 0 0 0

0 1 0 0

X1;t�1 X2;t�1 X1;t�2 X2;t�2

Z1;t Z2;t 0

Equation (12.4) implies that �Y.0/ is given by:

vec�Y.0/ D vec �

�X.0/ �X.1/

�X.1/ 0 �X.0/

�

D .I16 � ˆ ˝ ˆ/�1vec �

† 0

0 0

�

so that �X.0/ and �X.1/ become:

�X.0/ D �

2:4201 0:5759

0:5759 3:8978

�

�X.1/ D �

1:3996 �0:5711 �0:4972 �2:5599

�

The other covariance matrices can then be computed recursively according to Eq. (12.5):

�X.2/ D ˆ1�X.1/ C ˆ2�X.0/ D �

0:4695 �05191 0:0773 2:2770

�

;

�X.3/ D ˆ1�X.2/ C ˆ2�X.1/ D �

0:0662 �0:6145 �0:4208 �1:8441

�

Appendix: Autoregressive Final Form

Definition 12.1 defined the VARMA process fXtg as a solution to the corresponding multivariate stochastic difference equation (12.1). However, as pointed out by Zellner and Palm (1974) there is an equivalent representation in the form of n univariate ARMA processes, one for each Xit. Formally, these representations, also called autoregressive final form or transfer function form (Box and Jenkins 1976), can be written as

det ˆ.L/Xit D

ˆ � .L/‚.L/

i Zt

224 12 VARMA Processes

where the index i indicates the i-th row of ˆ�.L/‚.L/. Thereby ˆ�.L/ denotes the adjugate matrix of ˆ.L/.5 Thus each variable in Xt may be investigated separately as an univariate ARMA process. Thereby the autoregressive part will be the same for each variable. Note, however, that the moving-average processes will be correlated across variables.

The disadvantage of this approach is that it involves rather long AR and MA lags as will become clear from the following example.6 Take a simple two-dimensional VAR of order one, i.e. Xt D ˆXt�1CZt , Zt � WN.0;†/. Then the implied univariate processes will be ARMA(2,1) processes. After some straightforward manipulations we obtain:

.1 � .�11 C �22/L C .�11�22 � �12�21/L2/X1t D Z1t � �22Z1;t�1 C �12Z2;t�1;

.1 � .�11 C �22/L C .�11�22 � �12�21/L2/X2t D �21Z1;t�1 C Z2t � �11Z2;t�1:

It can be shown by the means given in Sects. 1.4.3 and 1.5.1 that the right hand sides are observationally equivalent to MA(1) processes.

5The elements of the adjugate matrix A� of some matrix A are given by ŒA� ij D .�1/iCj Mij where Mij is the minor (minor determinant) obtained by deleting the i-th column and the j-th row of A (Meyer 2000, p. 477). 6The degrees of the AR and the MA polynomial can be as large as np and .n�1/pCq, respectively.

13Estimation of Vector Autoregressive Models

13.1 Introduction

In this chapter we derive the Least-Squares (LS) estimator for vectorautoregressive (VAR) models and its asymptotic distribution. For this end, we have to make several assumption which we maintain throughout this chapter.

Assumption 13.1. The VAR process fXtg is generated by

ˆ.L/Xt D Zt Xt � ˆ1Xt�1 � � � � � ˆpXt�p D Zt with Zt � WN.0;†/;

† nonsingular, and admits a stationary and causal representation with respect to fZtg:

Xt D Zt C ‰1Zt�1 C ‰2Zt�2 C : : : D 1 X

jD0 ‰jZt D ‰.L/Zt

with P1

jD0 k‰jk < 1.

Assumption 13.2. The residual process fZtg is not only white noise, but also independently and identically distributed:

Zt � IID.0;†/:

Assumption 13.3. All fourth moments of Zt exist. In particular, there exists a finite constant c > 0 such that

E �

Zit ZjtZkt Zlt � � c for all i; j; k; l D 1;2; : : : ; n; and for all t:

225

226 13 Estimation of VAR Models

Note that the moment condition is automatically fulfilled by Gaussian processes. For the ease of exposition, we omit a constant in the VAR. Thus, we consider the demeaned process.

13.2 The Least-Squares Estimator

Let us denote by � .k/ ij the .i; j/-th element of the matrix ˆk , k D 1;2; : : : ; p, then the

i-th equation, i D 1; : : : ; n, can be written as

Xit D �.1/i1 X1;t�1 C : : : C �.1/in Xn;t�1 C : : : C �.p/i1 X1;t�p C : : : C �.p/in Xn;t�p C Zit:

We can view this equation as a regression equation of Xit on all lagged variables X1;t�1; : : : ; Xn;t�1; : : : ; X1;t�p; : : : ; Xn;t�p with error term Zit . Note that the regres- sors are the same for each equation. The np regressors have coefficient vector �

� .1/ i1 ; : : : ;�

.1/ in ; : : : ;�

.p/ i1 ; : : : ;�

.p/ in

�0 . Thus, the complete VAR(p) model has n2p

coefficients in total to be estimated. In addition, there are n.n C 1/=2 independent elements of the covariance matrix † that have to be estimated too.

It is clear that the n different equations are linked through the regressors and the errors terms which in general have non-zero covariances �ij D EZit Zjt. Hence, it seems warranted to take a systems approach and to estimate all equations of the VAR jointly. Below, we will see that an equation-by-equation approach is, however, still appropriate.

Suppose that we have T C p observations with t D �p C 1; : : : ;0;1; : : : ; T, then we can write the regressor matrix for each equation compactly as a T � np matrix X:

X D

X1;0 : : : Xn;0 : : : X1;�pC1 : : : Xn;�pC1 X1;1 : : : Xn;1 : : : X1;�pC2 : : : Xn;�pC2 :::

::: :::

X1;T�1 : : : Xn;T�1 : : : X1;T�p : : : Xn;T�p

Using this notation, we can write the VAR for observations t D 1;2; : : : ; T as

.X1; X2; : : : ; XT/ „ ƒ‚ …

DY D .ˆ1;ˆ2; : : : ;ˆp/ „ ƒ‚ …

Dˆ

X0 X1 : : : XT�1 X�1 X0 : : : XT�2 :::

::: :::

:::

X�pC1 X�pC2 : : : XT�p

„ ƒ‚ …

DX0

C .Z1; Z2; : : : ; ZT/ „ ƒ‚ …

13.2 The Least-Squares Estimator 227

or more compactly

Y D ˆX0 C Z:

There are two ways to bring this equation system in the usual multivariate regression framework. One can either arrange the data according to observations or according to equations. Ordered in terms of observations yields:

vec Y D vec.ˆX0/ C vec Z D .X ˝ In/ vec ˆ C vec Z (13.1)

with vec Y D .X11; X21; : : : ; Xn1; X12; X22; : : : ; Xn2; : : : ; X1T; X2T; : : : ; XnT/0. If the data are arranged equation by equation, the dependent variable is vec Y 0 D .X11; X12; : : : ; X1T; X21; X22; : : : ; X2T; : : : ; Xn1; Xn2; : : : ; XnT/

0. As both representa- tions, obviously, contain the same information, there exists a nT � nT permutation or commutation matrix KnT such that vec Y

0 D Knt vec Y. Using the computation rules for the Kronecker product, the vec operator, and the permutation matrix (see Magnus and Neudecker 1988), we get for the ordering in terms of equations

vec Y 0 D KnT vec Y D KnT �

vec.ˆX0/ C vec Z�

D KnT.X ˝ In/ vec ˆ C KnT vec Z D .In ˝ X/Kn2p vec ˆ C KnT vec Z D .In ˝ X/ vec ˆ0 C vec Z0 (13.2)

where Kn2p is the corresponding n 2�p permutation matrix relating vec ˆ and vec ˆ0.

The error terms of the different equations are correlated because, in general, the covariances �ij D EZit Zjt are nonzero. In the case of an arrangement by observation the covariance matrix of the error term vec Z is

V vec Z D E.vec Z/.vec Z/0

�21 : : : �1n 0 :: : 0 : : : 0 : : : 0 ::: :::

::: ::: :::

::: : : : ::: :::

:::

�n1 : : : � 2 n 0 :: : 0 : : : 0 : : : 0

0 0 0 �21 : : : �1n : : : 0 : : : 0 ::: :::

::: ::: :::

::: : : : ::: :::

:::

0 : : : 0 �n1 : : : � 2 n : : : 0 : : : 0

::: :::

::: ::: :::

::: :::

:::

0 : : : 0 0 : : : 0 : : : �21 : : : �1n ::: :::

::: ::: :::

::: : : : ::: :::

:::

0 : : : 0 0 : : : 0 : : : �n1 : : : � 2 n

D IT ˝ †:

228 13 Estimation of VAR Models

In the second case, the arrangement by equation, the covariance matrix of the error term vec Z0 is

V vec Z0 D E.vec Z0/.vec Z0/0

�21 : : : 0 �12 : : : 0 : : : �1n : : : 0 ::: :::

::: ::: :::

::: : : : ::: :::

:::

0 : : : �21 0 : : : �12 : : : 0 : : : �1n

�21 : : : 0 � 2 2 : : : 0 : : : �2n : : : 0

::: :::

::: ::: :::

::: : : : ::: :::

:::

0 : : : �21 0 : : : � 2 2 : : : 0 : : : �2n

::: :::

::: ::: :::

::: :::

:::

�n1 : : : 0 �n2 : : : 0 : : : � 2 n : : : 0

::: :::

::: ::: :::

::: : : : ::: :::

:::

0 : : : �n1 0 : : : �n2 : : : 0 : : : � 2 n

D † ˝ IT:

Given that the covariance matrix is not a multiple of the identity matrix, efficient estimation requires the use of generalized least squares (GLS). The GLS estimator minimizes the weighted sum of squared errors

S.vec ˆ/ D .vec Z/0.IT ˝ †/�1.vec Z/ �! min ˆ :

The solution of this minimization problem can be found in standard econometric textbooks like (Dhrymes 1978; Greene 2008; Hamilton 1994b) and is given by

.vec bˆ/GLS D �

.X ˝ In/0.IT ˝ †/�1.X ˝ In/ ��1

.X ˝ In/0.IT ˝ †/�1 vec Y D �.X0 ˝ In/.IT ˝ †�1/.X ˝ In/

��1 .X0 ˝ In/.IT ˝ †�1/ vec Y

D �.X0 ˝ †�1/.X ˝ In/ ��1

.X0 ˝ †�1/ vec Y D �.X0X/ ˝ †�1��1 .X0 ˝ †�1/ vec Y D �.X0X/�1 ˝ †�.X0 ˝ †�1/ vec Y D �..X0X/�1X0/ ˝ In

�

vec Y

D .vec bˆ/OLS As the covariance matrix † cancels, the GLS and the OLS-estimator deliver numerically exactly the same solution. The reason for this result is that the regressors are the same in each equation. If this does not hold, for example when some coefficients are set a priori to zero, efficient estimation would require the use of GLS.

13.2 The Least-Squares Estimator 229

Further insights can be gained by rewriting the estimation problem in terms of the arrangement by equation (see Eq. (13.2)). For this purpose, multiply the above estimator from the left by the commutation matrix Kn2p

.vec bˆ0/OLS D Kn2p.vec bˆ/OLS D Kn2p �

..X0X/�1X0/ ˝ In �

vec Y

D �In ˝ ..X0X/�1X0/ �

KnT vec Y D �

In ˝ ..X0X/�1X0/ �

vec Y 0:

This can be written in a more explicit form as

.vec bˆ0/OLS D

.X0X/�1X0 0 :: : 0 0 .X0X/�1X0 : : : 0 :::

::: :::

:::

0 0 : : : .X0X/�1X0

vec Y 0

.X0X/�1X0Y1 0 : : : 0 0 .X0X/�1X0Y2 : : : 0 :::

::: :::

:::

0 0 : : : .X0X/�1X0Yn

where Yi, i D 1; : : : ; n, stacks the observations of the i-th variable such that Yi D .Xi1; Xi2; : : : ; XiT/

0. Thus, the estimation of VAR as a system can be broken down into the estimation of n regression equations with dependent variable Xit. Each of these equations can then be estimated by OLS.

Thus, we have proven that

vec bˆ D .vec bˆ/GLS D .vec bˆ/OLS D �

..X0X/�1X0/ ˝ In �

vec Y; (13.3)

vec bˆ0 D .vec bˆ0/GLS D .vec bˆ0/OLS D �

In ˝ ..X0X/�1X0/ �

vec Y 0: (13.4)

The least squares estimator can also be rewritten without the use of the vec-operator:

bˆ D YX.X0X/�1:

Under the assumptions stated in the Introduction Sect. 13.1, these estimators are consistent and asymptotically normal.

Theorem 13.1 (Asymptotic Distribution of OLS Estimator). Under the assumption stated in the Introduction Sect. 13.1, it holds that

plim bˆ D ˆ

1Alternatively, one could start from scratch and investigate the minimization problem S.vec ˆ0/ D .vec Z0/0.†�1 ˝ IT/.vec Z0/ ! minˆ.

230 13 Estimation of VAR Models

and that

by observation: p

T �

vec bˆ � vec ˆ �

d��! N �0; � �1p ˝ † �

;

respectively,

by equation: p

T �

vec bˆ0 � vec ˆ0 �

d��! N �0;† ˝ � �1p �

where � p D plim 1T .X0X/.

Proof. See Sect. 13.3. ut

In order to make use of this result in practice, we have to replace the matrices † and � p by some estimate. A natural consistent estimate of � p is given according to Proposition 13.1 by

b� p D X0X

T :

In analogy to the multivariate regression model, a natural estimator for † can be obtained from the Least-Squares residuals bZ:

b† D 1 T

T X

tD1 bZtbZ

0 t D

bZbZ0

T D .Y �

bˆX0/.Y � bˆX0/0 T

The property of this estimator is summarized in the proposition below.

Theorem 13.2. Under the condition of Theorem 13.1

plim p

�

b† � ZZ 0

�

D 0

Proof. See Sect. 13.3. ut

An alternative, but asymptotically equivalent estimator e† is obtained by adjust- ing b† for the degrees of freedom:

e† D T T � np

b†: (13.5)

If the VAR contains a constant, as is normally the case in practice, the degrees of freedom correction should be T � np � 1.

13.3 Proofs of Asymptotic Normality 231

Small sample inference with respect to the parameters ˆ can therefore be carried out using the approximate distribution

vec bˆ � N �

vec ˆ;b† ˝ .X0X/�1 �

: (13.6)

This implies that hypothesis testing can be carried out using the conventional t- and F-statistics. From a system perspective, the appropriate degree of freedom for the t-ratio would be nT � n2p � n, taking a constant in each equation into account. However, as that the system can be estimated on an equation by equation basis, it seems reasonable to use T � np � 1 instead. This corresponds to a multivariate regression setting with T observation and np C 1 regressors, including a constant.

However, as in the univariate case the Gauss Markov theorem does not apply because the lagged regressors are correlated with past error terms. This results in biased estimates in small samples. The amount of the bias can be assessed and corrected either by analytical or bootstrap methods. For an overview, a comparison of the different corrections proposed in the literature, and further references see Engsteg and Pedersen (2014).

13.3 Proofs of the Asymptotic Properties of the Least-Squares Estimator

Lemma 13.1. Given the assumptions made in Sect. 13.1, the process fvec Zt�jZ0t�ig, i; j 2 Z and i ¤ j, is white noise.

Proof. Using the independence assumption of fZtg, we immediately get

E vec Zt�jZ0t�i D E.Zt�i ˝ Zt�j/ D 0; V.vec Zt�jZ0t�i/ D E

�

.Zt�i ˝ Zt�j/.Zt�i ˝ Zt�j/0 �

D E�.Zt�iZ0t�i/ ˝ .Zt�jZ0t�j/ � D † ˝ †;

�vec Zt�j Z0t�i .h/ D E�.Zt�i ˝ Zt�j/.Zt�i�h ˝ Zt�j�h/0

�

D E�.Zt�iZ0t�i�h/ ˝ .Zt�jZ0t�j�h/ � D 0; h ¤ 0:

Under the assumption put forward in the Introduction, 1 T .X0X/ converges in

probability for T ! 1 to a np � np matrix � p. This matrix consists of p2 blocks where each .i; j/-th block corresponds to the covariance matrix �.i � j/. Thus we have the following proposition:

Proposition 13.1. Under the assumption stated in the Introduction Sect. 13.1

232 13 Estimation of VAR Models

X0X T

p��! � p D

�.0/ �.1/ : : : �.p � 1/ �0.1/ �.0/ : : : �.p � 2/ :::

::: :::

:::

�0.p � 1/ �0.p � 2/ : : : �.0/

with � p being nonsingular.

Proof. Write 1 T .X0X/ as

X0X T

b�.0/ b�.1/ : : : b�.p � 1/ b�0.1/ b�.0/ : : : b�.p � 2/ :::

::: :::

::: b�0.p � 1/ b�0.p � 2/ : : : b�.0/

where

b�.h/ D 1 T

T�1 X

tD0 XtX

0 t�h; h D 0;1; : : : ; p � 1:

We will show that each component b�.h/ of 1 T

X0X converges in probability to �.h/. Taking the causal representation of fXtg into account

b�.h/ D 1 T

T�1 X

tD0 XtX

0 t�h D

T�1 X

tD0

1 X

jD0

1 X

iD0 ‰jZt�jZ0t�h�i‰

0 i

D 1 X

jD0

1 X

iD0 ‰j

T�1 X

tD0 Zt�jZ0t�h�i

‰0i

D 1 X

jD0

1 X

iDh ‰j

T�1 X

tD0 Zt�jZ0t�i

‰0i�h:

According to Lemma 13.1 above fZt�jZ0t�ig, i ¤ j, is white noise. Thus,

T�1 X

tD0 Zt�jZ0t�i

p��! 0; i ¤ j;

13.3 Proofs of Asymptotic Normality 233

according to Theorem 11.1. Hence, for m fixed,

Gm.h/ D m X

jD0

mCh X

iDh i¤j

‰j

T�1 X

tD0 Zt�jZ0t�i

‰0i�h p��! 0:

Taking absolute values and expectations element-wise,

E jG1.h/ � Gm.h/j D E

j>m or i>mCh i¤j

‰j

T�1 X

tD0 Zt�jZ0t�i

‰0i�h

� X

j>m or i>mCh i¤j

j‰jj

T�1 X

tD0 EjZt�jZ0t�ij

j‰0i�hj

� X

j>m or i>mCh i¤j

j‰jj �

EjZ1Z02j � j‰0i�hj

� X

j>m or i>m i¤j

j‰jj �

EjZ1Z02j � j‰0i j

� X

j>m

j‰jj

EjZ1Z02j X

j‰0i j !

C 0

j‰jj EjZ1Z02j 1

i>m

j‰0i j

As the bound is independent of T and converges to 0 as m ! 1, we have lim

m!1 lim supT!1 E jG1.h/ � Gm.h/j D 0:

The Basic Approximation Theorem C.14 then establishes that

G1.h/ p��! 0:

Henceforth

b�.h/ D G1.h/ C 1 X

jDh ‰j

T�1 X

tD0 Zt�jZ0t�j

‰0j�h

D G1.h/ C 1 X

jDh ‰j

T�1 X

tD0 ZtZ

0 t

‰0j�h C remainder

234 13 Estimation of VAR Models

where the remainder only depends on initial conditions2 and is therefore negligible as T ! 1. As

T�1 X

tD0 ZtZ

0 t

p��! †;

we finally get

b�.h/ p��!

1 X

jDh ‰j†‰

0 j�h D �.h/:

The last equality follows from Theorem 10.2. ut

Proposition 13.2. Under the assumption stated in the Introduction Sect. 13.1

1p T

T X

tD1 vec.ZtX

0 t�1; Zt X

0 t�2; : : : ; ZtX

0 t�p/

D 1p T

vec.ZX/ D 1p T .X0 ˝ In/ vec Z

d��! N.0; � p ˝ †/

Proof. The idea of the proof is to approximate fXtg by some simpler process fX.m/t g which allows the application of the CLT for dependent processes (Theorem C.13). This leads to an asymptotic distribution which by the virtue of the Basic Approxima- tion Theorem C.14 converges to the asymptotic distribution of the original process. Define X

.m/ t as the truncated process from the causal presentation of Xt:

X .m/ t D Zt C ‰1Zt�1 C : : : C ‰mZt�m; m D p; p C 1; p C 2; : : :

Using this approximation, we can then define the process fY.m/t g as

Y .m/ t D vec

�

Zt X .m/0 t�1 ; ZtX

.m/0 t�2 ; : : : ; Zt X

.m/0 t�p �

X .m/ t�1

X .m/ t�2 :::

X .m/ t�p

˝ Zt:

2See the proof of Theorem 11.2.2 in Brockwell and Davis (1991) for details.

13.3 Proofs of Asymptotic Normality 235

Due to the independence of fZtg this process is a mean zero white noise process, but is clearly not independent. It is easy to see that the process is actually .m C p/- dependent with variance Vm given by

Vm D EY.m/t Y.m/0t D E

X .m/ t�1

X .m/ t�2 :::

X .m/ t�p

˝ Zt

X .m/ t�1

X .m/ t�2 :::

X .m/ t�p

˝ Zt

X .m/ t�1

X .m/ t�2 : : :

X .m/ t�p

X .m/ t�1

X .m/ t�2 : : :

X .m/ t�p

˝ EZt Z0t

�.m/.0/ �.m/.1/ : : : �.m/.p � 1/ �.m/.1/0 �.m/.0/ : : : �.m/.p � 2/

::: :::

�.m/.p � 1/0 �.m/.p � 2/0 : : : �.m/.0/

˝ †

D �.m/p ˝ †

where � .m/ p is composed of

� .m/ .h/ D EX.m/t�1X.m/0t�1�h

D E.Zt�1 C ‰1Zt�2 C : : : C ‰mZt�1�m/ .Zt�1�h C ‰1Zt�2�h C : : : C ‰mZt�1�m�h/0

D m X

jDh ‰j†‰

0 j�h; h D 0;1; : : : ; p � 1:

Thus, we can invoke the CLT for .m C p/-dependent process (see Theorem C.13) to establish that

p T

T X

tD1 Y .m/ t

d��! N.0; Vm/:

For m ! 1, �.m/.h/ converges to �.h/ and thus �.m/p to � p. Therefore, Vm �! � p ˝ †.

236 13 Estimation of VAR Models

The variance of the approximation error is equal to

1p T

T X

tD1

�

vec.ZtX 0 t�1; : : : ; ZtX

0 t�p/ � Y.m/t

�

D 1 T V

T X

tD1 vec

�

Zt.Xt�1 � X.m/t�1/0; : : : ; Zt.Xt�p � X.m/t�p/0 �

D 1 T V

T X

tD1 vec

@Zt

1 X

jDmC1 ‰jZt�1�j

; : : : ; Zt

1 X

jDmC1 ‰jZt�p�j

D V 0

@vec

@Zt

1 X

jDmC1 ‰jZt�1�j

; : : : ; Zt

1 X

jDmC1 ‰jZt�p�j

D E

P1 jDmC1 ‰jZt�1�j

::: P1

jDmC1 ‰jZt�p�j

A ˝ Zt

P1 jDmC1 ‰jZt�1�j

::: P1

jDmC1 ‰jZt�p�j

A ˝ Zt

P1 jDmC1 ‰j†‰

0 j : : : : : :

::: :::

:::

: : : : : : P1

jDmC1 ‰j†‰ 0 j

A ˝ †:

The absolute summability of ‰j then implies that the infinite sums converge to

zero as m ! 1. As X.m/t m.s.��! Xt for m ! 1, we can apply the Basic

Approximation Theorem C.14 to reach the required conclusion

1p T

T X

tD1 vec.Zt X

0 t�1; Zt X

0 t�2; : : : ; Zt X

0 t�p/

d��! N.0; � p ˝ †/: ut

Proof of Theorem 13.1

Proof. We prove the Theorem for the arrangement by observation. The prove for the arrangement by equation can be proven in a completely analogous way. Inserting the regression formula (13.1) into the least-squares formula (13.3) leads to:

vec bˆ D ...X0X/�1X0/ ˝ In/.X ˝ In/ vec ˆ C ...X0X/�1X0/ ˝ In/ vec Z D vec ˆ C ...X0X/�1X0/ ˝ In/ vec Z: (13.7)

13.3 Proofs of Asymptotic Normality 237

Bringing vec ˆ to the left hand side and taking the probability limit, we get using Slutzky’s Lemma C.10 for the product of probability limits

plim.vec bˆ � vec ˆ/ D plim vec �ZX.X0X/�1�

D vec

plim ZX T

plim

�

X0X T

��1! D 0:

The last equality follows from the observation that Proposition 13.1 implies plim X

0X T

D � p nonsingular and that Proposition 13.2 implies plim ZXT D 0. Thus, we have established that the Least-Squares estimator is consistent.

Equation (13.7) further implies: p

T.vec bˆ � vec ˆ/ D p

T �

..X0X/�1X0/ ˝ In �

vec Z

�

X0X T

��1 ˝ In

1p T .X0 ˝ In/ vec Z

As plim X 0X T

D � p nonsingular, the above expression converges in distribution according to Theorem C.10 and Proposition 13.2 to a normally distributed random variable with mean zero and covariance matrix

.� �1p ˝ In/.� p ˝ †/.� �1p ˝ In/ D � �1p ˝ † ut

Proof of Theorem 13.2

Proof.

b† D .Y � bˆX0/.Y � bˆX0/0

D .Y � ˆX 0 C .ˆ � bˆ/X0/.Y � ˆX0 C .ˆ � bˆ/X0/0

D 1 T .Z C .ˆ � bˆ/X0/.Z C .ˆ � bˆ/X0/0

D ZZ 0

T C ZX

T .ˆ � bˆ/0 C .ˆ � bˆ/X

0Z0

T C .ˆ � bˆ/X

0X T .ˆ � bˆ/0

Applying Theorem C.7 and the results of Propositions 13.1 and 13.2 shows that

ZX.ˆ � bˆ/0p T

p��! 0

238 13 Estimation of VAR Models

and

.ˆ � bˆ/X 0X T

p T.ˆ � bˆ/0 p��! 0:

Hence,

p T

.Y � bˆX0/.Y � bˆX0/0 T

� ZZ 0

D p

�

b† � ZZ 0

�

p��! 0

13.4 The Yule-Walker Estimator

An alternative estimation method can be derived from the Yule-Walker equations. Consider first a VAR(1) model. The Yule-Walker equation in this case simply is:

�.0/ D ˆ�.�1/ C † �.1/ D ˆ�.0/

�.0/ D ˆ�.0/ˆ0 C † �.1/ D ˆ�.0/:

The solution of this system of equations is:

ˆ D �.1/�.0/�1

† D �.0/ � ˆ�.0/ˆ0 D �.0/ � �.1/�.0/�1�.0/�.0/�1�.1/0

D �.0/ � �.1/�.0/�1�.1/0:

Replacing the theoretical moments by their empirical counterparts, we get the Yule- Walker estimator for ˆ and †:

bˆ D b�.1/b�.0/�1; b† D b�.0/ � bˆb�.0/bˆ0:

13.4 The Yule-Walker Estimator 239

In the general case of a VAR(p) model the Yule-Walker estimator is given as the solution of the equation system

b�.h/ D p X

jD1 bˆjb�.h � j/; k D 1; : : : ; p;

b† D b�.0/ � bˆ1b�.�1/ � : : : � bˆpb�.�p/

As the least-squares and the Yule-Walker estimator differ only in the treatment of the starting values, they are asymptotically equivalent. In fact, they yield very similar estimates even for finite samples (see e.g. Reinsel (1993)). However, as in the univariate case, the Yule-Walker estimator always delivers, in contrast to the least- square estimator, coefficient estimates with the property det.In � Ô 1z�: : :� Ô pzp/ ¤ 0 for all z 2 C with jzj � 1. Thus, the Yule-Walker estimator guarantees that the estimated VAR possesses a causal representation. This, however, comes at the price that the Yule-Walker estimator has a larger small-sample bias than the least-squares estimator, especially when the roots of ˆ.z/ get close to the unit circle (Tjøstheim and Paulsen 1983; Shaman and Stine 1988; Reinsel 1993). Thus, it is generally preferable to use the least-squares estimator in practice.

14Forecasting with VAR Models

14.1 Forecasting with Known Parameters

The discussion of forecasting with VAR models proceeds in two steps. First, we assume that the parameters of the model are known. Although this assumption is unrealistic, it will nevertheless allow us to introduce and analyze important concepts and ideas. In a second step, we then investigate how the results established in the first step have to be amended if the parameters are estimated. The analysis will focus on stationary and causal VAR(1) processes. Processes of higher order can be accommodated by rewriting them in companion form. Thus we have:

Xt D ˆXt�1 C Zt; Zt � WN.0;†/;

Xt D Zt C ‰1Zt�1 C ‰2Zt�2 C : : : D 1 X

jD0 ‰jZt�j;

where ‰j D ˆj. Consider then the following forecasting problem: Given observa- tions fXT; XT�1; : : : ; X1g, find a linear function, called predictor or forecast function, PT XTCh, h � 1, which minimizes the expected quadratic forecast error

E.XTCh � PT XTCh/0 .XTCh � PT XTCh/ D E tr.XTCh � PT XTCh/.XTCh � PT XTCh/0:

Thereby “tr” denotes the trace operator which takes the sum of the diagonal elements of a matrix. As we rely on linear forecasting functions, PT XTCh can be expressed as

PT XTCh D A1XT C A2XT�1 C : : : C AT X1 (14.1)

241

242 14 Forecasting with VAR Models

with matrices A1; A2; : : : ; AT still to be determined. In order to simplify the exposition, we already accounted for the fact that the mean of fXtg is zero.1 A justification for focusing on linear least-squares forecasts is given in Chap. 3. The first order conditions for the least-squares minimization problem are given by the normal equations:

E �

XTCh � PT XTCh �

X0s D E �

XTCh � A1XT � : : : � AT X1 �

X0s

D EXTCh X0s � A1EXT X0s � : : : � AT EX1X0s D 0; 1 � s � T: These equations state that the forecast error .XTCh � PT XTCh/ must be uncorrelated with the available information Xs, s D 1;2; : : : ; T. The normal equations can be written as

.A1; A2; : : : ; AT/

�.0/ �.1/ : : : �.T � 1/ �0.1/ �.0/ : : : �.T � 2/ :::

::: :::

:::

�0.T � 1/ �0.T � 2/ : : : �.0/

D ��.h/ �.h C 1/ : : : �.T C h � 1/ �

Denoting by � T the matrix

� T D

�.0/ �.1/ : : : �.T � 1/ �0.1/ �.0/ : : : �.T � 2/ :::

::: :::

:::

�0.T � 1/ �0.T � 2/ : : : �.0/

;

the normal equations can be written more compactly as

.A1; A2; : : : ; AT/ � T D �

�.h/ �.h C 1/ : : : �.T C h � 1/ �

Using the assumption that fXtg is a VAR(1), �.h/ can be expressed as �.h/ D ˆh�.0/ (see Eq. (12.3)) so that the normal equations become

.A1; A2; : : : ; AT/

�.0/ ˆ�.0/ : : : ˆT�1�.0/ �.0/ˆ0 �.0/ : : : ˆT�2�.0/ :::

::: :::

:::

�.0/ˆ0T�1 �.0/ˆ0T�2 : : : �.0/

D �ˆh�.0/ ˆhC1�.0/ : : : ˆTCh�1�.0/ �

1If the mean is non-zero, a constant A0 must be added to the forecast function.

14.1 Forecasting with Known Parameters 243

The easily guessed solution is given by A1 D ˆh and A2 D : : : D AT D 0. Thus, the sought-after forecasting function for the VAR(1) process is

PT XTCh D ˆhXT: (14.2) The forecast error XTCh � PT XTCh has expectation zero. Thus, the linear least-

squares predictor delivers unbiased forecasts. As

XTCh D ZTCh C ˆZTCh�1 C : : : C ˆh�1ZTC1 C ˆhXT; the expected squared forecast error (mean squared error) MSE.h/ is

MSE.h/ D E�XTCh � ˆhXT ��

XTCh � ˆhXT �0

D † C ˆ†ˆ0 C : : : C ˆh�1†ˆ0h�1 D h�1 X

jD0 ˆ

j †ˆ

0j : (14.3)

In order to analyze the case of a causal VAR(p) process with T > p, we transform the model into the companion form. For h D 1, we can apply the result above to get:

PT YTC1 D ˆYT D

PT XTC1 XT

XT�1 :::

XT�pC2

ˆ1 ˆ2 :: : ˆp�1 ˆp In 0 :: : 0 0 0 In : : : 0 0 :::

::: :::

0 0 : : : In 0

XT XT�1 XT�2 :::

XT�pC1

This implies that

PT XTC1 D ˆ1XT C ˆ2XT�1 C : : : C ˆpXT�pC1: (14.4) The forecast error is XTC1 � PT XTC1 D Zt which has mean zero and covariance variance matrix †. In general we have that PT YTCh D ˆhYT so that PT XTCh is equal to

PT XTCh D ˆ.h/1 XT C ˆ.h/2 XT�1 C : : : C ˆ.h/p XT�pC1 where ˆ

.h/ i , i D 1; : : : ; p, denote the blocks in the first row of ˆh. Alternatively, the

forecast for h > 1 can be computed recursively. For h D 2 this leads to:

PT XTC2 D PT .ˆ1XTC1/ C PT .ˆ2XT/ C : : : C PT �

ˆpXTC2�p � C PT .ZTC2/

D ˆ1 �

ˆ1XT C ˆ2XT�1 C : : : C ˆpXTC1�p �

C ˆ2XT C : : : C ˆpXTC2�p D �ˆ21 C ˆ2

�

XT C .ˆ1ˆ2 C ˆ3/ XT�1 C : : : C �

ˆ1ˆp�1 C ˆp �

XTC2�p C ˆ1ˆpXTC1�p:

244 14 Forecasting with VAR Models

For h > 2 we proceed analogously. This way of producing forecasts is sometimes called iterated forecasts.

In general, the forecast error of a causal VAR(p) process can be expressed as

XTCh � PT XTCh D ZTCh C ‰1ZTCh�1 C : : : C ‰h�1ZTC1

D h�1 X

jD0 ˆjZTCh�j:

The MSE.h/ then is:

MSE.h/ D † C ‰1†‰01 C : : : C ‰h�1†‰0h�1 D h�1 X

jD0 ‰j†‰

0 j: (14.5)

Example

Consider again the VAR(2) model of Sect. 12.3. The forecast function in this case is then:

PT XTC1 D ˆ1Xt C ˆ2Xt�1

D �

0:8 �0:5 0:1 �0:5

�

Xt C ��0:3 �0:3

�0:2 0:3 �

Xt�1;

PT XTC2 D .ˆ21 C ˆ2/Xt C ˆ1ˆ2Xt�1

D �

0:29 �0:45 �0:17 0:50

�

Xt C ��0:14 �0:39 0:07 �0:18

�

Xt�1;

PT XTC3 D .ˆ31 C ˆ1ˆ2 C ˆ2ˆ1/Xt C .ˆ21ˆ2 C ˆ22/Xt�1

D �

0:047 �0:310 �0:016 �0:345

�

Xt C �

0:003 �0:222 �0:049 0:201

�

Xt�1:

Based on the results computed in Sect. 12.3, we can calculate the corresponding mean squared errors (MSE):

MSE.1/ D † D �

1:0 0:4

0:4 2:0

�

;

MSE.2/ D † C ‰1†‰01 D �

1:82 0:80

0:80 2:47

�

;

MSE.3/ D † C ‰1†‰01 C ‰2†‰02 D �

2:2047 0:3893

0:3893 2:9309

�

A practical forecasting exercise with additional material is presented in Sect. 14.4.

14.2 Forecasting with Estimated Parameters 245

14.1.1 Wold Decomposition Theorem

At this stage we note that Wold’s theorem or Wold’s Decomposition carries over to the multivariate case (see Sect. 3.2 for the univariate case). This Theorem asserts that there exists for each purely non-deterministic stationary process2 a decomposition, respectively representation, of the form:

Xt D � C 1 X

jD0 ‰jZt�j;

where ‰0 D In, Zt � WN.0;†/ with † > 0 and P1

jD0 k‰jk2 < 1. The innovations fZtg have the property Zt D Xt � ePt�1Xt and consequently Zt D ePt Zt. Thereby ePt denotes the linear least-squares predictor based on the infinite past fXt; Xt�1; : : :g. The interpretation of the multivariate case is analogous to the univariate one.

14.2 Forecasting with Estimated Parameters

In practice the parameters of the VAR model are usually unknown and have therefore to be estimated. In the previous Section we have demonstrated that

PT XTCh D ˆ1PT XTCh�1 C : : : C ˆpPT XTCh�p where PT XTCh�j D YTCh�j if j � h. Replacing the true parameters by their estimates, we get the forecast function

bPT XTCh D bˆ1bPT XTCh�1 C : : : C bˆpbPT XTCh�p: where a hat indicates the use of estimates. The forecast error can then be decom- posed into two components:

XTCh � bPT XTCh D .XTCh � PT XTCh/ C �

PT XTCh � bPT XTCh �

D h�1 X

jD0 ˆjZTCh�j C

�

PT XTCh � bPT XTCh �

: (14.6)

Dufour (1985) has shown that, under the assumption of symmetrically distributed Zt’s (i.e. if Zt and �Zt have the same distribution) the expectation of the forecast error is zero even when the parameters are replaced by their least-squares estimates.

2A stationary stochastic process is called deterministic if it can be perfectly forecasted from its infinite past. It is called purely non-deterministic if there is no deterministic component (see Sect. 3.2).

246 14 Forecasting with VAR Models

This result holds despite the fact that these estimates are biased in small samples. Moreover, the results do not assume that the model is correctly specified in terms of the order p. Thus, under quite general conditions the forecast with estimated

coefficients remains unbiased so that E �

XTCh � bPT XTCh �

D 0. If the estimation is based on a different sample than the one used for forecasting,

the two terms in the above expression are uncorrelated so that its mean squared error is by the sum of the two mean squared errors:

bMSE.h/ D h�1 X

jD0 ‰j†‰

0 j

CE �

PT XTCh � bPT XTCh ��

PT XTCh � bPT XTCh �0 : (14.7)

The last term can be evaluated by using the asymptotic distribution of the coeffi- cients as an approximation. The corresponding formula turns out to be cumbersome. The technical details can be found in Lütkepohl (2006) and Reinsel (1993). The formula can, however, be simplified considerably if we consider a forecast horizon of only one period. We deduce the formula for a VAR of order one, i.e. taking Xt D ˆXt�1 C Zt , Zt � WN.0;†/.

PT XTCh � bPT XTCh D .ˆ � bˆ/XT D vec �

.ˆ � bˆ/XT �

D .X0T ˝ In/ vec.ˆ � bˆ/:

This implies that

�

PT XTCh � bPT XTCh ��

PT XTCh � bPT XTCh �0

D E.X0T ˝ In/ vec.ˆ � bˆ/.vec.ˆ � bˆ//0.XT ˝ In/

D E.X0T ˝ In/ � �11 ˝ †

T .XT ˝ In/ D

T E.X0T �

�1 1 XT/ ˝ †

D 1 T E.trX0T �

�1 1 XT/ ˝ † D

T tr.� �11 E.XT X

0 T// ˝ †

D 1 T .tr.In/ ˝ †/ D

T †:

Thereby, we have used the asymptotic normality of the least-squares estimator (see Theorem 13.1) and the assumption that forecasting and estimation uses different realizations of the stochastic process. Thus, for h D 1 and p D 1, we get

bMSE.1/ D † C n T † D T C n

T †:

14.3 Modeling of VAR Models 247

Higher order models can be treated similarly using the companion form of VAR(p). In this case:

bMSE.1/ D † C np T † D T C np

T †: (14.8)

This is only an approximation as we applied asymptotic results to small sample entities. The expression shows that the effect of the substitution of the coefficients by their least-squares estimates vanishes as the sample becomes large. However, in small sample the factor TCnp

T can be sizeable. In the example treated in Sect. 14.4,

the covariance matrix †, taking the use of a constant into account and assuming 8 lags, has to be inflated by TCnpC1

T D 196C4�8C1

196 D 1:168. Note also that the

precision of the forecast, given †, diminishes with the number of parameters.

14.3 Modeling of VAR Models

The previous section treated the estimation of VAR models under the assumption that the order of the VAR, p, is known. In most cases, this assumption is unrealistic as the order p is unknown and must be retrieved from the data. We can proceed analogously as in the univariate case (see Sect. 5.1) and iteratively test the hypothesis that coefficients corresponding to the highest lag, i.e. ˆp D 0, are simultaneously equal to zero. Starting from a maximal order pmax, we test the null hypothesis that ˆpmax D 0 in the corresponding VAR.pmax/ model. If the hypothesis is not rejected, we reduce the order by one to pmax � 1 and test anew the null hypothesis ˆpmax �1 D 0 using the smaller VAR.pmax � 1/ model. One continues in this way until the null hypothesis is rejected. This gives, then, the appropriate order of the VAR. The different tests can be carried out either as Wald-tests (F-tests) or as likelihood-ratio tests (�2-tests) with n2 degrees of freedom.

An alternative procedure to determine the order of the VAR relies on some information criteria. As in the univariate case, the most popular ones are the Akaike (AIC), the Schwarz or Bayesian (BIC) and the Hannan-Quinn criterion (HQC). The corresponding formula are:

AIC(p): ln det e†p C 2pn2

T ;

BIC(p): ln det e†p C pn2

T ln T;

HQC(p): ln det e†p C 2pn2

T ln .ln T/;

where e†p denotes the degree of freedom adjusted estimate of the covariance matrix † for a model of order p (see equation(13.5)). n2p is the number of estimated coefficients. The estimated order is then given as the minimizer of one of these

248 14 Forecasting with VAR Models

criteria. In practice the Akaike’s criterion is the most popular one although it has a tendency to deliver orders which are too high. The BIC and the HQ-criterion on the other hand deliver the correct order on average, but can lead to models which suffer from the omitted variable bias when the estimated order is too low. Examples are discussed in Sects. 14.4 and 15.4.5.

Following Lütkepohl (2006), Akaike’s information criterion can be rationalized as follows. Take as a measure of fit the determinant of the one period approximate mean-squared errors bMSE.1/ from Eq. (14.8) and take as an estimate of † the degrees of freedom corrected version in Eq. (13.5). The resulting criterion is called according to Akaike (1969) the final prediction error (FPE):

FPE.p/ D det �

T C np T

� T T � np

e†

�

D �

T C np T � np

�n

det e†: (14.9)

Taking logs and using the approximations TCnp T�np � 1 C

2np T

and log.1 C 2np T / � 2np

T ,

we arrive at

AIC.p/ � log FPE.p/:

14.4 Example: A VAR Model for the U.S. Economy

In this section, we illustrate how to build and use VAR models for forecasting key macroeconomic variables. For this purpose, we consider the following four variables: GDP per capita (fYtg), price level in terms of the consumer price index (CPI) (fPtg), real money stock M1 (fMtg), and the three month treasury bill rate (fRtg). All variables are for the U.S. and are, with the exception of the interest rate, in logged differences.3 The components of Xt are with the exception of the interest rate stationary.4 Thus, we aim at modeling Xt D .� log Yt;� log Pt;� log Mt; Rt/0. The sample runs from the first quarter 1959 to the first quarter 2012. We estimate our models, however, only up to the fourth quarter 2008 and reserve the last thirteen quarters, i.e. the period from the first quarter 2009 to first quarter of 2012, for an out- of-sample evaluation of the forecast performance. This forecast assessment has the advantage to account explicitly of the sampling variability in estimated parameter models.

The first step in the modeling process is the determination of the lag-length. Allowing for a maximum of twelve lags, the different information criteria produce the values reported in Table 14.1. Unfortunately, the three criteria deliver different

3Thus, � log Pt equals the inflation rate. 4Although the unit root test indicate that Rt is integrated of order one, we do not difference this variable. This specification will not affect the consistency of the estimates nor the choice of the lag-length (Sims et al. 1990), but has the advantage that each component of Xt is expressed in percentage points which facilitates the interpretation.

14.4 Example: VAR Model 249

Table 14.1 Information criteria for the VAR models of different orders

Order AIC BIC HQ

0 �14.498 �14.429 �14.470 1 �17.956 �17.611 �17.817 2 �18.638 �18.016 �18.386 3 �18.741 �17.843 �18.377 4 �18.943 �17.768 �18.467 5 �19.081 �17.630 �18.493 6 �19.077 �17.349 �18.377 7 �19.076 �17.072 �18.264 8 �19.120 �16.839 �18.195 9 �18.988 �16.431 �17.952 10 �18.995 �16.162 �17.847 11 �18.900 �15.789 �17.639 12 �18.884 �15.497 �17.512 Minimum in bold

orders: AIC suggests 8 lags, HQ 5 lags, and BIC 2 lags. In such a situation it is wise to keep all three models and to perform additional diagnostic tests.5 One such test is to run a horse-race between the three models in terms of their forecasting performance.

We evaluate the forecasts according to the two criteria: the root-mean-squared- error (RMSE) and the mean-absolute-error (MAE)6:

RMSE W v

TCh X

TC1 .bXit � Xit/2 (14.10)

MAE W 1 h

TCh X

TC1 jbXit � Xitj (14.11)

where bXit and Xit denote the forecast and the actual value of variable i in period t. Forecasts are computed for a horizon h starting in period T. We can gain further insights by decomposing the mean-squared-error additively into three components:

TCh X

TC1 .bXit � Xit/2 D

TCh X

TC1 bXit

� Xi !2

C.� bXi

� �Xi/2 C 2.1 � �/� bXi �Xi:

5Such tests would include an analysis of the autocorrelation properties of the residuals and tests of structural breaks. 6Alternatively one could use the mean-absolute-percentage-error (MAPE). However, as all vari- ables are already in percentages, the MAE is to be preferred.

250 14 Forecasting with VAR Models

The first component measures how far the mean of the forecasts 1 h

PTCh TC1 bXit is away

from the actual mean of the data Xi. It therefore measures the bias of the forecasts. The second one compares the standard deviation of the forecast �

bXi to those of the

data �Xi . Finally, the last component is a measure of the unsystematic forecast errors where � denotes the correlation between the forecast and the data. Ideally, each of the three components should be close to zero: there should be no bias, the variation of the forecasts should correspond to those of the data, and the forecasts and the data should be highly positively correlated. In order to avoid scaling problems, all three components are usually expressed as a proportion of 1

PTCh TC1.bXit � Xit/2:

bias proportion:

��

1 h

PTCh TC1 bXit

�

� Xi �2

1 h

PTCh TC1.bXit � Xit/2

(14.12)

variance proportion: .� bXi

� �Xi/2 1 h

PTCh TC1.bXit � Xit/2

(14.13)

covariance proportion: 2.1 � �/�

bXi �Xi

1 h

PTCh TC1.bXit � Xit/2

(14.14)

We use these models to produce dynamic or iterated forecasts PT XTC1; PT XTC2; : : : ;PT XTCh. Forecasts for h � 2 are computed iteratively by inserting for the lagged variables the forecasts obtained in the previous steps. For details see Chap. 14. Alternatively, one may consider a recursive or rolling out-of-sample strategy where the model is reestimated each time a new observation becomes available. Thus, we would evaluate the one-period-ahead forecasts PT XTC1;PTC1XTC2; : : : ;PTCh�1XTCh, the two-period-ahead forecasts PT XTC2;PTC1XTC3; : : : ;PTCh�2XTCh, and so on. The difference between the recursive and the rolling strategy is that in the first case all observations are used for estimation whereas in the second case the sample is rolled over so that its size is kept fixed at T.

Figure 14.1 displays dynamic or iterated forecasts for the four variables expressed in log-levels, respectively in levels for the interest rate. Forecast are evaluated according to the performance measures explained above. The corresponding values are reported in Table 14.2. All models see a quick recovery after the recession in 2008 and are thus much too optimistic. The lowest RMSE for log Yt is 5.678 for the VAR(8) model. Thus, GDP per capita is predicted to be on average almost 6 % too high over the forecast period. This overly optimistic forecast is reflected in a large bias proportion which amounts to more than 95 %. The situation looks much better for the price level. All models see an increase in inflation starting in 2009. Especially, the two higher order models fare much better. Their RMSE is just over 1 %. The bias proportion is practically zero for the VAR(8) model. The forecast results of the real money stock are mixed. All models predict a quick recovery. This took indeed place, but first at a more moderate pace. Starting in

14.4 Example: VAR Model 251

4.40

4.44

4.48

4.52

4.56

4.60

4.64

2008 2009 2010 2011 2012 2013 2014

actual VAR(2) VAR(5) VAR(8)

5.32

5.36

5.40

5.44

5.48

5.52

5.56

5.60

2008 2009 2010 2011 2012 2013 2014

actual VAR(2) VAR(5) VAR(8)

2.5

2.6

2.7

2.8

2.9

3.0

2008 2009 2010 2011 2012 2013 2014

actual VAR(2) VAR(5) VAR(8)

-2

-1

2008 2009 2010 2011 2012 2013 2014

actual VAR(2) VAR(5) VAR(8)

a b

c d

Fig. 14.1 Forecast comparison of alternative models. (a) log Yt . (b) log Pt . (c) log Mt . (d) Rt

mid-2010 the unconventional monetary policy of quantitative easing, however, led to an unforeseen acceleration so that the forecasts turned out to be systematically too low for the later period. Interestingly, the smallest model fared significantly better than the other two. Finally, the results for the interest rates are very diverse. Whereas the VAR(2) model predicts a rise in the interest rate, the other models foresee a decline. The VAR(8) model even predicts a very drastic fall. However, all models miss the continuation of the low interest rate regime and forecasts an increase starting already in 2009. This error can again be attributed to the unforeseen low interest rate monetary policy which was implemented in conjunction with the quantitative easing. This misjudgement resulted in a relatively large bias proportion.

Up to now, we have just been concerned with point forecasts. Point forecasts, however, describe only one possible outcome and do not reflect the inherent uncertainty surrounding the prediction problem. It is, thus, a question of scientific integrity to present in addition to the point forecasts also confidence intervals. One straightforward way to construct such intervals is by computing the matrix of mean- squared-errors MSE using Eq. (14.5). The diagonal elements of this matrix can be interpreted as a measure of the forecast error variances for each variable. Under the assumption that the innovations fZtg are Gaussian, such confidence intervals can be easily computed. However, in practice this assumption is likely to be violated. This problem can be circumvented by using the empirical distribution function of the residuals to implement a bootstrap method similar to the computation of the

252 14 Forecasting with VAR Models

Table 14.2 Forecast evaluation of alternative VAR models

VAR(2) VAR(5) VAR(8)

log Yt RMSE 8.387 6.406 5.678

Bias proportion 0.960 0.961 0.951

Variance proportion 0.020 0.010 0.001

Covariance proportion 0.020 0.029 0.048

MAE 8.217 6.279 5.536

log Pt RMSE 3.126 1.064 1.234

Bias proportion 0.826 0.746 0.001

Variance proportion 0.121 0.001 0.722

Covariance proportion 0.053 0.253 0.278

MAE 2.853 0.934 0.928

log Mt RMSE 5.616 6.780 9.299

Bias proportion 0.036 0.011 0.002

Variance proportion 0.499 0.622 0.352

Covariance proportion 0.466 0.367 0.646

MAE 4.895 5.315 7.762

Rt RMSE 2.195 2.204 2.845

Bias proportion 0.367 0.606 0.404

Variance proportion 0.042 0.337 0.539

Covariance proportion 0.022 0.057 0.057

MAE 2.125 1.772 2.299

RMSE and MAE for log Yt , log Pt , and log Mt are multiplied by 100

Value-at-Risk in Sect. 8.4. Figure 14.2 plots the forecasts of the VAR(8) model together with a 80 % confidence interval computed from the bootstrap approach. It shows that, with the exception of the logged price level, the actual realizations fall out of the confidence interval despite the fact that the intervals are already relatively large. This documents the uniqueness of the financial crisis and gives a hard time for any forecasting model.

Instead of computing a confidence interval, one may estimate the probability distribution of possible future outcomes. This provides a complete description of the uncertainty related to the prediction problem (Christoffersen 1998; Diebold et al. 1998; Tay and Wallis 2000; Corradi and Swanson 2006). Finally, one should be aware that the innovation uncertainty is not the only source of uncertainty. As the parameters of the model are themselves estimated, there is also a coefficient uncer- tainty. In addition, we have to face the possibility that the model is misspecified.

The forecasting performance of the VAR models may seem disappointing at first. However, this was only be a first attempt and further investigations are usually necessary. These may include the search for structural breaks (See Bai et al. 1998; Perron 2006). This topic is treated in Sect. 18.1. Another reason for the poor

14.4 Example: VAR Model 253

4.40

4.44

4.48

4.52

4.56

4.60

4.64

2008 2009 2010 2011 2012 2013 2014

Actual forecast

5.30

5.35

5.40

5.45

5.50

5.55

5.60

5.65

5.70

2008 2009 2010 2011 2012 2013 2014

Actual forecast

2.5

2.6

2.7

2.8

2.9

3.0

2008 2009 2010 2011 2012 2013 2014

Actual forecast

-4

-2

2008 2009 2010 2011 2012 2013 2014

actual forecast

a b

c d

Fig. 14.2 Forecast of VAR(8) model and 80 % confidence intervals (red dotted lines). (a) log Yt . (b) log Pt . (c) log Mt. (d) Rt

forecasting may be due to the over-parametrization of VAR models. The VAR(8) model, for example, 32 lagged dependent variables plus a constant in each of the four equations which leads to a total 132 parameters. This problem can be dealt with by applying Bayesian shrinkage techniques. This approach, also known as Bayesian VAR (BVAR), was particularly successful when using the so-called Minnesota prior (See Doan et al. 1984; Litterman 1986; Kunst and Neusser 1986; Banbura et al. 2010). This prior is presented in Sect. 18.2.

Besides these more fundamental issues, one may rely on more technical reme- dies. One such remedy is the use of direct rather iterated forecasts. This difference is best explained in the context of the VAR(1) model Xt D ˆXt�1CZt, Zt � WN.0;†/. The iterated forecast for XTCh uses the OLS-estimate bˆ to compute the forecast bˆhXT (see Chap. 14). Alternatively, one may estimate instead of the VAR(1), the model Xt D ‡Xt�h C Zt and compute the direct forecast for XTCh as b‡XT . Although b‡ has larger variance than bˆ if the VAR(1) is correctly specified, it is robust to misspecification (see Bhansali 1999; Schorfheide 2005; Marcellino et al. 2006).

Another interesting and common device is intercept correction or residual adjustment. Thereby the constant terms are adjusted in such a way that the residuals of the most recent observation become zero. The model is thereby set back on track. In this way the forecaster can guard himself against possible structural breaks. Residual adjustment can also serve as a device to incorporate anticipated events, like announced policies, which are not yet incorporated into the model. See Clements and Hendry (1996, 2006) for further details and additional forecasting devices.

15Interpretation and Identification of VARModels

Although the estimation of VAR models poses no difficulties as outlined in the previous chapter, the individual coefficients are almost impossible to interpret. On the one hand, there are usually many coefficients, a VAR(4) model with three variables, for example, already has twelve coefficients per equation and thus 36 coefficients in total to interpret; on the other hand, there is in general no unambiguous relation of the VAR coefficients to the coefficients of a particular model. The last problem is known as the identification problem. To overcome this identification problem, many techniques have been developed which should allow to give the estimated VAR model an explicit economic interpretation.

15.1 Wiener-Granger Causality

As a first technique for the understanding of VAR processes, we analyze the concept of causality which was introduced by Granger (1969). The concept is also known as Wiener-Granger causality because Granger’s idea goes back to the work of Wiener (1956). Take a multivariate time series fXtg and consider the forecast of X1;TCh, h � 1, given XT; XT�1; : : : where fXtg has not only X1t as a component, but also another variable or group of variables X2t. Xt may contain even further variables than X1;t and X2;t . The mean-squared forecast error is denoted by MSE1.h/. Consider now an alternative forecast of X1;TCh given eXT;eXT�1; : : : where feXtg is obtained from fXtg by eliminating the component X2;t . The mean-squared error of this forecast is denoted by eMSE1.h/. According to Granger, we can say that the second variable X2;t causes or is causal for X1;t if and only if

MSE.h/1 < eMSE1.h/ for some h � 1:

255

256 15 Interpretation of VAR Models

This means that the information contained in fX2tg and its past improves the forecast of fX1tg in the sense of the mean-squared forecast error. Thus the concept of Wiener- Granger causality makes only sense for purely non-deterministic processes and rest on two principles1:

• The future cannot cause the past. Only the past can have a causal influence on the future.2

• A specific cause contains information not available otherwise.

The concept of Wiener-Granger causality played an important role in the debate between monetarists and Keynesians over the issue whether the money stock has an independent influence on real activity. It turned out that this question can only be resolved within a specific context. Sims (1980a), for example, showed that the relationship between the growth rate of the money stock and changes in real activity depends on whether a short interest rate is accounted for in the empirical analysis or not. Another problem of the concept is that it is not unambiguously possible to infer a causal relationship just from the chronology of two variables as demonstrated by Tobin (1970). This and other conceptual issues (see Zellner (1979) and the discussion in the next chapter) and econometric problems (Geweke 1984) led to a decline in the practical importance of this concept.

We propose two econometric implementations of the causality concept. The first one is based on a VAR, the second one is non-parametric and uses the cross-correlations. In addition, we propose an interpretation in terms of the causal representation, respectively the Wold Decomposition Theorem (see Chap. 14)

15.1.1 VAR Approach

If one restricts oneself to linear least-squares forecasts, the above definition can be easily operationalized in the context of VAR models with only two variables (see also Sims 1972). Consider first a VAR(1) model. Then according to the explanations in Chap. 14 the one-period forecast is:

PT XTC1 D �

PT X1;TC1 PT X2;TC1

�

D ˆXT D �

�11 �12 �21 �22

��

X1;T X2;T

�

and therefore

PT X1;TC1 D �11X1T C �12X2T:

1Compare this to the concept of a causal representation developed in Sects. 12.3 and 2.3. 2Sometimes also the concept of contemporaneous causality is considered. This concept is, however, controversial and has therefore not gained much success in practice and will, therefore, not be pursued.

15.1 Wiener-Granger Causality 257

If �12 D 0 then the second variable does not contribute to the one-period forecast of the first variable and can therefore be omitted: MSE1.1/ D eMSE1.1/. Note that

ˆh D �

�h11 0

� �h22

�

;

where � is a placeholder for an arbitrary number. Thus the second variable is not only irrelevant for the one-period forecast, but for any forecast horizon h � 1. Thus, the second variable is not causal for the first variable in the sense of Wiener-Granger causality.

These arguments can be easily extended to VAR(p) models. According to Eq. (14.4) we have that

PT X1;TC1 D �.1/11 X1T C �.1/12 X2T C : : : C �.p/11 X1;T�pC1 C �.p/12 X2;T�pC1

where � .k/ ij denotes .i; j/-th element, i D 1;2, of the matrix ˆk , k D 1; : : : ; p. In

order for the second variable to have no influence on the forecast of the first one, we must have that �

.1/ 12 D �.2/12 D : : : D �.p/12 D 0. This implies that all matrices

ˆk, k D 1; : : : ; p, must be lower triangular, i.e. they must be of the form �� 0

� � �

As the multiplication and addition of lower triangular matrices is again a lower triangular matrix, the second variable is irrelevant in forecasting the first one at any forecast horizon. This can be seen by computing the corresponding forecast function recursively as in Chap. 14.

Based on this insight it is straightforward to test the null hypothesis that the second variable does not cause the first one within the VAR(p) context:

H0 W fX2tg does not cause fX1tg.

In terms of the VAR model this hypothesis can be stated as:

H0 W �.1/12 D �.2/12 D : : : D �.p/12 D 0:

The alternative hypothesis is that the null hypothesis is violated. As the method of least-squares estimation leads under quite general conditions to asymptotically normal distributed coefficient estimates, it is straightforward to test the above hypothesis by a Wald-test (F-test). In the context of a VAR(1) model a simple t- test is also possible.

If more than two variables are involved the concept of Wiener-Granger causality is no longer so easy to implement. Consider for expositional purposes a VAR(1) model in three variables with coefficient matrix:

258 15 Interpretation of VAR Models

ˆ D 0

�11 �12 0

�21 �22 �23

�31 �32 �33

A :

The one-period forecast function of the first variable then is

PT X1;TC1 D �11X1T C �12X2T:

Thus, the third variable X3T is irrelevant for the one-period forecast of the first variable. However, as the third variable has an influence on the second variable, �23 ¤ 0, and because the second variable has an influence on the first variable, �12 ¤ 0, the third variable will provide indirectly useful information for the forecast of the first variable for forecasting horizons h � 2. Consequently, the concept of causality cannot immediately be extended from two to more than two variables.

It is, however, possible to merge variables one and two, or variables two and three, into groups and discuss the hypothesis that the third variable does not cause the first two variables, seen as a group; likewise that the second and third variable, seen as a group, does not cause the first variable. The corresponding null hypotheses then are:

H0 W �23 D �13 D 0 or H0 W �12 D �13 D 0:

Under these null hypotheses we get again lower (block-) triangular matrices:

�11 �12 ::: 0

�21 �22 ::: 0

: : : : : : ::: : : :

�31 �32 ::: �33

�11 ::: 0 0

: : : : : : : : : : : :

�21 ::: �22 �23

�31 ::: �32 �33

Each of these hypotheses can again be checked by a Wald-test (F-test).

15.1.2 Wiener-Granger Causality and Causal Representation

We can get further insights into the concept of causality by considering a bivariate VAR, ˆ.L/Xt D Zt, with causal representation Xt D ‰.L/Zt. Partitioning the matrices according to the two variables fX1tg and fX2tg, Theorem 12.1 of Sect. 12.3 implies that

�

ˆ11.z/ ˆ12.z/ ˆ21.z/ ˆ22.z/

��

‰11.z/ ‰12.z/ ‰21.z/ ‰22.z/

�

D �

1 0

0 1

�

15.1 Wiener-Granger Causality 259

where the polynomials ˆ12.z/ and ˆ21.z/ have no constant terms. The hypothesis that the second variable does not cause the first one is equivalent in this framework to the hypothesis that ˆ12.z/ D 0. Multiplying out the above expression leads to the condition

ˆ11.z/‰12.z/ D 0:

Because ˆ11.z/ involves a constant term, the above equation implies that ‰12.z/ D 0. Thus the causal representation is lower triangular. This means that the first variable is composed of the first shock, fZ1tg only whereas the second variable involves both shocks fZ1tg and fZ2tg. The univariate causal representation of fX1tg is therefore the same as the bivariate one.3 Finally, note the similarity to the issue of the identification of shocks discussed in subsequent sections.

15.1.3 Cross-Correlation Approach

In the case of two variables we also examine the cross-correlations to test for causality. This non-parametric test has the advantage that one does not have to rely on an explicit VAR model. This advantage becomes particularly relevant, if a VMA model must be approximated by a high order AR model. Consider the cross- correlations

�12.h/ D corr.X1t; X2;t�h/:

If �12.h/ ¤ 0 for h > 0, we can say that the past values of the second variable are useful for forecasting the first variable such that the second variable causes the first one in the sense of Wiener and Granger. Another terminology is that the second variable is a leading indicator for the first one. If in addition, �12.h/ ¤ 0, for h < 0, so that the past values of the first variable help to forecast the second one, we have causality in both directions.

As the distribution of the cross-correlations of two independent variables depends on the autocorrelation of each variable, see Theorem 11.4, Haugh (1976) and Pierce and Haugh (1977) propose a test based on the filtered time series. Analogously to the test for independence (see Sect. 11.2), we proceed in two steps:

Step 1: Estimate in the first step a univariate AR(p) model for each of the two time series fX1tg and fX2tg. Thereby chose p such that the corresponding residuals f OZ1t g and f OZ2tg are white noise. Note that although f OZ1tg and f OZ2tg are both not autocorrelated, the cross-correlations �Z1;Z2.h/ may still be non-zero for arbitrary orders h.

3As we are working with causal VAR’s, the above arguments also hold with respect to the Wold Decomposition.

260 15 Interpretation of VAR Models

Step 2: As f OZ1tg and f OZ2tg are the forecast errors based on forecasts which rely only on the own past, the concept of causality carries over from the original variables to the residuals. The null hypothesis that the second variable does not cause the first variable in the sense of Wiener and Granger can then be checked by the Haugh-Pierce statistic:

Haugh-Pierce statistic: T L X

hD1 O�2Z1;Z2.h/ � �2L: (15.1)

Thereby O�2Z1;Z2.h/, h D 1;2; : : :, denotes the squared estimated cross-correlation coefficients between f OZ1tg and f OZ2tg. Under the null hypothesis that the second variable does not cause the first one, this test statistic is distributed as a �2

distribution with L degrees of freedom.

15.2 Structural and Reduced Form

15.2.1 A Prototypical Example

The discussion in the previous section showed that the relation between VAR models and economic models is ambiguous. In order to better understand the quintessence of the problem, we first analyze a simple macroeconomic example. Let fytg and fmtg denote the output and the money supply of an economy4 and suppose that the relation between the two variables is represented by the following simultaneous equation system:

AD-curve: X1t D yt D a1mt C �11yt�1 C �12mt�1 C vyt policy reaction curve: X2t D mt D a2yt C �21yt�1 C �22mt�1 C vmt

Note the contemporaneous dependence of yt on mt in the AD-curve and a corre- sponding dependence of mt on yt in the policy reaction curve. These equations are typically derived from economic reasoning and may characterize a model explicitly derived from economic theory. In statistical terms the simultaneous equation system is called the structural form. The error terms fvytg and fvmtg are interpreted as demand shocks and money supply shocks, respectively. They are called structural shocks and are assumed to follow a multivariate white noise process:

Vt D �

vyt vmt

�

� WN .0;�/ with � D

!2y 0

0 !2m

4If one is working with actual data, the variables are usually expressed in log-differences to achieve stationarity.

15.2 Structural and Reduced Form 261

Note that the two structural shocks are assumed to be contemporaneously uncor- related which is reflected in the assumption that � is a diagonal matrix. This assumption in the literature is uncontroversial. Otherwise, there would remain some unexplained relationship between them. The structural shocks can be interpreted as the statistical analog of an experiment in the natural sciences. The experiment corresponds in this case to a shift of the AD-curve due to, for example, a temporary non-anticipated change in government expenditures or money supply. The goal of the analysis is then to trace the reaction of the economy, in our case represented by the two variables fytg and fmtg, to these isolated and autonomous changes in aggregate demand and money supply. The structural equations imply that the reaction is not restricted to contemporaneous effects, but is spread out over time. We thus represent this reaction by the impulse response function.

We can write the system more compactly in matrix notation:

�

1 �a1 �a2 1

��

yt mt

�

D �

�11 �12

�21 �22

��

yt�1 mt�1

�

C �

1 0

0 1

��

vyt

vmt

�

AXt D �Xt�1 C BVt

where A D �

1 �a1 �a2 1

�

, � D �

�11 �12 �21 �22

�

and B D �

1 0

0 1

�

Assuming that a1a2 ¤ 1, we can solve the above simultaneous equation system for the two endogenous variables yt and mt to get the reduced form of the model:

X1t D yt D �11 C a1�21 1 � a1a2

yt�1 C �12 C a1�22 1 � a1a2

mt�1 C vyt

1 � a1a2 C a1vmt 1 � a1a2

D �11yt�1 C �12mt�1 C Z1t X2t D mt D

�21 C a2�11 1 � a1a2

yt�1 C �22 C a2�12 1 � a1a2

mt�1 C a2vyt

1 � a1a2 C vmt 1 � a1a2

D �21yt�1 C �22mt�1 C Z2t:

Thus, the reduced form has the structure of a VAR(1) model with error term fZtg D f.Z1t; Z2t/0g. The reduced form can also be expressed in matrix notation as:

Xt D A�1�Xt�1 C A�1BVt D ˆXt�1 C Zt

262 15 Interpretation of VAR Models

where

Zt � WN.0;†/ with † D A�1B�B0A0�1: Whereas the structural form represents the inner economic relations between the variables (economic model), the reduced form given by the VAR model summarizes their outer directly observable characteristics. As there is no unambiguous relation between the reduced and structural form, it is impossible to infer the inner economic relationships from the observations alone. This is known in statistics as the identification problem. Typically, a whole family of structural models is compatible with a particular reduced form. The models of the family are thus observationally equivalent to each other as they imply the same distribution for fXtg. The identification problem can be overcome if one is willing to make additional a priori assumptions. The nature and the type of these assumption and their interpretation is subject of the rest of this chapter.

In our example, the parameters characterizing the structural and the reduced form are

fa1; a2;�11;�12;�21;�22;!2y ;!2mg

and

f�11;�12;�21;�22;�21;�12;�22 g:

As there are eight parameters in the structural form, but only seven parameters in the reduced form, there is no one-to-one relation between structural and reduced form. The VAR(1) model delivers estimates for the seven reduced form parameters, but there is no way to infer from these estimates the parameters of the structural form. Thus, there is a fundamental identification problem.

The simple counting of the number of parameters in each form tells us that we need at least one additional restriction on the parameters of the structural form. The simplest restriction is a zero restriction. Suppose that a2 equals zero, i.e. that the central bank does not react immediately to current output. This seems reasonable because national accounting figures are usually released with some delay. With this assumption, we can infer the structural parameters from the reduced ones:

�21 D �21; �22 D �22; �mt D Z2t ) !2m D �22; ) a1 D �12=�22; ) !2y D �21 � �212=�22 �11 D �11 � .�12=�22/�21; �12 D �12 � .�12=�22/�22:

Remark 15.1. Note that, because Zt D A�1BVt, the reduced form disturbances Zt are a linear combination of the structural disturbances, in our case the demand dis- turbance vyt and the money supply disturbance vmt . In each period t the endogenous variables output yt and money supply mt are therefore hit simultaneously by both

15.2 Structural and Reduced Form 263

shocks. It is thus not possible without further assumptions to assign the movements in Zt and consequently in Xt to corresponding changes in the fundamental structural shocks vyt and vmt .

Remark 15.2. As Cooley and LeRoy (1985) already pointed out, the statement “money supply is not causal in the sense of Wiener and Granger for real economic activity”, which, in our example is equivalent to �12 D 0, is not equivalent to the statement “money supply does not influence real economic activity“ because �12 can be zero without a1 being zero. Thus, the notion of causality is not very meaningful in inferring the inner (structural) relationships between variables.

15.2.2 Identification: The General Case

We now present the general identification problem in the context of VAR.5 The starting point of the analysis consists of a linear model, derived ideally from economic theory, in its structural form:

AXt D �1Xt�1 C : : : C �pXt�p C BVt (15.2)

where Vt are the structural disturbances. These disturbances usually have an economic interpretation, for example as demand or supply shocks. A is a n � n matrix which is normalized such that the diagonal consists of ones only. The matrix B is also normalized such that its diagonal contains only ones. The process of structural disturbances, fVtg, is assumed to be a multivariate white noise process with a diagonal covariance matrix �:

Vt � WN.0;�/ with � D EVtV 0t D

!21 0 : : : 0

0 !22 : : : 0 :::

::: :::

:::

0 0 : : : !2n

The assumption that the structural disturbance are uncorrelated with each other is not viewed as controversial as otherwise there would be unexplained relationships between them. In the literature one encounters an alternative completely equivalent normalization which leaves the coefficients in B unrestricted but assumes the covariance matrix of Vt, �, to be equal to the identity matrix In.

The reduced form is obtained by solving the equation system with respect to Xt. Assuming that A is nonsingular, the premultiplication of Eq. (15.2) by A�1 leads to the reduced form which corresponds to a VAR(p) model:

5A thorough treatment of the identification problem in econometrics can be found in Rothenberg (1971), and for the VAR context in Rubio-Ramírez et al. (2010).

264 15 Interpretation of VAR Models

Xt D A�1�1Xt�1 C : : : C A�1�pXt�p C A�1BVt D ˆ1Xt�1 C : : : C ˆpXt�p C Zt: (15.3)

The relation between the structural disturbances Vt and the reduced form distur- bances Zt is in the form of a simultaneous equation system:

AZt D BVt: (15.4)

While the structural disturbances are not directly observed, the reduced form disturbances are given as the residuals of the VAR and can thus be considered as given. The relation between the lagged variables is simply

�j D Aˆj; j D 1;2; : : : ; p:

Consequently, once A and B have been identified, not only the coefficients of the lagged variables in the structural form are identified, but also the impulse response functions (see Sect. 15.4.1). We can therefore concentrate our analysis of the identification problem on Eq. (15.4).

With these preliminaries, it is now possible to state the identification problem more precisely. Equation (15.4) shows that the structural form is completely determined by the parameters .A; B;�/. Taking the normalization of A and B into account, these parameters can be viewed as points in Rn.2n�1/. These parameters determine the distribution of Zt D A�1BVt which is completely characterized by the covariance matrix of Zt, †, as the mean is equal to zero.

6 Thus, the parameters of the reduced form, i.e. the independent elements of † taking the symmetry into account, are points in Rn.nC1/=2. The relation between structural and reduced form can therefore be described by a function g W Rn.2n�1/ ! Rn.nC1/=2:

† D g.A; B;�/ D A�1B�B0A0�1: (15.5)

Ideally, one would want to find the inverse of this function and retrieve, in this way, the structural parameters .A; B;�/ from †. This is, however, in general not possible because the dimension of the domain space of g, n.2n � 1/, is strictly greater, for n � 2, than the dimension of its range space, n.n C 1/=2. This discrepancy between the dimensions of the domain and the range space of g is known as the identification problem. To put it in another way, there are only n.n C 1/=2 (nonlinear) equations for n.2n � 1/ unknowns.7

6As usual, we concentrate on the first two moments only. 7Note also that our discussion of the identification problem focuses on local identification, i.e. the invertibility of g in an open neighborhood of †. See Rothenberg (1971) and Rubio-Ramírez et al. (2010) for details on the distinction between local and global identification.

15.2 Structural and Reduced Form 265

To overcome the identification problem, we have to bring in additional infor- mation. A customary approach is to impose a priori assumptions on the structural parameters. The Implicit Function Theorem tells us that we need

3n.n � 1/=2 D n.2n � 1/ � n.n C 1/=2 (15.6)

such restrictions, so-called identifying restrictions, to be able to invert the function g. Note that this is only a necessary condition and that the identification problem becomes more severe as the dimension of the VAR increases because the number of restrictions grows at a rate proportional to n2.

This result can also be obtained by noting that the function g in Eq. (15.5) is invariant under the following transformation h:

h W .A; B;�/ �! .RA; RB�1=2Q��1=2; D�D�1/

where R, Q and D are arbitrary invertible matrices such that R respects the normalization of A and B, Q is an orthogonal matrix, and D is a diagonal matrix. It can be verified that

.g ı h/.A; B;�/ D g.A; B;�/:

The dimensions of the matrices R, Q, and D are n2 � 2n, n.n � 1/=2, and n, respectively. Summing up gives 3n.n � 1/=2 D n2 � 2n C n.n � 1/=2 C n degrees of freedom as before8.

The empirical economics literature proposed several alternative identification schemes:

(i) Short-run restrictions place restrictions, usually zero restrictions, on the immediate impact of structural shocks (among many others, Sims 1980b; Blanchard 1989; Blanchard and Watson 1986; Christiano et al. 1999). See Sect. 15.3 for details.

(ii) Long-run restrictions place restrictions, usually zero-restrictions, on the long– run impact structural shocks (Blanchard and Quah 1989; Galí 1992). See Sect. 15.5 for details.

(iii) Maximization of the contribution to the forecast error variance of some variable at some horizon with respect to a particular shock (Faust 1998; Uhlig 2004; Francis et al. 2014; Uhlig, 2003, What drives real GNP? unpublished). This method has seen an interesting application in the identification of news shocks (see Barsky and Sims 2011). Further details will discussed in Sect. 15.4.2.

(iv) Sign restrictions restrict the set of possible impulse response functions (see Sect. 15.4.1) to follow a given sign pattern (Faust 1998; Uhlig 2005; Fry and

8See Neusser (2016) for further implications of viewing the identification problem from an invariance perspective.

266 15 Interpretation of VAR Models

Pagan 2011; Kilian and Murphy 2012; Rubio-Ramírez et al. 2010; Arias et al. 2014; Baumeister and Hamilton 2015). This approach is complementary to the two previous identification schemes and will be discussed in Sect. 15.6.

(v) Identification through heteroskedasticity (Rigobon 2003) (vi) Restrictions derived from a dynamic stochastic general equilibrium (DSGE)

model. These restrictions often come as nonlinear cross-equation restrictions and are viewed as the hallmark of rational expectations models (Hansen and Sargent 1980). Typically, the identification issue is overcome by imposing a priori restrictions via a Bayesian approach (Negro and Schorfheide (2004) among many others).

(vii) Identification using information on global versus idiosyncratic shocks in the context of multi-country or multi-region VAR models (Canova and Ciccarelli 2008; Dees et al. 2007)

(viii) Instead of identifying all parameters, researchers may be interested in iden- tifying only one equation or a subset of equations. This case is known as partial identification. The schemes presented above can be extended in a straightforward manner to the partial identification case.

These schemes are not mutually exclusive, but can be combined with each other. In the following we will only cover the identification through short- and long- run restrictions, because these are by far the most popular ones. The economic importance of these restrictions for the analysis of monetary policy has been emphasized by Christiano et al. (1999).

15.2.3 Identification: The Case n D 2 Before proceeding further, it is instructive to analyze the case n D 2 in more detail.9 Assume for simplicity A D I2, then the equation system (15.5) can be written explicitly as

�21 D !21 C .B/212!22 �12 D .B/21!21 C .B/12!22 �22 D .B/221!21 C !22

with unknowns .B/12;.B/21;! 2 1 , and !

2 2 . Note that the assumption of † being

positive definite implies that .B/12.B/21 ¤ 1. Thus, we can solve out !21 and !22 and reduce the three equations to only one:

�

.B/12 � b�121 ��

.B/21 � b�112 � D r�212 � 1 (15.7)

where b21 and b12 denote the least-squares regression coefficients of Z2t on Z1t , respectively of Z1t on Z2t , i.e. b21 D �12=�21 and b12 D �12=�22 . r12 is the correlation

9The exposition is inspired by Leamer (1981).

15.2 Structural and Reduced Form 267

Fig. 15.1 Identification in a two-dimensional structural VAR with �12 > 0

(b12,0)(0,b21)

(0,0)

(B)21

(B)12

C = (b 21 −1,b

12 −1)

b 21 −1

b 12 −1

coefficient between Z2t and Z1t, i.e. r12 D �12=.�1�2/. Note that imposing a zero restriction by setting .B/12, for example, equal to zero, implies that .B/21 equals b21; and vice versa, setting .B/21 D 0, implies .B/12 D b12. As a final remark, the right hand side of Eq. (15.7) is always positive as the inverse of the squared correlation coefficient is bigger than one. This implies both product terms must be of the same sign.

Equation (15.7) delineates all possible combinations of .B/12 and .B/21 which are compatible with a given covariance matrix †. Its graph represents a rectangular hyperbola in the parameter space ..B/12;.B/21/ with center C D .b�121 ; b�112 / and asymptotes .B/12 D b�121 and .B/21 D b�112 and is plotted in Fig. 15.1.10 The hyperbola consist of two disconnected branches with a pole at the center C D .b�121 ; b�112 /. At this point, the relation between the two parameters changes sign. The figure also indicates the two possible zero restrictions .B/12 D 0 and .B/21 D 0, called short-run restrictions. These two restrictions are connected and its path completely falls within one quadrant. Thus, along this path the sign of the parameters remain unchanged.

Suppose that instead of fixing a particular parameter, we only want to restrict its sign. Assuming that .B/12 � 0 implies that .B/21 must lie in one of the two disconnected intervals .�1; b21 and .b�112 ; C1/.11 Although not very explicit, some economic consequences of this topological particularity are discussed in Fry and Pagan (2011). Alternatively, assuming .B/12 � 0 implies .B/21 2 Œb21; b�112 /. Thus, .B/21 is unambiguously positive. Sign restrictions for .B/21 can be discussed in a similar manner. Section 15.6 discuses sign restrictions more explicitly.

10Moon et al. (2013; section 2) provided an alternative geometric representation. 11That these two intervals are disconnected follows from the fact that † is positive definite.

268 15 Interpretation of VAR Models

15.3 Identification via Short-Run Restrictions

Short-run restrictions represent the most common identification scheme encoun- tered in practice. They impose direct linear restrictions on the structural parameters A and B and restrict in this way the contemporaneous effect of the structural shocks on the variables of the system. The most common type of such restrictions are zero restrictions which set certain coefficients a priori to zero. These zero restrictions are either derived from an explicit economic theory or are based on some ad hoc arguments. As explained above, it is necessary to have at least 3n.n � 1/=2 restrictions at hand. If there are more restrictions, we have an overidentified system. This is, however, rarely the case in practice because the number of necessary restrictions grows at a rate proportional to n2. The case of overidentification is, thus, not often encountered and as such is not treated.12 The way to find appropriate restrictions in a relatively large system is documented in Sect. 15.4.5. If the number of restrictions equals 3n.n � 1/=2, we say that the system is exactly identified.

Given the necessary number of a priori restrictions on the coefficients A and B, there are two ways to infer A, B, and �. The first one views the relation (15.4) as a simultaneous equation system in Zt with error terms Vt and to estimate the coefficients by instrumental variables as in Blanchard and Watson (1986).13 The second way relies on the method of moments and solves the nonlinear equation system (15.5) as in Bernanke (1986). In the case of exact identification both methods are numerically equivalent.

In our example treated of Sect. 15.2.1, we had n D 2 so that three restrictions were necessary (six parameters, but only three equations). These three restrictions were obtained by setting B D I2 which gives two restrictions (i.e. b12 D b21 D 0). The third restriction was to set the immediate reaction of money supply to a demand shock to zero, i.e. to set a2 D 0. We then showed that these three restrictions are also sufficient to solve the nonlinear equation system (15.5).

Sims (1980b) proposed in his seminal article the VAR approach as an adequate alternative to then popular structural simultaneous approach. In particular, he suggested a simple recursive identification scheme. This scheme takes A D In so that the equation system (15.5) simplifies to:

† D B�B0: Next we assume that B is a lower triangular matrix:

B D

1 0 :: : 0

� 1 :: : 0 ::: ::: :::

:::

� � : : : 1

12The case of overidentification is, for example, treated in Bernanke (1986). 13A version of the instrumental variable (IV) approach is discussed in Sect. 15.5.2.

15.3 Identification via Short-Run Restrictions 269

where � is just a placeholder. The matrices B and � are uniquely determined by the Cholesky decomposition of the matrix †. The Cholesky decomposition factorizes a positive-definite matrix † uniquely into the product B�B0 where B is a lower triangular matrix with ones on the diagonal and a diagonal matrix � with strictly positive diagonal entries (Meyer 2000). As Zt D BVt, Sims’ identification gives rise to the following interpretation. v1t is the only structural shock which has an effect on X1t in period t. All other shocks have no contemporaneous effect. Moreover, Z1t D v1t so that the residual from the first equation is just equal to the first structural shock and that �21 D !21 . The second variable X2t is contemporaneously only affected by v1t and v2t , and not by the remaining shocks v3t; : : : ;vnt. In particular, Z2t D b21v1t C v2t so that b21 can be retrieved from the equation �21 D b21!21. This identifies the second structural shock v2t and !

2 2 . Due to the triangular nature of B, the system

is recursive and all structural shocks and parameters can be identified successively. The application of the Cholesky decomposition as an identification scheme rests crucially on the ordering of the variables .X1t; X2t; : : : ; Xnt/

0 in the system. Sims’ scheme, although easy to implement, becomes less plausible as the number

of variables in the system increases. For this reason the more general scheme with A ¤ In and B not necessarily lower triangular are more popular. However, even for medium sized systems such as n D 5, 30 restrictions are necessary which stresses the imagination even of brilliant economists as the estimation of Blanchard’s model in Sect. 15.4.5 shows.

Focusing on the identification of the matrices A and B brings also an advantage in terms of estimation. As shown in Chap. 13, the OLS-estimator of the VAR coefficient matrices ˆ1;: : : ;ˆp equals the GLS-estimator independently of †. Thus, the estimation of the structural parameters can be broken down into two steps. In the first step, the coefficient matrices ˆ1;: : : ;ˆp are estimated using OLS. The residuals are then used to estimate † which leads to an estimate of the covariance matrix (see Eq. (13.6)). In the second step, the coefficients of A, B, and � are then estimated given the estimate of †, b†, by solving the nonlinear equation system (15.5) taking the specific identification scheme into account. Thereby † is

replaced by its estimate b†. As p

T �

vech.b†/ � vech.†/ �

converges in distribution

to a normal distribution with mean zero, OA, OB and b� are also asymptotically normal because they are obtained by a one-to-one mapping from b†.14 The Continuous Mapping Theorem further implies that OA, OB and b� converge to their true means and that their asymptotic covariance matrix can be obtained by an application of the delta-method (see Theorem E.1 in the Appendix E). Further details can be found in Bernanke (1986), Blanchard and Watson (1986), Giannini (1991), Hamilton (1994b), and Sims (1986).

14The vech operator transforms a symmetric n � n matrix † into a 1 2 n.n C 1/ vector by stacking

the columns of † such that each element is listed only once.

270 15 Interpretation of VAR Models

15.4 Interpretation of VAR Models

15.4.1 Impulse Response Functions

The direct interpretation of VAR models is rather difficult because it is composed of many coefficients so that it becomes difficult to understand the dynamic interactions between the variables. It is therefore advantageous to simulate the dynamic effects of the different structural shocks by computing the impulse response functions. They show the effect over time of the structural shocks on the variables at issue. These effects can often be related to the underlying economic model and are thus at the heart of the VAR analysis. The examples in Sect. 15.4.4 and 15.4.5 provide some illustrations of this statement.

The impulse response functions are derived from the causal representation15 of the VAR process (see Sect. 12.3):

Xt D Zt C ‰1Zt�1 C ‰2Zt�2 C : : : D A�1BVt C ‰1A�1BVt�1 C ‰2A�1BVt�2 C : : : (15.8)

The effect of the j-th structural disturbance on the i-th variable after h periods, denoted by

@Xi;tCh @vjt

is thus given by the .i; j/-th element of the matrix ‰hA �1B:

@Xi;tCh @vjt

D ‰hA�1B

i;j :

Clearly, the impulse response functions depends on the identification scheme chosen. There are n2 impulse response functions if the system consists of n variables. Usually, the impulse response functions are represented graphically as a plot against h.

15.4.2 Forecast Error Variance Decomposition

Another instrument for the interpretation of VAR models is the forecast error variance decomposition (“FEVD”) or variance decomposition for short which decomposes the total forecast error variance of a variable into the variances of the structural shocks. It is again based on the causal representation of the VAR(p) model. According to Eq. (14.3) in Chap. 14 the variance of the forecast error or mean squared error (MSE) is given by:

MSE.h/ D E.XtCh � PtXtCh/.XtCh � Pt XtCh/0

15Sometimes the causal representation is called the MA.1) representation.

15.4 Interpretation of VAR Models 271

D h�1 X

jD0 ‰j†‰

0 j D

h�1 X

jD0 ‰jA

�1B�B0A0�1‰0j:

Given a specific identification scheme and estimates of the structural parameters, it is possible to attribute the MSE to the variance of the structural disturbances. Thereby it is customary to write the contribution of each disturbance as a percentage of the total variance. For this purpose let us write the MSE.h/ explicitly as

MSE.h/ D

m .h/ 11 � : : : � � m.h/22 : : : � :::

::: :::

:::

� � : : : m.h/nn

Our interest lies exclusively on the variances, m .h/ ii , i D 1; : : : ; n, so that we represent

the uninteresting covariance terms by the placeholder �. These variances can be seen as a linear combination of the !2i ’s because the covariance matrix of the structural disturbances � D diag �!21; : : : ;!2n

�

is a diagonal matrix:

m .h/ ii D d.h/i1 !21 C : : : C d.h/in !2n

or in matrix form

D e0i

h�1 X

jD0 ‰j†‰

0 j

A ei D e0i

h�1 X

jD0 ‰jA

�1B�B0A0�1‰0j

A ei

where the vector ei has entries equal to zero, except for the i-th entry which is equal to one. Given the positive definiteness of †, the weights d

.h/ ij , i; j D 1; : : : ; n and

h D 1;2; : : :, are strictly positive. They can be computed as

d .h/ ij D

h�1 X

kD0 Œ‰kA

�1B 2ij

In order to arrive at the percentage value of the contribution of the j-th disturbance to the MSE of the i-th variable at forecast horizon h, denoted by f

.h/ ij , we divide each

summand in the above expression by the total sum:

f .h/ ij D

d .h/ ij !

2 j

m .h/ ii

; i; j D 1; : : : ; n; for h D 0;1;2; : : :

272 15 Interpretation of VAR Models

The corresponding matrix expression is

D e0i �

Ph�1 jD0 ‰jA

�1B�1=2eje0j� 1=2B0A0�1‰0j

�

e0i �

Ph�1 jD0 ‰j†‰

0 j

�

Usually, these numbers are multiplied by 100 to give percentages and are either displayed graphically as a plot against h or in table form (see the example in Sect. 15.4.5). The forecast error variance f

.h/ ij thus shows which percentage of the

forecast variance of variable i at horizon h can be attributed to the j-th structural shock and thus measures the contribution of each of these shocks to the overall fluctuations of the variables in question.

The FEVD can be used as an alternative identification scheme, sometimes called max share identification. Assume for the ease of exposition that A D In. The VAR disturbances and the structural shocks are then simply related as Zt D BVt (compare Eq. (15.4)). Moreover, take � D In, but leave B unrestricted. This corresponds to a different, but equivalent normalization which economizes on the notation. Then the j-th structural disturbance can be identified by assuming that it maximizes the forecast error variance share with respect to variable i. Noting that, given †, B can be written as B D RQ with R being the unique Cholesky factor of † and Q being an orthogonal matrix, this optimization problem can be casted as

max qj

e0i

h�1 X

jD0 ‰jRqjq

0 jR

0‰0j

A ei s.t. q 0 jqj D 1

where qj is the j-th column of Q, i.e. qj D Qej. The constraint q0jqj D 1 normalizes the vector to have length 1. From Zt D BVt it then follows that corresponding structural disturbance is equal to Vjt D q0jR�1Zt. Because e0j Q0R�1†R0�1Qek D 0 for j ¤ k, this shock is orthogonal to the other structural disturbances. For practical applications it is advisable for reasons of numerical stability to transform to optimization problem into an equivalent eigenvalue problem (see Faust 1998; appendix).

15.4.3 Confidence Intervals

The impulse response functions and the variance decomposition are the most important tools for the analysis and interpretation of VAR models. It is, therefore, of importance not only to estimate these entities, but also to provide corresponding confidence intervals to underpin the interpretation from a statistical perspective. In the literature two approaches have been established: an analytic and a bootstrap approach. The analytic approach relies on the fact that the coefficient matrices ‰h, h D 1;2; : : :, are continuously differentiable functions of the estimated VAR

15.4 Interpretation of VAR Models 273

coefficients: vec.‰h/ D Fh.ˇ/ where ˇ denotes as in Chap. 13 the vectorized form of the VAR coefficient matrices ˆ1;: : : ;ˆp.

16 The relation between VAR coefficients and the causal representation (MA(1) representation) was established in Sect. 12.3. This discussion shows that the functions Fh W Rpn2 �! Rn2 are highly nonlinear. In Chap. 13 it was shown that

p T �

Ǒ � ˇ �

d��! N �0;† ˝ ��1p �

that we can apply the Continuous Mapping Theorem (see Theorem E.1 or Serfling (1980; 122–124)), sometimes also called the Delta method to get

p T �

Fh. Ǒ/ � Fh.ˇ/ �

d��! N �

�

@Fh.ˇ/

@̌ 0

�

† ˝ ��1p �

�

@Fh.ˇ/

@̌ 0

�0�

where in practice the covariance matrix is estimated by replacing ˇ and † by their estimate. The computation of the gradient matrices @Fh.ˇ/

@ˇ0 is rather involved,

especially when h becomes large. Details can be found in Lütkepohl (1990, 2006) and Mittnik and Zadrozny (1993).

The use of this asymptotic approximation has two problems. First, the complexity of the relationship between the ˆi’s and the ‰h’s augments with h so that the quality of the approximation diminishes with h for any given sample size. This is true even when Ǒ is exactly normally distributed. Second, the distribution of Ǒ is approximated poorly by the normal distribution. This is particularly relevant

when the roots of ˆ.L/ are near the unit circle. In this case, the bias towards zero can become substantial (see the discussion in Sect. 7.2). These two problems become especially relevant as h increases. For these reasons the analytic approach has becomes less popular.

The bootstrap approach (Monte Carlo or Simulation approach), as advocated by Runkle (1987), Kilian (1998) and Sims (1999), has become the most favored approach. This is partly due to the development of powerful computer algorithms, and the increased speed in computations. The so-called naive bootstrap approach consists of several steps.17

First step: Using a random number generator new disturbances are created. This can be done in two ways. The first one assumes a particular distribution for Vt: Vt � N.0;b�/, for example. The realizations V1; : : : ; VT are then independent draws from this distribution. The second one, takes random draws with replacement from the identified realizations OV1; : : : ; OVT .18 The second way has the advantage that non explicit distributional assumption is made which results in a better approximation of the true distribution of OVt .

16Recall that the vec operator stacks the columns of a n � m matrix to get one nm vector. 17The bootstrap is a resampling method. Efron and Tibshirani (1993) provide a general introduction to the bootstrap. 18The draws can also be done blockwise. This has the advantage that possible remaining temporal dependences are taken in account.

274 15 Interpretation of VAR Models

Second step: Given the fixed starting values X�pC1; : : : ; X0, the estimated coef- ficients matrices bˆ1;: : : ;bˆp and the new disturbances drawn in step one, a new realization of the time series for fXtg is generated.

Third step: Estimate the VAR model, given the newly generated realizations for fXtg, to obtain new estimates for the coefficient matrices.

Fourth step: Generate a new set of impulse response functions given the new estimates, taking the identification scheme as fixed.

The steps one to four are repeated several times to generate a whole family of impulse response functions which form the basis for the computation of the confi- dence bands. In many applications, these confidence bands are constructed in a naive fashion by connecting the confidence intervals for individual impulse responses at different horizons. This, however, ignores the fact that the impulse responses at different horizons are correlated which implies that the true coverage probability of the confidence band is different from the presumed one. Thus, the joint probability distribution of the impulse responses should serve as the basis of the computation of the confidence bands. Recently, several alternatives have been proposed which take this feature in account. Lütkepohl et al. (2013) provides a comparison of several methods.

The number of repetitions should be at least 500. The method can be refined somewhat if the bias towards zero of the estimates of the ˆ’s is taken into account. This bias can again be determined through simulation methods (Kilian 1998). A critical appraisal of the bootstrap can be found in Sims (1999) where additional improvements are discussed. The bootstrap of the variance decomposition works in similar way.

15.4.4 Example 1: Advertisement and Sales

In this example we will analyze the dynamic relationship between advertisement expenditures and sales by a VAR approach. The data we will use are the famous data from the Lydia E. Pinkham Medicine Company which cover yearly observations from 1907 to 1960. These data were among the first ones which have been used to quantify the effect of advertisement expenditures on sales. The data are taken from Berndt (1991; Chapter 8) where details on the specificities of the data and a summary of the literature can be found.

We denote the two-dimensional logged time series of advertisement expenditures and sales by fXtg D f.ln.advertisementt/; ln.salest//0g. We consider VAR models of order one to six. For each VAR(p), p D 1;2; : : : ;6, we compute the corresponding information criteria AIC and BIC (see Sect. 14.3). Whereas AIC favors a model of order two, BIC proposes the more parsimonious model of order one. To be on the safe side, we work with a VAR model of order two whose estimates are reported below:

15.4 Interpretation of VAR Models 275

Xt D

0.145 (0.634)

0.762 (0.333)

0.451 (0.174)

0.642 (0.302)

-0.068 (0.091)

1.245 (0.159)

Xt�1

-0.189 (0.180)

0.009 (0.333)

-0.176 (0.095)

-0.125 (0.175)

Xt�2 C Zt;

where the estimated standard deviations of the coefficients are reported in parenthe- sis. The estimate b† of the covariance matrix † is

b† D �

0:038 0:011

0:011 0:010

�

The estimated VAR(2) model is taken to be the reduced form model. The struc- tural model contains two structural shocks: a shock to advertisement expenditures, VAt, and a shock to sales, VSt. The disturbance vector of the structural shock is thus fVtg D f.VAt; VSt/0g. It is related to Zt via relation (15.4), i.e. AZt D BVt. To identify the model we thus need 3 restrictions.19 We will first assume that A D I2 which gives two restrictions. A plausible further assumption is that shocks to sales have no contemporaneous effects on advertisement expenditures. This zero restriction seems justified because advertisement campaigns have to be planed in advance. They cannot be produced and carried out immediately. This argument then delivers the third restriction as it implies that B is a lower triangular. This lower triangular matrix can be obtained from the Cholesky decomposition of b†:

bB D �

1 0

0:288 1

�

and b� D �

0:038 0

0 0:007

�

The identifying assumptions then imply the impulse response functions plotted in Fig. 15.2.

The upper left figure shows the response of a sudden transitory increase in advertisement expenditures by 1 % (i.e. of a 1-% increase of VAt) to itself. This shock is positively propagated to the future years, but is statistically zero after four years. After four years the shock even changes to negative, but statistically insignificant, expenditures. The same shock produces an increase of sales by 0.3 % in the current and next year as shown in the lower left figure. The effect then deteriorates and becomes even negative after three years. The right hand figures display the reaction

19Formula (15.6) for n D 2 gives 3 restrictions.

276 15 Interpretation of VAR Models

-1

-0.5

0.5

effect of an advertisement shock on advertisement expenditures

period

-1

effect of a sales shock on advertisement expenditures

-1

-0.5

0.5

1 effect of an advertisement shock on sales

0 10 20 30

period

0 10 20 30

period

0 10 20 30

period

0 10 20 30 -1

3 effect of a sales shock on sales

zero restriction

Fig. 15.2 Impulse response functions for advertisement expenditures and sales with 95-% confidence intervals computed using the bootstrap procedure

of a sudden transitory increase of sales by 1 %. Again, we see that the shock is positively propagated. Thereby the largest effect is reached after two years and then declines monotonically. The reaction of advertisement expenditures is initially equal to zero by construction as it corresponds to the identifying assumption with regard to B. Then, the effect starts to increase and reaches a maximum after three years and then declines monotonically. After 15 years the effect is practically zero. The 95-% confidence intervals are rather large so that all effects are no longer statistically significant after a few number of years.

15.4 Interpretation of VAR Models 277

15.4.5 Example 2: IS-LM Model with Phillips Curve

In this example we replicate the study of Blanchard (1989) which investigates the US business cycle within a traditional IS-LM model with Phillips curve.20 The starting point of his analysis is the VAR(p) model:

Xt D ˆ1Xt�1 C : : : C ˆpXt�p C C Dt C Zt where fXtg is a five-dimensional time series Xt D .Yt; Ut; Pt; Wt; Mt/0. The individual elements of Xt denote the following variables:

Yt : : : growth rate of real GDP

Ut : : : unemployment rate

Pt : : : inflation rate

Wt : : : growth rate of wages

Mt : : : growth rate of money stock:

The VAR has attached to it a disturbance term Zt D .Zyt; Zut; Zpt; Zwt; Zmt/0. Finally, fDtg denotes the deterministic variables of the model such as a constant, time trend or dummy variables. In the following, we assume that all variables are stationary.

The business cycle is seen as the result of five structural shocks which impinge on the economy:

Vdt : : : aggregate demand shock

Vst : : : aggregate supply shock

Vpt : : : price shock

Vwt : : : wage shock

Vmt : : : money shock:

We will use the IS-LM model to rationalize the restrictions so that we will be able to identify the structural form from the estimated VAR model. The disturbance of the structural and the reduced form models are related by the simultaneous equation system:

AZt D BVt where Vt D .Vyt; Vst; Vpt; Vwt; Vmt/0 and where A and B are 5 � 5 matrices with ones on the diagonal. Blanchard (1989) proposes the following specification:

20The results do not match exactly those of Blanchard (1989), but are qualitatively similar.

278 15 Interpretation of VAR Models

(AD): Zyt D Vdt C b12Vst (OL): Zut D �a21Zyt C Vst (PS): Zpt D �a34Zwt � a31Zyt C b32Vst C Vpt

(WS): Zwt D �a43Zpt � a42Zut C b42Vst C Vwt (MR): Zmt D �a51Zyt � a52Zut � a53Zpt � a54Zwt C Vmt:

In matrix notation the above simultaneous equation system becomes:

1 0 0 0 0

a21 1 0 0 0 a31 0 1 a34 0 0 a42 a43 1 0

a51 a52 a53 a54 1

Zyt Zut Zpt Zwt Zmt

1 b12 0 0 0 0 1 0 0 0

0 b32 1 0 0 0 b42 0 1 0 0 0 0 0 1

Vdt Vst Vpt Vwt Vmt

The first equation is interpreted as an aggregate demand (AD) equation where the disturbance term related to GDP growth, Zyt, depends on the demand shock Vdt and the supply shock Vst. The second equation is related to Okun’s law (OL) which relates the unemployment disturbance Zut to the demand disturbance and the supply shock. Thereby an increase in GDP growth reduces unemployment in the same period by a21 whereas a supply shock increases it. The third and the fourth equation represent a price (PS) and wage setting (WS) system where wages and prices interact simultaneously. Finally, the fifth equation (MR) is supposed to determine the money shock (MR). No distinction is made between money supply and money demand shocks. A detailed interpretation of these equations is found in the original article by Blanchard (1989).

Given that the dimension of the system is five (i.e. n D 5), formula (15.6) instructs us that we need 3 � .5 � 4/=2 D 30 restrictions. Counting the number of zero restrictions implemented above, we see that we only have 28 zeros. Thus we lack two additional restrictions. We can reach the same conclusion by counting the number of coefficients and the number of equations. The coefficients are a21; a31; a34; a42; a43; a51; a52; a53; a54, b12; b32; b42 and the diagonal elements of �, the covariance matrix of Vt . We therefore have to determine 17 unknown coefficients out of .5 � 6/=2 D 15 equations. Thus we find again that we are short of two restrictions. Blanchard discusses several possibilities among which the restrictions b12 D 1:0 and a34 D 0:1 seem most plausible.

The sample period runs through the second quarter in 1959 to the second quarter in 2004 encompassing 181 observations. Following Blanchard, we include a constant in combination with a linear time trend in the model. Whereas BIC suggests a model of order one, AIC favors a model of order two. As a model of order one seems rather restrictive, we stick to the VAR(2) model whose estimated coefficients are reported below21:

21To save space, the estimated standard errors of the coefficients are not reported.

15.4 Interpretation of VAR Models 279

bˆ1 D

0:07 �1:31 0:01 0:12 0:02 �0:02 1:30 0:03 �0:00 �0:00 �0:07 �1:47 0:56 0:07 0:03 0:07 0:50 0:44 0:07 0:06

�0:10 1:27 �0:07 0:04 0:49

bˆ2 D

0:05 1:79 �0:41 �0:13 0:05 �0:02 �0:35 0:00 0:01 �0:00 �0:04 1:38 0:28 0:05 �0:00 0:07 �0:85 0:19 0:10 �0:04

�0:02 �0:77 �0:07 0:11 0:17

bC D

2:18 �0:0101 0:29 0:0001

0:92 �0:0015 4:06 �0:0035

�0:98 �0:0025

b† D

9:94 �0:46 �0:34 0:79 0:29 �0:46 0:06 �0:02 �0:05 � 0:06 �0:34 �0:02 1:06 0:76 0:13 0:79 �0:05 0:76 5:58 0:76 0:29 �0:06 0:13 0:76 11:07

The first column of bC relates to the constants, whereas the second column gives the coefficients of the time trend. From these estimates and given the identifying restrictions established above, the equation b† D A�1B�B0A0�1 uniquely determines the matrices A, B and �:

bA D

1 0 0 0 0

0:050 1 0 0 0

0:038 0 1 0:1 0

0 1:77 �0:24 1 0 0:033 1:10 0:01 �0:13 1

bB D

1 1 0 0 0

0 1 0 0 0

0 �1:01 1 0 0 0 1:55 0 1 0

0 0 0 0 1

280 15 Interpretation of VAR Models

b� D

9:838 0 0 0 0

0 0:037 0 0 0

0 0 0:899 0 0

0 0 0 5:162 0

0 0 0 0 10:849

In order to give better interpretation of the results we have plotted the impulse response functions and their 95-% confidence bands in Fig. 15.3. The results show that a positive demand shock has only a positive and statistically significant effect on GDP growth in the first three quarters, after that the effect becomes even slightly negative and vanishes after sixteen quarters. The positive demand shock reduces unemployment significantly for almost fifteen quarters. The maximal effect is achieved after three to four quarters. Although the initial effect is negative, the positive demand shock also drives inflation up which then pushes up wage growth. The supply shock also has a positive effect on GDP growth, but it takes more than four quarters before the effect reaches its peak. In the short-run the positive supply

0 10 20 30 −0.2

0 0.2 0.4 0.6 0.8

demand shock on Yt

0 10 20 30 −4 −3 −2 −1

0 1 2 3

supply shock on Yt

0 10 20 30 −0.8 −0.6 −0.4 −0.2

0 0.2 0.4

price shock on Yt

0 10 20 30 −0.4

−0.2

0.2

0.4

wage shock on Yt

0 10 20 30 −0.15 −0.1

−0.05 0

0.05 0.1

0.15

money shock on Yt

0 10 20 30 −0.2

−0.15

−0.1

−0.05

0.05

demand shock on Ut

0 10 20 30 −1.5

−1

−0.5

0.5

1.5

supply shock on Ut

0 10 20 30 −0.2 −0.1

0 0.1 0.2 0.3 0.4

price shock on Ut

0 10 20 30 −0.05

0.05

0.1

0.15

wage shock on Ut

0 10 20 30 −0.06

−0.04

−0.02

0.02

0.04

0.06

money shock on Ut

0 10 20 30 −0.1

−0.05

0.05

0.1

0.15

demand shock on Pt

0 10 20 30 −4

−3

−2

−1

1 supply shock on Pt

0 10 20 30 −0.5

0.5

1.5

price shock on Pt

0 10 20 30 −0.1

−0.05 0

0.05 0.1

0.15 0.2

wage shock on Pt

0 10 20 30 −0.05

0.05

0.1

0.15

money shock on Pt

0 10 20 30 −0.1

−0.05 0

0.05 0.1

0.15 0.2

demand shock on Wt

0 10 20 30 −4

−3

−2

−1

supply shock on Wt

0 10 20 30 −0.4 −0.2

0 0.2 0.4 0.6 0.8

price shock on Wt

0 10 20 30 −0.2

0 0.2 0.4 0.6 0.8

wage shock on Wt

0 10 20 30 −0.1

−0.05 0

0.05 0.1

0.15 0.2

money shock on Wt

0 10 20 30 −0.4

−0.3

−0.2

−0.1

0.1

demand shock on Mt

0 10 20 30 −3 −2 −1

0 1 2 3 4

supply shock on Mt

0 10 20 30 −1

−0.5

0.5

price shock on Mt

0 10 20 30 −0.2 −0.1

0 0.1 0.2 0.3 0.4

wage shock on Mt

0 5 10 15 20 25 30 −0.2

0 0.2 0.4 0.6 0.8

1 1.2

money shock on Mt

Fig. 15.3 Impulse response functions for the IS-LM model with Phillips curve with 95-% confidence intervals computed using the bootstrap procedure (compare with Blanchard (1989))

15.4 Interpretation of VAR Models 281

shock even reduces GDP growth. In contrast to the demand shock, the positive supply shock increases unemployment in the short-run. The effect will only reduce unemployment in the medium- to long-run. The effect on price and wage inflation is negative.

Finally, we compute the forecast error variance decomposition according to Sect. 15.4.2. The results are reported in Table 15.1. In the short-run, the identifying restrictions play an important role as reflected by the plain zeros. The demand shock accounts for almost all the variance of GDP growth in the short-run. The value of 99:62 % for forecast horizon of one quarter, however, diminishes as h increases to 40 quarters, but still remains with a value 86:13 very high. The supply shock on the

Table 15.1 Forecast error variance decomposition (FEVD) in terms of demand, supply, price, wage, and money shocks (percentages)

Horizon Demand Supply Price Wage Money

Growth rate of real GDP

1 99.62 0.38 0 0 0

2 98.13 0.94 0.02 0.87 0.04

4 93.85 1.59 2.13 1.86 0.57

8 88.27 4.83 3.36 2.43 0.61

40 86.13 6.11 4.29 2.58 0.89

Unemployment rate

1 42.22 57.78 0 0 0

2 52.03 47.57 0.04 v0.01 0.00

4 64.74 33.17 1.80 0.13 0.16

8 66.05 21.32 10.01 1.99 0.63

40 39.09 16.81 31.92 10.73 0.89

Inflation rate

1 0.86 4.18 89.80 5.15 0

2 0.63 13.12 77.24 8.56 0.45

4 0.72 16.79 68.15 13.36 0.97

8 1.79 19.34 60.69 16.07 2.11

40 2.83 20.48 55.84 17.12 3.74

Growth rate of wages

1 1.18 0.10 0.97 97.75 0

2 1.40 0.10 4.30 93.50 0.69

4 2.18 2.75 9.78 84.49 0.80

8 3.80 6.74 13.40 74.72 1.33

40 5.11 8.44 14.19 70.14 2.13

Growth rate of money stock

1 0.10 0.43 0.00 0.84 98.63

2 1.45 0.44 0.02 1.02 97.06

4 4.22 1.09 0.04 1.90 92.75

8 8.31 1.55 0.81 2.65 86.68

40 8.47 2.64 5.77 4.55 78.57

282 15 Interpretation of VAR Models

contrary does not explain much of the variation in GDP growth. Even for a horizon of 40 quarters, it contributes only 6.11 %. The supply shock is, however, important for the variation in the unemployment rate, especially in the short-run. It explains more than 50 % whereas demand shocks account for only 42.22 %. Its contribution diminishes with the increase of the forecast horizon giving room for price and wage shocks. The variance of the inflation rate is explained in the short-run almost exclusively by price shocks. However, as the forecast horizon is increased supply and wage shocks become relatively important. The money growth rate does not interact much with the other variables. Its variation is almost exclusively explained by money shocks.

15.5 Identification via Long-Run Restrictions

15.5.1 A Prototypical Example

Besides short-run restrictions, essentially zero restrictions on the coefficients of A and/or B, Blanchard and Quah (1989) proposed long-run restrictions as an alternative option. These long-run restrictions have to be seen as complementary to the short-run ones as they can be combined. Long-run restrictions constrain the long-run effect of structural shocks. This technique makes only sense if some integrated variables are involved, because in the stationary case the effects of all shocks vanish eventually. To explain this, we discuss the two-dimensional example given by Blanchard and Quah (1989).

They analyze a two-variable system consisting of logged real GDP denoted by fYtg and the unemployment rate fUtg. Logged GDP is typically integrated of order one (see Sect. 7.3.4 for an analysis for Swiss GDP) whereas Ut is considered to be stationary. Thus they apply the VAR approach to the stationary process fXtg D f.�Yt; Ut/0g. Assuming that fXtg is already demeaned and follows a causal VAR process, we have the following representations:

Xt D �

�Yt Ut

�

D ˆ1Xt�1 C : : : C ˆpXt�p C Zt

D ‰.L/Zt D Zt C ‰1Zt�1 C ‰2Zt�2 C : : : :

For simplicity, we assume that A D I2, so that

Zt D BVt D �

1 b12 b21 1

��

vdt

vst

�

where Vt D .vdt;vst/0 � WN.0;�/ with � D diag.!2d;!2s /. Thereby fvdtg and fvstg denote demand and supply shocks, respectively. The causal representation of fXtg implies that the effect of a demand shock in period t on GDP growth in period t C h is given by:

15.5 Identification via Long-Run Restrictions 283

@�YtCh @vdt

D Œ‰hB 11

where Œ‰hB 11 denotes the upper left hand element of the matrix ‰hB. YtCh can be written as YtCh D �YtCh C �YtCh�1 C : : : C �YtC1 C Yt so that the effect of the demand shock on the level of logged GDP is given by:

@YtCh @vdt

D h X

jD0 Œ‰jB 11 D

h X

jD0 ‰jB

Blanchard and Quah (1989) propose, in accordance with conventional economic theory, to restrict the long-run effect of the demand shock on the level of logged GDP to zero:

lim h!1

@YtCh @vdt

D 1 X

jD0 Œ‰jB 11 D 0:

This implies that

1 X

jD0 ‰jB D

1 X

jD0 ‰j

A B D ‰.1/B D �

0 � � �

�

;

where � is a placeholder. This restriction is sufficient to infer b21 from the relation Œ‰.1/ 11b11 C b21Œ‰.1/ 12 D 0 and the normalization b11 D 1:

b21 D � Œ‰.1/ 11

Œ‰.1/ 12 D �Œˆ.1/

�1 11 Œˆ.1/�1 12

The second part of the equation follows from the identity ˆ.z/‰.z/ D I2 which gives ‰.1/ D ˆ.1/�1 for z D 1. The long-run effect of the supply shock on Yt is left unrestricted and is therefore in general nonzero. Note that the implied value of b21 depends on ˆ.1/, and thus on ˆ1;: : : ;ˆp. The results are therefore, in contrast to short-run restrictions, much more sensitive to correct specification of the VAR.

The relation Zt D BVt implies that

† D B �

!2d 0

0 !2s

�

or more explicitely

† D �

�21 �12

�12 � 2 2

�

D �

1 b12 b21 1

��

!2d 0

0 !2s

��

1 b21 b12 1

�

284 15 Interpretation of VAR Models

Taking b21 as already given from above, this equation system has three equations and three unknowns b12;!

2 d;!

2 s which is a necessary condition for a solution to

exist.

Analytic Solution of the System Because b21 is already determined from the long-run restriction, we rewrite the equation system22 explicitly in terms of b21:

�21 D !2d C b212!2s �12 D b21!2d C b12!2s �22 D b221!2d C !2s :

Using the last two equations, we can express !2d and ! 2 s as functions of b12:

!2d D �12 � b12�22 b21 � b12b221

!2s D �22 � b21�12 1 � b12b21

These expressions are only valid if b21 ¤ 0 and b12b21 ¤ 1. The case b21 D 0 is not interesting with regard to content. It would simplify the original equation system strongly and would results in b12 D �12=�22 , !2d D .�21�22 � �212/=�22 > 0 and !2s D �22 > 0. The case b12b21 D 1 contradicts the assumption that † is a positive-definite matrix and can therefore be disregarded.23

Inserting the solutions of !2d and ! 2 s into the first equation, we obtain a quadratic

equation in b12:

�

b21� 2 2 � b221�12

�

b212 C �

b221� 2 1 � �22

�

b12 C �

�12 � b21�21 � D 0:

The discriminant � of this equation is:

� D �b221�21 � �22 �2 � 4�b21�22 � b221�12

��

�12 � b21�21 �

D �b221�21 C �22 �2 � 4b21�12

�

b221� 2 1 C �22

� C 4b221�212 D �b221�21 C �22 � 2b21�12

�2 > 0:

22This equation system is similar to the one analyzed in Sect. 15.2.3. 23If b12b21 D 1, det † D �21�22 � �212 D 0. This implies that Z1t and Z2t are perfectly correlated, i.e. �2Z1t;Z2t D �212=.�21�22/ D 1.

15.5 Identification via Long-Run Restrictions 285

The positivity of the discriminant implies that the quadratic equation has two real solutions b

.1/ 12 and b

.2/ 12 :

b .1/ 12 D

�2 2

� b21�12 b21�

2 2 � b221�12

D 1 b21 ;

b .2/ 12 D

�12 � b21�21 �22 � b21�12

The first solution b .1/ 12 can be excluded because it violates the assumption b12b21 ¤ 1

which stands in contradiction to the positive-definiteness of the covariance matrix †. Inserting the second solution back into the solution for !2d and !

2 s , we finally get:

!2d D �21�

2 2 � �212

b221� 2 1 � 2b21�12 C �22

> 0

!2s D �

�22 � b21�12 �2

b221� 2 1 � 2b21�12 C �22

> 0:

Because † is a symmetric positive-definite matrix, �21� 2 2 � �212 and the denominator

b221� 2 1 � 2b21�12 C �22 are strictly positive. Thus, both solutions yield positive

variances and we have found the unique admissible solution.

15.5.2 The General Approach

The general case of long-run restrictions has a structure similar to the case of short- run restrictions. Take as a starting point the structural VAR (15.2) from Sect. 15.2.2:

AXt D �1Xt�1 C : : : C �pXt�p C BVt; Vt � WN.0;�/; A.L/Xt D BVt

where fXtg is stationary and causal with respect to fVtg. As before the matrix A is normalized to have ones on its diagonal and is assumed to be invertible, � is a diagonal matrix with � D diag.!21; : : : ;!2n/, and B is a matrix with ones on the diagonal. The matrix polynomial A.L/ is defined as A.L/ D A � �1L � : : : � �pLp. The reduced form is given by

ˆ.L/Xt D Zt; Zt � WN.0;†/

where AZt D BVt and Aˆj D �j, j D 1; : : : ; p, respectively Aˆ.L/ D A.L/.

286 15 Interpretation of VAR Models

The long-run variance of fXtg, J, (see Eq. (11.1) in Chap. 11) can be derived from the reduced as well as from the structural form, which gives the following expressions:

J D ˆ.1/�1† ˆ.1/�10 D ˆ.1/�1A�1B�B0A0�1ˆ.1/�10 D A.1/�1B�B0A.1/�10

D ‰.1/† ‰.1/0 D ‰.1/A�1B�B0A0�1‰.1/0

where Xt D ‰.L/Zt denotes the causal representation of fXtg. The long-run variance J can be estimated by adapting the methods in Sect. 4.4 to the multivariate case. Thus, the above equation system has a similar structure as the system (15.5) which underlies the case of short-run restrictions. As before, we get n.n C 1/=2 equations with 2n2 � n unknowns. The nonlinear equation system is therefore undetermined for n � 2. Therefore, 3n.n � 1/=2 additional equations or restrictions are necessary to achieve identification. Hence, conceptually we are in a similar situation as in the case of short-run restrictions.24

In practice, it is customary to achieve identification through zero restrictions where some elements of ‰.1/A�1B, respectively ˆ.1/�1A�1B, are set a priori to zero. Setting the ij-th element Œ‰.1/A�1B ij D Œˆ.1/�1A�1B ij equal to zero amounts to set the cumulative effect of the j-th structural disturbance Vjt on the i-th variable equal to zero. If the i-th variable enters Xt in first differences, as was the case for Yt in the previous example, this zero restriction restrains the long-run effect on the level of that variable.

An interesting simplification arises if one assumes that A D In and that ‰.1/B D ˆ.1/�1B is a lower triangular matrix. In this case, B and � can be estimated from the Cholesky decomposition of the estimated long-run variance OJ. Let OJ D OL OD OL0 be the Cholesky decomposition where OL is lower triangular matrix with ones on the diagonal and OD is a diagonal matrix with strictly positive diagonal entries. As OJ D Ô .1/�1B O�B0 Ô .1/�10, the matrix of structural coefficients can then be estimated as OB D Ô .1/OL OU�1. The multiplication by the inverse of the diagonal matrix U D diag. Ô .1/OL/ is necessary to guarantee that the normalization of OB (diagonal elements equal to one) is respected. � is then estimated as O� D OU OD OU.

Instead of using a method of moments approach, it is possible to use an instrumental variable (IV) approach. For this purpose we rewrite the reduced form of Xt in the Dickey-Fuller form (see Eqs. (7.1) and (16.4)):

�Xt D �ˆ.1/Xt�1 C eˆ1�Xt�1 C : : : C eˆp�1�Xt�pC1 C Zt;

where eˆj D � Pp

iDjC1 ˆi, j D 1;2; : : : ; p�1. For the ease of exposition, we assume B D In so that AZt D Vt. Multiplying this equation by A yields:

A�Xt D �Aˆ.1/Xt�1 C Aeˆ1�Xt�1 C : : : C Aeˆp�1�Xt�pC1 C Vt: (15.9)

24See Rubio-Ramírez et al. (2010) for a unified treatment of both type of restrictions.

15.5 Identification via Long-Run Restrictions 287

Consider for simplicity the case that Aˆ.1/ is a lower triangular matrix. This implies that the structural shocks V2t; V3t; : : : ; Vnt have no long-run impact on the first variable X1t. It is, therefore, possible to estimate the coefficients A12; A13; : : : ; A1n by instrumental variables taking X2;t�1; X3;t�1; : : : ; Xn;t�1 as instruments.

For n D 2 the Dickey-Fuller form of the equation system (15.9) is: �

1 A12 A21 1

��

� QX1t � QX2t

�

D � �

ŒAˆ.1/ 11 0 ŒAˆ.1/ 21 ŒAˆ.1/ 22

�� QX1;t�1 QX2;t�1

�

C �

V1t V2t

�

;

respectively

� QX1t D �A12� QX2t � ŒAˆ.1/ 11 QX1;t�1 C V1t � QX2t D �A21� QX1t

� ŒAˆ.1/ 21 QX1;t�1 � ŒAˆ.1/ 22 QX2;t�1 C V2t:

Thereby � QX1t and � QX2t denote the OLS residuals from a regression of �X1t, respectively �X2t on .�X1;t�1;�X2;t�1; : : : ;�X1;t�pC1;�X2;t�pC1/. QX2;t�1 is a valid instrument for � QX2t because this variable does not appear in the first equation. Thus, A12 can be consistently estimated by the IV-approach. For the estimation of A21, we can use the residuals from the first equation as instruments because V1t and V2t are assumed to be uncorrelated. From this example, it is easy to see how this recursive method can be generalized to more than two variables. Note also that the IV-approach can also be used in the context of short-run restrictions.

The issue whether a technology shock leads to a reduction of hours worked in the short-run, led to a vivid discussion on the usefulness of long-run restrictions for structural models (Galí 1999; Christiano et al. 2003, 2006; Chari et al. 2008). From an econometric point of view, it turned out, on the one hand, that the estimation of ˆ.1/ is critical for the method of moments approach. The IV-approach, on the other hand, depends on the strength or weakness of the instrument used (Pagan and Robertson 1998; Gospodinov 2010).

It is, of course, possible to combine both short- and long-run restrictions simultaneously. An interesting application of both techniques was presented by Galí (1992). In doing so, one must be aware that both type of restrictions are consistent with each other and that counting the number of restrictions gives only a necessary condition.

Example 3: Identifying Aggregate Demand and Supply Shocks

In this example, we follow Blanchard and Quah (1989) and investigate the behavior of the growth rate of real GDP and the unemployment rate for the US over the period from the first quarter 1979 to the second quarter 2004 (102 observations). The AIC and the BIC suggest models of order two and one, respectively. Because some

288 15 Interpretation of VAR Models

coefficients of bˆ2 are significant at the 10 % level, we prefer to use the VAR(2) model which results in the following estimates25:

bˆ1 D �

0:070 �3:376 �0:026 1:284

�

bˆ2 D �

0:029 3:697

�0:022 �0:320 �

b† D �

7:074 �0:382 �0:382 0:053

�

These results can be used to estimate ˆ.1/ D I2 � ˆ1 � ˆ2 and consequently also ‰.1/ D ˆ.1/�1:

bˆ.1/ D �

0:901 �0:321 0:048 0:036

�

b‰.1/ D bˆ.1/�1 D �

0:755 6:718

�1:003 18:832 �

Assuming that Zt D BVt and following the argument in Sect. 15.5.1 that the demand shock has no long -run impact on the level of real GDP, we can retrieve an estimate for b21:

Ob21 D �Œb‰.1/ 11=Œb‰.1/ 12 D �0:112:

The solution of the quadratic equation for b12 are �8:894 and 43.285. As the first solution results in a negative variance for !22 , we can disregard this solution and stick to the second one. The second solution makes also sense economically, because a positive supply shock leads to positive effects on GDP. Setting b12 D 43:285 gives the following estimates for covariance matrix of the structural shocks �:

b� D � O!2d 0 0 O!2s

�

D �

4:023 0

0 0:0016

�

The big difference in the variance of both shocks clearly shows the greater importance of demand shocks for business cycle movements.

Figure 15.4 shows the impulse response functions of the VAR(2) identified by the long-run restriction. Each figure displays the dynamic effect of a demand and a supply shock on real GDP and the unemployment rate, respectively, where the size of the initial shock corresponds to one standard deviation. The result conforms well with standard economic reasoning. A positive demand shock increases real GDP

25The results for the constants are suppressed to save space.

15.6 Sign Restrictions 289

0 10 20 30 40 −2

period

effect of demand shock to GDP

0 10 20 30 40 −4

−2

8 effect of supply shock to GDP

period

0 10 20 30 40 −1

−0.5

0.5

period

effect of demand shock to the unemployment rate

0 10 20 30 40 −0.4

−0.2

0.2

0.4 effect of supply shock to the unemployment rate

period

Fig. 15.4 Impulse response functions of the Blanchard-Quah model (Blanchard and Quah 1989) with 95-% confidence intervals computed using the bootstrap procedure

and lowers the unemployment rate in the short-run. The effect is even amplified for some quarters before it declines monotonically. After 30 quarters the effect of the demand has practically vanished so that its long-run effect becomes zero as imposed by the restriction. The supply shock has a similar short-run effect on real GDP, but initially increases the unemployment rate. Only when the effect on GDP becomes stronger after some quarters will the unemployment rate start to decline. In the long-run, the supply shock has a positive effect on real GDP but no effect on unemployment. Interestingly, only the short-run effects of the demand shock are statistically significant at the 95-% level.

15.6 Sign Restrictions

In recent years the use of sign restrictions has attracted a lot of attention. Pioneering contributions have been provided by Faust (1998), Canova and De Nicoló (2002), and Uhlig (2005). Since then the literature has abounded by many applications in many contexts. Sign restrictions try to identify the impact of the structural shocks by requiring that the signs of the impulse response coefficients have to follow a given pattern. The motivation behind this development is that economists are often more confident about the sign of an economic relationship than about its exact magnitude.

290 15 Interpretation of VAR Models

This seems to be true also for zero restrictions, whether they are short- or long-run restrictions. This insight already led Samuelson (1947) to advocate a calculus of qualitative relations in economics. Unfortunately, this approach has been forgotten in the progression of economic analysis.26 With the rise in popularity of sign restrictions his ideas may see a revival.

To make the notion of sign restrictions precise, we introduce a language based on the following notation and definitions. Sign restrictions will be specified as sign pattern matrices. These matrices make use of the sign function of a real number x, sgn.x/, defined as

sgn.x/ D 8

1; if x > 0; �1; if x < 0; 0; if x D 0.

Definition 15.1. A sign pattern matrix or pattern for short is a matrix whose elements are solely from the set f1; �1;0g. Given a sign pattern matrix S, the sign pattern class of S is defined by

S.S/ D fM 2 M.n/ j sgn.M/ D Sg

where M.n/ is the set of all n � n matrices and where sgn is applied elementwise to M. Clearly, S.S/ D S.sgn.S//.

Remark 15.1. Often the set fC; �;0g is used instead of fC1; �1;0g to denote the sign patterns.

Remark 15.2. In some instances we do not want to restrict all signs but only a subset of it. In this case, the elements of the sign pattern matrix S may come from the larger set f�1;0;1; #g where # stands for an unspecified sign. S is then called a generalized sign pattern matrix or a generalized pattern. The addition C and the multiplication � of (generalized) sign pattern matrices are defined in a natural way.

In order not to overload the discussion, we set A D In and focus on the case where Zt D BVt.27 Moreover, we use a different, but completely equivalent normalization. In particular, we relax, on the one hand, the assumption that B has only ones on its diagonal, but assume on the other hand that � D In. Note that † D EZtZ0t D BB0 is a strictly positive definite matrix. Assuming that a causal representation of fXtg in terms of fZtg and thus also in terms of fVtg exists, we can represent Xt as

26It is interesting to note that Samuelson’s ideas have fallen on fruitful grounds in areas like computer science or combinatorics (see Brualdi and Shader 1995; Hall and Li 2014). 27The case with general A matrices can be treated analogously.

15.6 Sign Restrictions 291

Xt D BVt C ‰1BVt�1 C ‰2BVt�2 C : : : D 1 X

hD0 ‰hBVt�h D ‰.L/BVt:

Denote by B.†/ the set of invertible matrices B such that † D BB0, i.e. B.†/ D fB 2 GL.n/ j BB0 D †g.28 Thus, B.†/ is the set of all feasible structural factorizations (models).

Sign restrictions on the impulse responses Œ‰hB i;j can then be defined in terms of a sequence of (generalized) sign pattern matrices fShg, h D 0;1;2; : : : Definition 15.2 (Sign Restrictions). A causal VAR allows a sequence of (general- ized) sign pattern matrices fShg if and only if there exists B 2 B.†/ such that

‰hB 2 S.Sh/; for all h D 0;1;2; : : : (15.10)

Remark 15.3. As k‰jBk converges to zero for j ! 1, it seems reasonable to impose sign restrictions only up to some horizon hmax < 1. In this case, Sh, h > hmax, is equal to the generalized sign pattern matrix whose elements consist exclusively of #’s. A case of particular interest is given by hmax D 0. In this case, we drop the index 0 and say that the VAR allows (generalized) sign pattern matrix S.

Remark 15.4. With this notation we can also represent (short-run) zero restrictions if the sign patterns are restricted to 0 and #.

A natural question to ask is how restrictive a prescribed sign pattern is. This amounts to the question whether a given VAR can be compatible with any sign pattern. As is already clear from the discussion of the two-dimensional case in Sect. 15.2.3, this is not the case. As the set of feasible parameters can be represented by a rectangular hyperbola, there will always be one quadrant with no intersection with the branches of the hyperbola. In the example plotted in Fig. 15.1, this is quadrant III. Thus, configurations with .B/21 < 0 and .B/12 < 0 are not feasible given .†/12 > 0. This argument can be easily extended to models of higher dimensions. Thus, there always exist sign patterns which are incompatible with a given †.

As pointed out in Sect. 15.3 there always exists a unique lower triangular matrix R, called the Cholesky factor of †, such that † D RR0. Thus, B.†/ ¤ ; because R 2 B.†/.

Lemma 15.1. Let the Cholesky factor of † be R, then

B.†/ D fB 2 GL.n/ j 9Q 2 O.n/ W B D RQg:

28GL.n/ is known as the general linear group. It is the set of all invertible n � n matrices.

292 15 Interpretation of VAR Models

Proof. Suppose B D RQ with Q 2 O.n/, then BB0 D RQQ0R0 D †. Thus, B 2 B.†/. If B 2 B.†/, define Q D R�1B. Then, QQ0 D R�1BB0.R0/�1 D R�1†.R0/�1 D R�1RR0.R0/�1 D In. Thus, Q 2 O.n/. ut

This lemma establishes that there is a one-to-one function '† from the group of orthogonal matrices O.n/ onto the set of feasible structural factorizations B.†/. From the proof we see that '†.Q/ D RQ and '�1† .B/ D R�1B. Moreover, for any two matrices B1 and B2 in B.†/ with B1 D RQ1 and B2 D RQ2, there exists an orthogonal matrix Q equal to Q02Q1 such that B1 D B2Q. As '† and '�1† are clearly continuous, '† is a homeomorphism. See Neusser (2016) for more details and further implications.

To make the presentation more transparent, we focus on sign restrictions only and disregard zero restrictions. Arias et al. (2014) show how sign and zero restrictions can be treated simultaneously. Thus, the entries of fShg are elements of f�1; C1; #g only. Assume that a VAR allows sign patterns fShg, h D 0;1; : : : ; hmax. Then according to Definition 15.2, there exists B 2 B.†/ such that ‰hB 2 S.Sh/ for all h D 0;1; : : : ; hmax. As the (strict) inequality restrictions delineate an open subset of B 2 B.†/, there exist other nearby matrices which also fulfill the sign restrictions. Sign restrictions therefore do not identify one impulse response sequence, but a whole set. Thus, the impulse responses are called set identified.

This set is usually difficult to characterize algebraically so that one has to rely on computer simulations. Conditional on the estimated VAR, thus conditional on fb‰jg and b†, Lemma 15.1 suggests a simple and straightforward algorithm (see Rubio- Ramírez et al. 2010; Arias et al. 2014; for further details):

Step 1: Draw at random an element Q from the uniform distribution on the set of orthogonal matrices O.n/.

Step 2: Convert Q into a random element of B.b†/ by applying ' b†

to Q. As ' b†

is a homeomorphism this introduces a uniform distribution on B.†/.

Step 3: Compute the impulse responses with respect to ' b† .Q/, i.e. compute

b‰h ' b† .Q/.

Step 4: Keep those models with impulse response functions which satisfy the prescribed sign restrictions b‰h '

b† .Q/ 2 S.Sh/, h D 0;1; : : : ; hmax.

Step 5: Repeat steps 1–4 until a satisfactory number of feasible structural models with impulse responses obeying the sign restrictions have been obtained.

The implementation of this algorithm requires a way to generate random draws Q from the uniform distribution on O.n/.29 This is not a straightforward task because the elements of Q are interdependent as they must ensure the orthonormality of the columns of Q. Edelman and Rao (2005) proposes the following efficient

29It can be shown that this probability measure is the unique measure � on O.n/ which satisfies the normalization �.O.n// D 1 and the (left)-invariance property �.QQ/ D �.Q/ for every Q O.n/ and Q 2 O.n/. In economics, this probability measure is often wrongly referred to as the Haar measure. The Haar measure is not normalized and is, thus, unique only up to a proportionality factor.

15.6 Sign Restrictions 293

two-step procedure. First, draw n�n matrices X such that X � N.0; In ˝In/, i.e. each element of X is drawn independently from a standard normal distribution. Second, perform the QR-decomposition which factorizes each matrix X into the product of an orthogonal matrix Q and an upper triangular matrix R normalized to have positive diagonal entries.30

As the impulse responses are only set identified, the way to report the results and how to conduct inference becomes a matter of discussion. Several methods have been proposed in the literature:

(i) One straightforward possibility consists in reporting, for each horizon h, the median of the impulse responses. Although simple to compute, this method presents some disadvantages. The median responses will not correspond to any of the structural models. Moreover, the orthogonality of the structural shocks will be lost. Fry and Pagan (2011) propose the median–target method as an ad hoc remedy to this shortage. They advocate to search for the admissible structural model whose impulse responses come closest to the median ones.

(ii) Another possibility is to search for the admissible structural model which maximizes the share of the forecast error variance at some horizon of a given variable after a particular shock.31 An early application of this method can be found in Faust (1998). This method remains, however, uninformative about the relative explanatory power of alternative admissible structural models.

(iii) The penalty function approach by Mountford and Uhlig (2009) does not accept or reject particular impulse responses depending on whether it is in accordance with the sign restrictions (see step 4 in the above algorithm). Instead, it associates for each possible impulse response function and every sign restriction a value which rewards a “correct” sign and penalizes a “wrong” sign. Mountford and Uhlig (2009) propose the following ad hoc penalty function: f .x/ D 100x if sgn.x/ is wrong and f .x/ D x if sgn.x/ is correct. The impulse response function which minimizes the total (standardized) penalty is then reported.

(iv) The exposition becomes more coherent if viewed from a Bayesian perspective. From this perspective, the uniform distribution on O.n/, respectively on B.b†/, is interpreted as diffuse or uninformative prior distribution.32 The admissible structural models which have been retained in step 5 of the algorithm are then seen as draws from the corresponding posterior distribution. The most likely model is then given by the model which corresponds to the mode of the poste- rior distribution. This model is associated an impulse response function which

30Given a value of n, the corresponding MATLAB commands are [Q,R]=qr(randn(n)); Q = Q*diag(sign(diag(R))); (see Edelman and Rao 2005). 31The minimization of the forecast error variance share have also been applied as an identification device outside the realm of sign restrictions. See Sect. 15.4.2. 32Whether this distribution is always the “natural” choice in economics has recently been disputed by Baumeister and Hamilton (2015).

294 15 Interpretation of VAR Models

can then be reported. This method also allows the construction of 100.1�˛/-% highest posterior density credible sets (see Inoue and Kilian 2013; for details). As shown by Moon and Schorfheide (2012) these sets cannot, even in a large sample context, be interpreted as approximate frequentist confidence intervals. Recently, however, Moon et al. (2013) proposed a frequentist approach to the construction of error bands for sign identified impulse responses.

16Cointegration

As already mentioned in Chap. 7, many raw economic time series are nonstationary and become stationary only after some transformation. The most common of these transformations is the formation of differences, perhaps after having taken logs. In most cases first differences are sufficient to achieve stationarity. The stationarized series can then be analyzed in the context of VAR models as explained in the previous chapters. However, many economic theories are formalized in terms of the original series so that we may want to use the VAR methodology to infer also the behavior of the untransformed series. Yet, by taking first differences we loose probably important information on the levels. Thus, it seems worthwhile to develop an approach which allows us to take the information on the levels into account and at the same time take care of the nonstationary character of the variables. The concept of cointegration tries to achieve this double requirement.

In the following we will focus our analysis on variables which are integrated of order one, i.e. on time series which become stationary after having taken first differences. However, as we have already mentioned in Sect. 7.5.1, a regression between integrated variables may lead to spurious correlations which make statisti- cal inferences and interpretations of the estimated coefficients a delicate issue (see Sect. 7.5.3). A way out of this dilemma is presented by the theory of cointegrated processes. Loosely speaking, a multivariate process is cointegrated if there exists a linear combination of the processes which is stationary although each process taken individually may be integrated. In many cases, this linear combination can be directly related to economic theory which has made the analysis of cointegrated processes an important research topic. In the bivariate case, already been dealt with in Sect. 7.5.2, the cointegrating relation can be immediately read off from the cointegrating regression and the cointegration test boils down to a unit root test for the residuals of the cointegrating regression. However, if more than two variables are involved, the single equation residual based test is, as explained in Sect. 7.5.2, no longer satisfactory. Thus, a genuine multivariate is desirable.

295

296 16 Cointegration

The concept of cointegration goes back to the work of Engle and Granger (1987) which is itself based on the precursor study of Davidson et al. (1978). In the meantime the literature has grown tremendously. Good introductions can be found in Banerjee et al. (1993), Watson (1994) or Lütkepohl (2006). For the more statistically inclined reader Johansen (1995) is a good reference.

16.1 A Theoretical Example

Before we present the general theory of cointegration within the VAR context, it is instructive to introduce the concept in the well-known class of present discounted value models. These models relate some variable Xt to present discounted value of another variable Yt:

Xt D �.1 � ˇ/ 1 X

jD0 ˇjPtYtCj C ut; 0 < ˇ < 1;

where ut � WN.0;�2u / designates a preference shock. Thereby, ˇ denotes the subjective discount factor and � is some unspecified parameter. The present discounted value model states that the variable Xt is proportional to the sum of future YtCj, j D 0;1;2; : : :, discounted by the factor ˇ. We can interpret Xt and Yt as the price and the dividend of a share, as the interest rate on long- and short-term bonds, or as consumption and income. In order to operationalize this model, we will assume that forecasts are computed as linear mean-squared error forecasts. The corresponding forecast operator is denoted by Pt. Furthermore, we will assume that the forecaster observes Yt and its past Yt�1; Yt�2; : : : The goal of the analysis is the investigation of the properties of the bivariate process f.Xt; Yt/0g. The analysis of this important class models presented below is based on Campbell and Shiller (1987).1

The model is closed by assuming some specific time series model for fYtg. In this example, we will assume that fYtg is an integrated process of order one (see Definition 7.1 in Sect. 7.1) such that f�Ytg follows an AR(1) process:

�Yt D �.1 � �/ C ��Yt�1 C vt; j�j < 1 and vt � WN.0;�2v/:

This specification of the fYtg process implies that Pt�YtCh D �.1 � �h/ C �h�Yt. Because PtYtCh D Pt�YtCh C : : : C Pt�YtC1 C Yt, h D 0;1;2; : : :, the present discounted value model can be manipulated to give:

1A more recent interesting application of this model is given by the work of Beaudry and Portier (2006).

16.1 A Theoretical Example 297

Xt D �.1 � ˇ/

Yt C ˇPt YtC1 C ˇ2PtYtC2 C : : : C ut

D �.1 � ˇ/Œ Yt C ˇ Yt C ˇ Pt�YtC1 C ˇ2Yt C ˇ2Pt�YtC1 C ˇ2Pt�YtC2 C ˇ3Yt C ˇ3Pt�YtC1 C ˇ3Pt�YtC2 C ˇ3Pt�YtC3 C : : : C ut

D �.1 � ˇ/ �

1 � ˇ Yt C ˇ

1 � ˇPt�YtC1 C ˇ2

1 � ˇPt�YtC2 C : : : �

C ut

This expression shows that the integratedness of fYtg is transferred to fXtg. Bringing Yt to the left we get the following expression:

St D Xt � �Yt D � 1 X

jD1 ˇjPt�YtCj C ut:

The variable St is occasionally referred to as the spread. If � is greater than zero, expected increases in �YtCj, j � 1, have a positive impact on the spread today. For � D 1, St can denote the log of the price-dividend ratio of a share, or minus the logged savings ratio as in the permanent income model of Campbell (1987). If investors expect positive (negative) change in the dividends tomorrow, they want to buy (sell) the share thereby increasing (decreasing) its price already today. In the context of the permanent income hypothesis expected positive income changes lead to a reduction in today’s saving rate. If, on the contrary, households expect negative income changes to occur in the future, they will save already today (“saving for the rainy days”).

Inserting for Pt�YtCj, j D 0;1; : : :, the corresponding forecast equation Pt�YtCh D �.1 � �h/ C �h�Yt , we get:

St D ˇ��.1 � �/

.1 � ˇ/.1 � ˇ�/ C ˇ��

1 � ˇ��Yt C ut:

The remarkable feature about this relation is that fStg is a stationary process because both f�Ytg and futg are stationary, despite the fact that fYtg and fXtg are both integrated processes of order one. The mean of St is:

ESt D ˇ��

1 � ˇ:

From the relation between St and �Yt and the AR(1) representation of f�Ytg we can deduce a VAR representation of the joint process f.St;�Yt/0g:

298 16 Cointegration

�

St �Yt

�

D �.1 � �/

ˇ�

.1�ˇ/.1�ˇ�/ C ˇ��

1�ˇ� 1

0 ˇ��2

1�ˇ� 0 �

�

St�1 �Yt�1

�

ut C ˇ��1�ˇ�vt vt

Further algebraic transformation lead to a VAR representation of order two for the level variables f.Xt; Yt/0g:

�

Xt Yt

�

D c C ˆ1 �

Xt�1 Yt�1

�

C ˆ2 �

Xt�2 Yt�2

�

C Zt

D �.1 � �/

�

.1�ˇ/.1�ˇ�/ 1

0 � C �� 1�ˇ�

0 1 C �

�

Xt�1 Yt�1

�

0 �� 1�ˇ�

0 ��

�

Xt�2 Yt�2

�

1 �

1�ˇ� 0 1

�

ut vt

�

Next we want to check whether this stochastic difference equation possesses a stationary solution. For this purpose, we must locate the roots of the equation det ˆ.z/ D det.I2 � ˆ1z � ˆ2z2/ D 0 (see Theorem 12.1). As

det ˆ.z/ D det

1 �

�� 1�ˇ�

�

z C �� 1�ˇ� z

0 1 � .1 C �/z C �z2

D 1 � .1 C �/z C �z2;

the roots are z1 D 1=� and z2 D 1. Thus, only the root z1 lies outside the unit circle whereas the root z2 lies on the unit circle. The existence of a unit root precludes the existence of a stationary solution. Note that we have just one unit root, although each of the two processes taken by themselves are integrated of order one.

The above VAR representation can be further transformed to yield a representa- tion of process in first differences f.�Xt;�Yt/0g:

�

�Xt �Yt

�

D �.1 � �/

�

.1�ˇ/.1�ˇ�/ 1

� �

1 �� 0 0

��

Xt�1 Yt�1

�

0 ��

1�ˇ� 0 �

�

�Xt�1 �Yt�1

�

1 �

1�ˇ� 0 1

�

ut vt

�

This representation can be considered as a generalization of the Dickey-Fuller regression in first difference form (see Eq. (7.1)). In the multivariate case, it is known as the vector error correction model (VECM) or vector error correction representation. In this representation the matrix

16.1 A Theoretical Example 299

… D �ˆ.1/ D ��1 � 0 0

�

is of special importance. This matrix is singular and of rank one. This is not an implication which is special to this specification of the present discounted value model, but arises generally as shown in Campbell (1987) and Campbell and Shiller (1987). In the VECM representation all variables except .Xt�1; Yt�1/0 are stationary by construction. This implies that �….Xt�1; Yt�1/0 must be stationary too, despite the fact that f.Xt; Yt/0g is not stationary as shown above. Multiplying �….Xt�1; Yt�1/0 out, one obtains two linear combinations which define stationary processes. However, as … has only rank one, there is just one linearly independent combination. The first one is Xt�1 � �Yt�1 and equals St�1 which was already shown to be stationary. The second one is degenerate because it yields zero. The phenomenon is called cointegration.

Because … has rank one, it can be written as the product of two vectors ˛ and ˇ:

… D ˛ˇ0 D ��1 0

��

�� 0 :

Clearly, this decomposition of … is not unique because Q̨ D a˛ and Q̌ D a�1ˇ, a ¤ 0, would also qualify for such a decomposition as … D Q̨ Q̌0. The vector ˇ is called a cointegration vector. It has the property that fˇ0.Xt; Yt/0g defines a stationary process despite the fact that f.Xt; Yt/0g is non-stationary. The cointegration vector thus defines a linear combination of Xt and Yt which is stationary. The matrix ˛, here only a vector, is called the loading matrix and its elements the loading coefficients.

The VAR and the VECM representations are both well suited for estimation. However, if we want to compute the impulse responses, we need a causal represen- tation. Such a causal representation does not exist due to the unit root in the VAR process for f.Xt; Yt/0g (see Theorem 12.1). To circumvent this problem we split the matrix ˆ.z/ into the product of two matrices M.z/ and V.z/. M.z/ is a diagonal matrix which encompasses all unit roots on its diagonal. V.z/ has all its roots outside the unit circle so that V �1.z/ exists for jzj < 1. In our example, we get:

ˆ.z/ D M.z/V.z/

D �

1 0

0 1 � z �

1 �

�� 1�ˇ�

�

z C �� 1�ˇ� z

0 1 � �z

Multiplying ˆ.z/ with eM.z/ D �

1 � z 0 0 1

�

from the left, we find:

eM.z/ˆ.z/ D eM.z/M.z/V.z/ D .1 � z/I2V.z/ D .1 � z/V.z/:

300 16 Cointegration

The application of this result to the VAR representation of f.Xt; Yt/g leads to a causal representation of f.�Xt;�Yt/g:

ˆ.L/

�

Xt Yt

�

D M.L/V.L/ �

Xt Yt

�

D c C Zt

eM.L/ˆ.L/

�

Xt Yt

�

D .1 � L/V.L/ �

Xt Yt

�

D �.1 � �/ �

1 � L 0 0 1

�

.1�ˇ/.1�ˇ�/ 1

C �

1 � L 0 0 1

�

V.L/

�

�Xt �Yt

�

D �

�.1 � �/ �

C �

1 � L 0 0 1

�

�Xt �Yt

�

D � �

�

C V �1.L/ �

1 � L 0 0 1

�

�Xt �Yt

�

D � �

�

C ‰.L/Zt:

The polynomial matrix ‰.L/ can be recovered by the method of undetermined coefficients from the relation between V.L/ and ‰.L/:

V.L/‰.L/ D �

1 � L 0 0 1

�

In this exposition, we abstain from the explicit computation of V �1.L/ and ‰.L/. However, the following relation holds:

V.1/ D �

1 �� 0 1 � �

�

H) V �1.1/ D

1 �

1�� 0 1 1��

Implying that

‰.1/ D V �1.1/ �

0 0

0 1

�

D .1 � �/�1 �

0 �

0 1

�

The cointegration vector ˇ D .1; ��/0 and loading matrix ˛ D .�1;0/0 therefore have the following properties:

ˇ 0 ‰.1/ D �0 0

�

and ‰.1/˛ D �

�

16.1 A Theoretical Example 301

Like in the univariate case (see Theorem 7.1 in Sect. 7.1.4), we can also construct the Beveridge-Nelson decomposition in the multivariate case. For this purpose, we decompose ‰.L/ as follows:

‰.L/ D ‰.1/ C .L � 1/e‰.L/

with e‰j D P1

iDjC1 ‰i. This result can be used to derive the multivariate Beveridge- Nelson decomposition (see Theorem 16.1 in Sect. 16.2.3):

�

Xt Yt

�

D �

X0 Y0

�

C � �

�

t C ‰.1/ t X

jD1 Zj C stationary process

D �

X0 Y0

�

C � �

�

t C 1 1 � �

�

0 �

0 1

�

1 �

1�ˇ� 0 1

t X

jD1

�

uj vj

�

C stationary process

D �

X0 Y0

�

C � �

�

t C 1 1 � �

�

0 �

0 1

� t X

jD1

�

uj vj

�

C stationary process: (16.1)

The Beveridge-Nelson decomposition represents the bivariate integrated process f.Xt; Yt/0g as a sum of three components: a linear trend, a multivariate random walk and a stationary process. Multiplying the Beveridge-Nelson decomposition from the left by the cointegration vector ˇ D .1; ��/0, we see that both the trend and the random walk component are eliminated and that only the stationary component remains.

Because the first column of ‰.1/ consists of zeros, only the second structural shock, namely fvtg, will have a long-run (permanent) effect. The long-run effect is �=.1 � �/ for the first variable, Xt, and 1=.1 � �/ for the second variable, Yt. The first structural shock (preference shock) futg has non long-run effect, its impact is of a transitory nature only. This decomposition into permanent and transitory shocks is not typical for this model, but can be done in general as part of the so-called common trend representation (see Sect. 16.2.4).

Finally, we will simulate the reaction of the system to a unit valued shock in vt . Although this shock only has a temporary influence on �Yt, it will have a permanent effect on the level Yt. Taking � D 0:8, we get long-run effect (persistence) of 1=.1 � �/ D 5 as explained in Sect. 7.1.3. The present discounted value model then implies that this shock will also have a permanent effect on Xt too. Setting � D 1, this long-run effect is given by �.1�ˇ/P1jD0 ˇj.1��/�1 D �=.1��/ D 5. Because this long-run effect is anticipated in period t, the period of the occurrence of the shock, Xt will increase by more than one. The spread turns, therefore, into positive. The error correction mechanism will then dampen the effect on future changes of Xt

302 16 Cointegration

0 5 10 15 20 25 30 1

1.5

2.5

3.5

4.5

5.5

Variable Y Variable X

Fig. 16.1 Impulse response functions of the present discounted value model after a unit shock to Yt (� D 1;ˇ D 0:9;� D 0:8)

so that the spread will return steadily to zero. The corresponding impulse responses of both variables are displayed in Fig. 16.1.

Figure 16.2 displays the trajectories of both variables after a stochastic simulation where both shocks futg and fvtg are drawn from a standard normal distribution. One can clearly discern the non-stationary character of both series. However, as it is typically for cointegrated series, they move more or less in parallel to each other. This parallel movement is ensured by the error correction mechanism. The difference between both series which is equal to the spread under this parameter constellation is mean reverting around zero.

16.2 Definition and Representation of Cointegrated Processes

16.2.1 Definition

We now want to make the concepts introduced earlier more precise and give a general definition of cointegrated processes and derive the different representations we have seen in the previous section. Given an arbitrary regular (purely non- deterministic) stationary process fUtgt2Z of dimension n, n � 1, with mean zero and some distribution for the starting random variable X0, we can define recursively a process fXtg, t D 0;1;2; : : : as follows:

Xt D � C Xt�1 C Ut; t D 1;2; : : :

16.2 Definition and Representation 303

0 10 20 30 40 50 60 70 80 90 100 240

250

260

270

280

290

300

310

320

330

time

V al

u es

f o r

X t an

d Y

Variable X Variable Y

Fig. 16.2 Stochastic simulation of the present discounted value model under standard normally distributed shocks (� D 1;ˇ D 0:9;� D 0:8)

Thereby, � denotes an arbitrary constant vector of dimension n. If Ut � WN.0;†/, then fXtg is a multivariate random walk with drift �. In general, however, fUtg is autocorrelated and possesses a Wold representation Ut D ‰.L/Zt (see Sect. 14.1.1) such that

�Xt D � C Ut D � C ‰.L/Zt D � C Zt C ‰1Zt�1 C ‰2Zt�2 C : : : ; (16.2)

where Zt � WN.0;†/ and P1

jD0 k‰jk2 < 1 with ‰0 D In. We now introduce the following definitions.

Definition 16.1. A regular stationary process fUtg with mean zero is integrated of order zero, I.0/, if and only if it can be represented as

Ut D ‰.L/Zt D Zt C ‰1Zt�1 C ‰2Zt�2 C : : :

such that Zt � WN.0;†/, P1

jD0 jk‰jk < 1, and ‰.1/ D P1

jD0 ‰j ¤ 0. Definition 16.2. A stochastic process fXtg is integrated of order d, I.d/, d D 0; 1;2; : : :, if and only if �d.Xt � E.Xt// is integrated of order zero.

In the following we concentrate on I(1) processes. The definition of an I(1) process implies that fXtg equals Xt D X0 C �t C

Pt jD1 Uj and is thus non-

stationary even if � D 0. The condition ‰.1/ ¤ 0 corresponds to the one in the univariate case (compare Definition 7.1 in Sect. 7.1). On the one hand, it precludes the case that a trend-stationary process is classified as an integrated process. On the

304 16 Cointegration

other hand, it implies that fXtg is in fact non-stationary. Indeed, if the condition is violated so that ‰.1/ D 0, we could express ‰.L/ as .1 � L/e‰.L/. Thus we could cancel 1 � L on both sides of Eq. (16.2) to obtain a stationary representation of fXtg, given some initial distribution for X0. This would then contradict our primal assumption that fXtg is non-stationary. The condition

P1 jD0 jk‰jk < 1 is stronger

than P1

jD0 k‰jk2 < 1 which follows from the Wold’s Theorem. It guarantees the existence of the Beveridge-Nelson decomposition (see Theorem 16.1 below).2 In particular, the condition is fulfilled if fUtg is a causal ARMA process which is the prototypical case.

Like in the univariate case, we can decompose an I(1) process additively into several components.

Theorem 16.1 (Beveridge-Nelson Decomposition). If fXtg is an integrated process of order one, it can be decomposed as

Xt D X0 C �t C ‰.1/ t X

jD1 Zj C Vt;

where Vt D e‰.L/Z0 � e‰.L/Zt with e‰j D P1

iDjC1 ‰i, j D 0;1;2; : : : and fVtg stationary.

Proof. Following the proof of the univariate case (see Sect. 7.1.4):

‰.L/ D ‰.1/ C .L � 1/e‰.L/ with e‰j D

P1 iDjC1 ‰i. Thus,

Xt D X0 C �t C t X

jD1 Uj D X0 C �t C

t X

jD1 ‰.L/Zj

D X0 C �t C t X

jD1

�

‰.1/ C .L � 1/e‰.L/� Zj

D X0 C �t C ‰.1/ t X

jD1 Zj C

t X

jD1 .L � 1/e‰.L/Zj

D X0 C �t C ‰.1/ t X

jD1 Zj C e‰.L/Z0 � e‰.L/Zt:

The only point left is to show that e‰.L/Z0 � e‰.L/Zt is stationary. Based on Theorem 10.2, it is sufficient to show that the coefficient matrices are

2This condition could be relaxed and replaced by the condition P

jD0 j 2k‰j k2 < 1. In addition,

this condition is an important assumption for the application of the law of large numbers and for the derivation of the asymptotic distribution (Phillips and Solo 1992).

16.2 Definition and Representation 305

absolutely summable. This can be derived by applying the triangular inequality and the condition for integrated processes:

1 X

jD0 ke‰jk D

1 X

jD0

�

1 X

iDjC1 ‰i

�

� 1 X

jD0

1 X

iDjC1 k‰ik D

1 X

jD1 jk‰jk < 1:

The process fXtg can therefore be viewed as the sum of a linear trend, X0 C �t, with stochastic intercept, a multivariate random walk, ‰.1/

Pt jD0 Zt ,

and a stationary process fVtg. Based on this representation, we can then define the notion of cointegration (Engle and Granger 1987).

Definition 16.3 (Cointegration). A multivariate stochastic process fXtg is called cointegrated if fXtg is integrated of order one and if there exists a vector ˇ 2 Rn, ˇ ¤ 0, such that fˇ0Xt g, is integrated of order zero, given a corresponding distribution for the initial random variable X0. ˇ is called the cointegrating or cointegration vector. The cointegrating rank is the maximal number, r, of linearly independent cointegrating vectors ˇ1; : : : ;ˇr . These vectors span a linear space called the cointegration space.

The Beveridge-Nelson decomposition implies that ˇ is a cointegrating vector if and only if ˇ0‰.1/ D 0. In this case the random walk component PtjD1 Zj is annihilated and only the deterministic and the stationary component remain.3 For some issues it is of interest whether the cointegration vector ˇ also eliminates the trend component. This would be the case if ˇ0� D 0. See Sect. 16.3 for details.

The cointegration vectors are determined only up to some basis transformations. If ˇ1; : : : ;ˇr is a basis for the cointegration space then .ˇ1; : : : ;ˇr/R is also a basis for the cointegration space for any nonsingular r � r matrix R because ..ˇ1; : : : ;ˇr/R/

0‰.1/ D 0.

16.2.2 Vector Autoregressive (VAR) and Vector Error Correction Models (VECM)

Although the Beveridge-Nelson decomposition is very useful from a theoretical point of view, in practice it is often more convenient to work with alternative representations. Most empirical investigations of integrated processes start from a VAR(p) model which has the big advantage that it can be easily estimated:

Xt D c C ˆ1Xt�1 C : : : C ˆpXt�p C Zt; Zt � WN.0;†/ (16.3)

where ˆ.L/ D In � ˆ1L � : : : � ˆpLp and c is an arbitrary constant. Subtracting Xt�1 on both sides of the difference equation, the VAR model can be rewritten as:

3The distribution of X0 is thereby chosen such that ˇ 0X0 D ˇ0e‰.L/Z0.

306 16 Cointegration

�Xt D c C …Xt�1 C �1�Xt�1 C : : : C �p�1�Xt�pC1 C Zt (16.4)

where … D �ˆ.1/ D �In C ˆ1 C : : :ˆp and �i D � Pp

jDiC1 ˆj. We will make the following assumptions:

(i) All roots of the polynomial det ˆ.z/ are outside the unit circle or equal to one, i.e.

det ˆ.z/ D 0 H) � jzj > 1 or

z D 1;

(ii) The matrix … is singular with rank r, 1 � r < n. (iii) Rank.…/ D Rank.…2/.

Assumption (i) makes sure that fXtg is an integrated process with order of integration d � 1. Moreover, it precludes other roots on the unit circles than one. The case of seasonal unit roots is treated in Hylleberg et al. (1990) and Johansen and Schaumburg (1998).4 Assumption (ii) implies that there exists at least n � r unit roots and two n � r matrices ˛ and ˇ with full column rank r such that

… D ˛ˇ0:

The columns of ˇ thereby represent the cointegration vectors whereas ˛ denotes the so-called loading matrix. The decomposition of … in the product of ˛ and ˇ0 is not unique. For every non-singular r � r matrix R we can generate an alternative decomposition … D ˛ˇ0 D .˛R0�1/.ˇR/0. Finally, assumption (iii) implies that the order of integration is exactly one and not greater. The number of unit roots is therefore exactly n � r.5 This has the implication that ˆ.z/ can be written as

ˆ.z/ D U.z/M.z/V.z/

where the roots of the matrix polynomials U.z/ and V.z/ are all outside the unit circle and where M.z/ equals

M.z/ D �

.1 � z/In�r 0 0 Ir

�

This representation of ˆ.z/ is a special form of the Smith-McMillan factorization of polynomial matrices (see Kailath (1980) and Yoo (1987)). This factorization isolates the unit roots in one simple matrix so that the system can be analyzed more easily.

4The seasonal unit roots are the roots of zs � 1 D 0 where s denotes the number of seasons. These roots can be expressed as cos.2k�=s/ C � sin.2k�=s/, k D 0;1; : : : ; s � 1. 5For details see Johansen (1995), Neusser (2000) and Bauer and Wagner (2003).

16.2 Definition and Representation 307

These assumptions will allow us to derive from the VAR(p) model several repre- sentations where each of them brings with it a particular interpretation. Replacing … by ˛ˇ0 in Eq. (16.4), we obtain the vector error correction representation or vector error correction model (VECM):

�Xt D c C ˛ˇ0Xt�1 C �1�Xt�1 C : : : C �p�1�Xt�pC1 C Zt: (16.5)

Multiplying both sides of the equation by .˛0˛/�1˛0 and solving for ˇ0Xt�1, we get:

ˇ 0Xt�1 D .˛0˛/�1˛0

@�Xt � c � p�1 X

jD1 �j�Xt�j � Zt

A :

˛ has full column rank r so that ˛0˛ is a non-singular r � r matrix. As the right hand side of the equation represents a stationary process, also the left hand side must be stationary. This means that the r-dimensional process fˇ0Xt�1g is stationary despite the fact that fXtg is integrated and has potentially a unit root with multiplicity n.

The term error correction was coined by Davidson et al. (1978). They interpret the mean of ˇ0Xt, �� D Eˇ0 Xt, as the long-run equilibrium or steady state around which the system fluctuates. The deviation from equilibrium (error) is therefore given by ˇ0Xt�1��. The coefficients of the loading matrix ˛ should then guarantee that deviations from the equilibrium are corrected over time by appropriate changes (corrections) in Xt.

An Illustration To illustrate the concept of the error correction model, we consider the following simple system with ˛ D .˛1;˛2/0, ˛1 ¤ ˛2, and ˇ D .1; �1/0. For simplicity, we assume that the long-run equilibrium �� is zero. Ignoring higher order lags, we consider the system:

�X1t D ˛1.X1;t�1 � X2;t�1/ C Z1t �X2t D ˛2.X1;t�1 � X2;t�1/ C Z2t:

The autoregressive polynomial of this system is:

ˆ.z/ D �

1 � .1 C ˛1/z ˛1z �˛2z 1 � .1 � ˛2/z

�

The determinant of this polynomial is det ˆ.z/ D 1 � .2 C ˛1 � ˛2/z C .1 C ˛1 � ˛2/z2 with roots equal to z D 1 and z D 1=.1 C ˛1 � ˛2/. This shows that assumption (i) is fulfilled. As … D

�

˛1 �˛1 ˛2 �˛2

�

, the rank of … equals one which

implies assumption (ii). Finally,

308 16 Cointegration

…2 D �

˛21 � ˛1˛2 �˛21 C ˛1˛2 �˛2

2 C ˛1˛2 ˛22 � ˛1˛2

�

Thus, the rank of …2 is also one because ˛1 ¤ ˛2. Hence, assumption (iii) is also fulfilled.

We can gain an additional insight into the system by subtracting the second equation from the first one to obtain:

X1t � X2t D .1 C ˛1 � ˛2/.X1;t�1 � X2;t�1/ C Z1t � Z2t: The process ˇ0Xt D X1t � X2t is stationary and causal with respect to Z1t � Z2t if and only if j1 C ˛1 � ˛2j < 1, or equivalently if and only if �2 < ˛1 � ˛2 < 0. Note the importance of the assumption that ˛1 ¤ ˛2. It prevents that X1t � X2t becomes a random walk and thus a non-stationary (integrated) process. A sufficient condition is that �1 < ˛1 < 0 and 0 < ˛2 < 1 which imply that a positive (negative) error, i.e. X1;t�1 � X2;t�1 > 0.< 0/, is corrected by a negative (positive) change in X1t and a positive (negative) change in X2t. Although the shocks Z1t and Z2t push X1t � X2t time and again away from its long-run equilibrium, the error correction mechanism ensures that the variables are adjusted in such a way that the system moves back to its long-run equilibrium.

16.2.3 The Beveridge-Nelson Decomposition

We next want to derive from the VAR representation a causal representation or MA.1/ representation for f�Xtg. In contrast to a normal causal VAR model, the presence of unit roots precludes the simple application of the method of undetermined coefficients, but requires an additional effort. Multiplying the VAR representation in Eq. (16.3), ˆ.L/Xt D U.L/M.L/V.L/Xt D c C Zt , from the left by U�1.L/ we obtain:

M.L/V.L/Xt D U�1.1/c C U�1.L/Zt:

Multiplying this equation by eM.L/ D �

In�r 0 0 .1 � L/Ir

�

leads to:

V.L/�Xt D eM.1/U�1.1/c C eM.L/U�1.L/Zt which finally leads to

�Xt D V �1.1/eM.1/U�1.1/c C V �1.L/eM.L/U�1.L/Zt D � C ‰.L/Zt:

This is the MA.1/ representation of f�Xtg and corresponds to Eq. (16.2).

16.2 Definition and Representation 309

Because … D �ˆ.1/ D �U.1/M.1/V.1/, the following relation holds for the partitioned matrices:

ˆ.1/ D �

U11.1/ U12.1/ U21.1/ U22.1/

��

0 0

0 Ir

��

V11.1/ V12.1/ V21.1/ V22.1/

�

D �

U12.1/ U22.1/

�

V21.1/ V22.1/ �

This implies that we can define ˛ and ˇ as

˛ D � �

U12.1/ U22.1/

�

and ˇ D �

V21.1/ 0

V22.1/ 0 �

U.1/ and V.1/ are non-singular so that ˛ and ˇ have full column rank r. Based on this derivation we can formulate the following lemma.

Lemma 16.1. The columns of the so-defined matrix ˇ are the cointegration vectors for the process fXtg. The corresponding matrix of loading coefficients is ˛ which fulfills ‰.1/˛ D 0.

Proof. We must show that ˇ0‰.1/ D 0 which is the defining property of cointe- gration vectors. Denoting by .V.ij/.1//i;jD1;2 the appropriately partitioned matrix of V.1/�1, we obtain:

ˇ0‰.1/ D ˇ0 �

V.11/.1/ V.12/.1/ V.21/.1/ V.22/.1/

��

In�r 0 0 0

�

U�1.1/

D �V21.1/ V22.1/ �

�

V.11/.1/ 0 V.21/.1/ 0

�

U�1.1/

D �

V21.1/V .11/.1/ C V22.1/V.21/.1/

::: 0

�

U�1.1/ D 0n where the last equality is a consequence of the property of the inverse matrix.

With the same arguments, we can show that ‰.1/˛ D 0. ut

The equivalence between the VEC and the MA representation is known as Granger’s representation theorem in the literature. Granger’s representation theorem immediately implies the Beveridge-Nelson decomposition:

Xt D X0 C ‰.1/c t C ‰.1/ t X

jD1 Zj C Vt (16.6)

D X0 C V �1.1/eM.1/U�1.1/c t C V �1.1/eM.1/U�1.1/ t X

jD1 Zj C Vt (16.7)

where the stochastic process fVtg is stationary and defined as Vt D e‰.L/Z0 � e‰.L/Zt with e‰j D

P1 iDjC1 ‰i and ‰.L/ D V �1.L/eM.L/U�1.L/. As ˇ0‰.1/ D

ˇ0V �1.1/eM.1/U�1.1/ D 0, ˇ eliminates the stochastic trend (random walk), Pt

jD1 Zt, as well as the deterministic linear trend �t D V �1.1/eM.1/U�1.1/c t.

310 16 Cointegration

An interesting special case is obtained when the constant c is a linear combination of the columns of ˛, i.e. if there exists a vector g such that c D ˛g. Under this circumstance, ‰.1/c D ‰.1/˛g D 0 and the linear trend vanishes and we have E�Xt D 0. In this case, the data will exhibit no trend although the VAR model contains a constant. A similar consideration can be made if the VAR model is specified to contain a constant and a linear time trend d t. The Beveridge-Nelson decomposition would then imply that the data should follow a quadratic trend. However, in the special case that d is a linear combination of the columns of ˛, the quadratic trend disappears and only the linear remains because of the constant.

16.2.4 Common Trend and Triangular Representation

The ‰.1/ in the Beveridge-Nelson decomposition is singular. This implies that the multivariate random walk ‰.1/

P1 jD1 Zj does not consist of n independent univariate

random walks. Instead only n � r independent random walks make up the stochastic trend so that fXtg is driven by n � r stochastic trends. In order to emphasize this fact, we derive from the Beveridge-Nelson decomposition the so-called common trend representation (Stock and Watson 1988a).

As ‰.1/ has rank n � r, there exists a n � r matrix � such that ‰.1/� D 0. Denote by �? the n �.n � r/ matrix whose columns are orthogonal to �, i.e. �0�? D 0. The Beveridge-Nelson decomposition can then be rewritten as:

Xt D X0 C ‰.1/ �

�? ::: �

��

�? ::: �

��1 c t

C ‰.1/ �

�? ::: �

��

�? ::: �

��1 tX

jD1 Zj C Vt

D X0 C �

‰.1/�? ::: 0

��

�? ::: �

��1 c t

C �

‰.1/�? ::: 0

��

�? ::: �

��1 tX

jD1 Zj C Vt

D X0 C �

‰.1/�? ::: 0

�

Qc t C �

‰.1/�? ::: 0

� t X

jD1 eZj C Vt

where Qc D �

�? ::: �

��1 c and eZj D

�

�? ::: �

��1 Zj. Therefore, only the first

n � r elements of the vector Qc are relevant for the deterministic linear trend. The remaining elements are multiplied by zero and are thus irrelevant. Similarly, for the multivariate random walk only the first n � r elements of the process feZt g are responsible for the stochastic trend. The remaining elements of eZt are multiplied

16.3 Johansen’s Cointegration Test 311

by zero and thus have no permanent, but only a transitory influence. The above representation decomposes the shocks orthogonally into permanent and transitory ones (Gonzalo and Ng 2001). The previous lemma shows that one can choose for � the matrix of loading coefficients ˛.

Summarizing the first n � r elements of Qc and eZt to Qc1 and eZ1t , respectively, we arrive at the common trend representation:

Xt D X0 C BQc1 t C B t X

jD1 eZ1j C Vt

where the n � .n � r/ matrix B is equal to ‰.1/�?. Applying these results to our introductory example, we arrive at

B D 1 1 � �

�

; Qc D �.1 � �/; and eZ1t D ut:

This again demonstrates that the trend, the linear as well as the stochastic trend, are exclusively stemming from the nonstationary variables fYtg (compare with Eq. (16.1)).

Finally, we want to present a triangular representation which is well suited to deal with the nonparametric estimation approach advocated by Phillips (1991) and Phillips and Hansen (1990) (see Sect. 16.4). In this representation we normalize the cointegration vector such ˇ D .Ir; �b0/0. In addition, we partition the vector Xt into X1t and X2t such that X1t contains the first r and X2t the last n � r elements. Xt D .X01t; X2t/0 can then be expressed as:

X1t D b0X2t C �1Dt C u1t (16.8a) �X2t D �2�Dt C u2t (16.8b)

where Dt summarizes the deterministic components such as constant and linear time trend. fu1tg and fu2tg denote potentially autocorrelated and cross-correlated stationary time series.

16.3 Johansen’s Test for Cointegration

In Sect. 7.5.2 we have already discussed a regression based test for cointegration among two variables. It was based on a unit root of the residuals from a bivari- ate regression of one variable against the other. In this regression, it turned out to be irrelevant which of the two variables was chosen as the regressor and which one as the regressand. This method can, in principle, be extended to more than two variables. However, with more than two variables, the choice of the regressand

312 16 Cointegration

becomes more crucial as not all variables may be part of the cointegrating relation. Moreover, more than one independent cointegrating relation may exist. For these reasons, it is advantageous to use a method which treats all variables symmetrically. The cointegration test developed by Johansen fulfills this criterion because it is based on a VAR which does not single out a particular variable. This test has received wide recognition and is most often used in practice. The test serves two purposes. First, we want to determine the number r of cointegrating relationships. Second, we want to test properties of the cointegration vector ˇ and the loading matrix ˛.

The exposition of the Johansen test follows closely the work of Johansen where the derivations and additional details can be found (Johansen 1988, 1991, 1995). We start with a VAR(p) model with constant c in VEC form (see Eq. (16.4)):

�Xt D c C …Xt�1 C �1�Xt�1 C : : : C �p�1�Xt�pC1 C Zt; t D 1;2; : : : ; T (16.9)

where Zt � IIDN.0;†/ and given starting values X0 D x0; : : : ; X�pC1 D x�pC1. The problem can be simplified by regressing �Xt as well as Xt�1 against c;�Xt�1; : : : ;�Xt�pC1 and working with the residuals from these regressions.6 This simplification results in a VAR model of order one. We therefore start our analysis without loss of generality with a VAR(1) model without constant term:

�Xt D …Xt�1 C Zt where Zt � IIDN.0;†/.7

The phenomenon of cointegration manifests itself in the singularity of the matrix …. In particular, we want to determine the rank of … which gives the number of linearly independent cointegrating relationships. Denoting by r, the rank of …, 0 � r � n, , we can formulate a sequence of hypotheses:

H.r/ W rank.…/ � r; r D 0;1; : : : ; n:

Hypothesis H.r/, thus, implies that there exists at most r linearly independent cointegrating vectors. The sequence of hypotheses is nested in the sense that H.r/ implies H.r C 1/:

H.0/ � H.1/ � : : : � H.n/:

The hypothesis H.0/ means that rank.…/ D 0. In this case, … D 0 and there are no cointegration vectors. fXtg is thus driven by n independent random walks and

6If the VAR model (16.9) contains further deterministic components besides the constant, these components have to be accounted for in these regressions. 7This two-stage least-squares procedure is also known as partial regression and is part of the Frisch- Waugh-Lowell Theorem (Davidson and MacKinnon 1993; 19–24).

16.3 Johansen’s Cointegration Test 313

the VAR can be transformed into a VAR model for f�Xtg which in our simplified version just means that �Xt D Zt � IIDN.0;†/. The hypothesis H.n/ places no restriction on … and includes in this way the case that the level of fXtg is already stationary. Of particular interest are the hypotheses between these two extreme ones where non-degenerate cointegrating vectors are possible. In the following, we not only want to test for the number of linearly independent cointegrating vectors, r, but we also want to test hypotheses about the structure of the cointegrating vectors summarized in ˇ.

Johansen’s test is conceived as a likelihood-ratio test. This means that we must determine the likelihood function for a sample X1; X2; : : : ; XT where T denotes the sample size. For this purpose, we assume that fZtg � IIDN.0;†/ so that logged likelihood function of the parameters ˛, ˇ, and † conditional on the starting values is given by :

`.˛;ˇ;†/ D � Tn 2

ln.2�/ C T 2

ln det.†�1/

� 1 2

T X

tD1 .�Xt � ˛ˇ0Xt�1/0†�1.�Xt � ˛ˇ0Xt�1/

where … D ˛ˇ0. For a fixed given ˇ, ˛ can be estimated by a regression of �Xt on ˇ0Xt�1:

Ǫ D Ǫ.ˇ/ D S01ˇ.ˇ0S11ˇ/�1

where the moment matrices S00; S11; S01 and S10 are defined as:

S00 D 1

T X

tD1 .�Xt/.�Xt/

S11 D 1

T X

tD1 Xt�1X0t�1

S01 D 1

T X

tD1 .�Xt/X

0 t�1

S10 D S001:

The covariance matrix of the residuals then becomes:

b† D b†.ˇ/ D S00 � S01ˇ.ˇ0S11ˇ/�1ˇ0S10:

Using these results, we can concentrate the log-likelihood function to obtain:

314 16 Cointegration

`.ˇ/ D `. Ǫ.ˇ/;ˇ;b†.ˇ// D � Tn 2

ln.2�/ � T 2

ln det.b†.ˇ// � Tn 2

D � Tn 2

ln.2�/ � Tn 2

� T 2

ln det �

S00 � S01ˇ.ˇ0S11ˇ/�1ˇ0S10 �

: (16.10)

The expression Tn 2

in the above equation is derived as follows:

T X

tD1 .�Xt � Ǫˇ0Xt�1/0b†�1.�Xt � Ǫˇ0Xt�1/

D 1 2

T X

tD1 .�Xt � Ǫˇ0Xt�1/.�Xt � Ǫˇ0Xt�1/0b†�1

D 1 2

tr �

.TS00 � T Ǫˇ0S10 � TS01ˇ Ǫ 0 C T Ǫˇ0S11ˇ Ǫ 0/b†�1 �

D T 2

@ .S00 � Ǫˇ0S10/ „ ƒ‚ …

Db†

b†�1

A D Tn

2 :

The log-likelihood function is thus maximized if

det.b†.ˇ// D det �S00 � S01ˇ.ˇ0S11ˇ/�1ˇ0S10 �

D det S00 det �

ˇ0.S11 � S10S�100 S01/ˇ �

det.ˇ0S11ˇ/

is minimized over ˇ.8 The minimum is obtained by solving the following general- ized eigenvalue problem (Johansen 1995):

det �

S11 � S10S�100 S01 � D 0:

This eigenvalue problem delivers n eigenvalues

1 � O 1 � O 2 � : : : � O n � 0

8Thereby we make use of the following equality for partitioned matrices:

det

�

A11 A12 A21 A22

�

D det A11 det.A22 � A21A�111 A12/ D det A22 det.A11 � A12A�122 A21/

where A11 and A22 are invertible matrices (see for example Meyer 2000; p. 475).

16.3 Johansen’s Cointegration Test 315

with corresponding n eigenvectors Ǒ1; : : : ; Ǒn. These eigenvectors are normalized such that Ǒ0S11 Ǒ D In. Therefore we have that

argminˇ det.b†.ˇ// D .det S00/ n Y

iD1 O i:

Remark 16.1. In the case of cointegration, … is singular with rankb… D r. To estimate r, it seems natural to investigate the number of nonzero eigenvalues of b… D S01S�111 . However, because eigenvalues may be complex, it is advantageous not to investigate the eigenvalues of b… but those of b…0b… which are all real and positive due to the symmetry of b…0b…. These eigenvalues are called the singular values of b….9 Noting that

0 D det � S11 � S10S�100 S01 � D det S11 det

�

In � S�1=211 S10S�100 S01S�1=211 �

D det S11 det �

In � .S�1=200 S01S�1=211 /0.S�1=200 S01S�1=211 / �

;

the generalized eigenvalue problem above therefore just determines the singular values of S

�1=2 00 S01S

�1=2 11 D S�1=200 b…S1=211 .

Remark 16.2. Based on the observation that, for n D 1, D S01S10 S11S00

equals the squared empirical correlation coefficient between �Xt and Xt�1, we find that the eigenvalues j, j D 1; : : : ; n, are nothing but the squared canonical correlation coefficients (see Johansen 1995; Reinsel 1993). Thereby, the largest eigenvalue, O 1, corresponds to the largest squared correlation coefficient that can be achieved between linear combinations of �X1 and Xt�1. Thus ˇ1 gives the linear combination of the integrated variable Xt�1 which comes closest in the sense of correlation to the stationary variable f�Xtg. The second eigenvalue 2 corresponds to the maximal squared correlation coefficient between linear combinations of �Xt and Xt�1 which are orthogonal to the linear combination corresponding to 1. The remaining squared canonical correlation coefficients are obtained by iterating this procedure n times.

If the dimension of the cointegrating space is r then Ǒ consists of those eigen- vectors which correspond to the r largest eigenvalues O 1; : : : ; O r . The remaining eigenvalues rC1; : : : ; n should be zero. Under the null hypothesis H.r/, the log- likelihood function (16.10) can be finally expressed as:

9An appraisal of the singular values of a matrix can be found in Strang (1988) or Meyer (2000).

316 16 Cointegration

`. Ǒr/ D � Tn

2 ln � � Tn

2 � T 2

ln det S00 � T

r X

iD1 ln.1 � i/:

The expression for the optimized likelihood function can now be used to construct the Johansen likelihood-ratio test. There are two versions of the test depending on the alternative hypothesis:

trace test: H0 W H.r/ against H.n/; max test: H0 W H.r/ against H.r C 1/:

The corresponding likelihood ratio test statistics are therefore:

trace test: 2.`. Ǒn/ � `. Ǒr// D �T n X

jDrC1 ln.1 � O j/ � T

n X

jDrC1 O j;

max test: 2.`. ǑrC1/ � `. Ǒr// D �T ln.1 � O rC1/ � T O rC1:

In practice it is useful to adopt a sequential test strategy based on the trace test. Given some significance level, we test in a first step the null hypothesis H.0/ against H.n/. If, on the one hand, the test does not reject the null hypothesis, we conclude that r D 0 and that there is no cointegrating relation. If, on the other hand, the test rejects the null hypothesis, we conclude that there is at least one cointegrating relation. We then test in a second step the null hypothesis H.1/ against H.n/. If the test does not reject the null hypothesis, we conclude that there exists one cointegrating relation, i.e. that r D 1. If the test rejects the null hypothesis, we examine the next hypothesis H.2/, and so on. In this way we obtain a test sequence. If in this sequence, the null hypothesis H.r/ is not rejected, but H.r C 1/ is, we conclude that exist r linearly independent cointegrating relations as explained in the diagram below.

H.0/ against H.n/ rejection��! H.1/ against H.n/ rejection��! H.2/ against H.n/ : : :

r D 0 r D 1 r D 2 If in this sequence we do not reject H.r/ for some r, it is useful to perform the max test H.r/ against H.r C1/ as a robustness check. The asymptotic distributions of the test statistics are, like in the Dickey-Fuller unit root test, nonstandard and depend on the specification of the deterministic components.

16.3 Johansen’s Cointegration Test 317

16.3.1 Specification of the Deterministic Components

As mentioned previously, the asymptotic distribution of Johansen’s test depends on the specification of the deterministic components. Thus, some care must be devoted to this issue. We illustrate this point by decomposing the model additively into a linear deterministic and a stochastic component in vector error correction form (see Johansen (1995; 80–84) and Lütkepohl (2006; section 6.4)):

Xt D �0 C �1t C Yt (16.11) �Yt D …Yt�1 C Zt D ˛ˇ0Yt�1 C Zt: (16.12)

For the ease of exposition, we have omit the autoregressive corrections. Eliminating Yt using Yt D Xt � �0 � �1t and �Yt D �Xt � �1 leads to

�Xt � �1 D ˛ˇ0 .Xt�1 � �0 � �1.t � 1// C Zt:

This equation can be rewritten as

�Xt D c0 C c1.t � 1/ C ˛ˇ0Xt�1 C Zt (16.13)

with c0 D �1 � ˛ˇ0�0 and c1 D �˛ˇ0�1

D c0 C ˛.ˇ0; �ˇ0�1/X0t�1 C Zt (16.14)

where X0t D .X0t; t/0. Equation (16.13) is just the vector error correction model (16.4) augmented by the linear trend term c1t. If the term c1 would be left unrestricted arguments similar to those in Sect. 16.2.3 would show that Xt exhibits a determinis- tic quadratic trend with coefficient vector ‰.1/c1. This, however, contradicts the specification in Eq. (16.11). However, if we recognize that c1 in Eq. (16.13) is actually restricted to lie in the span of ˛, i.e. that c1 D ˛�1 with �1 D �ˇ0�1, no quadratic trend would emerge in the levels because ‰.1/˛ D 0 by Granger’s representation Theorem 16.1. Alternatively, one may view the time trend as showing up in the error correction term, respectively being part of the cointegrating relation, as in Eq. (16.14).

Similarly, one may consider the case that Xt has a constant mean �0, i.e. that �1 D 0 in Eq. (16.11). This leads to the same error correction specification (16.13), but without the term c1t. Leaving the constant c0 unrestricted, this will generate a linear trend ‰.1/c0t as shown in Sect. 16.2.3. In order to reconcile this with the assumption of a constant mean, we must recognize that c0 D ˛�0 with �0 D �ˇ0�0.

318 16 Cointegration

Table 16.1 Trend specifications in vector error correction models

Case Deterministic term in VECM or VAR Restriction Trend in Xt E�Xt E.ˇ

0Xt/

I None – Zero Zero Zero

II c0 c0 D ˛�0 Constant Zero Constant III c0 None Linear Constant Constant

IV c0 C c1t c1 D ˛�1 Linear Constant Linear V c0 C c1t None Quadratic Linear Linear

Table inspired by Johansen (2007)

Based on these arguments, we can summarize the discussion by distinguishing five different cases displayed in Table 16.1.10 This table also shows the implications for E�Xt and E.ˇ

0Xt/. These can read off from Eqs. (16.13) and (16.14). The corresponding asymptotic distributions of the trace as well as the max test

statistic in these five cases are tabulated in Johansen (1995), MacKinnon et al. (1999), and Osterwald-Lenum (1992).11 The finite sample properties of theses tests can be quite poor. Thus, more recently, bootstrap methods have been proven to provide a successful alternative in practice (Cavaliere et al. 2012).

16.3.2 Testing Hypotheses on Cointegrating Vectors

As mentioned previously, the cointegrating vectors are not unique, only the coin- tegrating space is. This makes the cointegrating vectors often difficult to interpret economically, despite some basis transformation. It is therefore of interest to see whether the space spanned by the cointegrating vectors summarized in the columns of Ǒ can be viewed as a subspace spanned by some hypothetical vectors H D .h1; : : : ; hs/, r � s < n. If this hypothesis is true, the cointegrating vectors should be linear combinations of the columns of H so that the null hypothesis can be formulated as

H0 W ˇ D H' (16.15) for some s � r matrix '. Under this null hypothesis, this amounts to solve an analogous general eigenvalue problem:

det �

'H0S11H � H0S10S�100 S01H � D 0:

The solution of this problem is given by the eigenvalues 1 > Q 1 � Q 2 � : : : Q s > 0 with corresponding normalized eigenvectors. The likelihood ratio test statistic for this hypothesis is then

10It is instructive to compare theses cases to those of the unit root test (see Sect. 7.3.1). 11The tables by MacKinnon et al. (1999) allow for the possibility of exogenous integrated variables.

16.4 Estimation and Testing of Cointegrating Relationships 319

T r X

jD1 ln 1 � Q j 1 � O j

This test statistic is asymptotically distributed as a �2 distribution with r.n � s/ degrees of freedom.

With similar arguments it is possible to construct a test of the null hypothesis that the cointegrating space spanned by the columns of Ǒ contains some hypothetical vectors K D .h1; : : : ; hs/, 1 � s � r. The null hypothesis can then be formulated as

H0 W K' D ˇ (16.16)

for some s � r matrix '. Like in the previous case, this hypothesis can also be tested by the corresponding likelihood ratio test statistic which is asymptotically distributed as a �2 distribution with s.n � r/ degrees of freedom. Similarly, it is possible to test hypotheses on ˛ and joint hypotheses on ˛ and ˇ (see Johansen 1995; Kunst and Neusser 1990; Lütkepohl 2006).

16.4 Estimation and Testing of Cointegrating Relationships

Johansen’s approach has become very popular because it presents an integrated framework for testing and estimating cointegrating relationships based on the maximum likelihood method. However, it requires the specification of a concrete VAR model. This proves sometimes difficult it practice, especially when the true data generating process is not purely autoregressive. Similar to the Phillips-Perron test discussed in Sect. 7.3.2, Phillips and Hansen (1990) propose a nonparametric approach for the estimation and hypothesis testing of cointegrating relationships. This approach is especially appropriate if the long-run relationships are the prime objective of the investigation as f.e. in Neusser and Kugler (1998).

The Phillips and Hansen approach is based on the triangular representation of cointegrated processes given in the equation system (16.8). Thereby the r cointegration vectors are normalized such that ˇ D .Ir; �b0/0 where b is the regression coefficient matrix from a regression of X1t on X2t controlling for deterministic components Dt (see Eq. (16.8a)).

12 The least-squares estimate of b is (super) consistent as already noted in Sect. 7.5.2. However, the estimator is not directly suitable for hypothesis testing because the conventional test statistics do not have the usual asymptotic distributions. The idea of Phillips and Hansen (1990) is to correct the conventional least-squares estimates to account for serial correlation and for the endogeneity arising from the cointegrating relationship. This leads to the fully-modified ordinary least-squares estimator (FMOLS estimator).

12The choice of the variables used for normalization turns out to be important in practice. See the application in Sect. 16.5.

320 16 Cointegration

As the endogeneity shows up in the long-run correlation between the variables, the proposed modification uses of the long-run variance J of ut D .u01t; u02t/0. According to Sect. 11.1 this entity defined as:

J D �

J11 J12 J21 J22

�

D 1 X

hD�1 �.h/ D ƒ C ƒ0 � †

where

ƒ D 1 X

hD0 �.h/ D

�

ƒ11 ƒ12 ƒ21 ƒ22

�

;

† D E.utu0t/ D �

�11 �12

�21 �22

�

The fully-modified ordinary least-squares estimator of .b;�1/ is then constructed as follows. Estimate the Eqs. (16.8a) and (16.8b) by ordinary least-squares to compute the residuals Out D .Ou01t; Ou02t/0. From these residuals estimate † as b† D

PT tD1 utu

0 t

and the long-run variance J and its one-sided counterpart ƒ. Estimates of J and ƒ, denoted by OJ and Oƒ, can be obtained by applying a kernel estimator as explained in Sect. 4.4, respectively Sect. 11.1. The estimators b†, OJ and Oƒ are consistent because ordinary least-squares is. The corresponding estimates are then used to correct the data for X1t and to construct the bias correction term Oƒ.C/21 :

X .C/ 1t D X1t � OJ12 OJ�122 Ou2t;

Oƒ.C/21 D Oƒ21 � OJ12 OJ�122 Oƒ22: The fully-modified ordinary least-squares estimator (FMOLS estimator) is then given by

�

Ob; O�1 �

T X

tD1 X .C/ 1t .X

0 2t; D

0 t/

� T. Oƒ.C/021 ;0/ !

T X

tD1 .X02t; D

0 t/

0.X02t; D 0 t/

!�1 :

It turns out that this estimator is asymptotically equivalent to full maximum likelihood with limiting distributions free of nuisance parameters.

The main advantage of the FMOLS estimator is that conventional Wald test statistics, appropriately modified, have limiting �2 distributions. This brings statisti- cal inference back to the realm of traditional econometric analysis. Consider testing the null hypothesis

16.5 An Example 321

H0 W R vec b D q:

where q is a vector of dimension g and R selects the appropriate elements of vec b. Thus, in effect we are considering hypothesis of the form H0 W b D b0. The hypothesis b D 0 is thereby of particular interest. The Wald test statistic is then defined as

W D

.R vec Ob � q/0 2

@ OJ11:2 ˝

T X

tD1 .X02t; D

0 t/

0.X02t; D 0 t/

!�11

A R0 3

�1

.R vec Ob � q/

where OJ11:2 D OJ11� OJ12 OJ�122 OJ21. It can be shown that the so defined modified Wald test statistic is asymptotically distributed as �2 with g degrees of freedom (see Phillips and Hansen 1990; Hansen 1992).

16.5 An Example

This example reproduces the study by Neusser (1991) with actualized data for the United States over the period first quarter 1950 to fourth quarter 2005. The starting point is a VAR model which consists of four variables: real gross domestic product (Y), real private consumption (C), real gross investment (I), and the ex- post real interest rate (R). All variables, except the real interest rate, are in logs. First, we identify a VAR model for these variables where the order is determined by Akaike’s (AIC), Schwarz’ (BIC) or Hannan-Quinn’ (HQ) information criteria. The AIC suggests seven lags whereas the other criteria propose a VAR of order two. As the VAR(7) consists of many statistically insignificant coefficients, we prefer the more parsimonious VAR(2) model which produces the following estimates:

Xt D

Yt Ct It Rt

0.185 (0.047)

0.069 (0.043)

0.041 (0.117)

-0.329 (0.097)

0.951 (0.086)

0.254 (0.091)

0.088 (0.033)

0.042 (0.032)

0.157 (0.079)

0.746 (0.084)

0.065 (0.031)

-0.013 (0.030)

0.283 (0.216)

0.250 (0.229)

1.304 (0.084)

0.026 (0.081)

0.324 (0.178)

-0.536 (0.189)

-0.024 (0.069)

0.551 (0.067)

Xt�1

322 16 Cointegration

-0.132 (0.085)

-0.085 (0.093)

-0.089 (0.033)

-0.016 (0.031)

-0.213 (0.078)

0.305 (0.085)

-0.066 (0.031)

0.112 (0.029)

-0.517 (0.214)

0.040 (0.233)

-0.364 (0.084)

0.098 (0.079)

-0.042 (0.176)

0.296 (0.192)

0.005 (0.069)

0.163 (0.065)

Xt�2 C Zt

where the estimated standard errors of the corresponding coefficients are reported in parenthesis. The estimate covariance matrix †, b†, is

b† D 10�4 0

0:722 0:428 1:140 0:002

0:428 0:610 1:026 �0:092 1:140 1:026 4:473 �0:328 0:002 �0:092 �0:328 3:098

The sequence of the hypotheses starts with H.0/ which states that there exists no cointegrating relation. The alternative hypothesis is always H.n/ which says that there are n cointegrating relations. According to Table 16.2 the value of the trace test statistic is 111.772 which is clearly larger than the 5 % critical value of 47.856. Thus, the null hypothesis H.0/ is rejected and we consider next the hypothesis H.1/. This hypothesis is again clearly rejected so that we move on to the hypothesis H.2/. Because H.3/ is not rejected, we conclude that there exists 3 cointegrating relations. To check this result, we test the hypothesis H.2/ against H.3/ using the max test. As this test also rejects H.2/, we can be pretty confident that there are three cointegrating relations given as:

Ǒ D

1:000 0:000 0:000

0:000 1:000 0:000

0:000 0:000 1:000

�258:948 �277:869 �337:481

Table 16.2 Evaluation of the results of Johansen’s cointegration test

Trace statistic Max statistic

Null hypothesis Eigenvalue Test statistic

Critical value

Test statistic

Critical value

H.0/ W r D 0 0.190 111.772 47.856 47.194 27.584 H.1/ W r � 1 0.179 64.578 29.797 44.075 21.132 H.2/ W r � 2 0.081 20.503 15.495 18.983 14.265 H.3/ W r � 3 0.007 1.520 3.841 1.520 3.841

Critical 5 % values are taken from MacKinnon et al. (1999)

16.5 An Example 323

This matrix is actually the outcome from the EVIEWS econometrics software package. It should be noted that EVIEWS, like other packages, chooses the normalization mechanically. This can become a problem if the variable on which the cointegration vectors are normalized is not part of the cointegrating relation.

In this form, the cointegrating vectors are economically difficult to interpret. We therefore ask whether they are compatible with the following hypotheses:

ˇC D

1:0

�1:0 0:0

0:0

; ˇI D

1:0

0:0

�1:0 0:0

; ˇR D

0:0

1:0

These hypotheses state that the log-difference (ratio) between consumption and GDP, the log-difference (ratio) between investment and GDP, and the real interest rate are stationary. They can be rationalized in the context of the neoclassical growth model (see King et al. 1991; Neusser 1991). Each of them can be brought into the form of Eq. (16.16) where ˇ is replaced by its estimate Ǒ. The corresponding test statistics for each of the three cointegrating relations is distributed as a �2

distribution with one degree of freedom,13 which gives a critical value of 3.84 at the 5 % significance level. The corresponding values for the test statistic are 12.69, 15.05 and 0.45, respectively. This implies that we must reject the first two hypotheses ˇC and ˇI . However, the conjecture that the real interest is stationary, cannot be rejected. Finally, we can investigate the joint hypothesis ˇ0 D .ˇC;ˇI;ˇR/ which can be represented in the form (16.15). In this case the value of the test statistic is 41.20 which is clearly above the critical value of 7.81 inferred from the �23 distribution.

14 Thus, we must reject this joint hypothesis. As a matter of comparison, we perform a similar investigation using the fully-

modified approach of Phillips and Hansen (1990). For this purpose we restrict the analysis to Yt, Ct , and It because the real interest rate cannot be classified unambiguously as being stationary, respectively integrated of order one. The long- run variance J and its one-sided counterpart ƒ are estimated using the quadratic spectral kernel with VAR(1) prewhitening as advocated by Andrews and Monahan (1992) (see Sect. 4.4). Assuming two cointegrating relations and taking Yt and Ct as the left hand side variables in the cointegrating regression (Eq. (16.8a)), the following results are obtained:

0.234 (0.166)

0.215 (0.171)

It C

6.282 (0.867)

5.899 (0.892)

0.006 (0.002)

0.007 (0.002)

t C Ou1t

13The degrees of freedom are computed according to the formula: s.n � r/ D 1.4 � 3/ D 1. 14The degrees of freedom are computed according to the formula: r.n � s/ D 3.4 � 3/ D 3.

324 16 Cointegration

where the estimated standard deviations are reported in parenthesis. The specifica- tion allows for a constant and a deterministic trend as well as a drift in the equation for �It (Eq. (16.8b), not shown).

Given these results we can test a number of hypotheses to get a better understand- ing of the cointegrating relations. First we test the hypothesis of no cointegration of Yt, respectively Ct with It. Thus, we test H0 W b.1/ D b.2/ D 0. The value of the corresponding Wald test statistic is equal to 2.386 which is considerably less than the 5 % critical value of 5.992. Therefore we can not reject the null hypothesis no cointegration Another interesting hypothesis is H0 W b.1/ D b.2/ which would mean that Yt and Ct are cointegrated with cointegration vector .1; �1/. As the corresponding Wald statistic is equal to 0.315, this hypothesis can not be rejected at the 5 % critical value of 3.842. This suggests a long-run relation between Yt and Ct .

Repeating the analysis with Ct and It as the left hand side variables leads to the following results:

0.834 (0.075)

2.192 (0.680)

Yt C

0.767 (0.561)

-11.27 (5.102)

0.002 (0.001)

-0.008 (0.006)

t C Ou1t

As before first the hypothesis H0 W b.1/ D b.2/ D 0 is tested. The corresponding value of the test statistic is 137.984 which is clearly above the 5 % critical value. Thus, the null hypothesis of no cointegration is rejected. Next, the hypothesis H0 W b.1/ D b.2/ D 1 is tested. This hypothesis is rejected as the value 7.717 of the test statistic is above the critical value. If these hypotheses are not tested jointly, but individually, the null hypothesis b.1/ D 1 can be rejected, but b.2/ D 1 can not. These findings conform reasonably well with those based on the Johansen approach.

The diverse result between the two specifications demonstrates that the sensi- tivity of cointegration analysis with respect to the normalization. It is important that the variable on which the cointegrating vector is normalized is indeed in the cointegrating space. Otherwise, insensible results may be obtained.

17State-Space Models and the Kalman Filter

The state space representation is a flexible technique originally developed in automatic control engineering to represent, model, and control dynamic systems. Thereby we summarize the unobserved or partially observed state of the system in period t by an m-dimensional vector Xt. The evolution of the state is then described by a VAR of order one usually called the state equation. A second equation describes the connection between the state and the observations given by a n-dimensional vector Yt. Despite its simple structure, state space models encompass a large variety of model classes: VARMA, respectively VARIMA models,1 unobserved-component models, factor models, structural time series models which decompose a given time series into a trend, a seasonal, and a cyclical component, models with measurement errors, VAR models with time-varying parameters, etc.

From a technical point of view, the main advantage of state space modeling is the unified treatment of estimation, forecasting, and smoothing. At the center of the analysis stands the Kalman-filter named after its inventor Rudolf Emil Kálmán (Kalman 1960, 1963). He developed a projection based algorithm which recursively produces a statistically optimal estimate of the state. The versatility and the ease of implementation have made the Kalman filter an increasingly popular tool also in the economically oriented times series literature. Here we present just an introduction to the subject and refer to Anderson and Moore (1979), Brockwell and Davis (1991; Chapter 12), Brockwell and Davis (1996; Chapter 8), Hamilton (1994b; Chapter 13), Hamilton (1994a), Hannan and Deistler (1988), or Harvey (1989), and in particular to Durbin and Koopman (2011) and Kim and Nelson (1999) for extensive reviews and further details.

1VARIMA models stand for vector autoregressive integrated moving-average models.

325

326 17 Kalman Filter

17.1 The State Space Model

We consider a dynamical system whose state at each point in time t is determined by a vector Xt. The evolution of the system over time is then described by a state equation. The state is, however, unobserved or only partly observed to the outside observer. Thus, a second equation, called the observation equation, is needed to describe the connection of the state to the observations. This relation may be subject to measurement errors. The equation system consisting of state and observation equation is called a state space model which is visualized in Fig. 17.1. The state equation typically consists of a VAR model of order one whereas the observation equation has the structure of multivariate linear regression model.2

Despite the simplicity of each of these two components, their combination is very versatile and able to represent a great variety of models.

In the case of time invariant coefficients3 we can set up these two equations as follows:

state equation: XtC1 D FXt C VtC1; t D 1;2; : : : (17.1) observation equation: Yt D A C GXt C Wt; t D 1;2; : : : (17.2)

Thereby Xt denotes an m-dimensional vector which describes the state of the system in period t. The evolution of the state is represented as a vector autoregressive model

internal dynamics: Xt+1 = F Xt + Vt+1 state equation

� present Xt �

future Xt+1

�

error Vt+1

�

� observations: Yt = A + GXt + Wt data possibly contaminated by measurement errors Wt

Fig. 17.1 State space model

2We will focus on linear dynamic models only. With the availability of fast and cheap computing facilities, non-linear approaches have gained some popularity. See Durbin and Koopman (2011) for an exposition. 3For the ease of exposition, we will present first the time-invariant case and analyze the case of time-varying coefficients later.

17.1 The State Space Model 327

of order one with coefficient matrix F and disturbances VtC1.4 As we assume that the state Xt is unobservable or at least partly unobservable, we need a second equation which relates the state to the observations. In particular, we assume that there is a linear time-invariant relation given by A and G of the n-dimensional vector of observations, Yt , to the state Xt. This relation may be contaminated by measurement errors Wt . The system is initialized in period t D 1.

We make the following simplifying assumption of the state space model repre- sented by Eqs. (17.1) and (17.2).

(i) fVtg � WN.0; Q/ where Q is a constant nonnegative definite m � m matrix. (ii) fWtg � WN.0; R/ where R is a constant nonnegative definite n � n matrix.

(iii) The two disturbances are uncorrelated with each other at all leads and lags, i.e.:

E.WsV 0 t/ D 0; for all t and s.

(iv) Vt and Wt are multivariate normally distributed. (v) X1 is uncorrelated with Vt as well as with Wt , t D 1;2; : : :

Remark 17.1. In a more general context, we can make both covariance matrices Q and R time-varying and allow for contemporaneous correlations between Vt and Wt (see example Sect. 17.4.1).

Remark 17.2. As both the state and the observation equation may include identities, the covariance matrices need not be positive definite. They can be non-negative definite.

Remark 17.3. Neither fXtg nor fYtg are assumed to be stationary.

Remark 17.4. The specification of the state equation and the normality assumption imply that the sequence fX1; V1; V2; : : :g is independent so that the conditional distribution XtC1 given Xt; Xt�1; : : : ; X1 equals the conditional distribution of XtC1 given Xt. Thus, the process fXtg satisfies the Markov property. As the dimension of the state vector Xt is arbitrary, it can be expanded in such a way as to encompass every component Xt�1 for any t (see, for example, the state space representation of a VAR(p) model with p > 1). However, there remains the problem of the smallest dimension of the state vector (see Sect. 17.3.2).

Remark 17.5. The state space representation is not unique. Defining, for example, a new state vector QXt by multiplying Xt with an invertible matrix P, i.e. QXt D PXt, all properties of the system remain unchanged. Naturally, we must redefine all the system matrices accordingly: QF D PFP�1, QQ D PQP0, QG D GP�1.

4In control theory the state equation (17.1) is amended by an additional term HUt which represents the effect of control variables Ut . These exogenous controls are used to regulate the system.

328 17 Kalman Filter

Given X1, we can iterate the state equation forward to arrive at:

Xt D Ft�1X1 C t�1 X

jD1 Fj�1VtC1�j; t D 1;2; : : :

Yt D A C GFt�1X1 C t�1 X

jD1 GFj�1VtC1�j C Wt; t D 1;2; : : :

The state equation is called stable or causal if all eigenvalues of F are inside the unit circle which is equivalent that all roots of det.Im � Fz/ D 0 are outside the unit circle (see Sect. 12.3). In this case the state equation has a unique stationary solution:

Xt D 1 X

jD0 Fj�1VtC1�j: (17.3)

The process fYtg is therefore also stationary and we have:

Yt D A C 1 X

jD0 GFj�1VtC1�j C Wt: (17.4)

In the case of a stationary state space model, we may do without an initialization period and take t 2 Z.

In the case of a stable state equation, we can easily deduce the covariance function for fXtg, �X.h/, h D 0;1;2; : : : According to Sect. 12.4 it holds that:

�X.0/ D F�X.0/F0 C Q; �X.h/ D Fh�X.0/; h D 1;2; : : :

where �X.0/ is uniquely determined given the stability assumption. Similarly, we can derive the covariance function for the observation vector, �Y.h/, h D 0;1;2; : : ::

�Y.0/ D G�X.0/G0 C R; �Y.h/ D GFh�X.0/G0; h D 1;2; : : :

17.1.1 Examples

The following examples should illustrate the versatility of the state space model and demonstrate how many economically relevant models can be represented in this form.

17.1 The State Space Model 329

VAR(p) Process

Suppose that fYtg follows a n-dimensional VAR(p) process given by ˆ.L/Yt D Zt , respectively by Yt D ˆ1Yt�1 C : : : C ˆpYt�p C Zt , with Zt � WN.0;†/. Then the companion form of the VAR(p) process (see Sect. 12.2) just represents the state equation (17.1):

XtC1 D

YtC1 Yt

Yt�1 :::

Yt�pC1

ˆ1 ˆ2 :: : ˆp�1 ˆp In 0 :: : 0 0 0 In : : : 0 0 :::

::: :::

0 0 : : : In 0

Yt Yt�1 Yt�2 :::

Yt�p

ZtC1 0

0 :::

D FXt C VtC1;

with VtC1 D .Z0tC1;0;0; : : : ;0/0 and Q D �

† 0

0 0

�

: The observation equation is just

an identity because all components of Xt are observable:

Yt D .In;0;0; : : : ;0/Xt D GXt: Thus, G D .In;0;0; : : : ;0/ and R D 0. Assuming that Xt is already mean adjusted, A D 0.

ARMA(1,1) Process

The representation of ARMA processes as a state space model is more involved when moving-average terms are involved. Let fYtg be an ARMA(1,1) process defined by the stochastic difference equation Yt D �Yt�1 C Zt C �Zt�1 with Zt � WN.0;�2/ and �� ¤ 0.

Define fXtg as the AR(1) process defined by the stochastic difference equation Xt � �Xt�1 D Zt and Xt D .Xt; Xt�1/0 as the state vector, then we can write the observation equation as:

Yt D .1;�/Xt D GXt with R D 0. The state equation is then

XtC1 D �

XtC1 Xt

�

D �

� 0

1 0

��

Xt Xt�1

�

C �

ZtC1 0

�

D FXt C VtC1;

where Q D �

�2 0

0 0

�

. It is easy to verify that the so defined process fYtg satisfies the stochastic difference equation Yt D �Yt�1 C Zt C �Zt�1. Indeed Yt � �Yt�1 D .1;�/Xt ��.1;�/Xt�1 D Xt C�Xt�1 ��Xt�1 ��Xt�2 D .Xt ��Xt�1/C�.Xt�1 � �Xt�2/ D Zt C �Zt�1.

330 17 Kalman Filter

If j�j < 1, the state equation defines a causal process fXtg so that the unique stationary solution is given by Eq. (17.3). This implies a stationary solution for fYtg too. It is thus easy to verify if this solution equals the unique solution of the ARMA stochastic difference equation.

The state space representation of an ARMA model is not unique. An alternative representation in the case of a causal system is given by:

XtC1 D �Xt C .� C �/Zt D FXt C VtC1 Yt D Xt C Zt D Xt C Wt:

Note that in this representation the dimension of the state vector is reduced from two to one. Moreover, the two disturbances VtC1 D .� C �/Zt and Wt D Zt are perfectly correlated.

ARMA(p,q) Process

It is straightforward to extend the above representation to ARMA(p,q) models.5 Let fYtg be defined by the following stochastic difference equation:

ˆ.L/Yt D ‚.L/Zt with Zt � WN.0;�2/ and �p�q ¤ 0: Define r as r D maxfp; q C 1g and set �j D 0 for j > p and �j D 0 for j > q. Then, we can set up the following state space representation with state vector Xt and observation equation

Yt D .1;�1; : : : ;�r�1/Xt where the state vector equals Xt D .Xt; : : : ; Xt�rC2; Xt�rC1/0 and where fXtg follows an AR(p) process ˆ.L/Xt D Zt . The AR(p) process can be transformed into companion form to arrive at the state equation:

XtC1 D

�1 �2 : : : �r�1 �r 1 0 :: : 0 0

0 1 : : : 0 0 ::: ::: :::

::: :::

0 0 : : : 1 0

Xt C

Zt 0

0 :::

Missing Observations

The state space approach is best suited to deal with missing observations. However, in this situation the coefficient matrices are no longer constant, but time-varying. Consider the following simple example of an AR(1) process for which we have

5See also Exercise 17.2.

17.1 The State Space Model 331

only observations for the periods t D 1; : : : ;100 and t D 102;: : : ;200, but not for period t D 101 which is missing. This situation can be represented in state space form as follows:

XtC1 D �Xt C Zt Yt D GtXt C Wt

Gt D �

1; t D 1; : : : ;100;102; : : : ;200; 0; t D 101.

Rt D �

0; t D 1; : : : ;100;102; : : : ;200; c > 0; t D 101.

This means that Wt D 0 and that Yt D Xt for all t except for t D 101. For the missing observation, we have G101 D Y101 D 0. The variance for this observation is set to R101 D c > 0.

The same idea can be used to obtain quarterly data when only yearly data are available. This problem typically arises in statistical offices which have to produce, for example, quarterly GDP data from yearly observations incorporating quarterly information from indicator variables (see Sect. 17.4.1). More detailed analysis for the case of missing data can be found in Harvey and Pierce (1984) and Brockwell and Davis (1991; Chapter 12.3).

Time-Varying Coefficients

Consider the regression model with time-varying parameter vector ˇt :

Yt D x0tˇt C Wt (17.5) where Yt is an observed dependent variable, xt is a K-vector of exogenous regressors, and Wt is a white noise error term. Depending on the specification of the evolution of ˇt several models have been proposed in the literature:

Hildreth-Houck W ˇt D Ň C vt Harvey-Phillips: ˇt � Ň D F.ˇt � Ň/ C vt Cooley-Prescott: ˇt D ˇpt C v1t

ˇ p t D ˇpt�1 C v2t

where vt, v1t, and v2t are white noise error terms. In the first specification, proposed originally proposed by Hildreth and Houck (1968), the parameter vector is in each period just a random from a distribution with mean Ň and variance given by the variance of vt . Departures from the mean are seen as being only of a transitory nature. In the specification by Harvey and Phillips (1982), assuming that all eigenvalues of F are strictly smaller than one in absolute value, the parameter vector is a mean reverting VAR of order one. In this case, the departures from

332 17 Kalman Filter

the mean can have a longer duration depending on the eigenvalues of F. The last specification due to Cooley and Prescott (1973, 1976) views the parameter vector as being subject to transitory and permanent shifts. Whereas shifts in v1t have only a transitory effect on ˇt , movements in v2t result in permanent effects.

In the Cooley-Prescott specification, for example, the state is given by Xt D .ˇ0t;ˇpt 0/0 and the state equation can be written as:

XtC1 D �

ˇtC1 ˇ

p tC1

�

D �

0 1

�

„ƒ‚…

�

ˇt

ˇ p t

�

C �

v1t

v2t

�

The observation equation then becomes:

Yt D yt D �

xt 0

�0 Xt C Wt

Thus, A D 0 and Gt D .x0t;0/. Note that this an example of a state space model with time-varying coefficients. In Sect. 18.2, we will discuss time-varying coefficient models in the context of VAR models.

Structural Time Series Analysis

An important application of the state space representation in economics is the decomposition of a given time series into several components: trend, cycle, season and irregular component. This type of analysis is usually coined structural time series analysis (See Harvey 1989; Mills 2003). Consider, for example, the additive decomposition of a time series fYtg into a trend Tt, a seasonal component St, a cyclical component fCt g, and an irregular or cyclical component Wt :

Yt D Tt C St C Ct C Wt: The above equation relates the observed time series to its unobserved components and is called the basic structural model (BSM) (Harvey 1989).

The state space representation is derived in several steps. Consider first the case with no seasonal and no cyclical component. The trend is typically viewed as a random walk with time-varying drift ıt�1:

Tt D ıt�1 C Tt�1 C "t; "t � WN.0;�2" / ıt D ıt�1 C �t; �t � WN.0;�2� /:

The second equation models the drift as a random walk. The two disturbances f"tg and f�tg are assumed to be uncorrelated with each other and with fWtg. Defining the state vector X

.T/ t as X

.T/ t D .Tt;ıt/0, the state and the observation equations become:

X .T/ tC1 D

�

TtC1 ıtC1

�

D �

1 1

0 1

��

Tt ıt

�

C �

"tC1 �tC1

�

D F.T/X.T/t C V.T/tC1

Yt D .1;0/X.T/t C Wt

17.1 The State Space Model 333

with Wt � WN.0;�2W/. This representation is called the local linear trend (LLT) model and implies that fYtg follows an ARIMA(0,2,2) process (see Exercise 17.5.1).

In the special case of a constant drift equal to ı, �2 �

D 0 and we have that �Yt D ı C "t C Wt � Wt�1. f�Ytg therefore follows a MA(1) process with �.1/ D ��2W=.�2" C 2�2W/ D �.2 C �/�1 where � D �2" =�2W is called the signal- to-noise ratio. Note that the first order autocorrelation is necessarily negative. Thus, this model is not suited for time series with positive first order autocorrelation in its first differences.

The seasonal component is characterized by two conditions St D St�d and Pd

tD1 St D 0 where d denotes the frequency of the data.6 Given starting values S1; S0; S�1; : : : ; S�dC3, the subsequent values can be computed recursively as:

StC1 D �St � : : : � St�dC2 C �tC1; t D 1;2; : : : where a noise �t � WN.0;�2�/ is taken into account.7 The state vector related to the seasonal component, X

.S/ t , is defined as X

.S/ t D .St; St�1; : : : ; St�dC2/0 which gives

the state equation

X .S/ tC1 D

�1 �1 :: : �1 �1 1 0 :: : 0 0

0 1 : : : 0 0 :::

::: :::

0 0 : : : 1 0

X .S/ t C

�tC1 0 :::

D F.S/X.S/t C V.S/tC1

with Q.S/ D diag.�2�;0; : : : ;0/. Combining the trend and the seasonal model to an overall model with state vector

Xt given by Xt D �

X .T/ t

0; X.S/t 0 �0

, we arrive at the state equation:

XtC1 D �

F.T/ 0 0 F.S/

�

Xt C

V .T/ tC1

V .S/ tC1

D FXt C VtC1

with Q D diag.�2" ;�2ı ;�2�;0; : : : ;0/. The observation equation then is:

Yt D �

1 0 1 0 :: : 0 �

Xt C Wt with R D �2W .

Finally, we can add a cyclical component fCt g which is modeled as a harmonic process (see Sect. 6.2) with frequency C , respectively periodicity 2�= C:

Ct D A cos. Ct/ C B sin. Ct/

6Four in the case of quarterly and twelve in the case of monthly observations. 7Alternative seasonal models can be found in Harvey (1989) and Hylleberg (1986).

334 17 Kalman Filter

0 0.5 1 1.5 2 2.5 3 0

0.5

1.5

2.5

3.5

4.5

frequency

λc = π/4, ρ = 0.7

λc = π/4, ρ = 0.8

λc = π/4, ρ = 0.85

λc = π/12, ρ = 0.85

π/4π/12

Fig. 17.2 Spectral density of the cyclical component for different values of C and �

Following Harvey (1989; p.39), we let the parameters A and B evolve over time by introducing the recursion

�

CtC1 C�tC1

�

D � �

cos C sin C � sin C cos C

��

Ct C�t

�

V .C/ 1;tC1

V .C/ 2;tC1

where C0 D A and C�0 D B and where fC�t g is an auxiliary process. The dampening factor � allows for additional flexibility in the specification. The processes fV.C/1;t g and fV.C/2;t g are two mutually uncorrelated white noise processes. It is instructive to examine the spectral density (see Sect. 6.1) of the cyclical component in Fig. 17.2. It can be shown (see Exercise 17.5.2) that fCtg follows an ARMA(2,1) process.

The cyclical component can be incorporated into the state space model above by augmenting the state vector XtC1 by the cyclical components CtC1 and C�tC1 and the error term VtC1 by fV.C/1;tC1g and fV.C/2;tC1g. The observations equation has to be amended accordingly. Section 17.4.2 presents an empirical application of this approach.

Dynamic Factor Models Dynamic factor models are an interesting approach when it comes to modeling simultaneously a large cross-section of times series. The concept was introduced into macroeconomics by Sargent and Sims (1977) and was then developed further and popularized by Quah and Sargent (1993), Reichlin (2003) and Breitung and Eickmeier (2006), among others. The idea is to view each time series Yit , i D 1; : : : ; n, as the sum of a linear combination of some joint unobserved factors

17.1 The State Space Model 335

ft D .f1t; : : : ; frt/0 and an idiosyncratic component fWitg, i D 1; : : : n. Dynamic factor models are particularly effective when the number of factors r is small compared to the number of time series n. In practice, several hundred time series are related to a handful factors. In matrix notation we can write the observation equation for the dynamic factor model as follows:

Yt D ƒ0ft C ƒ1ft�1 C : : : C ƒqft�q C Wt where ƒi, i D 0;1; : : : ; q, are n � r matrices. The state vector Xt equals .f 0t ; : : : ; f

0 t�q/

0 if we assume that the idiosyncratic component is white noise, i.e. Wt D .W1t; : : : ; Wnt/0 � WN.0; R/. The observation equation can then be written compactly as:

Yt D GXt C Wt where G D .ƒ0;ƒ1; : : : ;ƒq/. Usually, we assume that R is a diagonal matrix. The correlation between the different time series is captured exclusively by the joint factors.

The state equation depends on the assumed dynamics of the factors. One possibility is to model fftg as a VAR(p) process with ˆ.L/ft D et, et � WN.0;†/, and p � q C 1, so we can use the state space representation of the VAR(p) process from above. For the case p D 2 and q D 2 we get:

XtC1 D 0

ftC1 ft

ft�1

A D 0

ˆ1 ˆ2 0

Ir 0 0 0 Ir 0

ft ft�1 ft�2

A C 0

etC1 0

A D FXt C VtC1

and Q D diag.†;0;0/. This scheme can be easily generalized to the case p > q C 1 or to allow for autocorrelated idiosyncratic components, assuming for example that they follow autoregressive processes.

The dimension of these models can be considerably reduced by an appropriate re–parametrization or by collapsing the state space adequately (Bräuning and Koopman 2014). Such a reduction can considerably increase the efficiency of the estimation.

Real Business Cycle Model (RBC Model) State space models are becoming increasingly popular in macroeconomics, espe- cially in the context of dynamic stochastic general equilibrium (DSGE) models. These models can be seen as generalizations of the real business cycle (RBC) models.8 In these models a representative consumer is supposed to maximize the utility of his consumption stream over his infinite life time. Thereby, the consumer has the choice to consume part of his income or to invest his savings (part of his

8Prototypical models can be found in King et al. (1988) or Woodford (2003). Canova (2007) and Dejong and Dave (2007) present a good introduction to the analysis of DSGE models.

336 17 Kalman Filter

income which is not consumed) at the market rate of interest. These savings can be used as a mean to finance investment projects which increase the economy wide capital stock. The increased capital stock then allows for increased production in the future. The production process itself is subject to a random shocks called technology shocks.

The solution of this optimization problem is a nonlinear dynamic system which determines the capital stock and consumption in every period. Its local behavior can be investigated by linearizing the system around its steady state. This equation can then be interpreted as the state equation of the system. The parameters of this equation F and Q are related, typically in a nonlinear way, to the parameters describing the utility and the production function as well as the process of technology shocks. Thus, the state equation summarizes the behavior of the theoretical model.

The parameters of the state equation can then be estimated by relating the state vector, given by the capital stock and the state of the technology, via the observation equation to some observable variables, like real GDP, consumption, investment, or the interest rate. This then completes the state space representation of the model which can be analyzed and estimated using the tools presented in Sect. 17.3.9

17.2 Filtering and Smoothing

As we have seen, the state space model provides a very flexible framework for a wide array of applications. We therefore want to develop a set of tools to handle this kind of models in terms of interpretation and estimation. In this section we will analyze the problem of inferring the unobserved state from the data given the parameters of the model. In Sect. 17.3 we will then investigate the estimation of the parameters by maximum likelihood.

In many cases the state of the system is not or only partially observable. It is therefore of interest to infer from the data Y1; Y2; : : : ; YT the state vector Xt. We can distinguish three types of problems depending on the information used:

(i) estimation of Xt from Y1; : : : ; Yt�1, known as the prediction problem; (ii) estimation of Xt from Y1; : : : ; Yt, known as the filtering problem;

(iii) estimation of Xt from Y1; : : : ; YT , known as the smoothing problem.

For the ease of exposition, we will assume that the disturbances Vt and Wt are normally distributed. The recursive nature of the state equation implies that Xt D Ft�1X1 C

Pt�2 jD0 F

jVt�j. Therefore, Xt is also normally distributed for all t, if X1 is normally distributed. From the observation equation we can infer also

9See Sargent (2004) or Fernandez-Villaverde et al. (2007) for systematic treatment of state space models in the context of macroeconomic models. In this literature the use of Bayesian methods is widespread (see An and Schorfheide 2007; Dejong and Dave 2007).

17.2 Filtering and Smoothing 337

that Yt is normally distributed, because it is the sum of two normally distributed random variables A C GXt and Wt . Thus, under these assumptions, the vector .X01; : : : ; X

0 T; Y

0 1; : : : ; Y

0 T/

0 is jointly normally distributed: 0

X1 :::

XT Y1 :::

� N ��

�Y

�

;

�

�X �YX �XY �Y

��

where the covariance matrices �X;�YX;�XY and �Y can be retrieved from the model given the parameters.

For the understanding of the rest of this section, the following theorem is essential (see standard textbooks, like Amemiya 1994; Greene 2008).

Theorem 17.1. Let Z be a n-dimensional normally distributed random variable with Z � N.�;†/. Consider the partitioned vector Z D .Z01; Z02/0 where Z1 and Z2 are of dimensions n1 � 1 and n2 � 1, n D n1 C n2, respectively. The corresponding partitioning of the covariance matrix † is

† D �

†11 †12 †21 †22

�

where †11 D VZ1, †22 D VZ2, and †12 D †021 D cov.Z1; Z2/ D E.Z1�EZ1/0.Z2� EZ2/. Then the partitioned vectors Z1 and Z2 are normally distributed. Moreover, the conditional distribution of Z1 given Z2 is also normal with mean and variance

E.Z1jZ2/ D EZ1 C †12†�122 .Z2 � EZ2/; V.Z1jZ2/ D †11 � †12†�122 †21:

This formula can be directly applied to figure out the mean and the variance of the state vector given the observations. Thus, setting Z1 D .X01; : : : ; X0t/0 and Z2 D .Y 01; : : : ; Y 0t�1/0, we get the predicted values; setting Z1 D .X01; : : : ; X0t/0 and Z2 D .Y 01; : : : ; Y 0t /0, we get the filtered values; setting Z1 D .X01; : : : ; X0t/0 and Z2 D .Y 01, : : : ; Y 0T/0, we get the smoothed values.

AR(1) Process with Measurement Errors

We illustrate the above ideas by analyzing a univariate AR(1) process with measurement errors10:

10Sargent (1989) provides an interesting application showing the implications of measurement errors in macroeconomic models.

338 17 Kalman Filter

XtC1 D �Xt C vtC1; vt � IIDN.0;�2v/ Yt D Xt C wt; wt � IIDN.0;�2w/:

For simplicity, we assume j�j < 1. Suppose that we only have observations Y1 and Y2 at our disposal. The joint distribution of .X1; X2; Y1; Y2/

0 is normal. The covariances can be computed by applying the methods discussed in Chap. 2:

X1 X2 Y1 Y2

� N

; �2v

1 � �2

1 � 1 �

� 1 � 1

1 � 1 C �2w.1��2/ �2v

�

� 1 � 1 C �2w.1��2/ �2v

The smoothed values are obtained by applying the formula from Theorem 17.1:

��

X1 X2

�ˇ

Y1; Y2

�

.1 C �2w.1��2/ �2v

/2 � �2 �

1 �

� 1

�

1 C �2w.1��2/ �2v

�� 1 C �2w.1��2/

�2v

�

Y1 Y2

�

Note that for the last observation, Y2 in our case, the filtered and the smoothed values are the same. For X1 the filtered value is

E.X1jY1/ D 1

1 C �2w.1��2/ �2v

Y1:

An intuition for this result can be obtained by considering some special cases. For � D 0, the observations are not correlated over time. The filtered value for X1 therefore corresponds to the smoothed one. This value lies between zero, the uncon- ditional mean of X1, and Y1 with the variance ratio �

2 w=�

2 v delivering the weights:

the smaller the variance of the measurement error the closer the filtered value is to Y1. This conclusion holds also in general. If the variance of the measurement error is relatively large, the observations do not deliver much information so that the filtered and the smoothed values are close to the unconditional mean.

For large systems the method suggested by Theorem 17.1 may run into numerical problems due to the inversion of the covariance matrix of Y, †22. This matrix can become rather large as it is of dimension nT � nT. Fortunately, there exist recursive solutions to this problem known as the Kalman filter, and also the Kalman smoother.

17.2 Filtering and Smoothing 339

17.2.1 The Kalman Filter

The Kalman filter circumvents the problem of inverting a large nT � nT matrix by making use of the Markov property of the system (see Remark 17.4). The distribution of Xt given the observations up to period t can thereby be computed recursively from the distribution of the state in period t � 1 given the information available up to period t � 1. Starting from some initial distribution in period 0, we can in this way obtain in T steps the distribution of all states. In each step only an n � n matrix must be inverted. To describe the procedure in detail, we introduce the following notation:

E.Xt jY1; : : : ; Yh/ D Xtjh V.XtjY1; : : : ; Yh/ D Ptjh:

Suppose, we have already determined the distribution of Xt conditional on the observations Y1; : : : ; Yt . Because we are operating in a framework of normally distributed random variables, the distribution is completely characterized by its conditional mean Xtjt and variance Ptjt. The goal is to carry forward these entities to obtain XtC1jtC1 and PtC1jtC1 having observed an additional data point YtC1. This problem can be decomposed into a forecasting and an updating step.

Step 1: Forecasting Step The state equation and the assumption about the disturbance term VtC1 imply:

XtC1jt D FXtjt (17.6) PtC1jt D FPtjt F0 C Q

The observation equation then allows to compute a forecast of YtC1 where we assume for simplicity that A D 0:

YtC1jt D GXtC1jt (17.7)

Step 2: Updating Step In this step the additional information coming from the additional observation YtC1 is processed to update the conditional distribution of the state vector. The joint conditional distribution of .X0tC1; Y

0 tC1/

0 given Y1; : : : ; Yt is

�

XtC1 YtC1

�ˇ

Y1; : : : ; Yt � N ��

XtC1jt YtC1jt

�

;

�

PtC1jt PtC1jt G0 GPtC1jt GPtC1jtG0 C R

��

As all elements of the distribution are available from the forecasting step, we can again apply Theorem 17.1 to get the distribution of the filtered state vector at time t C 1:

XtC1jtC1 D XtC1jt C PtC1jt G0.GPtC1jt G0 C R/�1.YtC1 � YtC1jt/ (17.8) PtC1jtC1 D PtC1jt � PtC1jtG0.GPtC1jtG0 C R/�1GPtC1jt (17.9)

340 17 Kalman Filter

where we replace XtC1jt, PtC1jt, and YtC1jt by FXtjt , FPtjt F0 C Q, and GFXtjt , respectively, which have been obtained from the forecasting step.

Starting from given values for X0j0 and P0j0, we can therefore iteratively compute Xtjt and Ptjt for all t D 1;2; : : : ; T. Only the information from the last period is necessary at each step. Inserting Eq. (17.8) into Eq. (17.6) we obtain as a forecasting equation:

XtC1jt D FXtjt�1 C FPtjt�1G0.GPtjt�1G0 C R/�1.Yt � GXtjt�1/

where the matrix

Kt D FPtjt�1G0.GPtjt�1G0 C R/�1

is know as the (Kalman) gain matrix. It prescribes how the innovation Yt � Ytjt�1 D Yt � GXtjt�1 leads to an update of the predicted state.

Initializing the Algorithm It remains to determine how to initialize the recursion. In particular, how to set the starting values for X0j0 and P0j0. If Xt is stationary and causal with respect to Vt, the state equation has the solution X0 D

P1 jD0 F

jVt�j. Thus,

X0j0 D E.X0/ D 0 P0j0 D V.X0/

where P0j0 solves the equation (see Sect. 12.4)

P0j0 D FP0j0F0 C Q:

According to Eq. (12.4), the solution of the above matrix equation is:

vec.P0j0/ D ŒI � F ˝ F �1vec.Q/:

If the process is not stationary, we can set X0j0 to zero and P0j0 to infinity. In practice, a very large number is sufficient.

17.2.2 The Kalman Smoother

The Kalman filter determines the distribution of the state at time t given the information available up to this time. In many instances, we want, however, make an optimal forecast of the state given all the information available, i.e. the whole sample. Thus, we want to determine XtjT and PtjT . The Kalman filter determines the smoothed distribution for t D T, i.e. XTjT and PTjT . The idea of the Kalman

17.2 Filtering and Smoothing 341

smoother is again to determine the smoothed distribution in a recursive manner. For this purpose, we let the recursion run backwards. Starting with the last observation in period t D T, we proceed back in time by letting t take successively the values t D T � 1; T � 2; : : : until the first observation in period t D 1.

Using again the linearity of the equations and the normality assumption, we get:

�

Xt XtC1

�ˇ

Y1; Y2; : : : ; Yt � N ��

Xtjt FXtjt

�

;

�

Ptjt Ptjt F0 FPtjt PtC1jt

��

This implies that

E.Xt jY1; : : : ; Yt; XtC1/ D Xtjt C Ptjt F0P�1tC1jt.XtC1 � XtC1jt/:

The above mean is only conditional on all information available up to time t and on the information at time t C 1. The Markov property implies that this mean also incorporates the information from the observations YtC1; : : : ; YT . Thus, we have:

E.XtjY1; : : : ; YT; XtC1/ D E.XtjY1; : : : ; Yt; XtC1/ D Xtjt C Ptjt F0P�1tC1jt.XtC1 � XtC1jt/

Applying the law of iterated expectations or means (see, f.e. Amemiya 1994; p. 78), we can derive XtjT :

XtjT D E.Xt jY1; : : : ; YT/ D E.E.XtjY1; : : : ; YT; XtC1/jY1; : : : ; YT/ D E.Xtjt C PtjtF0P�1tC1jt.XtC1 � XtC1jt/jY1; : : : ; YT/ D Xtjt C Ptjt F0P�1tC1jt.XtC1jT � XtC1jt/: (17.10)

The algorithm can now be implemented as follows. In the first step compute XT�1jT according to Eq. (17.10) as

XT�1jT D XT�1jT�1 C PT�1jT�1F0P�1TjT�1.XTjT � XTjT�1/:

All entities on the right hand side can readily be computed by applying the Kalman filter. Having found XT�1jT , we can again use Eq. (17.10) for t D T � 2 to evaluate XT�2jT :

XT�2jT D XT�2jT�2 C PT�2jT�2F0P�1T�1jT�2.XT�1jT � XT�1jT�2/:

Proceeding backward through the sample we can derive a complete sequence of smoothed states XTjT; XT�1jT; XT�2jT; : : : ; X1jT . These calculations are based on the computations of Xtjt , XtC1jt , Ptjt , and PtC1jt which have already been obtained from

342 17 Kalman Filter

the Kalman filter. The smoothed covariance matrix PtjT is given as (see Hamilton 1994b; Section 13.6):

PtjT D Ptjt C Ptjt FP�1tC1jt.PtC1jT � PtC1jt/P�1tC1jtF0Ptjt:

Thus, we can compute also the smoothed variance with the aid of the values already determined by the Kalman filter.

AR(1) Process with Measurement Errors (Continued)

We continue our illustrative example of an AR(1) process with measurement errors and just two observations. First, we determine the filtered values for the state vector with the aid of the Kalman filter. To initialize the process, we have to assign a distribution to X0. For simplicity, we assume that j�j < 1 so that it makes sense to assign the stationary distribution of the process as the distribution for X0:

X0 � N �

0; �2V

1 � �2 �

Then we compute the forecasting step as the first step of the filter (see Eq. (17.6)):

X1j0 D �X0j0 D 0

P1j0 D �2 �2v

1 � �2 C � 2 v D

�2v 1 � �2

Y1j0 D 0:

P1j0 was computed by the recursive formula from the previous section, but is, of course, equal to the unconditional variance. For the updating step, we get from Eqs. (17.8) and (17.9):

X1j1 D �

�2v 1 � �2

��

�2v 1 � �2 C �

2 w

��1 Y1 D

1 C �2w.1��2/ �2v

P1j1 D �

�2v 1 � �2

�

� �

�2v 1 � �2

�2 � �2v

1 � �2 C � 2 w

��1

D � 2 v

1 � �2

@1 � 1 1 C �2w.1��2/

�2v

These two results are then used to calculate the next iteration of the algorithm. This will give the filtered values for t D 2 which would correspond to the smoothed values because we just have two observations. The forecasting step is:

17.3 Estimation of State Space Models 343

X2j1 D �

1 C �2w.1��2/ �2v

P2j1 D �2�2v 1 � �2

@1 � 1 1 C �2w.1��2/

�2v

A C �2v

Next we perform the updating step to calculate X2j2 and P2j2. It is easy to verify that this leads to the same results as in the first part of this example.

An interesting special case is obtained when we assume that � D 1 so that the state variable is a simple random walk. In this case the unconditional variance of Xt and consequently also of Yt are no longer finite. As mentioned previously, we can initialize the Kalman filter by X0j0 D 0 and P0j0 D 1. This implies:

Y1j0 D X1j0 D X0j0 D 0 P1j0 D P0j0 C �2V D 1:

Inserting this result in the updating Eqs. (17.8) and (17.9), we arrive at:

X1j1 D P1j0

P1j0 C �2W .Y1 � Y1j0/ D

P0j0 C �2V P0j0 C �2V C �2W

P1j1 D P1j0 � P2 1j0

P1j0 C �2W D .P0j0 C �2V/

�

1 � P1j0 P1j0 C �2W

�

D .P0j0 C � 2 V/�

2 W

P0j0 C �2V C �2W :

Letting P0j0 go to infinity, leads to:

X1j1 D Y1 P1j1 D �2W:

This shows that the filtered variance is finite for t D 1 although P1j0 was infinite.

17.3 Estimation of State Space Models

Up to now we have assumed that the parameters of the system are known and that only the state is unknown. In most economic applications, however, also the parameters are unknown and have therefore to be estimated from the data. One big advantage of the state space models is that they provide an integrated approach to forecasting, smoothing and estimation. In particular, the Kalman filter turns out to be an efficient and quick way to compute the likelihood function. Thus, it seems natural to estimate the parameters of state space models by the method of

344 17 Kalman Filter

maximum likelihood. Kim and Nelson (1999) and Durbin and Koopman (2011) provide excellent and extensive reviews of the estimation of state space models using the Kalman filter.

More recently, due to advances in computational methods, in particular with respect to sparse matrix programming, other approaches can be implemented. For example, by giving the states a matrix representation Chan and Jeliazkov (2009) derive a viable and efficient method for the estimation of state space models.

17.3.1 The Likelihood Function

The joint unconditional density of the observations .Y 01; : : : ; Y 0 T/

0 can be factorized into the product of conditional densities as follows:

f .Y1; : : : ; YT/ D f .YT jY1; : : : ; YT�1/f .Y1; : : : ; YT�1/

D ::: D f .YT jY1; : : : ; YT�1/f .YT�1jY1; : : : ; YT�2/ : : : f .Y2jY1/f .Y1/

Each conditional density is the density of a normal distribution and is therefore given by:

f .Yt jY1; : : : ; Yt�1/ D .2�/�n=2.det �t/�1=2

exp

�

�1 2 .Yt � Ytjt�1/0��1t .Yt � Ytjt�1/

�

where �t D GPtjt�1G0 CR. The Gaussian likelihood function L is therefore equal to:

L D .2�/�.Tn/=2

T Y

tD1 det.�t/

!�1=2

exp

�1 2

T X

tD1 .Yt � Ytjt�1/0��1t .Yt � Ytjt�1/

Note that all the entities necessary to evaluate the likelihood function are provided by the Kalman filter. Thus, the evaluation of the likelihood function is a byproduct of the Kalman filter. The maximum likelihood estimator (MLE) is then given by the maximizer of the likelihood function, or more conveniently the log-likelihood function. Usually, there is no analytic solution available so that one must resort to numerical methods. An estimation of the asymptotic covariance matrix can be obtained by evaluating the Hessian matrix at the optimum. Under the usual

17.3 Estimation of State Space Models 345

assumptions, the MLE is consistent and delivers asymptotically normally distributed estimates (Greene 2008; Amemiya 1994).

The direct maximization of the likelihood function is often not easy in prac- tice, especially for large systems involving many parameters. The expectation- maximization algorithm, EM algorithm for short, represents a valid, though slower alternative. As the name indicates, it consists of two steps which have to be carried out iteratively. Based on some starting values for the parameters, the first step (expectation step) computes estimates, XtjT , of the unobserved state vector Xt using the Kalman smoother. In the second step (maximization step), the likelihood function is maximized taking the estimates of Xt, XtjT , as additional observations. The treatment of XtjT as additional observations, allows to reduce the maximization step to a simple multivariate regression. Indeed, by treating XtjT as if they were known, the state equation becomes a simple VAR(1) which can be readily estimated by linear least-squares to obtain the parameters F and Q. The parameters A, G and R are also easily retrieved from a regression of Yt on XtjT . Based on these new parameter estimates, we go back to step one and derive new estimates for XtjT which are then used in the maximization step. One can show that this procedure maximizes the original likelihood function (see Dempster et al. 1977; Wu 1983). A more detailed analysis of the EM algorithm in the time series context is provided by Brockwell and Davis (1996).11

Sometimes it is of interest not only to compute parameter estimates and to derive from them estimates for the state vector via the Kalman filter or smoother, but also to find confidence intervals for the estimated state vector to take the uncertainty into account. If the parameters are known, the methods outlined previously showed how to obtain these confidence intervals. If, however, the parameters have to be estimated, there is a double uncertainty: the uncertainty from the filter and the uncertainty arising from the parameter estimates. One way to account for this additional uncertainty is by the use of simulations. Thereby, we draw a given number of parameter vectors from the asymptotic distribution and compute for each of these draws the corresponding estimates for the state vector. The variation in these estimates is then a measure of the uncertainty arising from the estimation of the parameters (see Hamilton 1994b; Section 13.7).

11The analogue to the EM algorithm in the Bayesian context is given by the Gibbs sampler. In contrast to the EM algorithm, we compute in the first step not the expected value of the states, but we draw a state vector from the distribution of state vectors given the parameters. In the second step, we do not maximize the likelihood function, but draw a parameter from the distribution of parameters given the state vector drawn previously. Going back and forth between these two steps, we get a Markov chain in the parameters and the states whose stationary distribution is exactly the distribution of parameters and states given the data. A detailed description of Bayesian methods and the Gibbs sampler can be found in Geweke (2005). Kim and Nelson (1999) discuss this method in the context of state space models.

346 17 Kalman Filter

17.3.2 Identification

As emphasized in Remark 17.5 of Sect. 17.1, the state space representations are not unique. See, for example, the two alternative representations of the ARMA(1,1) model in Sect. 17.1. This non-uniqueness of state space models poses an identi- fication problem because different specifications may give rise to observationally equivalent models.12 This problem is especially serious if all states are unob- servable. In practice, the identification problem gives rise to difficulties in the numerical maximization of the likelihood function. For example, one may obtain large differences for small variations in the starting values; or one may encounter difficulties in the inversion of the matrix of second derivatives.

The identification of state space models can be checked by transforming them into VARMA models and by investigating the issue in this reparameterized setting (Hannan and Deistler 1988). Exercise 17.5.6 invites the reader to apply this method to the AR(1) model with measurement errors. System identification is a special field in systems theory and will not be pursued further here. A systematic treatment can be found in the textbook by Ljung (1999).

17.4 Examples

17.4.1 Disaggregating Yearly Data into Quarterly Ones

The official data for quarterly GDP are released in Switzerland by the State Secretariat for Economic Affairs (SECO). They estimate these data taking the yearly values provided by the Federal Statistical Office (FSO) as given. This division of tasks is not uncommon in many countries. One of the most popular methods for disaggregation of yearly data into quarterly ones was proposed by Chow and Lin (1971).13 It is a regression based method which can take additional information in the form of indicator variables (i.e. variables which are measured at the higher frequency and correlated at the lower frequency with the variable of interest) into account. This procedure is, however, rather rigid. The state space framework is much more flexible and ideally suited to deal with missing observations. Applications of this framework to the problem of disaggregation were provided by Bernanke et al. (1997:1) and Cuche and Hess (2000), among others. We will illustrate this approach below.

Starting point of the analysis are the yearly growth rates of GDP and indicator variables which are recorded at the quarterly frequency and which are correlated with GDP growth. In our application, we will consider the growth of industrial pro- duction .IP/ and the index on consumer sentiment .C/ as indicators. Both variables

12Remember that, in our context, two representations are equivalent if they generate the same mean and covariance function for fYt g. 13Similarly, one may envisage the disaggregation of yearly data into monthly ones or other forms of disaggregation.

17.4 Examples 347

are available on a quarterly basis from 1990 onward. For simplicity, we assume that the annualized quarterly growth rate of GDP, fQtg, follows an AR(1) process with mean �:

Qt � � D �.Qt�1 � �/ C wt; wt � WN.0;�2w/

In addition, we assume that GDP is related to industrial production and consumer sentiment by the following two equations:

IPt D ˛IP C ˇIPQt C vIP;t Ct D ˛C C ˇC Qt C vC;t

where the residuals vIP;t and vC;t are uncorrelated. Finally, we define the relation between quarterly and yearly GDP growth as:

Jt D 1

4 Qt C

4 Qt�1 C

4 Qt�2 C

4 Qt�3; t D 4;8;12: : :

We can now bring these equations into state space form. Thereby the observation equation is given by

Yt D At C Gt Xt C Wt with observation and state vectors

Yt D

Jt IPt Ct

A ; t D 4;8;12; : : : I

IPt Ct

A ; t ¤ 4;8;12; : : :

Xt D

Qt � � Qt�1 � � Qt�2 � � Qt�3 � �

and time-varying coefficient matrices

348 17 Kalman Filter

At D

�

˛IP

˛C

A ; t D 4;8;12; : : : I

˛IP

˛C

A ; t ¤ 4;8;12; : : :

Gt D

1 4

1 4 1 4 1 4

ˇIP 0 0 0

ˇC 0 0 0

A ; t D 4;8;12; : : : I

0 0 0 0

ˇIP 0 0 0

ˇC 0 0 0

A ; t ¤ 4;8;12; : : :

Rt D

0 0 0

0 �2IP 0

0 0 �2C

A ; t D 4;8;12; : : : I

1 0 0

0 �2IP 0

0 0 �2C

A ; t ¤ 4;8;12; : : :

The state equation becomes:

XtC1 D FXt C VtC1 where

F D

� 0 0 0

1 0 0 0

0 1 0 0

0 0 1 0

Q D

�2w 0 0 0

0 0 0 0

On my homepage http://www.neusser.ch/ you will find a MATLAB code which maximizes the corresponding likelihood function numerically. Figure 17.3 plots the different estimates of GDP growth and compares them with the data released by State Secretariat for Economic Affairs (SECO).

http://www.neusser.ch/

17.4 Examples 349

Q1−91 Q4−92 Q4−94 Q4−96 Q4−98 Q4−00 Q4−02 Q4−04 Q4−06 −3

−2

−1

4 p er

ce n t

yearly GDP filterted quarterly GDP smoothed quarterly GDP

quarterly estimates of GDP published by SECO

Fig. 17.3 Estimates of quarterly GDP growth rates for Switzerland

17.4.2 Structural Time Series Analysis

A customary practice in business cycle analysis is to decompose a time series into several components. As an example, we estimate a structural time series model which decomposes a times series additively into a local linear trend, a business cycle component, a seasonal component, and an irregular component. This is the specification studied as the basic structural model (BSM) in Sect. 17.1.1. We carry over the specification explained there to apply it to quarterly real GDP of Switzerland. Figure 17.4 shows the smoothed estimates of the various components. In the left upper panel the demeaned logged original series (see Fig. 17.4a) is plotted. One clearly discern the trend and the seasonal variations. The right upper panel shows the local linear trend (LLT). As one can see the trend is not a straight line, but exhibits pronounced waves of low frequency. The business cycle component showed in Fig. 17.4c is much more volatile. The large drop of about 2.5 % in 2008/09 corresponds to the financial markets. The lower right panel plots the seasonal component (see Fig. 17.4d). From a visual inspections, one can infer that the volatility of the seasonal component is much larger than the cyclical component (compare the scale of the two components) so that movements in GDP are dominated by seasonal fluctuations.14 Moreover, the seasonal component changes its character over time.

14The irregular component which is not shown has only very small variance.

350 17 Kalman Filter

1980 1985 1990 1995 2000 2005 2010 −0.4

−0.3

−0.2

−0.1

0.1

0.2

0.3

1980 1985 1990 1995 2000 2005 2010 2015 −0.4

−0.3

−0.2

−0.1

0.1

0.2

0.3

1980 1985 1990 1995 2000 2005 2010 2015 −0.03

−0.02

−0.01

0.01

0.02

1980 1985 1990 1995 2000 2005 2010 2015 −0.05

−0.04

−0.03

−0.02

−0.01

0.01

0.02

0.03

a b

c d

Fig. 17.4 Components of the basic structural model (BSM) for real GDP of Switzerland. (a) Logged Swiss GDP (demeaned). (b) Local linear trend (LLT). (c) Business cycle component. (d) Seasonal component

17.5 Exercises

Exercise 17.5.1. Consider the basic structural time series model for fYtg:

Yt D Tt C Wt; Wt � WN.0;�2W/ Tt D ıt�1 C Tt�1 C "t; "t � WN.0;�2"/ ıt D ıt�1 C �t; �t � WN.0;�2� /

where the error terms Wt , "t and �t are all uncorrelated with other at all leads and lags.

(i) Show that fYtg follows an ARIMA(0,2,2) process. (ii) Compute the autocorrelation function of f�2Yt g.

Exercise 17.5.2. If the cyclical component of the basic structural model for fYtg is: �

Ct C�t

�

D � �

cos C sin C � sin C cos C

��

Ct�1 C�t�1

�

V .C/ 1;t

V .C/ 2;t

17.5 Exercises 351

where fV.C/1;t g and fV.C/2;t g are mutually uncorrelated white-noise processes.

(i) Show that fCtg follows an ARMA(2,1) process with ACF given by �h.h/ D �h cos Ch.

Exercise 17.5.3. Write the ARMA(p,q) process Yt D �1Xt�1 C : : : C �pXt�p C Zt C �1Zt�1 C: : :C�qZt�q as a state space model such that the state vector Xt is given by:

Xt D

Yt Yt�1 :::

Yt�p Zt�1 Zt�2 :::

Zt�q

Exercise 17.5.4. Show that Xt and Yt have a unique stationary and causal solution if all eigenvalues of F are absolutely strictly smaller than one. Use the results from Sect. 12.3.

Exercise 17.5.5. Find the Kalman filter equations for the following system:

Xt D �Xt�1 C wt Yt D Xt C vt

where and � are scalars and where

�

vt wt

�

� N ��

�

;

�

�2v �vw �vw �

2 w

��

Exercise 17.5.6. Consider the state space model of an AR(1) process with mea- surement error analyzed in Sect. 17.2:

XtC1 D �Xt C vtC1; vt � IIDN.0;�2v/ Yt D Xt C wt; wt � IIDN.0;�2w/:

For simplicity assume that j�j < 1.

352 17 Kalman Filter

(i) Show that fYt g is an ARMA(1,1) process given by Yt ��Yt�1 D Zt C�Zt�1 with Zt � WN.0;�2Z/.

(ii) Show that the parameters of the state space, �;�2v;� 2 w and those of the

ARMA(1,1) model are related by the equation

��2Z D ��2w 1

1 C �2 D ��2w

�2v C .1 C �2/�2w (iii) Why is there an identification problem?

18Generalizations of Linear Time Series Models

Autoregressive moving-average models have become the predominant approach in the analysis of economic, especially macroeconomic time series. The success of these parametric models is due to a mature and by now well-understood statistical theory which has been the subject of this book. The main assumption behind this theory is its linear structure. Although convenient, the assumption of a constant linear structure turned out to be unrealistic in many empirical applications. The evolution of economies and the economic dynamics are often not fully captured by constant coefficient linear models. Many time series are subject to structural breaks which manifest themselves as a sudden change in the model coefficients by going from one period to another. The detection and dating of such structural breaks is the subject of Sect. 18.1. Alternatively, one may think of the model coefficients as varying over time. Such models have proven to be very flexible and able to generate a variety of non-linear features. We present in Sects. 18.2 and 18.3 two variants of such models. In the first one, the model parameters vary in a systematic way with time. They are, for example, following an autoregressive process. In the second one, the parameters switch between a finite number of states according to a hidden Markov chain. These states are often identified as regimes which have a particular economic meaning, for example as booms and recessions. Further parametric and nonparametric methods for modeling and analyzing nonlinear time series can be found in Fan and Yao (2003).

18.1 Structural Breaks

There is a extensive literature dealing with the detection and dating of structural breaks in the context of time series. This literature is comprehensibly summarized in Perron (2006), among others. A compact account can also be found in Aue and Horváth (2011) where additional testing procedures, like the CUSUM test, are

353

354 18 Generalizations of Linear Models

presented. In this short exposition we follow Bai et al. (1998) and focus on Chow type test procedures. For the technical details the interested reader is referred to these papers.

18.1.1 Methodology

Consider, for the ease of exposition, a VAR(1) process which allows for a structural break at some known date tb:

Xt D dt.tb/ �

c.1/ C ˆ.1/Xt�1 � C .1 � dt.tb//

�

c.2/ C ˆ.2/Xt�1 � C Zt (18.1)

where

dt.tb/ D �

1; t � tbI 0; t > tb:

Thus, before time tb the coefficients of the VAR process are given by c .1/ and ˆ.1/

whereas after tb they are given by c .2/ and ˆ.2/. The error process fZtg is assumed to

be IID.0;†/ with † positive definite.1 Suppose further that the roots of ˆ.1/.z/ as well as those of ˆ.2/.z/ are outside the unit circle. The process therefore is stationary and admits a causal representation with respect to fZtg before and after date tb.

The assumption of a structural break at some known date tb can then be investigated by testing the hypothesis

H0 W c.1/ D c.2/ and ˆ.1/ D ˆ.2/ against H1 W c.1/ ¤ c.2/ or ˆ.1/ ¤ ˆ.2/:

The standard way to test such a hypothesis is via the F-statistic. Given a sample ranging from period 0 to period T, the strategy is to partition all variables and matrices along the break date tb. Following the notation and the spirit of Sect. 13.2, define Y D vec.Y.1/; Y.2// where Y.1/ D .X1; X2; : : : ; Xtb/ and Y.2/ D .Xtb C1; Xtb C2; : : : ; XT/, Z D .Z1; Z2; : : : ; ZT/, and

X.1/ D

1 X1;0 : : : Xn;0 1 X1;1 : : : Xn;1 :::

::: :::

:::

1 X1;tb �1 : : : Xn;tb �1

X.2/ D

1 X1;tb : : : Xn;tb 1 X1;tb C1 : : : Xn;tb C1 :::

::: :::

:::

1 X1;T�1 : : : Xn;T�1

;

1Generalization to higher order VAR models is straightforward. For changes in the covariance matrix † see Bai (2000). For the technical details the reader is referred to the relevant literature.

18.1 Structural Breaks 355

then the model (18.1) can be written as

Y D vec.Y.1/; Y.2// D �

X.1/ ˝ In 0 0 X.2/ ˝ In

�

„ ƒ‚ …

vec.c.1/;ˆ.1/; c.2/;ˆ.2// „ ƒ‚ …

C vec Z:

The least-squares estimator becomes

Ǒ D vec.Oc.1/;bˆ.1/; Oc.2/;bˆ.2// D .X0X/�1X0Y

..X.1/ 0 X.1//�1X.1/0/ ˝ In 0

0 ..X.2/ 0 X.2//�1X.2/0/ ˝ In

vec.Y.1/; Y.2//:

This amounts to estimate the model separately over the two sample periods. Note that as in Sect. 13.2 the GLS estimator is numerically identical to the OLS estimator because the same regressors are used for each equation. The corresponding Wald- test can be implemented by defining R D .In2Cn; �In2Cn/ and computing the F-statistic

F.tb/ D .R Ǒ/0ŒR.X0.IT ˝ b†�1tb /X/R0 �1.R Ǒ/ (18.2)

where Ǒ D vec.Oc.1/;bˆ.1/; Oc.2/;bˆ.2// and where b†tb is computed from the least- squares residuals bZt as b†tb D 1T

PT tD1bZtbZ

0 t given break date tb. Under the standard

assumptions made in Sect. 13.2, the test statistic F.tb/=.n 2 C n/ converges for

T ! 1 to a chi-square distribution with n2 C n degrees of freedom.2 This test is known in the literature as the Chow test.

The previous analysis assumed that the potential break date tb is known. This assumption often turns out to be unrealistic in practice. The question then arises how to determine a potential break date. Quandt (1960) proposed a simple procedure: compute the Chow-test for all possible break dates and take as a candidate break date the date where the F-statistic reaches its maximal value. Despite its simplicity, Quandt’s procedure could not be implemented coherently because it was not clear which distribution to use for the construction of the critical values. This problem remained open for more than thirty years until the contribution of Andrews (1993).3

Denote by bxc the value of x rounded to the nearest integer towards minus infinity, then the maximum Wald statistic and the logarithm of the Andrews and Ploberger (1994) exponential Wald statistic can be written as follows:

2As the asymptotic theory requires that tb=T does not go to zero, one has to assume that both the number of periods before and after the break go to infinity. 3A textbook version of the test can be found in Stock and Watson (2011).

356 18 Generalizations of Linear Models

sup F W sup �2.��;1��/

F.bT�c/

exp F W log Z 1��

�� exp.1

2 F.bT�c//d�

where �� denotes the percentage of the sample which is trimmed. Usually, �� takes the value of 0:15 or 0:10. Critical values for low degrees of freedom are tabulated in Andrews (1993, 2003) and Stock and Watson (2011). It is possible to construct an asymptotic confidence interval for the break date. The corresponding formulas can be found in Bai et al. (1998; p. 401–402).

18.1.2 An Example

The use of the structural break test is demonstrated using historical data for the United Kingdom. The data consist of logged per capita real GDP, logged per capita real government expenditures, logged per capita real government revenues, the inflation based on the consumer price index, and a long-term interest rate over a sample period from 1830 to 2003. The basis for the analysis consists of a five variable VAR(2) model including a constant term and a linear trend. Three alternative structural break modes are investigated: break in the intercept, break in the intercept and the time trend, and break in all coefficients, including the VAR coefficients. The corresponding F-statistics are plotted in Fig. 18.1 against all possible break dates allowing for a trimming value of 10 %. The horizontal lines show for all three alternative break modes the corresponding critical values for the supF test given 5 % significance levels. These critical values have been obtained from Monte Carlo simulations as in Andrews (1993, 2003) and are given as 18.87, 28.09, and 97.39.4

Figure 18.1 shows that for all three modes a significant structural break occurs. The corresponding values of the supF statistics are 78.06, 104.75, and 285.22. If only the deterministic parts are allowed to change, the break date is located in 1913. If all coefficients are allowed to change, the break is dated in 1968. However, all three F-statistics show a steep increase in 1913. Thus, if only one break is allowed 1913 seems to be the most likely one.5 The breaks are quite precisely dated. The corresponding standard errors are estimated to be two years for the break in the intercept only and one year for the other two break modes.

4Assuming a trimming value of 0:10 Andrews (2003; table I) reports critical values of 18.86 for p D 5 which corresponds to changes in the intercept only and 27.27 for p D 10 which corresponds to changes in intercept and time trend. 5See Perron (2006) for a discussion of multiple breaks.

18.2 Time-Varying Parameters 357

1840 1860 1880 1900 1920 1940 1960 1980 2000

time

100

150

200

250

300 F

s ta

tis tic

break year:1968

break year:1913

break in all coefficients break in intercept break in intercept and trend

critical value for break in all coefficients

critical value for break in intercept and trend critical value for

break in intercept

Fig. 18.1 Analysis of breaks dates with the sup F test statistic for historical UK time series

18.2 Time-Varying Parameters

This section discusses time-varying coefficient vector autoregressive models (TVC- VAR models). This model class retains the flavor of VAR models but assumes that they are only valid locally. Consider for this purpose a VAR(1) model with time- varying autoregressive coefficients ˆt :

XtC1 D ˆtXt C ZtC1; Zt � IID.0;†/ with † > 0; t 2 Z: (18.3)

This model can be easily generalized to higher order VAR’s (see below) or, alternatively, one may think of Eq. (18.3) as a higher order VAR in companion form. The autoregressive coefficient matrix is assumed to be stochastic. Thus, ˆt is a random n � n matrix. Models of this type have been widely discussed in the probabilistic literature because they arise in many diverse contexts. In economics, Eq. (18.3) can be interpreted as the probabilistic version describing the value of a perpetuity, i.e. the present discounted value of a permanent commitment to pay a certain sum each period. Thereby Zt denotes the random periodic payments and ˆt the random cumulative discount factors. The model also plays an important role in the characterization of the properties of volatility models as we have seen in Sect. 8.1 (see in particular the proofs of Theorems 8.1 and 8.3). In this presentation, the above model is interpreted as a locally valid VAR process.

A natural question to ask is under which conditions Eq. (18.3) admits a stationary solution. An answer to this question can be found by iterating the equation backwards in time:

358 18 Generalizations of Linear Models

Xt D ˆt�1Xt�1 C Zt D ˆt�1.ˆt�2Xt�2 C Zt�1/ C Zt D Zt C ˆt�1Zt�1 C ˆt�1ˆt�2Xt�2 D Zt C ˆt�1Zt�1 C ˆt�1ˆt�2Zt�2 C ˆt�1ˆt�2ˆt�3Xt�3

: : :

D k X

jD0

j Y

iD1 ˆt�i

Zt�j C

kC1 Y

iD1 ˆt�i

Xt�k�1; k D 0;1;2; : : :

where it is understood that Q0

iD1 D In. This suggests as a solution candidate

Xt D lim k!1

k X

jD0

j Y

iD1 ˆt�i

Zt�j

D Zt C ˆt�1Zt�1 C ˆt�1ˆt�2Zt�2 C ˆt�1ˆt�2ˆt�3Zt�3 C : : : (18.4)

Based on results obtained by Brandt (1986) and extended by Bougerol and Picard (1992b), we can cite the following theorem.

Theorem 18.1 (Solution TVC-VAR(1)). Let f.ˆt; Zt/g be a strictly stationary ergodic process such that

(i) E.logC kˆt k/ < 1 and E.logC kZt k/ < 1 where xC denotes maxfx;0g; (ii) the top Lyapounov exponent � defined as

� D inf n2N

�

n C 1 log kˆ0ˆ�1 : : :ˆ�nk ��

is strictly negative.

Then Xt as defined in Eq. (18.4) converges a.s. and fXtg is the unique strictly stationary solution of equation (18.3).

Remark 18.1. The Lyapunov exponent measures the rate of separation of nearly trajectories in a dynamic system. The top Lyapunov exponent gives the largest of these rates. It is used to characterize the stability of a dynamic system (see Colonius and Kliemann (2014)).

Remark 18.2. Although Theorem 18.1 states only sufficient conditions, these assumptions can hardly be relaxed.

The solution (18.4), if it exists, is similar to a causal representation. The matrix sequence fQhiD1 ˆtCh�ighD0;1;2;::: D fIn;ˆt;ˆtC1ˆt;ˆtC2ˆtC1ˆt; : : :g represents the effect of an impulse in period t to XtCh, h D 0;1;2; : : : and can therefore be interpreted as impulse response functions. In contrast to the impulse response

18.2 Time-Varying Parameters 359

functions studied so far, they are clearly random and time-dependent because the effect of Zt depends on future coefficients. In particular, the effect of Zt on XtCh, h � 1, is not the same as the effect of Zt�h on Xt. Nevertheless it is possible to construct meaningful impulse response functions by Monte Carlo simulations. One may then report the mean of the impulse responses or some quantiles for different time periods.6 Alternatively, one may ignore the randomness and time- dependency and define “local” impulse responses as ˆht , h D 0;1;2; : : :. Note, however, that the impulse responses so defined still vary with time. Irrespectively how the impulse responses are constructed, they can be interpreted in the same way as in the case of constant coefficients. In particular, we may use some of the identification schemes discussed in Chap. 15 and compute the impulse responses with respect to structural shocks. Similar arguments apply to the forecast error variance decomposition (FEVD).

The model is closed by fixing the law of motion for ˆt . As already mentioned in Sect. 17.1.1 there are several possibilities. In this presentation we adopt the following flexible autoregressive specification:

ˇtC1 � Ň D F.ˇt � Ň/ C VtC1 Vt � WN.0; Q/ (18.5)

where ˇt D vec ˆt denotes the n2 vector of stacked coefficients. Q is assumed to be fixed and is, usually, specified as a diagonal matrix. If the eigenvalues of F are inside the unit circle, the autoregressive model is mean-reverting and Ň can be interpreted as the average coefficient vector. The formulation in Eq. (18.5) is, however, not restricted to this case and allows explicitly the possibility that fˇtg follows a random walk. This specification has become very popular in the empirical macroeconomic literature and was initially adopted by Cogley and Sargent (2001) to analyze the dynamics of inflation across different policy regimes.7

The model consisting of Eqs. (18.3) and (18.5) can be easily reformulated as a state space model by defining �t D ˇt � Ň as the state vector. The state and the measurement equation can then be written as:

state equation: �tC1 D F�t C VtC1 (18.6) measurement equation: Xt D .X0t�1 ˝ In/ Ň C .X0t�1 ˝ In/�t C Zt: (18.7)

Conditional on initial values for the coefficients and their covariances, the state space model can be estimated by maximum likelihood by applying the Kalman filter (see Sect. 17.3 and Kim and Nelson (1999)). One possibility to initialize the Kalman filter is to estimate the model for some initial sample period assuming fixed coefficients and extract from these estimates the corresponding starting values.

6Potter (2000) discusses the primal problems of defining impulse responses in a nonlinear context. 7They allow for a correlation between Vt and Zt .

360 18 Generalizations of Linear Models

As it turns out, allowing time-variation only in the coefficients of the VAR model overstates the role attributed to structural changes. We therefore generalize the model to allow for time-varying volatility. More specifically, we also allow † in Eq. (18.3) to vary with time. The modeling of the time-variation in † is, however, not a straightforward task because we must ensure that in each period †t is a symmetric positive definite matrix. One approach is to specify a process especially designed for modeling the dynamics of covariance matrices. This so-called Wishart autoregressive process was first introduced to economics by Gouriéroux et al. (2009) and successfully applied by Burren and Neusser (2013). It leads to a nonlinear state space system which can be estimated with the particle filter, a generalization of the Kalman filter.

Another more popular approach was initiated by Cogley and Sargent (2005) and Primiceri (2005). It is based on the Cholesky factorization of the time-varying covariance matrix †t. Using the same notation as in Sect. 15.3 †t is decomposed as

†t D Bt�tB0t (18.8)

where Bt is a time-varying lower triangular matrix with ones on the diagonal and �t a time-varying diagonal matrix with strictly positive diagonal elements.8 The logged diagonal elements of �t are then assumed to evolve as independent univariate random walks. This specification can be written in matrix terms as

�t D �t�1 exp.Dt/ (18.9)

where Dt is a diagonal matrix with diag.Dt/ � WN.0;�D/. In the above formulation exp denotes the matrix exponential.9 Taking the matrix logarithm, we get exactly the formulation of Cogley and Sargent (2005) and Primiceri (2005). For the time evolution of Bt we propose a similar specification:

Bt D Bt�1 exp.Ct/ (18.10)

where Ct is a strictly lower triangular matrix, i.e. Ct is a lower triangular matrix with zeros on the diagonal. The non-zero entries of Ct , denoted by ŒCt i>j, are assumed to follow a multivariate white noise process with diagonal covariance matrix †B, i.e. ŒCt i>j � WN.0;†B/. It can be shown that the matrix exponential of strictly lower triangular matrices are triangular matrices with ones on the diagonal. As the set of triangular matrices with ones on the diagonal form a group, called the unipotent group and denoted by SLTn, the above specification is well-defined. Moreover, this formulation is a very natural one as the set of strictly lower triangular matrices

8It is possible to consider other short-run type identification schemes (see Sect. 15.3) than the Cholesky factorization. 9The matrix exponential of a matrix A is defined as exp.A/ D P1iD0 1iŠ Ai where A is any matrix. Its inverse log.A/ is defined only for kAk < 1 and is given by log.A/ D P1iD1 .�1/

i�1

i Ai.

18.2 Time-Varying Parameters 361

is the tangent space of SLTn at the identity (see Baker 2002; for details). Thus, Eq. (18.10) can be interpreted as a log-linearized version of Bt. The technique proposed for the evolution of Bt in Eq. (18.10) departs from Primiceri (2005) who models each element of the inverse of Bt and therefore misses a coherent system theoretic approach. See Neusser (2016) for details.

Although this TVC-VAR model with time-varying volatility can in principle also be estimated by maximum likelihood, this technique can hardly be implemented successfully in practice. The main reason is that the likelihood function of such a model, even when the dimension and the order of the VAR is low, is a very high dimensional nonlinear object with probably many local maxima. Moreover, as the variances governing the time-variation are small, at least for some of the coefficients, the likelihood function is flat in some regions of the parameter space. These features make maximization of the likelihood function a very difficult, if not impossible, task in practice. For these reasons, Bayesian techniques have been used almost exclusively. There is, however, also a conceptional issue involved. As the Bayesian approach does not strictly distinguish between fixed “true” parameters and random samples, it is better suited to handle TVC-VAR models which treat the parameters as random. In this monograph, we will not tackle the Bayesian approach but refer to the relevant literature. See for example Primiceri (2005), Negro and Primiceri (2015), Cogley and Sargent (2005), Canova (2007) and Koop and Korobilis (2009) among others.

The Minnesota Prior

Although, we will not discuss the Bayesian approach to VAR modeling, it is nevertheless instructive to portray the so-called Minnesota prior applied by Doan et al. (1984) to TVC-VAR models. This prior has gained some reputation in connection to forecasting with VAR models and as a way to specify the initial distribution for the Kalman filter in time-varying models. The combination of the prior distribution with the likelihood function delivers via Bayes’ rule a posterior distribution of the parameters which can then be analyzed using simulation methods.

The Minnesota prior is based on the a priori belief that each variable follows a random walk with no interaction among the variables nor among the coefficients of the VAR equations. We expose one version of the Minnesota prior in the general context of a TVC-VAR model of order p with time-varying constant term ct:

Xt D ct C ˆ.1/t�1Xt�1 C : : : C ˆ.p/t�1Xt�p C Zt: (18.11)

This model can be written compactly as

Xt D .X0t�1 ˝ In/ vec ˆt�1 C Zt D .X0t�1 ˝ In/ˇt�1 C Zt (18.12)

362 18 Generalizations of Linear Models

where Xt�1 D .1; X0t�1; : : : ; X0t�p/0, ˆt�1 D .ct;ˆ.1/t�1; : : : ;ˆ.p/t�1/, and ˇt D vec ˆt . Assuming for ˇt the same autoregressive form as in Eq. (18.5), the state space representation (18.6) and (18.7) also applies to the TVC-VAR(p) model with Xt�1 replaced by Xt�1. Note that the dimension of the state equation can become very high because ˇt is a n C n2p vector.

Taking date 0 as the initial date, the prior distribution of the autoregressive parameters is supposed to be normal:

ˇ0 D vec ˆ0 � N. Ň; P0j0/

where Ň D vec.0; In;0; : : : ;0/. This implies that the mean for all coefficients, including the constant term, is assumed to be zero except for the own lag coefficients of order one Œˆ

.1/ 0 ii, i D 1; : : : ; n, which are assumed to be one. The covariance

matrix P0j0 is taken as being diagonal so that there is no correlation across coefficients. Thus, the prior specification amounts to assuming that each variable follows a random walk with no interaction with other variables.

The strength of this belief is governed by a number of so-called hyperparameters which regulate the diagonal elements of P0j0. The first one, �2, controls the confidence placed on the assumption that Œˆ

.1/ 0 ii D 1:

ˆ .1/ 0

ii � N.1;�2/; i D 1;2; : : : ; n:

A small (large) value of �2 thus means more (less) confidence. As the lag order increases more confidence is placed on the assumption Œˆ

.h/ 0 ii D 0:

ˆ .h/ 0

ii � N

�

0; �2

�

; h D 2; : : : ; p and i D 1; : : : ; n

Instead of the harmonic decline other schemes have been proposed. For h D 1; : : : ; p the off-diagonal elements of ˆ

.h/ 0 are assumed to have prior distribution

ˆ .h/ 0

ij � N

0; w2�2 O�2i

h O�2j

; i; j D 1; : : : ; n; i ¤ j; h D 1;2; : : : ; p:

Thereby O�2i =O�2j represents a correction factor which accounts for the magnitudes of Xit relative to Xjt. Specifically, O�2i is the residual variance of a univariate AR(1) model. The hyperparameter w2 is assumed to be strictly smaller than one. This represents the belief that Xj;t�h is less likely to be important as an explanation for Xi;t, i ¤ j, than the own lag Xi;t�h. Finally, the strength of the belief that the constant terms are zero is

ci0 D N.0; g O�i/:

18.2 Time-Varying Parameters 363

This completes the specification for the prior belief on ˇ0. Combining all elements we can write P0j0 as a block diagonal matrix with diagonal blocks:

P0j0 D

P .c/ 0j0 0 0 P

.�/

0j0

where P .c/ 0j0 D g � diag.O�1; : : : ; O�n/ and P

.�/

0j0 D diag.vec.G ˝ ‡//. Thereby, G and ‡ are defined as

G D .�2;�2=2; : : : ;�2=p/

Œ‡ ij D (

1; i D jI w2.O�2i =O�2j /; i ¤ j:

According to Doan et al. (1984) the preferred values for the three hyperparameters are g D 700, �2 D 0:07, and w2 D 0:01.

Thus, for a bivariate TVC-VAR(2) model the mean vector is given by Ň D .0;0;1;0;0;1;0;0;0;0/0 with diagonal covariance matrix P0j0:

P0j0 D

P .c/ 0j0 0 0 0 P

.1/

0j0 0 0 0 P

.2/

0j0

with

P .c/ 0j0 D g

�O�2 1 0

0 O�22

�

;

P .1/

0j0 D �2 0

1 0 0 0

0 w2 O�2 2 =O�2 1

0 0

0 0 w2 O�21=O�22 0 0 0 0 1

and

P .2/

0j0 D �2

1 0 0 0

0 w2 O�22=O�21 0 0 0 0 w2 O�21=O�22 0 0 0 0 1

364 18 Generalizations of Linear Models

Next we specify the parameters of the state transition equation (18.5). Following Doan et al. (1984), F D �F InCpn2 with �F D 0:999 and Q D �QP0j0 with �Q D 10�7. The proportionality factor does, however, not apply to the constant terms. For these terms, the corresponding diagonal elements of Q, ŒQ ii, i D 1; : : : ; n, are set to �QŒP0j0 i.nC1/;i.nC1/, i D 1; : : : ; n. The reason for this correction is that the prior put on the constants is rather loose as expressed by the high value of g. The final component is a specification for †, the variance of Zt. This matrix is believed to be diagonal with † D �†diag.O�21; : : : ; O�2n / and �† D 0:9.

With these ingredients the state space model is completely specified. Given observations X1; : : : ; Xt , the Kalman filter produces a sequence of ˇtC1jt , t D 1;2; : : : and one-period ahead forecasts XtC1jt computed as

XtC1jt D .X0t ˝ In/ˇtC1jt:

Doan et al. (1984) suggest to compute an approximate h period ahead forecast by treating the forecast from the previous periods as if they were actual observations.

18.3 Regime Switching Models

The regime switching model is similar to the time-varying model discussed in the previous section. The difference is that the time-varying parameters are governed by a hidden Markov chain with a finite state space S D f1;2; : : : ; kg. Usually, the number of states k is small and is equal in practice to two or maximal three. The states have usually an economic connotation. For example, if k equals two, state 1 might correspond to a boom phase whereas state 2 to a recession. Such models have a long tradition in economics and have therefore been used extensively. Seminal references include Goldfeld and Quandt (1973, 1976), Hamilton (1994b), Kim and Nelson (1999), Krolzig (1997), and Maddala (1986). Frühwirt-Schnatter (2006) presents a detailed statistical analysis of regime switching models.

The starting point of our presentation of the regime switching model is again the TVC-VAR(1) as given in Eq. (18.3). We associate to each state j 2 S a coefficient matrix ˆ.j/. Thus, in the regime switching model the coefficients ˆt can only assume a finite number values ˆ.1/; : : : ;ˆ.k/ depending on the state of the Markov chain. The actual value assigned to ˆt is governed by a Markov chain defined through a fixed but unknown transition probability matrix P where

ŒP ij D P �

ˆt D ˆ.j/jˆt�1 D ˆ.i/ �

i; j D 1; : : : ; k: (18.13)

Thus, ŒP ij is the probability that ˆt assumes value ˆ .j/ given that it assumed in the

previous period the value ˆ.i/. The probability that ˆtCh is in state j given that ˆt was in state i is therefore ŒPh ij. The definition of the transition matrix in Eq. (18.13) implies that P is a stochastic matrix, i.e. that ŒP ij � 0 and

Pk jD1ŒP ij D 1.

Moreover, we assume that the chain is regular meaning that it is ergodic (irreducible)

18.3 Regime Switching Models 365

and aperiodic.10 This is equivalent to the existence of a fixed integer m > 0 such that Pm has only strictly positive entries (see Berman and Plemmons 1994; Chapter 8). Regular Markov chains have a unique ergodic (stationary) distribution vector � with strictly positive entries and determined by �0P D �0. This distribution is approached from any initial distribution vector �0, i.e. limt!1 �00P

t D �0. Moreover, limt!1 Pt D P1 where P1 is a transition matrix with all rows equal to �0.

Given this setup we can again invoke Theorem 18.1 and claim that a (strictly) stationary solution of the form of Eq. (18.4) exists if all the autoregressive matrices ˆ.j/, j D 1;2; : : : ; k, have eigenvalues strictly smaller than one.

Given observations xT; xT�1; : : : ; x1; x0 for XT; XT�1; : : : ; X1; X0, a maximum likelihood approach can be set up to estimate the unknown parameters ˆ.1/; : : : ;ˆ.k/;†; P.11 Collect these parameters into a vector � and denote by st 2 S the state of the Markov chain in period t and by Xt D .xt; xt�1; : : : ; x1; x0/ the information available up to period t. Write the conditional density of xt given st D j and observations Xt�1 as

f .xtjst D j;Xt�1I �/:

The joint density of .xt; st D j/ is

f .xt; st D jjXt�1I �/ D f .xt jst D j;Xt�1I �/ � P.st D jjXt�1I �/

where in analogy to the Kalman filter the expressions P.st D jjXt�1I �/, j D 1; : : : ; k, are called the predicted transition probabilities. The conditional marginal density of xt then becomes

f .xt jXt�1I �/ D k X

jD1 f .xt jst D j;Xt�1I �/ � P.st D jjXt�1I �/:

In the case of Zt � IID N.0;†/ the above density is a finite mixture of Gaussian distributions (see Frühwirt-Schnatter 2006; for details). The (conditional) log likelihood function, finally, is therefore given by

`.�/ D T X

tD1 log f .xt jXt�1I �/:

10A chain is called ergodic or irreducible if for every states i and j there is a strictly positive probability that the chain moves from state i to state j in finitely many steps. A chain is called aperiodic if it can return to any state i at irregular times. See, among others, Norris (1998) and Berman and Plemmons (1994) for an introduction to Markov chains and its terminology. 11The presentation of the maximum likelihood approach follows closely the exposition by Hamilton (1994b; chapter 22) where more details can be found.

366 18 Generalizations of Linear Models

In order to evaluate the likelihood function note that the joint density of .xt; st D j/ may also be factored as

f .xt; st D jjXt�1I �/ D P.st D jjXt I �/ � f .xt jXt�1I �/: Combining these expressions one obtains an expression for the filtered transition probabilities P.st D jjXtI �/:

P.st D jjXtI �/ D f .xtjst D j;Xt�1I �/ � P.st D jjXt�1I �/

f .xtjXt�1I �/

D f .xt jst D j;Xt�1I �/ � P.st D jjXt�1I �/ Pk

jD1 f .xtjst D j;Xt�1I �/ � P.st D jjXt�1I �/ (18.14)

Next period’s predicted transition probabilities are then obtained by multiplication with the transition matrix:

P.stC1 D 1jXtI �/ :::

P.stC1 D kjXt I �/

A D P0 �

P.st D 1jXtI �/ :::

P.st D kjXt I �/

A (18.15)

Given initial probabilities P.s1 D jjX0I �/, j D 1; : : : ; k, and a fixed value for �, Eqs. (18.14) and (18.15) can be iterated forward to produce a sequence of predicted transition probabilities .P.st D 1jXt�1I �/; : : : ; P.st D kjXt�1I �//0, t D 1;2; : : : ; T which can be used to evaluate the Gaussian likelihood function. Numerical procedures must then be used for the maximization of the likelihood function. This task is not without challenge because the likelihood function of Gaussian mixture models typically has singularities and many local maxima. Kiefer (1978) showed that there exists a bounded local maximum which yields a consistent and asymptotically normal estimate for � for which standard errors can be constructed in the usual way. In practice, problems encountered during the maximization can be alleviated by experimentation with alternative starting values. Thereby the initial probability .P.s1 D 1jX0I �/; : : : ; P.s1 D kjX0I �/ could either be treated as additional parameters as in Goldfeld and Quandt (1973) or set to the uniform distribution. For technical details and alternative estimation strategies, like the EM algorithm, see Hamilton (1994b; chapter 22) and in particular Frühwirt- Schnatter (2006).

By reversing the above recursion it is possible to compute smoothed transition probabilities P.st D jjXT I �/ (see Kim 1994):

P.st D jjXT I �/ D P.st D jjXt I �/ k X

iD1 ŒP ij

P.stC1 D ijXT I �/ P.stC1 D ijXtI �/

The iteration is initialized with P.sT D jjXT I �/ which has been computed in the forward recursion.

18.3 Regime Switching Models 367

The basic model can and has been generalized in several dimensions. The most obvious one is the inclusion of additional lags beyond the first one. The second one concerns the possibility of a regime switching covariance matrix †. These modifications can be accommodated using the methods outlined above. Thirdly, one may envision time-varying transition probabilities to account for duration dependence. In business cycle analysis, for example, the probability of moving out of a recession may depend on how long the economy has been in the recession regime. This idea can be implemented by modeling the transition probabilities via a logit specification:

ŒP ij D exp.z0t˛i/

1 C exp.z0t˛i/ i ¤ j

where zt includes a constant and a set of additional variables. These additional variables can be some exogenous variables, but more interestingly may include some lagged variables xt�d (Krolzig 1997). Note that the transition probabilities do not only depend on zt , but also on the state. The resulting model has some features shared with the smooth transition autoregressive model of Granger and Teräsvirta (1993). Early economic applications of regime switching models with time-varying transition probabilities can be found in Diebold et al. (1994), Filardo (1994), and Filardo and F.Gordon (1998).

An important aspect in practice is the determination of the number of regimes. Unfortunately, there is no direct test available for the null hypothesis k D m against the alternative k D m C 1. The reason is that the likelihood contains parameters which are only present under the alternative. The parameters describing the m C 1-th state are unidentified under the null hypothesis. The problem has been analyzed by Andrews and Ploberger (1994) in a general theoretical context. Alternatively, one may estimate the model under the null hypothesis and conduct a series of specification tests as proposed by Hamilton (1996). It has also been suggested to use the information criteria like AIC and BIC to determine the number of regimes (Frühwirt-Schnatter 2006; p. 346–347):

AIC D �2`.�/ C 2k.k � 1/ BIC D �2`.�/ C log.T/k.k � 1/

where k.k � 1/ are the free parameters in the transition matrix P.

AComplex Numbers

The simple quadratic equation x2 C 1 D 0 has no solution in the field of real numbers, R. Thus, it is necessary to envisage the larger field of complex numbers C. A complex number z is an ordered pair .a; b/ of real numbers where ordered means that we regard .a; b/ and .b; a/ as distinct if a ¤ b. Let x D .a; b/ and y D .c; d/ be two complex numbers. Then we endow the set of complex numbers with an addition and a multiplication in the following way:

addition: x C y D .a; b/ C .c; d/ D .a C c; b C d/ multiplication: xy D .a; b/.c; d/ D .ac � bd; ad C bc/:

These two operations will turn C into a field where .0;0/ and .1;0/ play the role of 0 and 1.1 The real numbers R are embedded into C because we identify any a 2 R with .a;0/ 2 C.

The number { D .0;1/ is of special interest. It solves the equation x2 C 1 D 0, i.e. {2 D �1. The other solution being �{ D .0; �1/. Thus any complex number .a; b/ may be written as .a; b/ D a C {b where a; b are arbitrary real numbers.2

1Substraction and division can be defined accordingly:

subtraction: .a; b/ � .c; d/ D .a � c; b � d/

division: .a; b/=.c; d/ D .ac C bd; bc � ad/ .c2 C d2/ ; c

2 C d2 ¤ 0:

2A more detailed introduction of complex numbers can be found in Rudin (1976) or any other mathematics textbook.

369

370 A Complex Numbers

An element z in this field can be represented in two ways:

z D a C {b Cartesian coordinates D re{� D r.cos � C { sin �/ polar coordinates:

In the representation in Cartesian coordinates a D Re.z/ D <.z/ is called the real part whereas b D Im.z/ D =.z/ is called the imaginary part of z.

A complex number z can be viewed as a point in the two-dimensional Cartesian coordinate system with coordinates .a; b/. This geometric interpretation is repre- sented in Fig. A.1.

The absolute value or modulus of z, denoted by jzj, is given by r D p

a2 C b2. Thus, the absolute value is nothing but the distance of z viewed as a point in the complex plane (the two-dimensional Cartesian coordinate system) to the origin (see Fig. A.1). � denotes the angle to the positive real axis (x-axis) measured in radians. It is denoted by � D arg z. It holds that tan � D b

a . Finally, the conjugate of z,

denoted by Nz, is defined by Nz D a � {b. Setting r D 1 and � D �, gives the following famous formula:

e{� C 1 D .cos � C { sin �/ C 1 D �1 C 1 D 0:

This formula relates the most famous numbers in mathematics.

Fig. A.1 Representation of a complex number

−2 −1 0 1 2 −2

−1

real part

im a g in

a ry

p a rt

unit circle: a2 + b2 = 1

b z = a + ib

−b conjugate of z:

a − ib

A Complex Numbers 371

From the definition of complex numbers in polar coordinates, we immediately derive the following implications:

cos � D e {� C e�{� 2

D a r ;

sin � D e {� � e�{� 2{

D b r :

Further implications are de Moivre’s formula and Pythagoras’ theorem (see Fig. A.1):

de Moivre’s formula �

re{� �n D rne{n� D rn.cos n� C { sin n�/

Pythagoras’ theorem 1 D e{� e�{� D .cos � C { sin �/.cos � � { sin �/ D cos2 � C sin2 �

From Pythagoras’ theorem it follows that r2 D a2 C b2. The representation in polar coordinates allows to derive many trigonometric formulas.

Consider the polynomial ˆ.z/ D �0 ��1z ��2z2 �: : :��pzp of order p � 1 with �0 D 1.3 The fundamental theorem of algebra then states that every polynomial of order p � 1 has exactly p roots in the field of complex numbers. Thus, the field of complex numbers is algebraically complete. Denote these roots by 1; : : : ; p, allowing that some roots may appear several times. The polynomial can then be factorized as

ˆ.z/ D �1 � �11 z ��

1 � �12 z �

: : : �

1 � �1p z �

This expression is well-defined because the assumption of a nonzero constant (�0 D 1 ¤ 0) excludes the possibility of roots equal to zero. If we assume that the coefficients �j, j D 0; : : : ; p, are real numbers, the complex roots appear in conjugate pairs. Thus if z D a C {b, b ¤ 0, is a root then Nz D a � {b is also a root.

3The notation with “��jzj” instead of “�jzj ” was chosen to conform to the notation of AR-models.

BLinear Difference Equations

Linear difference equations play an important role in time series analysis. We there- fore summarize the most important results.1 Consider the following linear difference equation of order p with constant coefficients. This equation is defined by the recursion:

Xt D �1Xt�1 C : : : C �pXt�p; �p ¤ 0; t 2 Z:

Thereby fXtg represents a sequence of real numbers and �1; : : : ;�p are p constant coefficients. The above difference equation is called homogeneous because it involves no other variable than Xt. A solution to this equation is a function F W Z ! R such that its values F.t/ or Ft reduce the difference equation to an identity.

It is easy to see that if fX.1/t g and fX.2/t g are two solutions than fc1X.1/t C c2X.2/t g, for any c1; c2 2 R, is also a solution. The set of solutions is therefore a linear space (vector space).

Definition B.1. A set of solutions ffX.1/t g; : : : ; fX.m/t gg, m � p, is called linearly independent if

c1X .1/ t C : : : C cmX.m/t D 0; for t D 0;1; : : : ; p � 1

implies that c1 D : : : D cm D 0. Otherwise we call the set linearly dependent. Given arbitrary starting values x0; : : : ; xp�1 for X0; : : : ; Xp�1, the difference

equation determines all further through the recursion:

Xt D �1Xt�1 C : : : C �pXt�p t D p; p C 1; : : : :

1For more detailed presentations see Agarwal (2000), Elaydi (2005) or Neusser (2009).

373

374 B Linear Difference Equations

Similarly for Xt mit t D �1; �2; : : :. Suppose we have p linearly independent solutions ffX.1/t g; : : : ; fX.p/t gg then there exists exactly p numbers c1; : : : ; cp such that the solution

Xt D c1X.1/t C c2X.2/t C : : : C cpX.p/t is compatible with arbitrary starting values x0; : : : ; xp�1. These starting values then determine uniquely all values of the sequence fXtg. Thus fXtg is the only solution compatible with starting values. The goal therefore consists in finding p linearly independent solutions.

We guess that the solutions are of the form Xt D z�t where z may be a complex number. If this guess is right then we must have for t D 0:

1 � �1z � : : : � �pzp D 0:

This equation is called the characteristic equation.2 Thus z must be a root of the polynomial ˆ.z/ D 1 � �1z � : : : � �pzp. From the fundamental theorem of algebra we know that there are exactly p roots in the field of complex numbers. Denote these roots by z1; : : : ; zp.

Suppose that these roots are different from each other. In this case ffz�t1 g; : : : ; fz�tp gg constitutes a set of p linearly independent solutions. To show this it is sufficient to verify that the determinant of the matrix

W D

1 1 :: : 1

z�11 z �1 2 : : : z

�1 p

z�21 z �2 2 : : : z

�2 p

::: :::

:::

z �pC1 1 z

�pC1 2 : : : z

�pC1 p

is different from zero. This determinant is known as Vandermonde’s determinant and is equal to det W D Q1�i<j�p.zi � zj/. This determinant is clearly different from zero because the roots are different from each other. The general solution to the difference equation therefore is

Xt D c1z�t1 C : : : C cpz�tp (B.1)

where the constants c1; : : : ; cp are determined from the starting values (initial conditions).

In the case where some roots of the characteristic polynomial are equal, the general solution becomes more involved. Let z1; : : : ; zr , r < p, be the roots which

2Sometimes one can find zp � �1zp�1 � : : : � �p D 0 as the characteristic equation. The roots are of the two characteristic equations are then reciprocal to each other.

B Linear Difference Equations 375

are different from each other and denote their corresponding multiplicities by m1; : : : ; mr . It holds that

Pr jD1 D p. The general solution is then given by

Xt D r X

jD1

�

cj0 C cj1t C : : : C cjmj�1 tmj �1 �

z�tj (B.2)

where the constants cji are again determined from the starting values (initial conditions).

CStochastic Convergence

This appendix presents the relevant concepts and theorems from probability theory. The reader interested in more details should consult corresponding textbooks, for example Billingsley (1986), Brockwell and Davis (1991), Hogg and Craig (1995), or Kallenberg (2002) among many others.

In the following, all real random variables or random vectors X are defined with respect to some probability space .�;A; P/. Thereby, � denotes an arbitrary space with �-field A and probability measure P. A random variable, respectively random vector, X is then defined as a measurable function from � to R, respectively Rn. The probability space � plays no role as it is introduced just for the sake of mathematical rigor. The interest rather focuses on the distributions induced by P ı X�1.

We will make use of the following important inequalities.

Theorem C.1 (Cauchy-Bunyakovskii-Schwarz Inequality). For any two random variables X and Y,

jE.XY/j � p EX2

p EY2:

The equality holds if and only if X D E.XY/ E.Y2/

Theorem C.2 (Minkowski’s Inequality). Let X and Y be two random variables with EjXj2 < 1 and EjYj2 < 1, then

�

EjX C Yj2�1=2 � �EjXj2�1=2 C �EjYj2�1=2 :

Theorem C.3 (Chebyschev’s Inequality). If EjXjr < 1 for r � 0 then for every r � 0 and any " > 0

PŒjXj � " � "�rEjXjr:

377

378 C Stochastic Convergence

Theorem C.4 (Borel-Cantelli Lemma). Let A1; A2; : : : 2 A be an infinite sequence of events in some probability space .�;A; P/ such that

P1 kD1 P.Ak/ < 1. Then,

PfAk i:o:g D 0. The event fAk i:o:g is defined by fAk i:o:g D lim supkfAkg D T1

kD1 S1

jDk Aj where i:o: stands for infinitely often.

On several occasions it is necessary to evaluate the limit of a sequence of random variables. In probability theory several concepts of convergence are dis- cussed: almost sure convergence, convergence in probability, convergence in r-th mean (convergence in quadratic mean), convergence in distribution. We only give definitions and the most important theorems leaving an in-depth discussion to the relevant literature. Although not explicitly mentioned, many of the theorem below also hold in an analogous way in a multidimensional context.

Definition C.1 (Almost Sure Convergence). For random variables X and fXtg defined on the same probability space .�;A; P/, we say that fXtg converges almost surely or with probability one to X if

P n

! 2 � W lim t!1 Xt.!/ D X.!/

D 1:

This fact is denoted by Xt a:s:��! X or lim Xt D Xa:s:

Theorem C.5 (Kolmogorov’s Strong Law of Large Numbers (SLLN)). Let X; X1; X2; : : : be identically and independently distributed random variables. Then, the arithmetic average XT D 1T

PT tD1 Xt converges almost surely to EX if and only

if EjXj < 1. Definition C.2 (Convergence in Probability). For random variables X and fXtg defined on the same probability space, we say that fXtg converges in probability to X if

lim t!1 PŒjXt � Xj > " D 0 for all " > 0:

This fact is denoted by Xt p�! X or plim Xt D X.

Remark C.1. If X and fXtg are real valued random vectors, we replace the absolute value in the definition above by the Euclidean norm k:k. This is, however, equivalent to saying that every component Xit converges in probability to Xi, the i-th component of X.

Definition C.3 (Convergence in r-th Mean). A sequence fXtg of random variables converges in r-th mean to a random variable X if

lim t!1 E.jXt � Xj

r/ D 0 for r > 0:

C Stochastic Convergence 379

We denote this fact by Xt r�! X. If r D 1 we say that the sequence converges

absolutely; and if r D 2 we say that the sequence converges in mean square which is denoted by Xt

m:s:��! X.

Remark C.2. In the case r D 2, the corresponding definition for random vectors is

lim t!1 E.kXt � Xk

2 / D lim

t!1 E.Xt � X/ 0 .Xt � X/ D 0:

Theorem C.6 (Riesz-Fisher). Let fXtg be a sequence of random variables such supt EjXtj2 < 1. Then there exists a random variable X with EjXj2 < 1 such that

Xt m:s:��! X if and only if EjXt � Xsj2 ! 0 for t; s ! 1:

This version of the Riesz-Fisher theorem provides a condition, known as the Cauchy criterion, which is often easier to verify when the limit is unknown.

Definition C.4 (Convergence in Distribution). A sequence fXtg of random vectors with corresponding distribution functions fFXt g converges in distribution, if there exists an random vector X with distribution function FX such that

lim t!1 FXt.x/ D FX.x/ for all x 2 C

where C denotes the set of points for which FX.x/ is continuous. We denote this fact by Xt

d�! X. Note that, in contrast to the previously mentioned modes of convergence,

convergence in distribution does not require that all random vectors are defined on the same probability space. The convergence in distribution states that, for large enough t, the distribution of Xt can be approximated by the distribution of X.

The following Theorem relates the four convergence concepts.

Theorem C.7. (i) If Xt a:s:��! X then Xt

p�! X. (ii) If Xt

p�! X then there exists a subsequence fXtn g such that Xtn a:s:��! X.

(iii) If Xt r�! X then Xt

p�! X by Chebyschev’s inequality (Theorem C.3). (iv) If Xt

p�! X then Xt d�! X.

(v) If X is a fixed constant, then Xt d�! X implies Xt

p�! X. Thus, the two concepts are equivalent under this assumption.

380 C Stochastic Convergence

These facts can be summarized graphically:

Xtn a:s:��! X *

Xt a:s:��! X H) Xt

p��! X H) Xt d��! X

* Xt

r��! X

A further useful theorem is:

Theorem C.8. If EXt �! � and VXt �! 0 then Xt m:s:��! � and consequently

Xt p�! �.

Theorem C.9 (Continuous Mapping Theorem). For any continuous function f W Rn �! Rm and random vectors fXtg and X defined on some probability space, the following implications hold:

(i) Xt a:s:��! X implies f .Xt/

a:s:��! f .X/. (ii) Xt

p�! X implies f .Xt/ p�! f .X/.

(iii) Xt d�! X implies f .Xt/

d�! f .X/. An important application of the Continuous Mapping Theorem is the so-called

Delta method which can be used to approximate the distribution of f .Xt/ (see Appendix E).

A further useful result is given by:

Theorem C.10 (Slutzky’s Lemma). Let fXtg and fYtg be two sequences of random vectors such that Xt

d�! X and Yt d�! c, c constant, then

(i) Xt C Yt d��! X C c,

(ii) Y 0t Xt d��! c0X.

(iii) Xt=Yt d��! X=c if c is a nonzero scalar.

Like the (cumulative) distribution function, the characteristic function provides an alternative way to describe a random variable.

Definition C.5 (Characteristic Function). The characteristic function of a real random vector X, denoted by 'X , is defined by

'X.s/ D Ee{ 0 X; 2 Rn; where { is the imaginary unit.

If, for example, X � N.�;�2/, then 'X.s/ D exp.{s� � 12�2s2/. The characteris- tic function uniquely determines the distribution of X. Thus, if two random variables

C Stochastic Convergence 381

have the same characteristic function, they have the same distribution. Moreover, convergence in distribution is equivalent to convergence of the corresponding characteristic functions.

Theorem C.11 (Convergence of Characteristic Functions, Lévy). Let fXtg be a sequence of real random variables with corresponding characteristic functions 'Xt then

Xt d��! X if and only if lim

t!1 'Xt. / D 'X. /; for all 2 R n:

In many cases the limiting distribution is a normal distribution. In which case one refers to the asymptotic normality.

Definition C.6 (Asymptotic Normality). A sequence of random variables fXtg with “means” �t and “variances” �

2 t > 0 is said to be asymptotically normally

distributed if

��1t .Xt � �t/ d��! X � N.0;1/:

Note that the definition does not require that �t D EXt nor that �2t D V.Xt/. Asymptotic normality is obtained if the Xt’s are identically and independently distributed with constant mean and variance. In this case the Central Limit Theorem (CLT) holds.

Theorem C.12 (Central Limit Theorem). Let fXtg be a sequence of identically and independently distributed random variables with constant mean � and constant variance �2 then

p T

XT � � �

d��! N.0;1/;

where XT D T �1 PT

tD1 Xt is the arithmetic average. It is possible to relax the assumption of identically distributed variables in various

ways so that there exists a variety of CLT’s in the literature. For our purpose it is especially important to relax the independence assumption. A natural way to do this is by the notion of m-dependence.

Definition C.7 (m-Dependence). A strictly stationary random process fXtg is called m-dependent for some nonnegative integer m if and only if the two sets of random variables fX�;� � tg and fX�;� � t C m C 1g are independent.

Note that for such processes �.j/ D 0 for j > m. This type of dependence allows to proof the following generalized Central Limit Theorem (see Brockwell and Davis 1991).

Theorem C.13 (CLT for m-Dependent Processes). Let fXtg be a strictly stationary mean zero m-dependent process with autocovariance function �.h/ such that Vm D Pm

hD�m �.h/ ¤ 0 then

382 C Stochastic Convergence

(i) limT!1 TV.XT/ D Vm and (ii)

p TXT is asymptotically normal N.0; Vm/.

Often it is difficult to derive the asymptotic distribution of fXtg directly. This situation can be handled by approximating the original process fXtg by a process fX.m/t g which is easier to handle in terms of its asymptotic distribution and where the precision of the approximation can be “tuned” by the parameter m.

Theorem C.14 (Basis Approximation Theorem). Let fXtg and fX.m/t g be two random vectors process such that

(i) X .m/ t

d��! X.m/ as t ! 1 for each m D 1;2; : : :, (ii) X.m/

d��! X as m ! 1, and (iii) limm!1 lim supt!1 PŒjXt � X.m/t j > D 0 for every > 0.

Then

Xt d��! X as t ! 1:

DBeveridge-Nelson Decomposition

The Beveridge-Nelson decomposition proves to be an indispensable tool. Based on the seminal paper by Phillips and Solo (1992), we proof the following Theorem for matrix polynomials where k:k denotes the matrix norm (see Definition 10.6 in Chapter 10). The univariate version is then a special case with the absolute value replacing the norm.

Theorem D.1. Any a lag polynomial ‰.L/ D P1jD0 ‰jLj where ‰j are n � n matrices with ‰0 D In can be represented by

‰.L/ D ‰.1/ � .In � L/e‰.L/ (D.1)

where e‰.L/ D P1jD0 e‰jL with e‰j D P1

iDjC1 ‰i. Moreover,

1 X

jD1 j2k‰jk2 < 1 implies

1 X

jD0 ke‰jk2 < 1 and k‰.1/k < 1:

Proof. The first part of the Theorem is obtained by the algebraic manipulations below:

‰.L/ � ‰.1/ D In C ‰1L C ‰2L2 C : : : � In � ‰1 � ‰2 � : : :

D ‰1.L � In/ C ‰2.L2 � In/ C ‰3.L3 � In/ C : : : D .L � In/‰1 C .L � In/‰2.L C In/

C .L � In/‰3.L2 C L C In/ C : : :

383

384 D BN-Decomposition

D �.In � L/..‰1 C ‰2 C ‰3 C : : :/ „ ƒ‚ …

e‰0

.‰2 C ‰3 C : : :/ „ ƒ‚ …

e‰1

L C .‰3 C : : :/ „ ƒ‚ …

e‰2

L2 C : : :/

Taking any ı 2 .1=2;1/, the second part of the Theorem follows from

1 X

jD0 ke‰jk2 D

1 X

jD0

�

1 X

iDjC1 ‰j

�

� 1 X

jD0

1 X

iDjC1 k‰jk

D 1 X

jD0

iDjC1 iık‰jki�ı

� 1 X

jD0

1 X

iDjC1 i2ık‰jk2

1 X

iDjC1 i�2ı

� .2ı � 1/�1 1 X

jD0

1 X

iDjC1 i2ık‰ik2

A j1�2ı

D .2ı � 1/�1 1 X

iD0

i�1 X

jD0 j1�2ı

A i2ık‰ik2

� Œ.2ı � 1/.2 � 2ı/ �1 1 X

jD0 j2ık‰ik2j2�2ı

D Œ.2ı � 1/.2 � 2ı/ �1 1 X

jD0 j2k‰ik2 < 1:

The first inequality follows from the triangular inequality for the norm. The second inequality is Hölder’s inequality (see, for example, Naylor and Sell 1982; p. 548) with p D q D 2. The third and the fourth inequality follow from the Lemma below. The last inequality, finally, follows from the assumption.

D BN-Decomposition 385

The last assertion follows from

k‰.1/k � 1 X

jD0 k‰jk D kInk C

1 X

jD1 jk‰jkj�1

� kInk C 0

1 X

jD1 j2k‰jk2

2 0

1 X

jD1 j�2 1

< 1:

The last inequality is again a consequence of Hölder’s inequality. The summability assumption then guarantees the convergence of the first term in the product. Cauchy’s condensation test finally establishes the convergence of the last term. ut

Lemma D.1. The following results are useful:

(i) For any b > 0, P1

iDjC1 i �1�b � b�1j�b.

(ii) For any c 2 .0;1/, PijD1 jc�1 � c�1ic.

Proof. Let k be a number greater than j, then k�1�b � j�1�b and

k�1�b � Z k

k�1 j�1�bdj D b�1.k � 1/�b � b�1k�b:

This implies that P1

kDjC1 k �1�b � b�1j�b. This proves part (i) by changing the

summation index back from k to j. Similarly, kc�1 � jc�1 and

kc�1 � Z k

k�1 jc�1dj D c�1kc � c�1.k � 1/c:

Therefore Pi

kD1 k c�1 � c�1ic which proves part (ii) by changing the summation

index back from k to j. ut

Remark D.1. An alternative common assumption is P1

jD1 jk‰jk < 1. It is, however, easy to see that this assumption is more restrictive as it implies the one assumed in the Theorem, but not vice versa. See Phillips and Solo (1992) for more details.

EThe Delta Method

It is often the case that it is possible to obtain an estimate ǑT of some parameter ˇ, but that one is really interested in a function f of ˇ. The Continuous Mapping Theorem then suggests to estimate f .ˇ/ by f . ǑT/. But then the question arises how the distribution of ǑT is related to the distribution of f . ǑT/.

Expanding the function into a first order Taylor approximation allows to derive the following theorem.

Theorem E.1. Let f ǑT g be a K-dimensional sequence of random variables with the property

p T. ǑT � ˇ/

d��! N.0;†/ then p

T �

f . ǑT/ � f .ˇ/ �

d��! N �0; rf .ˇ/ † rf .ˇ/0� ;

where f W RK �! RJ is a continuously differentiable function with Jacobian matrix (matrix of first order partial derivatives) rf .ˇ/ D @f .ˇ/=@̌ 0 .

Proof. See Serfling (Serfling 1980; 122–124). ut

Remark E.1. In the one-dimensional case where p

T. ǑT � ˇ/ d��! N.0;�2/ and

f W R �! R the above theorem becomes: p

T �

f . ǑT/ � f .ˇ/ �

d��! N �0;Œf 0.ˇ/ 2�2�

where f 0.ˇ/ is the first derivative evaluated at ˇ.

387

388 E The Delta Method

Remark E.2. The J �K Jacobian matrix of first order partial derivatives is defined as

rf .ˇ/ D @f .ˇ/=@̌ 0 D

@f1.ˇ/ @ˇ1

: : : @f1.ˇ/ @ˇK

::: :::

::: @fJ.ˇ/ @ˇ1

: : : @fJ.ˇ/ @ˇK

Remark E.3. In most applications ˇ is not known so that one evaluates the Jacobian matrix at ǑT .

Example: Univariate

Suppose we have obtained an estimate of ˇ equal to Ǒ D 0:6 together with an estimate for its variance O�2Ǒ D 0:2. We can then approximate the variance of f . Ǒ/ D 1= Ǒ D 1:667 by

OV.f . Ǒ// D "

�1 Ǒ2

O�2Ǒ D 1:543:

Example: Multivariate

In the process of computing the impulse response function of a VAR(1) model with

ˆ D �

�11 �12

�21 �22

�

one has to calculate ‰2 D ˆ2. If we stack all coefficients of ˆ into a vector ˇ D vec.ˆ/ D .�11;�21;�12;�22/0 then we get:

f .ˇ/ D vec ‰2 D vec ˆ2 D

.2/ 11

.2/ 21

.2/ 12

.2/ 22

�211 C �12�21 �11�21 C �21�22 �11�12 C �12�22 �12�21 C �222

;

where ‰2 D h

.2/ ij

. The Jacobian matrix then becomes:

rf .ˇ/ D

@ .2/ 11

@�11

@ .2/ 11

@�21

@ .2/ 11

@�12

@ .2/ 11

@�22

@ .2/ 21

@�11

@ .2/ 21

@�21

@ .2/ 21

@�12

@ .2/ 21

@�22

@ .2/ 12

@�11

@ .2/ 12

@�21

@ .2/ 12

@�12

@ .2/ 12

@�22

@ .2/ 22

@�11

@ .2/ 22

@�21

@ .2/ 22

@�12

@ .2/ 22

@�22

2�11 �12 �21 0

�21 �11 C �22 0 �21 �12 0 �11 C �22 �12 0 �12 �21 2�22

E The Delta Method 389

In Section 15.4.4 we obtained the following estimate for a VAR(1) model for fXtg D f.ln.At/; ln.St//0g:

Xt D Oc C Ô Xt�1 C OZt D ��0:141 0:499

�

C �

0:316 0:640

�0:202 1:117 �

Xt�1 C OZt:

The estimated covariance matrix of vec Ô , OV.vec Ô /, was:

OV. Ǒ/ D OV.vec Ô / D

0:0206 0:0069 �0:0201 �0:0067 0:0069 0:0068 �0:0067 �0:0066

�0:0201 �0:0067 0:0257 0:0086 �0:0067 �0:0066 0:0086 0:0085

We can then approximate the variance of f . Ǒ/ D vec.ˆ2/ by OV.f .vec Ô // D OV.vec Ô 2/ D rf .vec ˆ/j

ˆD Ô OV.vec Ô / rf .vec ˆ/j0ˆD Ô :

This leads :

OV.f .vec Ô // D

0:0245 0:0121 �0:0245 �0:0119 0:0121 0:0145 �0:0122 �0:0144

�0:0245 �0:0122 0:0382 0:0181 �0:0119 �0:0144 0:0181 0:0213

Bibliography

Abraham B, Ledolter J (1983) Statistical methods for forecasting. Wiley, New York Adelman I, Adelman FL (1959) The dynamic properties of the Klein-Goldberger model. Econo-

metrica 27:596–625 Agarwal RP (2000) Difference equations and inequalities, 2nd edn. Marcel Dekker, New York Akaike H (1969) Fitting autoregressive models for prediction. Ann Inst Stat Math 21:243–247 Amemiya T (1994) Introduction to statistics and econometrics. Harvard University Press,

Cambridge An S, Schorfheide F (2007) Bayesian analysis of DSGE models. Econ Rev 26:113–172 Anderson BDO, Moore JB (1979) Optimal filtering. Electrical Engeneering Series. Prentice-Hall,

Englewood Cliffs Andrews DWK (1991) Heteroskedasticity and autocorrelation consistent covariance matrix esti-

mation. Econometrica 59:817–858 Andrews DWK (1993) Tests for parameter instability and structural change with unknown change

point. Econometrica 61:821–856 Andrews DWK (2003) Tests for parameter instability and structural change with unknown change

point: A corrigendum. Econometrica 71:395–397 Andrews DWK, Monahan JC (1992) An improved heteroskedasticity and autocorrelation consis-

tent covariance matrix estimator. Econometrica 60:953–966 Andrews DWK, Ploberger W (1994) Optimal tests when a nuisance parameter is present only

under the alternative. Econometrica 62:1383–1414 Arias JE, Rubio-Ramírez J, Waggoner DF (2014) Inference based on SVARs identified with sign

and zero restrictions: Theory and applications. FRB Atlanta Working Paper 2014–1, Federal Reserve Bank of Atlanta

Ashley R, Granger CWJ, Schmalensee R (1980) Advertising and aggregate consumption: An analysis of causality. Econometrica 48(5):1149–1168

Aue A, Horváth L (2011) Structural breaks in time series. J Time Ser Anal 34:1–16 Bai J (2000) Vector autoregressive models with structural changes in regression coefficients and in

variance-covariance matrices. Ann Econ Finance 1:303–339 Bai J, Lumsdaine RL, Stock JH (1998) Testing for and dating common breaks in multivariate time

series. Rev Econ Stud 65:395–432 Baker A (2002) Matrix groups – an introduction to Lie Group theory. Springer, London Banbura M, Giannone D, Reichlin L (2010) Large Bayesian vector autoregressions. J Appl Econ

25:71–92 Banerjee A, Dolado J, Galbraith JW, Hendry DF (1993) Co-integration, error-correction, and the

econometric analysis of non-stationary data. Oxford University Press, Oxford Barsky R, Sims E (2011) News shocks and business cycles. J Monet Econ 58:273–289 Bauer D, Wagner M (2003) A canonical form for unit root processes on the state space framework,

diskussionsschrift 3-12, Volkswirtschaftliches Institut, Universität Bern

391

392 Bibliography

Baumeister C, Hamilton JD (2015) Sign restrictions, structural vector autoregressions, and useful prior information. Econometrica 83:1963–1999

Beaudry P, Portier F (2006) Stock prices, news and economic fluctuations. Am Econ Rev 96(4):1293–1307

Berman A, Plemmons RJ (1994) Nonnegative matrices in the mathematical sciences. No. 9 in Classics in Applied Mathematics, Society of Industrial and Applied Mathematics, Philadelphia

Bernanke BS (1986) Alternative explanations of money-income correlation. In: Brunner K, Meltzer A (eds) Real business cycles, real exchange rates, and actual policies, no. 25 in Carnegie-Rochester Conference Series on Public Policy. North-Holland, Amsterdam, pp 49–100

Bernanke BS, Gertler M, Watson MW (1997:1) Systematic monetary policy and the effects of oil price shocks. Brook Pap Econ Act 91–142

Berndt ER (1991) The practice of econometrics. Addison Wesley, Reading, Bhansali RJ (1999) Parameter estimation and model selection for multistep prediction of a time

series: A review. In: Gosh S (ed) Asymptotics, Nonparametrics, and Time Series. Marcel Dekker, New York, pp 201–225

Billingsley P (1986) Probability and measure, 2nd edn. Wiley, New York Blanchard OJ (1989) A traditional interpretation of macroeconomic fluctuations. Am Econ Rev

79:1146–1164 Blanchard OJ, Quah D (1989) The dynamic effects of aggregate demand and supply disturbances.

Am Econ Rev 79:655–673 Blanchard OJ, Watson MW (1986) Are business cycles all alike? In: Gordon R (ed) The American

business cycle: continuity and change. University of Chicago Press, Chicago, pp 123–179 Bollerslev T (1986) Generalized autoregressive conditional heteroskedasticity. J Econ 31:307–327 Bollerslev T (1988) On the correlation structure for the generalized autoregressive conditional

heteroskedastic process. J Time Ser Anal 9:121–131 Bollerslev T, Engle RF, Nelson DB (1994) ARCH models. In: Engle RF, McFadden DL (eds)

Handbook of econometrics, vol IV. Elsevier Science B.V., Amsterdam, pp 2959–3038 Bougerol P, Picard N (1992a) Stationarity of GARCH processes and some nonnegative time series.

J Econ 52:115–127 Bougerol P, Picard N (1992b) Strict stationarity of generalized autoregressive processes. Ann

Probab 20:1714–1730 Box GEP, Jenkins GM (1976) Time series analysis: forecasting and control, revised edn. Holden-

Day, San Francisco Brandner P, Neusser K (1992) Business cycles in open economies: Stylized facts for Austria and

Germany. Weltwirtschaftliches Arch 128:67–87 Brandt A (1986) The stochastic equation ynC1 D anyn C bn with stationary coefficients. Adv Appl

Probab 18:211–220 Bräuning F, Koopman SJ (2014) Forecasting macroeconomic variables using collapsed dynamic

factor analysis. Int J Forecast 30:572–584 Breitung J, Eickmeier S (2006) Dynamic factor models. In: Hübler O, Frohn J (eds) Modern

econometric analysis, chap 3. Springer, Berlin, pp 25–40 Brockwell PJ, Davis RA (1991) Time series: theory and methods, 2nd edn. Springer, New York Brockwell PJ, Davis RA (1996) Introduction to time series and forecasting. Springer, New York Brualdi RA, Shader BL (1995) Matrices of sign-solvable linear systems. No. 116 in Cambridge

tracts in mathematics. Cambridge University Press, Cambridge Burren D, Neusser K (2013) The role of sectoral shifts in the decline of real GDP volatility.

Macroecon Dyn 17:477–500 Campbell JY (1987) Does saving anticipate declining labor income? An alternative test of the

permanent income hypothesis. Econometrica 55:1249–1273 Campbell JY, Mankiw NG (1987) Are output fluctuations transitory? Q J Econ 102:857–880 Campbell JY, Perron P (1991) Pitfalls and opportunities: What macroeconomists should know

about unit roots. In: Blanchard OJ, Fischer S (eds) Macroeconomics annual 1991, vol 6. MIT Press, Cambridge, pp 141–201

Bibliography 393

Campbell JY, Shiller RJ (1987) Cointegration and tests of present value models. J Polit Econ 95:1062–1088

Campbell JY, Lo AW, MacKinlay AC (1997) The econometrics of financial markets. Princeton University Press, Princeton

Canova F (2007) Methods for applied macroeconomic research. Princeton University Press, Princeton

Canova F, Ciccarelli M (2008) Estimating multi-country VAR models. Working Paper 603, European Central Bank

Canova F, De Nicoló G (2002) Monetary disturbances matter for business fluctuations in the G–7. J Monet Econ 49:1131––1159

Cavaliere G, Rahbek A, Taylor AMR (2012) Bootstrap determination of the co-integration rank in vector autoregressive models. Econometrica 80:1721–1740

Chan JCC, Jeliazkov I (2009) Efficient simulation and integrated likelihood estimation in state space models. Int J Math Model Numer Optim 1:101–120

Chari VV, Kehoe PJ, McGrattan ER (2008) Are structural VARs with long-run restrictions useful in developing business cycle theory? J Monet Econ 55:1337–1352

Chow GC, Lin A (1971) Best linear unbiased interpolation, distribution, and extrapolation of time series by related series. Rev Econ Stat 53:372–375

Christiano LJ, Eichenbaum M (1990) Unit roots in real GNP: Do we know and do care? Carn-Roch Conf Ser Public Pol 32:7–62

Christiano LJ, Eichenbaum M, Evans CL (1999) Monetary policy shocks: what have we learned and to what end? Handbook of macroeconomics, vol 1A, chap 2. North-Holland, Amsterdam, pp 65–148

Christiano LJ, Eichenbaum M, Vigfusson RJ (2003) What happens after a technology shock? Working Paper No. 9819, NBER

Christiano LJ, Eichenbaum M, Vigfusson RJ (2006) Assessing structural VARs. International Finance Discussion Papers No. 866, Board of Governors of the Federal Reserve System

Christoffersen PF (1998) Evaluating interval forecasts. Int Econ Rev 39:841–862 Clements MP, Hendry DF (1996) Intercept corrections and structural change. J Appl Econ

11:475–494 Clements MP, Hendry DF (2006) Forecasting with breaks. In: Handbook of economic forecasting,

vol 1, Elsevier, Amsterdam, pp 605–657 Cochrane JH (1988) How big is the random walk in GNP? J Polit Econ 96(5):893–920 Cogley T, Sargent TJ (2001) Evolving post-world war II U.S. inflation dynamics. In: Bernanke BS,

Rogoff K (eds) NBER macroeconomics annual, vol 16. MIT Press, Cambridge, pp 331–373 Cogley T, Sargent TJ (2005) Drift and volatilities: Monetary policies and outcomes in the post

WWII U.S. Rev Econ Dyn 8:262–302 Colonius F, Kliemann W (2014) Dynamical systems and linear algebra. Graduate studies in

mathematics, vol 158. American Mathematical Society, Providence Colonius F, Kliemann W (2014) Dynamical Systems and Linear Algebra. Graduate Studies in

Mathematics, Vol. 158. American Mathematical Society, Providence, Rhode Island Cooley TF, LeRoy SF (1985) Atheoretical macroeconometrics - a critique. J Monet Econ

16:283–308 Cooley TF, Prescott EC (1973) Varying parameter regression. A theory and some applications.

Ann Econ Soc Meas 2:463–474 Cooley TF, Prescott EC (1976) Estimation in the presence of stochastic parameter variation.

Econometrica 44:167–184 Corradi V, Swanson NR (2006) Predictive density evaluation. In: Handbook of economic forecast-

ing, vol 1, Elsevier, Amsterdam, pp 197–284 Cuche NA, Hess MA (2000) Estimating monthly GDP in a general Kalman filter framework:

Evidence from Switzerland. Econ Financ Model 7:153–194 Davidson JEH, Hendry DF, Srba F, Yeo S (1978) Econometric modelling of the aggregate time-

series relationship between consumers’ expenditure and income in the United Kingdom. Econ J 88:661–692

394 Bibliography

Davidson R, MacKinnon JG (1993) Estimation and inference in econometrics. Oxford University Press, Oxford

Dees S, Mauro FD, Pesaran MH, Smith V (2007) Exploring the international linkages of the Euro area: A global VAR analysis. J Appl Econ 22:1–38

Deistler M, Neusser K (2012) Prognosen uni- und multivariater Zeitreihen. In: Mertens P (ed) Prognoserechnung. Physica-Verlag, Heidelberg, pp 225–256

Dejong DN, Dave C (2007) Structural macroeconomics. Princeton University Press, Princeton Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood estimation from incomplete data

via the EM algorithm. J R Stat Soc Ser B 39:1–38 Dhrymes PJ (1978) Introductory econometrics. Springer, New York Dickey D, Fuller WA (1976) Distribution of the estimators for autoregressive time series with a

unit root. J Am Stat Assoc 74:427–431 Dickey DA, Fuller WA (1981) Likelihood ratio statistics for autoregressive time series with a unit

root. Econometrica 49:1057–1072 Diebold FX, Lee JH, Weinbach GC (1994) Regime switching with time-varying transition

probabilities. In: Hargreaves CP (ed) Nonstationary time series analysis and cointegration. Oxford Universiy Press, Oxford, pp 283–302

Diebold FX, Gunther TA, Tay AS (1998) Evaluating density forecasts, with applications to financial risk management. Int Econ Rev 39:863–883

Doan T, Litterman RB, Sims CA (1984) Forecasting and conditional projection using realistic prior distributions. Econ Rev 3:1–100

Dufour JM (1985) Unbiasedness of predictions from estimated autoregressions. Econ Theory 1:387–402

Durbin J (1960) The fitting of time series models. Revue de l’Institut International de Statistique 28:233–244

Durbin J, Koopman SJ (2011) Time series analysis by state space methods, 2nd edn. Oxford University Press, Oxford

Edelman A, Rao NR (2005) Random matrix theory. Acta Numerica 14:1–65 Efron B, Tibshirani RJ (1993) An introduction to the bootstrap. Monographs on statistics and

applied probability, vol 57. Chapman & Hall/CRC, Boca Raton Elaydi S (2005) An introduction to difference equations, 3rd edn. Springer Science + Business

Media, New York Elder J, Kennedy PE (2001) Testing for unit roots: What should students be taught? J Econ Educ

32:137–146 Elliott G, Timmermann A (2008) Economic forecasting. J Econ Lit 46:3–56 Engle RF (1982) Autoregressive conditional heteroskedasticity with estimates of the variance of

the United Kingdom inflation. Econometrica 50:987–1007 Engle RF (2004) Risk and volatility: Econometric models and financial practice. Am Econ Rev

94:405–420 Engle RF, Bollerslev T (1986) Modeling the persistence of conditional variances. Econ Rev 5:1–50 Engle RF, Granger CWJ (1987) Co-integration and error correction: Representation, estimation

and testing. Econometrica 55:251–276 Engle RF, Lilien D, Robins R (1987) Estimating time varying risk premia in the term structure:

The ARCH–M model. Econometrica 55:391–407 Engsteg T, Pedersen TQ (2014) Bias–correction in vectorautoregressive models: A simulation

study. Econometrics 2:45–71 Epstein RJ (1987) A history of econometrics. North-Holland, Amsterdam Eurostat (2009) ESS guidelines on seasonal adjustment. Eurostat, Luxembourg Fan J, Yao Q (2003) Nonlinear time series. Springer, New York Faust J (1998) The robustness of identified VAR conclusions about money. Carn-Roch Conf Ser

Publis Policy 49:207–244 Fernandez-Villaverde J, Rubio-Ramírez JF, Sargent TJ, Watson MW (2007) ABCs and (Ds) of

understanding VARs. Am Econ Rev 97:1021–1026

Bibliography 395

Filardo AJ (1994) Business-cycle phases and their transitional dynamics. J Bus Econ Anal 12:299–308

Filardo AJ, FGordon S (1998) Business cycle duration. J Econ 85:99–123 Francis N, Owyang MT, Roush JE, DiCecio R (2014) A flexible finite-horizon alternative to long-

run restrictions with an application to technology shocks. Rev Econ Stat 96:638–647 Friedman M, Schwartz AJ (1963) A monetary history of the United States, 1867–1960. Priceton

University Press, Princeton Frisch R (1933) Propagation problems and impulse problems in dynamic economics. In: Economic

essays in honour of Gustav cassel. Frank Cass, London, pp 171–205 Frühwirt-Schnatter S (2006) Finite mixture and markov switching models. Springer Science +

Business Media LLC, New York Fry RA, Pagan AR (2011) Sign restrictions in structural vector autoregressions: A critical review.

J Econ Lit 49:938–960 Fuller WA (1976) Introduction to statistical time series. Wiley, New York Galí J (1992) How well does the IS-LM model fit postwar U.S. data? Q J Econ 107(2):709–738 Galí J (1999) Technology, employment, and the business cycle: Do technology shocks explain

aggregate fluctuations? Am Econ Rev 89:249–271 Geweke JF (1984) Inference and causality in economic time series models. In: Griliches Z,

Intriligator MD (eds) Handbook of econometrics, vol II. Elsevier, Amsterdam, pp 1101–1144 Geweke JF (2005) Contemporary bayesian econometrics and statistics. Wiley series in probability

and statistics. Wiley, New York Ghysels E, Osborn DR (2001) The econometric analysis of seasonal time series. Cambridge

University Press, Cambridge Giannini C (1991) Topics in structural VAR econometrics. Quaderni di Ricerca 21, Università degli

Studi di Anacona, Dipartimento di Economia Giraitis L, Kokoszka P, Leipus R (2000) Stationary ARCH models: Dependence structure and

central limit theorem. Econ Theory 16:3–22 Glosten LR, Jagannathan R, Runkle DE (1993) On the relation between expected value and the

volatility of the nominal excess returns on stocks. J Finance 48:1779–1801 Gohberg I, Lancaster P, Rodman L (1982) Matrix polynomials. Academic Press, New York Goldfeld SM, Quandt RE (1973) A Markov model for switching regressions. J Econ 1:3–15 Goldfeld SM, Quandt RE (1976) Studies in nonlinear estimation. Ballinger Publishing, Cambridge Gómez V, Maravall A (1996) Programs TRAMO and SEATS. Instructions for the user (with some

updates). Working Paper 9628, Servicio de Estudios, Banco de España Gonzalo J, Ng S (2001) A systematic framework for analyzing the dynamic effects of permanent

and transitory schocks. J Econ Dyn Control 25:1527–1546 Gospodinov N (2010) Inference in nearly nonstationary SVAR models with long-run identifying

restrictions. J Bus Econ Stat 28:1–12 Gouriéroux C (1997) ARCH models and financial applications. Springer, New York Gouriéroux C, Jasiak J, Sufana R (2009) The Wishart autoregressive process of multivariate

stochastic volatility. J Econ 150:167–181 Granger CWJ (1964) Spectral analysis of economic time series. Princeton University Press,

Princeton Granger CWJ (1966) The typical spectral shape of an economic variable. Econometrica

34:150–161 Granger CWJ (1969) Investigating causal relations by econometric models and cross-spectral

methods. Econometrica 37(3):424–438 Granger CWJ, Newbold P (1974) Spurious regression in econometrics. J Econ 2:111–120 Granger CWJ, Teräsvirta T (1993) Modelling nonlinear economic relationships. Oxford University

Press, Oxford Greene WH (2008) Econometric anlysis, 7th edn. Prentice Hall, New Jersey Haan WJ, Levin AT (1997) A practitioner’s guide to robust covariance matrix estimation.

In: Maddala GS, Rao CR (eds) Handbook of statistics: robust inference, vol 15. Elsevier, New York, pp 299–342

396 Bibliography

Hall FJ, Li Z (2014) Sign pattern matrices. In: Hogben L (ed) Handbook of linear algebra, 2nd edn, chap 42. Chapman & Hall/CRC, Boca Raton, pp 1–32

Hall P, Yao Q (2003) Inference in ARCH and GARCH models with heavy-tailed errors. Econometrica 71(1):285–317

Hall RE (1978) Stochastic implications of the life cycle-permanent income hypothesis: Theory and evidence. J Polit Econ 86:971–987

Hamilton JD (1994a) State-Space models. In: Engle RF, McFadden DL (eds) Handbook of econometrics, vol 4, chap 50. Elsevier, Amsterdam, pp 3039–3080

Hamilton JD (1994b) Time series analysis. Princeton University Press, Princeton Hamilton JD (1996) Specification testing in Markov-switching time-series models. J Econ

70:127–157 Hannan EJ, Deistler M (1988) The statistical theory of linear systems. Wiley, New York Hansen BE (1992) Efficient estimation and testing of cointegrating vectors in the presence of

deterministic trends. J Econ 53:321–335 Hansen LP, Sargent TJ (1980) Formulating and estimating dynamic linear rational expectations

models. J Econ Dyn Control 2:7–46 Hansen LP, Sargent TJ (1991) Two difficulties in interpreting vector autoregressions. In: Hansen

LP, Sargent TJ (eds) Rational expectations econometrics, underground classics in economics. Westview, Boulder, Colorado, pp 77–119

Hansen LP, Sargent TJ (1993) Saesonality and approximation errors in rational expectations models. J Econ 55:21–55

Harvey AC (1989) Forecasting, structural time series models and the kalman filter. Cambridge University Press, Cambridge

Harvey AC, Jaeger A (1993) Detrending, stylized facts and the business cycle. J Appl Econ 8:231–247

Harvey AC, Phillips GD (1982) Estimation of regression models with time varying parameters. In: Deistler M, Fürst E, Schödiauer G (eds) Games, economic dynamics and time series analysis. Physica-Verlag, Wien-Würzburg, pp 306–321

Harvey AC, Pierce RG (1984) Estimating missing observations in economic time series. J Am Stat Assoc 79:125–131

Haugh LD (1976) Checking the independence of two covariance stationary time series: A univari- ate residual cross-correlation approach. J Am Stat Assoc 71:378–385

Hauser MA, Pötscher BM, Reschenhofer E (1999) Measuring persistence in aggregate output: ARMA models, fractionally integrated ARMA models and nonparametric procedures. Empir Econ 24:243–269

Hildreth C, Houck JP (1968) Some estimators for a linear model with random coefficients. J Am Stat Assoc 63:584–595

Hodrick RJ, Prescott EC (1980) Post-war U.S. business cycles: An empirical investigation. Discussion Paper 451, Carnegie-Mellon University, Pittsburgh

Hogg RV, Craig AT (1995) Introduction to mathematical statistics, 5th edn. Prentice-Hall, Upper Saddle River

Hong EP (1991) The autocorrelation structure for the GARCH-M process. Econ Lett 37:129–132 Howrey EP (1968) A spectrum analysis of the long swing hypothesis. Int Econ Rev 9:228–252 Hylleberg S (1986) Seasonality in regression. Academic Press, Orlando, FL Hylleberg S, Engle RF, Granger CWJ, Yoo S (1990) Seasonal integration and cointegration. J Econ

44:215–238 Inoue A, Kilian L (2013) Inference on impulse response functions in structural VAR models. J Econ

177:1–13 Jensen ST, Rahbek A (2004) Asymptotic normality for non-stationary, explosive GARCH. Econ

Theory 20(6):1203–1226 Johansen S (1988) Statistical analysis of cointegration vectors. J Econ Dyn Control 12:231–254 Johansen S (1991) Estimation and hypothesis testing of cointegration vectors in Gaussian vector

autoregressive models. Econometrica 59:1551–1580

Bibliography 397

Johansen S (1995) Likelihood-based inference in cointegrated vector autoregressive models. Oxford University Press, Oxford

Johansen S (2007) Cointegration: A survey. In: Mills TC, Patterson K (eds) Econometric theory, palgrave handbook of econometrics, vol 1. Palgrave MacMillan, Basingstoke and New York, pp 540–577

Johansen S, Schaumburg E (1998) Likelihood analysis of seasonal cointegration. J Econ 88:301–339

Kailath T (1980) Linear systems. Prentice Hall, Englewood Cliffs Kallenberg O (2002) Foundations of modern probability, 2nd edn. Probability and its applications,

Springer, New York Kalman RE (1960) A new approach to linear filtering and prediction problems. J Basic Eng Trans

ASME Ser D 82:35–45 Kalman RE (1963) New methods in Wiener filtering theory. In: Bogdanoff JL, Kozin F (eds)

Proceedings of the first symposium of engeneering applications of random function theory and probability. Wiley, New York, pp 270–388

Kendall MG (1954) Note on the bias in the estimation of autocorelation. Biometrika 41:403–404 Kiefer NM (1978) Discrete parameter variation: Efficient estimation of a switching regression

model. Econometrica 46:427–434 Kilian L (1998) Small-sample confidence intervals for impulse response functions. Rev Econ Stat

80:186–201 Kilian L (2013) Structural vector autoregressions. In: Hashimzade N, Thornton M (eds) Handbook

of research methods and applications in empirical macroeconomics. Edward Elgar, Cheltenham, pp 515–554

Kilian L, Murphy DP (2012) Why agnostic sign restrictions are not enough: Understanding the dynamics of oil market VAR models. J Eur Econ Assoc 10:1166–1188

Kim CJ (1994) Dynamic linear models with Markov-switching. J Econ 60:1–22 Kim CJ, Nelson CR (1999) State-space models with regime-switching: classical and Gibbs-

sampling approaches with applications. MIT Press, Cambridge King RG, Rebelo ST (1993) Low frequency filtering and real business cycles. J Econ Dyn Control

17:201–237 King RG, Plosser CI, Rebelo ST (1988) Production, growth, and business cycles: I. The basic

neoclassical model. J Monet Econ 21:195–232 King RG, Plosser CI, Stock JH, Watson MW (1991) Stochastic trends and economic fluctuations.

Am Econ Rev 81:819–840 Klein LR (1950) Economic fluctuations in United States 1921–1941. Wiley, New York Klein LR (1985) New developments in project LINK. Am Econ Rev 75:223–227 Klein LR, Goldberger AS (1955) An econometric model of the United States, 1929–1952. North

Holland, Amsterdam Klüppelberg C, Lindner A, Maller R (2004) A continuous time GARCH process driven by a Lévy

process: Stationarity and second order behaviour. J Appl Probab 41:601–622 Koop G, Korobilis D (2009) Bayesian multivariate time series methods for empirical macroeco-

nomics. Found Trends Econ 3:267—-358 Kristensen D, Linton O (2006) A closed-form estimator for the GARCH(1,1)-model. Econ Theory

22:323–337 Krolzig HM (1997) Markov–switching vector autoregressions. Modelling, statistical inference, and

application to business cycle analysis. Springer, Berlin Kunst R, Neusser K (1986) A forecasting comparison of some var techniques. Int J Forecast

2:447–456 Kunst R, Neusser K (1990) Cointegration in a macroeconomic system. J Appl Econ 5:351–365 Kuznets S (1930) Secular movements in production and prices. Their nature and their bearing upon

cyclical fluctuations. Houghton Mifflin, Boston Kwiatkowski D, Phillips PC, Schmidt P, Shin Y (1992) Testing the null hypothesis of stationarity

against the alternative of a unit root. How sure are we that economic time series have a unit root? J Econ 54:159–178

398 Bibliography

Leamer EE (1981) Is it a demand curve, or is it a supply curve? Partial identification through inequality constraints. Rev Econ Stat 63:319–327

Lee SW, Hansen BE (1994) Asymptotic theory for the GARCH(1,1) quasi-maximum likelihood estimator. Econ Theory 10:29–52

Lindner AM (2009) Stationarity, mixing, distributional properties and moments of GARCH(p,q)- processes. In: Anderson TG, Davis RA, Kreiß JP, Mikosch T (eds) Handbook of financial time series. Springer, Berlin

Lippi M, Reichlin L (1993) The dynamic effects of aggregate demand and supply disturbances: Comment. Am Econ Rev 83:644–652

Litterman RB (1986) Forecasting with Bayesian vector autoregressions: Five years of experience. J Bus Econ Stat 4:25–38

Ljung L (1999) System identification: theory for the user, 2nd edn. Prentice Hall, Englewood Cliffs Lucas RE (1976) Econometric policy evaluation: A critique. In: Brunner K, Meltzer AH (eds) The

phillips curve and labor markets. Carnegie-Rochester Conference Series on Public Policy, vol 1. North-Holland, Amsterdam, pp 19–46

Lumsdaine RL (1986) Consistency and asymptotic normality of the quasi-maximum likelihood estimator in IGARCH(1,1) and covariance stationary GARCH(1,1) models. Econometrica 64:575–596

Lütkepohl H (1990) Asymptotic distributions of impulse response functions and forecast error variance decomposition. Rev Econ Stat 72:116–125

Lütkepohl H (2006) New introduction to multiple time series analysis. Springer, Berlin Lütkepohl H, Staszewska-Bystrova A, Winker P (2013) Comparison of methods for constructing

joint confidence bands for impulse response functions. Discussion Paper 1292, DIW Berlin MacKinnon JG (1991) Critical values for co-integration tests. In: Engle RF, Granger CWJ (eds)

Long-run economic relationships. Oxford University Press, Oxford, pp 267–276 MacKinnon JG (1996) Numerical distribution functions for unit root and cointegration tests. J Appl

Econ 11:601–618 MacKinnon JG, Smith AA Jr (1998) Approximate bias correction in econometrics. J Econ 85:

205–230 MacKinnon JG, Haug A, Michelis L (1999) Numerical distribution functions of the likelihood ratio

test for cointegration. J Appl Econ 14:563–577 Maddala GS (1986) Disequilibrium, self-selection, and switching models. In: Griliches Z, Intrili-

gator MD (eds) Handbook of econometrics, vol III. North-Holland, Amsterdam Magnus JR, Neudecker H (1988) Matrix differential calculus with applications in statistics and

econometrics. Wiley, Chichester Marcellino M, Stock JH, Watson MW (2006) A comparison of direct and iterated multistep AR

methods for forecasting macroeconomic time series. J Econ 135:499–526 Marriott FHC, Pope JA (1954) Bias in the estimation of autocorrelation. Biometrika 41:393–402 Mertens P, Rässler S (2005) Prognoserechnung - einführung und Überblick. In: Mertens P, Rässler

S (eds) Prognoserechnung. Physica-Verlag, Heidelberg, pp 1–37 Meyer CD (2000) Matrix analysis and applied linear algebra. Society for Industrial and Applied

Mathematics, Philadelphia Mills TC (2003) Modelling trends and cycles in economic time series. Palgrave texts in economet-

rics. Palgrave Macmillan, Hampshire Miron JA (1996) The economics of seasonal cycles. MIT Press, Cambridge Mittnik S, Zadrozny PA (1993) Asymptotic distributions of impulse responses, step responses and

variance decompositions of estimated linear dynamic models. Econometrica 61:857–870 Moon HR, Schorfheide F (2012) Bayesian and frequentist inference in partially identified models.

Econometrica 80:755–782 Moon HR, Schorfheide F, Granziera E (2013) Inference for VARs identified with sign restrictions,

working paper Mountford A, Uhlig H (2009) What are the effects of fiscal policy shocks? J Appl Econ 24:960–992 Muth J (1960) Optimal properties of exponentially weighted forecasts. J Am Stat Assoc

55:299–306

Bibliography 399

Naylor AW, Sell GR (1982) Linear operator theory in engeneering and science, applied mathemat- ical sciences, vol 40. Springer, New York

Negro Md, Primiceri GE (2015) Time varying structural vector autoregressions and monetary polic: A corrigendum. Rev Econ Stud, forthcoming

Negro Md, Schorfheide F (2004) Priors from general equilibrium models for VARs. Int Econ Rev 45:643–673

Nelson CR, Plosser CI (1982) Trends and random walks in macro-economic time series: Some evidence and implications. J Monet Econ 10:139–162

Nelson DB (1990) Stationarity and persistence in the GARCH(1,1) model. Econ Theory 6:318–334 Nelson DB (1991) Conditional heteroskedasticity in asset returns: A new approach. Econometrica

59:347–370 Neusser K (1991) Testing the long-run implications of the neoclassical growth model. J Monet

Econ 27:3–37 Neusser K (2000) An algebraic interpretation of cointegration. Econ Lett 67:273–281 Neusser K (2009) Difference equations for economists. http://www.neusser.ch/downloads/

DifferenceEquations.pdf Neusser K (2016) A topological view on the identification of structural vector autoregressions.

Econ. Lett. (forthcoming) Neusser K, Kugler M (1998) Manufacturing growth and financial development: Evidence from

oecd countries. Rev Econ Stat 80:638–646 Newey WK, West KD (1994) Automatic lag selection in covariance matrix estimation. Rev Econ

Stud 61:631–653 Ng S, Perron P (1995) Unit root tests in ARMA models with data dependent methods for the

selection of the truncation lag. J Am Stat Assoc 90:268–281 Nicholls DF, Pagan AR (1984) Estimating predictions, prediction errors and their standard

deviations using constructed variables. J Econ 24:293–310 Norris JR (1998) Markov chains. Cambridge series in statistical and probabilistic mathematics.

Cambridge University Press, Cambridge Ogaki M (1992) An introduction to the generalized method of moments. Working Paper No. 314,

University of Rochester Orcutt GH, Winokur HS Jr (1969) First order autoregression inference, estimation, and prediction.

Econometrica 37:1–14 Osterwald-Lenum M (1992) A note with quantiles of the asymptotic distribution of the maximum

likelihood cointegration rank test statistics. Oxford Bull Econ Stat 54:461–471 Pagan AR (1984) Econometric issues in the analysis of regressions with generated regressors. Int

Econ Rev 25:183–209 Pagan AR, Robertson OC (1998) Structural models of the liquidity effect. Rev Econ Stat

80:202–217 Perron P (1989) The great crash, the oil price shock, and the unit root hypothesis. Econometrica

57:1361–1401 Perron P (2006) Dealing with structural breaks. In: Hassani H, Mills TC, Patterson K (eds) Pal-

grave handbook in econometrics, vol 1. Econometric theory. Palgrave Macmillan, Hampshire, pp 278–352

Phillips PC (1987) Time series regression with a unit root. Econometrica 55:277–301 Phillips PCB (1986) Understanding spurious regressions in econometrics. J Econ 33:311–340 Phillips PCB (1991) Optimal inference in cointegrating systems. Econometrica 59:283–306 Phillips PCB (2004) HAC estimation by automated regression. Cowles Foundation Discussion

Paper 1470, Yale University Phillips PCB, Hansen BE (1990) Statistical inference in instrumental variables regresssion with

I(1) processes. Rev Econ Stud 57:99–125 Phillips PCB, Ouliaris S (1990) Asymptotic properties of residual based tests of cointegration.

Econometrica 58(1):165–193 Phillips PCB, Perron P (1988) Testing for a unit root in time series regression. Biometrika

75:335–346

http://www.neusser.ch/downloads/DifferenceEquations.pdf

400 Bibliography

Phillips PCB, Solo V (1992) Asymptotics for linear processes. Ann Stat 20:971–1001 Phillips PCB, Sul D (2007) Some empirics on economic growth under heterogeneous technology.

J Macroecon 29:455–469 Pierce DA, Haugh LD (1977) Causality in temporal systems - characterization and survey. J Econ

5:265–293 Potter SM (2000) Nonlinear impulse response functions. J Econ Dyn Control 24:1425–1446 Press H, Tukey JW (1956) Power spetral methods of analysis and their application to problems in

airplane dynamics. In: Flight Test Manual, NATO Advisory Group for Aeronautical Research and Development, pp 1–41

Priestley MB (1981) Spectral analysis and time series, vol 1&2. Academic Press, London Primiceri GE (2005) Time varying structural vector autoregressions and monetary policy. Rev Econ

Stud 72:821–852 Quah D (1990) Permanent and transitory movements in labor income: An explanation for ‘excess

smoothnes’ in consumption. J Polit Econ 98:449–475 Quah D, Sargent TJ (1993) A dynamic index model for large cross sections. In: Stock JH, Watson

MW (eds) Business cycles, indicators, and forecasting, Chapter 7. University of Chicago Press, Chicago

Quandt RE (1960) Tests of the hypothesis that a linear regression system obeys two separate regimes. J Am Stat Assoc 55:324–330

Reichlin L (2003) Factor models in large cross sections of time series. In: Dewatripont M, Hansen LP, JTurnovsky S (eds) Advances in econometrics, theory and applications, econometric society monographs, vol III. Cambridge University Press, Cambridge, pp 47—86

Reinsel GC (1993) Elements of multivariate time series analysis. Springer Series in statistics. Springer, New York

Rigobon R (2003) Identification through heteroskedasticity. Rev Econ Stat 85:777–792 Robinson EA (1982) A historical perspective of spectrum estimation. Proc IEEE 70:885–907 Rosenblatt M (2000) Gaussian and Non-Gaussian linear time series and random fields. Springer,

New York Rothenberg TJ (1971) Identification of parametric models. Econometrica 39:577–591 Rubio-Ramírez JF, Waggoner DF, Zha T (2010) Structural vector autoregressions: Theory of

identification and algorithms for inference. Rev Econ Stud 77:665–696 Rudin W (1976) Principles of mathematical analysis, 3rd edn. McGraw-Hill, New York Rudin W (1987) Real and complex analysis, 3rd edn. McGraw-Hill, Boston Runkle D (1987) Vector autoregressions and reality. J Bus Econ Stat 5:437–442 Said SE, Dickey DA (1984) Testing for unit roots in autoregressive-moving average models of

unknown order. Biometrika 71:599–607 Samuelson PA (1947) Foundations of economic analysis. Harvard University Press, Cambridge Samuelson PA (1965) Proof that properly anticipated prices fluctuate randomly. Ind Manage Rev

6:41–49 Sargent TJ (1987) Macroeconomic theory, 2nd edn. Academic Press, Orlando, FL Sargent TJ (1989) Two models of measurements and the investment accelerator. J Polit Econ

97:251–87 Sargent TJ (2004) Recursive macroeconomic theory, 2nd edn. MIT Press, Cambridge Sargent TJ, Sims CA (1977) Business cycle modelling without pretending to have too much a

priori economic theory. In: Sims CA (ed) New Methods in business cycle research. Federal Reserve Bank of Minneapolis, Minneapolis, pp 45–109

Schorfheide F (2005) VAR forecasting under misspecification. J Econ 128:99–136 Serfling RJ (1980) Approximation theorems of mathematical statistics. Wiley, New York Shaman P, Stine RA (1988) The bias of autoregressive coefficient estimators. J Am Stat Assoc

83:842–848 Silverman BW (1986) Density estimation. Chapman and Hall, London Sims CA (1972) Money, income, and causality. Am Econ Rev 62:540–552 Sims CA (1974) Seasonality in regression. J Am Stat Assoc 69:618–626

Bibliography 401

Sims CA (1980a) Comparison of interwar and postwar business cycles: Monetarism reconsidered. Am Econ Rev 70(2):250–257

Sims CA (1980b) Macroeconomics and reality. Econometrica 48:1–45 Sims CA (1986) Are forecasting models usable for policy analysis. Federal Reserve Bank of

Minneapolis Q Rev 10(1):2–16 Sims CA (1993) Rational expectations modeling with seasonally adjsuted data. J Econ 55:9–19 Sims CA (1999) Error bands for impulse responses. Econometrica 67:1113–1155 Sims CA, Stock JH, Watson MW (1990) Inference in linear time series with some unit roots.

Econometrica 58:113–144 Slutzky E (1937) The summation of random causes as the source of cyclic processes. Econometrica

5:105–146 Stock JH (1994) Unit roots, structural breaks and trends. In: Engle RF, McFadden DL (eds)

Handbook of econometrics, vol IV. Elsevier Science B.V., Amsterdam, pp 2739–2841 Stock JH, Watson MW (1988a) Testing for common trends. J Am Stat Assoc 83:1097–1107 Stock JH, Watson MW (1988b) Variable trends in economic time series. J Econ Perspect

2(3):147–174 Stock JH, Watson MW (2011) Introduction to econometrics, 3rd edn. Addison Wesley, Longman Strang G (1988) Linear algebra and its applications, 3rd edn. Harcourt Brace Jovanovich, San

Diego Sul D, Phillips PCB, Choi CY (2005) Prewhitening bias in HAC estimation. Oxford Bull Econ Stat

67:517–546 Tay AS, Wallis KF (2000) Density forecasting: A Survey. J Forecast 19:235–254 Tinbergen J (1939) Statistical testing of business cycle theories. League of Nations, Genf Tjøstheim D, Paulsen J (1983) Bias of some commonly-used time series estimates. Biometrika

70:389–399; corrigendum (1984), 71:656 Tobin J (1970) Money and income: Post hoc ergo propter hoc? Q J Econ 84:310–317 Uhlig H (2004) Do technology shocks lead to a fall in total hours worked? J Eur Econ Assoc

2:361–371 Uhlig H (2005) What are the effects of monetary policy on output? results from an agnostic

identification procedure. J Monet Econ 52:381–419 Uhlig H, Ravn M (2002) On adjusting the HP-filter for the frequency of observations. Rev Econ

Stat 84:371–376 Vogelsang TJ (1997) Wald-type tests for detecting breaks in the trend function of a dynamic time

series. Econ Theory 13:818–849 Watson MW (1994) Vector autoregressions and cointegration. Handbook of econometrics, vol 4,

chap 47. North-Holland, Amsterdam, pp 2843–2915 Weiss AA (1986) Asymptotic theory for ARCH models: Estimation and testing. Econ Theory

2:107–131 White H (1980) A heteroskedasticity consistent covariance matrix estimator and a direct test for

heteroskedasticity. Econometrica 48:817–838 Whittaker ET (1923) On a new method of graduation. Proc Edinbrough Math Soc 41:63–75 Wiener N (1956) The theory of prediction. In: Beckenbach EF (ed) Modern mathematics for

engineers. McGraw-Hill, New York, Series 1 Woodford M (2003) Interest and prices: foundations of a theory of monetary policy. Princeton

University Press, Princeton Wu CFJ (1983) On the convergence of the EM algorithm. Ann Stat 11:95–103 Yoo BS (1987) Multi-cointegrated time series and a generalized error correction model. Ph.D.,

University of California, San Diego Yule GU (1926) Why do we sometimes get nonsense correlations between time series? A study in

sampling and the nature of time series. J R Stat Soc 89:1–64 Yule GU (1927) On a method of investigating periodicities in disturbed series, with special

reference to Wolfer’ sunspot numbers. Philos Trans R Soc A 226:267–298 Zadrozny PA (2005) Necessary and sufficient restrictions for existence of a unique fourth moment

of a univariate GARCH(p,q) process, cESifo Working Paper No.1505

402 Bibliography

Zakoïan JM (1994) Threshold heteroskedastic models. J Econ Dyn Control 18:931–955 Zellner A (1979) Causality and econometrics. In: Brunner K, Meltzer A (eds) Three aspects of

policymaking: knowledge, data and institution, Carnegie-Rochester conference series on public policy. North-Holland, Amsterdam, pp 9–54

Zellner A, Palm F (1974) Time series analysis and simultaneous equation econometric models. J Econ 2:17–54

Zivot E, Andrews DW (1992) Further evidence on the great crash, the oil-price shock, and the unit-root hypothesis. J Bus Econ Stat 10:251–270

Index

A ACF, see also Autocorrelation

function ADF-test, 148 AIC, see Information criterion, 101, see

Information criterion, 247 AR process, 29

autocorrelation function, 29 autocovariance function, 29 stationary solution, 29

ARIMA process, 102, 134 ARMA model

estimation, 87 identification, 87

ARMA process, see also Autoregressive moving-average process

autocovariance function, 38 causality, 32 causality condition, 33 estimation, 95 invertibility, 37 invertibility condition, 37 maximum likelihood estimation, 95 state space representation, 330

Autocorrelation function, 14 confidence interval

MA(q) process, 76 AR(1) process, 77

estimation, 73 asymptotic distribution, 74 Bartlett’s formula, 74 confidence interval, 75

interpretation, 64 order, 14 properties, 21 random walk, 144 univariate, 14

Autocorrelation function, partial, 62 AR process, 63

estimation, 78 interpretation, 64 MA process, 52, 64

Autocovariance function, 13 ARMA process, 38 estimation, 73 linear process, 124 MA(1) process, 21 multivariate, 202 order, 13 properties, 20 random walk, 144 univariate, 13

Autoregressive conditional heteroskedasticity models, see Volatility

Autoregressive final form, 223 Autoregressive moving-average process, 25 Autoregressive moving-average prozess

mean, 25

B Back-shift operator, see also Lag operator Bandwidth, 80 Bartlett’s formula, 74 Basic structural model, 332, 349

cylical component, 333 local linear trend model, 333 seasonal component, 333

Bayesian VAR, 253 Beveridge-Nelson decomposition, 138, 383 Bias proportion, 250 Bias, small sample, 92, 231

correction, 92, 231 BIC, see Information criterion, 101, see

Information criterion, 247 Borel-Cantelli lemma, 377 Box-Pierce statistic, 75 BSM, see Basic structural model

403

404 Index

C Canonical correlation coefficients, 315 Cauchy-Bunyakovskii-Schwarz

inequality, 377 Causal representation, 32 Causality, see also Wiener-Granger

causality, 328 Wiener-Granger causality, 255

Central Limit Theorem m-dependence, 381

Characteristic function, 380 Chebyschev’s inequality, 377 Chow test, 355 Cointegration, 159

Beveridge-Nelson decomposition, 304, 309

bivariate, 159 common trend representation, 310 definition, 305 fully-modified OLS, 319, 323 Granger’s representation theorem, 309 normalization, 323 order of integration, 303 shocks, permanent and transitory, 311 Smith-McMillan factorization, 306 test

Johansen test, 312 regression test, 161

triangular representation, 311 VAR model, 305

assumptions, 305 VECM, 307 vector error correction, 307 Wald test, 321

Companion form, 218 Convergence

Almost sure convergence, 378 Convergence in r-th mean, 378 Convergence in distribution, 379 Convergence in probability, 378

Correlation function, 202 estimator, 208 multivariate, 202

Covariance function estimator, 208 properties, 203

covariance function, 202 Covariance proportion, 250 Covariance, long-run, 209 Cross-correlation, 203

distribution, asymptotic, 209 Cyclical component, 128, 333

D Dickey-Fuller distribution, 142 Durbin-Levinson algorithm, 48, 63 Dynamic factor model, 335 Dynamic multiplier, see Shocks, transitory

E EM algorithm, 345 Ergodicity, 10, 69 Estimation

ARMA model, 95 order, 99

Estimator maximum likelihood estimator, 95, 96 method of moments

GARCH(1,1) model, 187 moment estimator, 88 OLS estimator, 91

process, integrated, 141 Yule-Walker estimator, 88

Example AD-curve and Money Supply, 260 advertisement and sales, 274 ARMA processes, 34 cointegration

fully-modified OLS, 323 Johansen approach, 321

consumption expenditure and advertisement, 212

demand and supply shocks, 287 estimation of long-run variance, 83 estimation of quarterly GDP, 346 GDP and consumer sentiment index, 213 growth model, neoclassical, 323 inflation and short-term interest rate, 162 IS-LM model with Phillips curve, 277 modeling real GDP of Switzerland, 103 present discounted value model, 296 structural breaks, 356 Swiss Market Index, 188 term structure of interest rate, 164 unit root test, 152

Expectation, adaptive, 59 Exponential smoothing, 58

F Factor model, dynamic, see Dynamic factor

model FEVD, see also Forecast error variance

decomposition

Index 405

Filter gain function, 126 Gibbs phenomenon, 128 high-pass, 128 Hodrick-Prescott filter, 128 HP-filter, 128 Kuznets filter, 126 low-pass, 127 phase function, 126 TRAMO-SEATS, 131 transfer function, 125 X-11 filter, 131 X-12-Filter, 131

Filter, time invariant, 122 Filtering problem, 336 Final form, see Autoregressive final form FMOLS estimator, 319, 323

Wald test, 321, 324 Forecast error variance decomposition, 270 Forecast evaluation

Bias proportion, 250 Covariance proportion, 250 Mean-absolute-error, 249 Out-of-sample strategy, 250 Root-mean-squared-error, 249 Uncertainty, 251 Variance proportion, 250

Forecast function, 45 AR(p) process, 48 ARMA(1,1) process, 53 forecast error, 47 infinite past, 53 linear, 45 MA(q) process, 50 variance of forecast error, 48

Forecast, direct, 253 Forecast, iterated, 244, 250, 253 Fourier frequencies, 117 Fourier transform, discrete, 118 FPE, see Information criterion, 248 Frequency domain, 109 Fully-modified ordinary least-squares, 319

G Gain function, 126 Gauss Markov theorem, 92 Growth component, 128

H HAC variance, see also variance,

heteroskedastic and autocorrelation consistent

Harmonic process, 54, 115 Hodrick-Prescott filter, 128 HQC, see Information criterion, 101, see

Information criterion, 247

I Identification

Box-Jenkins, 64 Kalman filter, 346

Identification problem, 262 Impulse response function, 32, 37 Information criterion, 101, 247

AIC, 101, 247 BIC, 101, 247 Final prediction error, 248 FPE, 248 Hannan-Quinn, 247 HQC, 101 Schwarz, 101, 247

Innovation algorithm, 48 Innovations, 56 Integrated GARCH, 181 Integrated process, 102 Integrated regressors

rules of thumb, 162 Integration, order of, 134 Intercept correction, 253 Invertibility, 37

J Johansen test

distribution, asymptotic, 318 hypothesis tests over ˇ, 318 max test, 316 specification of deterministic part, 317 trace test, 316

K Kalman filter, 339

application basic structural model, 349 estimation of quarterly GDP, 346

AR(1) process, 337, 342 assumptions, 327 causal, 328 EM algorithm, 345 filtering problem, 336 forecasting step, 339 gain matrix, 340 identification, 346 initialization, 340 likelihood function, 344

406 Index

Kalman filter (cont.) Markov property, 327 measurement errors, 342 observation equation, 326 prediction problem, 336 smoother, 341 smoothing problem, 336 stable, 328 state equation, 326 stationarity, 328 updating step, 339

Kalman smoother, 341 Kernel function, 80

bandwidth, 80 optimal, 82 rule of thumb, 82

Bartlett, 80 boxcar, 80 Daniell, 80 lag truncation parameter, 80

optimal, 82 quadratic spectral, 80 Tukey-Hanning, 80

L Lag operator, 26

calculation rules, 26 definition, 26 polynomial, 26

Lag polynomial, 26 Lag truncation parameter, 80 Lag window, 117 Lead operator, 26 Leading indicator, 213, 259 Least-squares estimator, 91, 97 Likelihood function, 95, 344, 365

ARMA process, 95 Kalman filter, 344 regime switching model, 365

Ljung-Box statistic, 75 Loading matrix

definition, 306 Local linear trend model, 333 Long-run identification, 285

instrumental variables, 286

M m-dependence, 381 MA process, 17, 27

autocorrelation function, 28 autocovariance function, 21, 27

MAE, 249

Markov chain, 364 ergodic distribution, 365 regular, 365

Matrix norm, 205 absolute summability, 206 quadratic summability, 206 submultiplicativity, 206

Max share identification, see also VAR process, see also VAR process

Maximum likelihood estimator, 96, 184 ARMA(p,q) model, 95 asymptotic distribution, 98

AR process, 98 ARMA(1,1) process, 99 MA process, 99

GARCH(p,q) model, 186 Maximum likelihood method, 95 Mean, 67, 207

asymptotic distribution, 69, 71 distribution, asymptotic, 208 estimation, 67, 207 estimator, 208

Mean reverting, 133 Mean squared error matrix

estimated coefficients, 247 known coefficients, 244

Measurement errors, 337 Median–target method, 293 Memory, short, 28 Minnesota prior, 253, 361 Missing observations, 331 Mixture distributions, 365 Model, 10

N Normal distribution, multivariate

conditional, 337 Normal equations, 46

O Observation equation, 326 Observationally equivalent, 262 OLS estimator, 91

distribution, asymptotic, 92 Order of integration, 134 Ordinary-least-squares estimator, 91 Oscillation length, 111 Overfitting, 99

P PACF, see also Autocorrelation function,

partial

Index 407

Partial autocorrelation function computation, 63 estimation, 78

Particle filter, 360 Penalty function approach, 293 Period length, 111 Periodogram, 117 Perpetuity, 357 Persistence, 137 Phase function, 126 Portmanteau test, 76 PP-test, 149 Prediction problem, 336 Predictor, see also Forecast function Present discounted value model, 296

Beveridge-Nelson decomposition, 301 cointegration, 299 spread, 297 VAR representation, 298 vector error correction model, 298

Prewhitening, 83 Process, ARIMA, 134 Process, stochastic, 7, 201

ARMA process, 25 branching process, 11 deterministic, 54 difference-stationary, 134 finite memory, 18 finite-range dependence, 18 Gaussian process, 15 harmonic process, 54 integrated, 102, 134, 303

Beveridge-Nelson decomposition, 138

forecast, long-run, 135 impulse response function, 137 OLS estimator, 141 persistence, 137 variance of forecast error, 136

linear, 204 linearly regular, 57 memory, 15 moving-average process, 17 multivariate, 201 purely non-deterministic, 57 random walk, 19 random walk with drift, 19 singular, 54 spectral representation, 116 trend-stationary, 134

forecast, long-run, 135 impulse response function, 137 variance of forecast error, 136

white noise, 15

R Random walk, 11, 19

autocorrelation function, 144 autocovariance function, 144

Random walk with drift, 19 Real business cycle model, 336 Realization, 9 Regime switching model, 364

maximum likelihood estimation, 365 Restrictions

long-run, 282 short-run, 268 sign restrictions, 267

RMSE, 249

S Seasonal component, 333 Set identified, 292 Shocks

fundamental, 57 permanent, 37 structural, 260 transitory, 36

Short range dependence, see also Memory, short

Signal-to-noise ratio, 333 Singular values, 315 Smoothing, 341 Smoothing problem, 336 Smoothing, exponential, 58 Spectral average estimator, discrete, 118 Spectral decomposition, 109 Spectral density, 110, 115

ARMA process, 121 autocovariance function, 111 estimator, direct, 117 estimator, indirect, 117 Fourier coefficients, 111 spectral density, rational, 122 variance, long-run, 117

Spectral distribution function, 115 Spectral representation, 114, 116 Spectral weighting function, 119 Spectral window, 119 Spectrum estimation, 109 Spurious correlation, 158 Spurious regression, 158 State equation, 326 State space, 9 State space representation, 218, 326

ARMA processes, 330 ARMA(1,1), 329 missing observations, 331

408 Index

State space representation (cont.) stationarity, 328 time-varying coefficients, 331, 357

Cooley-Prescott, 331 Harvey-Phillips, 331 Hildreth-Houck, 331

VAR process, 329 Stationarity, 13

multivariate, 202 strict, 14 weak, 13

Stationarity, strict multivariate, 203

Strong Law of Large Numbers, 378 Structural breaks, 153, 252, 354

Chow test, 355 dating of breaks, 356 tests, 356

Structural change, 20 Structural time series analysis, 140 Structural time series model, 332, 349

basic structural model, 332 Summability

absolute, 206 quadratic, 206

Summability Condition, 383 Superconsistency, 142 Swiss Market Index (SMI), 188

T Test

autocorrelation, squared residuals, 183 cointegration

regression test, 161 Dickey-Fuller regression, 146 Dickey-Fuller test, 146, 147

augmented, 148 correction, autoregressive, 148

heteroskedasticity, 183 Engle’s Lagrange-multiplier test, 184

Independence, 210 Johansen test, 312

correlation coefficients, canonical, 315 distribution, asymptotic, 318 eigenvalue problem, 314 hypotheses, 312 hypothesis tests over ˇ, 318 likelihood function, 315 max test, 316 singular values, 315 trace test, 316

Kwiatkowski-Phillips-Schmidt-Shin-test, 157

Phillips-Perron test, 146, 149 stationarity, 157 uncorrelatedness, 210 unit root test

structural breaks, 153 testing strategy, 150

unit-root test, 146 white noise

Box-Pierce statistic, 75 Ljung-Box statistic, 75 Portmanteau test, 76

Time, 8 Time domain, 109 Time series model, 10 Time-varying coefficients, 331, 357

Minnesota prior, 361 regime switching model, 364

Times series analysis, structural, 140 Trajectory, 9 Transfer function, 125 Transfer function form, 223 Transition probability matrix, 364

U Underfitting, 99

V Value-at-Risk, 192 VaR, see Value-at-Risk VAR process

Bayesian VAR, 253 correlation function, 221 covariance function, 221 estimation

order of VAR, 247 Yule-Walker estimator, 238

forecast error variance decomposition, 270 forecast function, 241

mean squared error, 243, 244 form, reduced, 261, 263 form, structural, 260, 263 identification

forecast error variance share maximization, 272, 293

long-run identification, 282, 285 short-run identification, 268 sign restrictions, 267, 289 zero restrictions, 268

identification problem, 262, 264 Cholesky decomposition, 269

impulse response function, 270 bootstrap, 273

Index 409

confidence intervals, 272 delta method, 273

state space representation, 329 Structural breaks, 252 time-varying coefficients, 357 VAR(1) process, 216

stationarity, 216 variance decomposition

confidence intervals, 272 Variance proportion, 250 Variance, heteroskedastic and autocorrelation

consistent, 72 Variance, long-run, 72, 209

estimation, 79, 83 prewhitening, 83

multivariate, 209 spectral density, 117

VARMA process, 215 causal representation, 219 condition for causal representation, 219

VECM, see also Cointegration Vector autoregressive moving-average process,

see also VARMA process Vector autoregressive process, see also VAR

process Volatility

ARCH(p) model, 173 ARCH-in-mean model, 176 EGARCH model, 176 Forecasting, 182 GARCH(1,1) model, 177 GARCH(p,q) model, 174

ARMA process, 175

heavy-tail property, 175 GARCH(p,q) model, asymmetric, 176 heavy-tail property, 172 IGARCH, 181 models, 173 TARCH(p,q) model, 176 time-varying, 360 Wishart autoregressive process, 360

W Weighting function, 80 White noise, 15

multivariate, 204 univariate, 15

Wiener-Granger causality, 255 test

F-test, 257 Haugh-Pierce test, 260

Wishart autoregressive process, 360 Wold Decomposition Theorem, 55

multivariate, 245 univariate, 55

Y Yule-Walker equations

multivariate, 221 univariate, 88

Yule-Walker estimator, 88 AR(1) process, 89 asymptotic distribution, 89 MA process, 90

Preface
Contents
List of Figures
List of Tables
List of Definitions
List of Theorems
Notation and Symbols
Part I Univariate Time Series Analysis

1 Introduction

1.1 Some Examples
1.2 Formal Definitions

Examples of Stochastic Processes

1.3 Stationarity
1.4 Construction of Stochastic Processes

1.4.1 White Noise
1.4.2 Construction of Stochastic Processes: Some Examples
1.4.3 Moving-Average Process of Order One
1.4.4 Random Walk
1.4.5 Changing Mean

1.5 Properties of the Autocovariance Function

1.5.1 Autocovariance Function of MA(1) Processes

1.6 Exercises

2 ARMA Models

2.1 The Lag Operator
2.2 Some Important Special Cases

2.2.1 Moving-Average Process of Order q
2.2.2 First Order Autoregressive Process

2.3 Causality and Invertibility

Some Examples

2.4 Computation of Autocovariance Function

2.4.1 First Procedure

A Numerical Example

2.4.2 Second Procedure

A Numerical Example

2.4.3 Third Procedure

A Numerical Example

2.5 Exercises

3 Forecasting Stationary Processes

3.1 Linear Least-Squares Forecasts

3.1.1 Forecasting with an AR(p) Process
3.1.2 Forecasting with MA(q) Processes
3.1.3 Forecasting from the Infinite Past

Example of an ARMA(1,1) Process

3.2 The Wold Decomposition Theorem
3.3 Exponential Smoothing
3.4 Exercises
3.5 Partial Autocorrelation

3.5.1 Definition

Autoregressive Processes
Moving-Average Processes

3.5.2 Interpretation of ACF and PACF

3.6 Exercises

4 Estimation of Mean and ACF

4.1 Estimation of the Mean
4.2 Estimation of ACF

Example: {Xt} IID(0,σ2)
Example: MA(q) Process: Xt = Zt + θ1 Zt-1 + …+ θq Zt-q with Zt IID(0,σ2)
Example: AR(1) Process Xt - ϕXt-1 = Zt with Zt IID(0,σ2)

4.3 Estimation of PACF
4.4 Estimation of the Long-Run Variance

4.4.1 An Example

4.5 Exercises

5 Estimation of ARMA Models

5.1 The Yule-Walker Estimator

Example: AR(1) Process
Example: MA(q) Process

5.2 OLS Estimation of an AR(p) Model

Appendix: Proof of the Asymptotic Normality of the OLS Estimator

5.3 Estimation of an ARMA(p,q) Model

Example: AR(p) Process
Example: MA(q) Process
Example: ARMA(1,1) Process

5.4 Estimation of the Orders p and q
5.5 Modeling a Stochastic Process

Step 1: Transformations to Achieve Stationary Time Series
Step 2: Finding the Orders p and q
Step 3: Checking the Plausibility

5.6 Modeling Real GDP of Switzerland

6 Spectral Analysis and Linear Filters

6.1 Spectral Density

Some Examples

6.2 Spectral Decomposition of a Time Series
6.3 The Periodogram and the Estimation of Spectral Densities

6.3.1 Non-Parametric Estimation
6.3.2 Parametric Estimation

6.4 Linear Time-Invariant Filters

Examples of Filters

6.5 Some Important Filters

6.5.1 Construction of Low- and High-Pass Filters
6.5.2 The Hodrick-Prescott Filter
6.5.3 Seasonal Filters
6.5.4 Using Filtered Data

6.6 Exercises

7 Integrated Processes

7.1 Definition, Properties and Interpretation

7.1.1 Long-Run Forecast
7.1.2 Variance of Forecast Error
7.1.3 Impulse Response Function
7.1.4 The Beveridge-Nelson Decomposition

Examples

7.2 Properties of the OLS Estimator in the Case of Integrated Variables
7.3 Unit-Root Tests

7.3.1 Dickey-Fuller Test
7.3.2 Phillips-Perron Test
7.3.3 Unit-Root Test: Testing Strategy
7.3.4 Examples of Unit-Root Tests

7.4 Generalizations of Unit-Root Tests

7.4.1 Structural Breaks in the Trend Function
7.4.2 Testing for Stationarity

7.5 Regression with Integrated Variables

7.5.1 The Spurious Regression Problem
7.5.2 Bivariate Cointegration

An Example for Bivariate Cointegration

7.5.3 Rules to Deal with Integrated Times Series

Example: Term Structure of Interest

8 Models of Volatility

8.1 Specification and Interpretation

8.1.1 Forecasting Properties of AR(1)-Models
8.1.2 The ARCH(1) Model
8.1.3 General Models of Volatility

The Threshold GARCH Model
The Exponential GARCH Model
The ARCH-in-Mean Model

8.1.4 The GARCH(1,1) Model

The IGARCH Model
Forecasting

8.2 Tests for Heteroskedasticity

8.2.1 Autocorrelation of Quadratic Residuals
8.2.2 Engle's Lagrange-Multiplier Test

8.3 Estimation of GARCH(p,q) Models

8.3.1 Maximum-Likelihood Estimation
8.3.2 Method of Moment Estimation

8.4 Example: Swiss Market Index (SMI)

Value at Risk

Part II Multivariate Time Series Analysis

9 Introduction
10 Definitions and Stationarity

An Example
Appendix: Norm and Summability of Matrices

11 Estimation of Covariance Function

11.1 Estimators and Asymptotic Distributions
11.2 Testing Cross-Correlations of Time Series
11.3 Some Examples for Independence Tests

Two Independent AR Processes
Consumption Expenditure and Advertisement Expenses

Real Gross Domestic Product and Consumer Sentiment

12 VARMA Processes

12.1 The VAR(1) Process
12.2 Representation in Companion Form
12.3 Causal Representation

Example

12.4 Computation of Covariance Function

Example
Appendix: Autoregressive Final Form

13 Estimation of VAR Models

13.1 Introduction
13.2 The Least-Squares Estimator
13.3 Proofs of Asymptotic Normality

Proof of Theorem 13.1
Proof of Theorem 13.2

13.4 The Yule-Walker Estimator

14 Forecasting with VAR Models

14.1 Forecasting with Known Parameters

Example
14.1.1 Wold Decomposition Theorem

14.2 Forecasting with Estimated Parameters
14.3 Modeling of VAR Models
14.4 Example: VAR Model

15 Interpretation of VAR Models

15.1 Wiener-Granger Causality

15.1.1 VAR Approach
15.1.2 Wiener-Granger Causality and Causal Representation
15.1.3 Cross-Correlation Approach

15.2 Structural and Reduced Form

15.2.1 A Prototypical Example
15.2.2 Identification: The General Case
15.2.3 Identification: The Case n=2

15.3 Identification via Short-Run Restrictions
15.4 Interpretation of VAR Models

15.4.1 Impulse Response Functions
15.4.2 Variance Decomposition
15.4.3 Confidence Intervals
15.4.4 Example 1: Advertisement and Sales
15.4.5 Example 2: IS-LM Model with Phillips Curve

15.5 Identification via Long-Run Restrictions

15.5.1 A Prototypical Example

Analytic Solution of the System

15.5.2 The General Approach

Example 3: Identifying Aggregate Demand and Supply Shocks

15.6 Sign Restrictions

16 Cointegration

16.1 A Theoretical Example
16.2 Definition and Representation

16.2.1 Definition
16.2.2 VAR and VEC Models

An Illustration

16.2.3 Beveridge-Nelson Decomposition
16.2.4 Common Trend Representation

16.3 Johansen's Cointegration Test

16.3.1 Specification of the Deterministic Components
16.3.2 Testing Cointegration Hypotheses

16.4 Estimation and Testing of Cointegrating Relationships
16.5 An Example

17 Kalman Filter

17.1 The State Space Model

17.1.1 Examples

VAR(p) Process
ARMA(1,1) Process
ARMA(p,q) Process
Missing Observations
Time-Varying Coefficients
Structural Time Series Analysis
Dynamic Factor Models
Real Business Cycle Model (RBC Model)

17.2 Filtering and Smoothing

AR(1) Process with Measurement Errors
17.2.1 The Kalman Filter
17.2.2 The Kalman Smoother

AR(1) Process with Measurement Errors (Continued)

17.3 Estimation of State Space Models

17.3.1 The Likelihood Function
17.3.2 Identification

17.4 Examples

17.4.1 Estimation of Quarterly GDP
17.4.2 Structural Time Series Analysis

17.5 Exercises

18 Generalizations of Linear Models

18.1 Structural Breaks

18.1.1 Methodology
18.1.2 An Example

18.2 Time-Varying Parameters

The Minnesota Prior

18.3 Regime Switching Models

A Complex Numbers
B Linear Difference Equations
C Stochastic Convergence
D BN-Decomposition
E The Delta Method

Example: Univariate
Example: Multivariate

Bibliography
Index