Basic Literiture Review

182s8j12sa

SSRN-id2319260.pdf

Home >Business & Finance homework help >Basic Literiture Review

A Tale of Two Investors:

Estimating Optimism and Overconfidence∗

Giovanni Barone-Adesi†

Swiss Finance Institute

and University of Lugano

Loriano Mancini‡

Swiss Finance Institute

and EPFL

Hersh Shefrin§

Leavey School of Business

Santa Clara University

This version: July 18, 2013

First draft: June 2012

Separate Appendix Available Online

∗For helpful comments we thank Malcolm Baker, Kelley Bergsma, Menachem Brenner, Pierre Collin- Dufresne, Kent Daniel, Alex Edmans, Rob Engle, Stephen Figlewski, Simon Gilchrist, Paul Glasserman, Cam Harvey, Harrison Hong, Tom Howard, Danling Jiang, Bryan Kelly, Fabio Maccheroni, Roberto Marfè, Norman Schuerhoff, Raghu Sundaram, Fabio Trojani, Wei Xiong, Jianfeng Yu, Bill Zame and seminar participants at New York University, Santa Clara University and Princeton-Lausanne workshop. Barone- Adesi and Mancini acknowledge the financial support from the Swiss National Science Foundation NCCR- FinRisk. Shefrin acknowledges a course release grant from Santa Clara University.

†Giovanni Barone-Adesi, Institute of Finance, University of Lugano, Via G. Buffi 13, CH-6900 Lugano, Switzerland, E-mail: [email protected]. Telephone: +41 58 666 4753.

‡Loriano Mancini, Swiss Finance Institute at EPFL, Quartier UNIL-Dorigny, CH-1015 Lausanne, Switzer- land. E-mail: [email protected]. Telephone: +41 21 693 0107.

§Corresponding author: Hersh Shefrin, Department of Finance, Santa Clara University, 500 El Camino Real, Santa Clara, CA 95053, USA, E-mail: [email protected]. Telephone: 408 554 6893. Fax: 408 554 5206.

A Tale of Two Investors:

Estimating Optimism and Overconfidence

Abstract

We estimate investors’ sentiment from option and stock prices by anchoring objec-

tive beliefs to a neoclassical pricing kernel. Our estimates of sentiment correlate well

with other sentiment measures such as the Baker–Wurgler index, the Yale/Shiller crash

confidence index and the Duke/CFO survey responses, and yet contain additional infor-

mation. Our analysis points out three significant issues related to overconfidence. First,

the Baker–Wurgler index strongly reflects excessive optimism but not overconfidence.

Second, overconfidence drives the pricing kernel puzzle. Third, the dynamics of opti-

mism and overconfidence generate a perceived negative risk-return relationship, while

objectively the relationship is positive. Optimism and overconfidence about market

returns co-move together, inflating asset prices in good times and exacerbating market

crashes in bad times.

Keywords: Sentiment, Pricing Kernel, Optimism, Overconfidence

JEL Codes: G02, G12

1. Introduction

In a literature that goes back at least to Keynes (1936), behavioral economists have analyzed

how psychology leads sentiment to create gaps between security prices and fundamental val-

ues. These gaps distort capital allocation, impact investment portfolios, and weaken the

intermediation capacity of the financial sector. The behavioral line of inquiry has intensified

in recent decades, with attention devoted to such issues as over and underreaction in indi-

vidual stock prices and implied volatility in option prices, the closed end fund puzzle, the

equity premium puzzle, the new issues puzzle, dividend catering effects, and mood swing

effects on stock prices from seasonal effect and sporting events.1

Two features distinguish behavioral finance from neoclassical finance: psychology and

limits to arbitrage; Barberis and Thaler (2003). Psychology means heuristics, biases, and

framing effects; Kahneman, Slovic, and Tversky (1982), and Kahneman and Tversky (1979).

Limits to arbitrage include characteristics such as risk exposure and liquidity which lead

rational investors to limit the extent to which they exploit mispricing. Sentiment is the

manifestation of psychology. Limits to arbitrage enable sentiment to impact market prices.

Despite its importance as one of the distinguishing features of behavioral finance, there

is no uniform definition of sentiment in the literature. Instead, discussions of sentiment have

tended to be context dependent, and defined using proxies such as closed end fund discounts,

returns to new equity issues, and premiums on dividend paying stocks.

The main contribution of this paper is the estimation of a theoretically-based notion of

sentiment, using option prices, market returns, and risk free rates. We define sentiment

in terms of a change of measure that links objective and subjective beliefs. Two features

1Seminal contributions to the behavioral asset pricing literature include Shiller (1981), De Bondt and Thaler (1985, 1987), Lee, Shleifer, and Thaler (1991), Jegadeesh and Titman (1993, 2001), Loughran and Ritter (1995), Daniel, Hirshleifer, and Subrahmanyam (1998, 2001), Hong and Stein (1999), Kamstra, Kramer, and Levi (2000), Barberis, Huang, and Santos (2001), Poteshman (2001), and Baker and Wurgler (2004). Additional contributions that relate to the pricing kernel approach in this paper are Scheinkman and Xiong (2003), Edmans, Garćıa, and Norli (2007), Dumas, Kurshev, and Uppal (2009), and Xiong and Yan (2010). Grith, Härdle, and Krätschmer (2013) and Cheng, Raina, and Xiong (2013) provide recent related studies. Three prominent surveys of the behavioral finance literature are Hirshleifer (2001), Barberis and Thaler (2003) and Subrahmanyam (2007).

characterize our analysis. First, our definition of sentiment is general, formal, and pre-

cise. In contrast, the existing behavioral finance literature has no clear formal definition of

sentiment. Second, to estimate sentiment we use option prices, which provide the richest

source to identify variations in sentiment. We demonstrate that our measure of sentiment is

parsimonious, strongly reflects a disparate collection of other sentiment measures, and yet

contains additional information.

Based on Shefrin (2008), we define sentiment as a difference between the return density

functions of two investors. One investor is the representative investor whose beliefs set prices,

even though they might be biased. The other investor is a rational investor whose beliefs

are objectively correct. These are the two investors in the title of the paper. Sentiment is a

function of the change of measure that transforms the objective return probability density

function (pdf) into the representative investor’s pdf. This definition of sentiment is precise

and estimable.

We estimate sentiment by estimating the return pdfs of the two investors described

above, using empirical techniques developed by Barone-Adesi, Engle, and Mancini (2008).

An advantage of this approach is that we do not impose restrictions on pdf shapes. In our

empirical analysis, we use options and returns on the S&P 500. Thus, our sentiment measure

pertains to biased beliefs about returns to the S&P 500.

Our sentiment measure is general, and therefore encapsulates specific biases such as

excessive optimism and overconfidence. Excessive optimism occurs when the representative

investor overestimates mean returns. Overconfidence occurs when the representative investor

underestimates return volatility. Our analysis suggests that both biases are time varying

and become economically large when the economy expands and contracts. Notably, our

general approach also enables us to analyze other aspects of sentiment such as return biases

associated with skewness and tails.

Armed with a precise definition of sentiment and corresponding estimates, we use them to

analyze three specific questions. The first question pertains to the sentiment index developed

in Baker and Wurgler (2006), which is the most prominent empirical treatment of sentiment

to date. Baker and Wurgler developed their index from a Principal Component Analysis

of six specific sentiment proxies.2 While this approach does extract a common component

from these various indexes of sentiment, it leaves open the question of what the principal

component actually represents. Although Baker and Wurgler mention several psychological

heuristics and biases that underlie sentiment, such as excessive optimism, overconfidence,

and representativeness, they emphasize excessive optimism in their analysis.3 This state-

ment is intuitive but untested. Based on our formal tests, we find that the Baker–Wurgler

index does reflect excessive optimism about the market return. In an AR(2) regression of

the Baker–Wurgler series on excessive optimism and overconfidence, the coefficient for ex-

cessive optimism has a t-statistic of 3.7. At the same time, we also find that Baker–Wurgler

index fails to capture the component of overconfidence which is uncorrelated to excessive

optimism. These findings are important because they indicate that Baker and Wurgler’s

intuition about what their index measures is correct, but at the same time their sentiment

index is incomplete.

The second question we investigate pertains to the connection between the Baker–Wurgler

index and the risk-return relationship. Yu and Yuan (2011) report that the relationship

between risk and return, while positive when the Baker–Wurgler index is low, weakens when

the Baker–Wurgler index is high to the point where it becomes insignificant. Yu and Yuan

suggest that overconfidence plays a role in the risk-return dynamics, but do not include

a measure of overconfidence in their formal analysis. We use our two-investor framework

to explicitly analyze the role of overconfidence in the risk-return dynamics. We separate

the objective risk-return relationship and the relationship imbedded in the representative

investor’s beliefs. We find that for the objective pdf, the risk-return relationship is similar

to what Yu and Yuan find. However, we also find that for the representative investor the

risk-return relationship is negative for most of our sample period. Our analysis indicates that

the negative relationship stems from excessive optimism and overconfidence being positively

2The six specific series are: turnover on the New York Stock Exchange (NYSE); dividend premium; closed- end fund discount; number and first-day returns on IPOs; and the equity share in new issues. 3Baker and Wurgler (2007) state: “we view investor sentiment as simply optimism or pessimism about stocks in general . . . ” p. 132.

correlated and strong. What makes this point important is our finding that the Baker–

Wurgler index is effectively a measure only of excessive optimism. Thus, the risk-return

trade-off appears to be driven by the co-movements of excessive optimism and overconfidence,

not just the level of excessive optimism.

In a related vein, we investigate whether sentiment reflects more than excessive optimism

and overconfidence by analyzing biases associated with tail events. In the body of the paper

we focus on left tail bias, and expand the discussion in the online appendix to address biases

associated with the right tail, skewness, and kurtosis. Left tail bias is especially important

for default risk. To analyze this issue we examine recent work by Gilchrist and Zakraǰsek

(2012) who develop an excess bond premium measuring the component of corporate bond

spreads that is unrelated to firm-specific information on expected defaults. Gilchrist and

Zakraǰsek contend that a rise in the excess bond premium represents a reduction in the

effective risk-bearing capacity of the financial sector and, as a result, a contraction in the

supply of credit with adverse consequences for the macroeconomy. To test whether the

excess bond premium is effectively a dimension of sentiment, we compare the excess bond

premium to the left tail of the representative investor’s pdf. We find that the correlation

between the left tail of the representative investor’s pdf and the excess bond premium is

0.9. In addition, the representative investor’s left tail bias, i.e., the difference between the

representative investor’s and objective left tail probability, is highly correlated (0.7) with

the excess bond premium. This suggests that when the representative investor’s left tail

bias moves from negative to positive, the likelihood of an economic contraction increases

and the intermediation capacity of the financial sector diminishes. Not surprisingly, the

correlations of the excess bond premium with excessive optimism and overconfidence are

also significant, but nowhere near as strong as with left tail probabilities. This reinforces

our general contention that sentiment is more than excessive optimism and overconfidence.

The third question we investigate pertains to the pricing kernel puzzle. Shefrin (2008)

argues that nonzero sentiment lies at the root of the non-monotone decreasing pricing ker-

nel property which neoclassical economists regard as a puzzle. In contrast, the behavioral

approach predicts non-monotonicity, because that is the hallmark of strong sentiment. In

particular the theory establishes a link between the character of investors’ biases and the

shape of the pricing kernel. Notably, we find that this link is borne out in the time series

of our estimates of overconfidence and the manner in which the shape of our pricing kernel

estimates change over time.

Finally, we find that our estimate of sentiment correlates well with other measures of

sentiment. This finding goes beyond robustness. Our measure of sentiment is not only

well defined and theoretically grounded, but empirically parsimonious, cutting across a va-

riety of sentiment indicators in the literature such as the Duke/CFO survey responses, the

Yale/Shiller crash confidence index, and as we mentioned above, the Baker–Wurgler index,

and the Gilchrist–Zakraǰsek excess bond premium.

The remainder of the paper is organized as follows. Section 2 presents the intuition

underlying our approach. Section 3 describes our methodology for estimating the empirical

pricing kernel. Section 4 reviews the theoretical framework for analyzing investors’ sentiment.

Section 5 presents our estimates of sentiment. Section 6 relates our findings to external

measures of sentiment. Section 7 concludes.

2. Intuition Underlying Our Approach

The main contribution of this paper is the estimation of a theoretically-based notion of

sentiment, using option prices, market returns, and risk free rates. We define sentiment in

terms of a change of measure that links subjective and objective beliefs, and therefore the

connection between psychological biases and asset pricing. To develop the intuition under-

lying our approach, we provide a brief nontechnical introduction. Our starting point is the

standard neoclassical framework in which equilibrium prices are set as if by a representative

investor holding correct beliefs. The objective probability density function (pdf) associated

with correct beliefs, is depicted in the top panel of Figure 1, and is labeled Pobj.

In a behavioral framework, equilibrium prices are also set as if by a representative in-

vestor, but one whose beliefs possibly reflect biases in the investor population. Because of

limits to arbitrage, investor biases are not necessarily eliminated in equilibrium. In the top

panel of Figure 1, the function Prep denotes the pdf of the representative investor exhibit-

ing two biases, excessive optimism and overconfidence. Relative to the objective pdf Pobj,

excessive optimism means that the representative investor overestimates expected return.

Overconfidence means that the representative investor underestimates return standard de-

viation. In Figure 1, notice that the mode of Prep is to the right of the mode of Pobj, and

Prep attaches much less weight to tail events than Pobj.

Theoretically, excessive optimism is defined as expected return under Prep minus ex-

pected return under Pobj. Overconfidence is defined as return standard deviation under

Pobj minus return standard deviation under Prep. Operationally, we estimate Pobj and

Prep and then compute excessive optimism and overconfidence from their first and second

moments. To estimate Pobj, we use a dynamic model for S&P 500 returns. To estimate

Prep, we use S&P 500 index option prices (SPX) and the risk free rate to infer the risk

neutral pdf, and then apply a pricing kernel-based change of measure.

The pricing kernel lies at the heart of our process for inferring Prep. The pricing kernel is

a function whose values are ratios of state prices to probabilities, which in this case we take

to be objective probabilities Pobj. The bottom panel of Figure 1 displays three functions.

The function CRRAKernel is the pricing kernel from a neoclassical representative investor

model with CRRA preferences. As usual, the function is monotone decreasing, and measures

intertemporal marginal rate of substitution.

In contrast to CRRAKernel, the function BehavKernel in Figure 1 depicts a pricing

kernel associated with a representative investor whose beliefs exhibit excessive optimism

and overconfidence. Notice how overconfidence manifests itself in tail events where the

BehavKernel function lies below CRRAKernel, as the behavioral representative investor

underestimates tail event probabilities. Notably, in this example, the degree of overconfidence

leads BehavKernel to feature an upward sloping portion in the left region of the figure. For

the middle range, the combination of biases leads BehavKernel to lie above CRRAKernel,

so that BehavKernel has the shape of an inverted-U.

We use estimates of BehavKernel, CRRAKernel, and their difference to provide infor-

mation which allows us to infer values for excessive optimism and overconfidence, and to

disentangle their manifestation within prices. To estimate BehavKernel we use the ratio

of the estimate of the risk neutral pdf to our estimate of the objective pdf. To estimate

CRRAKernel we use a technique described later in the paper. To capture the differences be-

tween the two pricing kernels, we use the log of BehavKernel minus the log of CRRAKernel,

which is displayed as the function LogDiff in the bottom panel of Figure 1. We provide an

exact interpretation of LogDiff later in the paper.

Our empirical measures of excessive optimism and overconfidence are computed relative

to a process estimated from historical returns. We do not contend that historical returns are

completely free of investor bias. Instead we investigate the extent to which market prices

accurately reflect an econometrician’s best estimate of future returns.4 This approach is in

the same vein as, e.g., Xiong and Yan (2010) who theoretically discuss the econometrician’s

prediction of bond premia when investors have heterogenous beliefs.

The representative investor always holds the market portfolio, and therefore does not “lose

money” because of biases. To the extent that the representative investor corresponds to a

real investor, the biases cause the representative investor to be disappointed and surprised.

Optimism leads to disappointment in the realized risk premium, and overconfidence leads to

surprise about the amount of volatility.

3. Method to Estimate the Empirical Pricing Kernel

By a pricing kernel we mean a stochastic discount factor (SDF) defined as state price per

unit objective probability. Let Mt denote the empirical SDF associated with returns between

4In the theoretical framework underlying our analysis, biases are defined relative to a market in which all investors hold objectively correct beliefs about the stochastic process governing aggregate consumption growth. Although biases impact the return distribution of the market portfolio, the magnitude of the impact is small; see Theorem 17.2 in Shefrin (2008). In practice, biases might have a greater impact on the S&P 500 than the market portfolio. We follow the standard practice to use the S&P 500 as a proxy for the market portfolio.

date t and date T , conditional on the information available at date t ≤ T . Throughout the

paper, (T − t) is fixed and equal to one year. The empirical SDF is given by

Mt = e −rf (T−t) q(ST /St)

p(ST /St) (1)

where q is the risk neutral density, p the objective or historical density, rf the instantaneous

risk free rate, and St the S&P 500 index at date t, which is a proxy for the market portfolio. 5

The densities q and p are conditional on the information available at date t, but for ease of

notation we omit such a dependence. The risk free rate rf depends on t and T , and such

a dependence is omitted as well. Once the conditional densities q and p are estimated, we

can recover the SDF by simply taking their discounted ratio, (1). The advantage of this

procedure is that no constraint is imposed on the functional form of the SDF.

In order to estimate the empirical SDF, we use the same approach as in Barone-Adesi,

Engle, and Mancini (2008). For a given date t, we fit an asymmetric Glosten, Jagannathan,

and Runkle (1993) GARCH model to historical daily log-returns of the S&P 500 to capture

the index dynamic under the objective pdf p. The model has the form

log(Su/Su−1) = µu + ϵu (2)

σ2u = ω + βσ 2 u−1 + αϵ

2 u−1 + γIu−1ϵ

2 u−1 (3)

where ϵu = σu zu, zu is the standardized historical innovation at day u, Iu−1 = 1 when

ϵu−1 < 0, Iu−1 = 0 otherwise, and u = t0, . . . , t, with (t − t0) being the number of daily

data. When γ > 0, the model accounts for the leverage effect, namely bad news (ϵu−1 < 0)

5Multiple state variables could enter the SDF and there is considerable debate among researchers over the relevant state variables. As in Aı̈t-Sahalia and Lo (2000), Jackwerth (2000), Rosenberg and Engle (2002), and others, we consider the projection of the SDF into S&P 500 returns. As discussed in Cochrane (2005), this projected SDF has the same pricing implications as the original SDF for assets with payoffs that depend on S&P 500 returns such as call and put options on the S&P 500 considered in this paper.

raises future volatility more than good news (ϵu−1 ≥ 0) of the same absolute magnitude.6

The scaled return innovation, zu, is from its empirical density function, which is obtained

by dividing each estimated return innovations, ϵ̂u, by its estimated conditional volatility σ̂u.

This set of estimated scaled innovations gives an empirical density function that incorporates

excess skewness, kurtosis, and other extreme return behaviors that are not captured in a

normal density. This approach is called filtered historical simulation (FHS). The drift term

is specified as µu = 0.012+0.76 (E/P)u, where E/P is the inverse of the price-earnings ratio,

adjusted for inflation, developed by Campbell and Shiller (1998). The coefficients in µu are

obtained by regressing subsequent annualized ten-year returns for the Campbell–Shiller series

on a constant and E/P.7 The online appendix shows that our subsequent results hold true

when the excess risk premium (in excess of the risk free rate) is set to a constant value of 4%,

rather than based on E/P. The GARCH parameter estimates are obtained by maximizing

the Pseudo Maximum Likelihood, under the nominal, not necessarily true, assumption of

normal innovations, Bollerslev and Wooldridge (1992). This technique provides consistent

parameter estimates even when the true innovation density is not normal, e.g., White (1982),

and Gourieroux, Monfort, and Trognon (1984). Rosenberg and Engle (2002) use the same

approach to estimate the objective distribution of S&P 500 returns in their analyses.

For a given date t, a GARCH model (2)–(3) is calibrated to the cross section of out-of-

the-money call and put options on the S&P 500 to capture the index dynamic under the risk

neutral pdf q. For a given set of risk neutral GARCH parameters {ω∗, β∗, α∗, γ∗}, a return

path is simulated by drawing an estimated past innovation, say, z[1], updating the conditional

variance σ2t+1, drawing a second innovation z[2], updating the conditional variance σ 2 t+2, and

so on up to t+τ. The τ periods simulated gross return is St+τ/St = exp(τµ ∗ +

∑τ i=1 σt+i z[i]),

6The terminology leverage effect was introduced by Black (1976), who suggested that a large negative return increases the firm financial and operating leverage, and rises equity return volatility; see also Christie (1982). Campbell and Hentschel (1992) suggest an alternative explanation based on the market risk premium and volatility feedback effects; see also Bekaert and Wu (2000). 7The key result of Campbell and Shiller (1998) is that subsequent ten-year returns to stocks are negatively and statistically related to the price-earnings ratio. In this sense, the specification µu = 0.012 + 0.76 (E/P)u is forward looking. Updated data series of the price-earnings ratio are available from Robert Shiller’s website, http://www.econ.yale.edu/∼shiller/.

where the drift µ∗ ensures that the average gross return equals the risk free gross rate

erf , using the Empirical Martingale Simulation method of Duan and Simonato (1998). We

simulate L = 20,000 return paths from t to t+τ. The GARCH call option price at time t with

strike price K and time to maturity τ is given by e−rf τ ∑L

l=1 max(S (l) t+τ −K, 0)/L, where S

(l) t+τ

is the simulated index price at time t + τ in the l-th sample path. Put prices are computed

similarly. The risk neutral GARCH parameters {ω∗, β∗, α∗, γ∗} are varied, which changes

the simulated return paths, so as to best fit the cross-section of option prices at date t,

minimizing the mean square pricing error ∑Nt

j=1 et(Kj, τj) 2, where et(Kj, τj) is the difference

between the GARCH option price and the actual price of the option with strike Kj and time

to maturity τj, and Nt is the number of options at day t. The calibration is achieved when,

varying the risk neutral GARCH parameters, the reduction in the mean square pricing error

is negligible or below a given threshold.8

Having estimated objective and risk neutral GARCH parameters on a given date t, the

next step to recover the SDF is the estimation of the conditional densities p(ST /St) and

q(ST /St). For each date t, these conditional densities are estimated by Monte Carlo Simu-

lation. For the objective and risk neutral GARCH parameters, we simulate 50,000 return

paths of the index at a daily frequency from t to T , using the FHS and Empirical Martin-

gale Simulation methods described above. The conditional densities p and q are obtained

by nonparametric kernel density estimation, i.e., smoothing the corresponding simulated

distribution of ST /St. 9 Finally, the empirical SDF is estimated as in (1).

We consider two GARCH models under the risk neutral density q that lead to two

estimates of the empirical SDF. One we call Gauss and the other we call FHS. The Gaussian

model uses randomly drawn Gaussian innovations for the simulation of the return paths,

whereas the FHS model uses the historical, nonparametric innovations zu, as described above.

8To ensure the convergence of the calibration algorithm, the FHS innovations, zu, used to simulate the return paths are kept fix across all the iterations of the algorithm. Starting values for the risk neutral parameters are the GARCH parameters estimated under the objective measure. 9In our empirical application, we consider a range of gross returns approximately between 0.69 to 1.35. This range is well within the span of option moneyness in our sample. So there is no extrapolation bias when estimating risk neutral densities. Outside this range, objective and risk neutral densities are virtually zero, so we simply set them to zero.

We use both models in order to contrast the difference that FHS makes.

4. Theoretical Framework for Sentiment

In this section, we formally define sentiment and then discuss the estimation procedure.

4.1. Definition of Sentiment

Sentiment impacts the SDF by distorting state prices relative to a neoclassical counterpart.

Therefore, a behavioral SDF effectively decomposes into a neoclassical component and a

sentiment distortion. In a neoclassical framework featuring constant relative risk aversion

(CRRA), the SDF has the following form:

Mt(θ) = θ0 (ST /St) −θ1 (4)

where θ0 is a discount factor measuring the degree of impatience, θ1 is the coefficient of

relative risk aversion, and θ = (θ0, θ1). The logarithmic version of (4) is

log(Mt(θ)) = log(θ0) − θ1 log(ST /St). (5)

Shefrin (2008) provides a theoretical framework to formally define sentiment, which is

adopted here. In that framework, (5) generalizes to include an additional term Λt to reflect

the impact of sentiment. The equation for the log-SDF becomes

log(Mt) = Λt + log(θ0,t) − θ1,t log(ST /St) (6)

where the parameter θ is now time varying.10 Appendix A sketches a derivation of (6). We

define sentiment as the function Λt. This function is a scaled log-change of measure, where

10Λt is a function of T and ST /St, but we omit such dependencies for ease of notation. In addition, θ1,t is theoretically a function of T and ST /St, and θ0,t is a function of T. For simplicity, we omit such dependencies as well.

the change of measure transforms the objective pdf p into the representative investor’s pdf

pR. In other words, the function e Λt is proportional to the change of measure pR/p so that

pR = p e Λt θ0,t,p/θ0,t (7)

where θ0,t,p is a rescaling of θ0,t whose purpose is to ensure that pR integrates to one.

The log-change of measure log(pR/p) specifies the percentage error in probability density

which the representative investor assigns to the occurrence of a specific return. For example,

suppose that the representative investor underestimates by 2% the probability that the

market return will be 1%. In this case, the log-change of measure at 1% will be −2%.

In a Gaussian framework, a log-linear change of measure generates a variance preserving

shift in mean (with the form x 7→ xµ − 1 2 µ2). If the mean shifts to the right by µ, the

log-change of measure is a positively sloped linear function which, when applied to p, shifts

probability mass from low values to high values. If the mean shifts to the left, the log-change

of measure is a negatively sloped linear function. To put it another way, a positively sloped

log-linear change of measure gives rise to excessive optimism, while a negatively sloped log-

linear change of measure gives rise to excessive pessimism.

If the log-change of measure is non-linear, then applying the change of measure impacts

the second moment. A log-change of measure with a U-shape shifts probability mass from

the center to the tails, thereby increasing the variance. A log-change of measure with an

inverted U-shape shifts probability mass from the tails into the center, thereby lowering the

variance. To put it another way, a U-shape gives rise to underconfidence, whereas an inverted

U-shape gives rise to overconfidence. With respect to (6), if Λt is large enough, then the

shape of the sentiment function will dominate the shape of the fundamental component. For

example, if the log-change of measure has an inverted U-shape which is sufficiently strong,

then Λt will overpower the other terms in (6), and dominate the shape of the log-SDF.

If the market reflects a mix of optimists and pessimists with optimism and overconfidence

being positively correlated, then log-sentiment can feature an oscillating pattern which is

sharply downward sloping in the left tail, upward sloping in the middle region, and downward

sloping in the right tail. It is this shape which characterizes the empirical findings for the

shape of the pricing kernel in Aı̈t-Sahalia and Lo (2000) and Rosenberg and Engle (2002).

In neoclassical pricing theory, the risk neutral pdf q can be obtained from the objective

pdf p by applying a change of measure using the normalized pricing kernel; e.g., p. 51 of

Cochrane (2005). Of course, this relationship can be inverted to express p as a function of

q. In the behavioral framework, an analogous relationship holds between the representative

investor’s pdf pR and q, rather than between p and q. The expression for pR as a function

of q is

pR(ST /St) = q(ST /St) (ST /St) θ1,t E

pR t [(ST /St)

−θ1,t] (8)

where E pR t is the time-t conditional expectation with respect to pR.

4.2. Estimation of Sentiment

We use least square regressions to decompose the log-SDF in (6) into its constituent com-

ponents, the sentiment function Λt and a fundamental component corresponding to a neo-

classical log-SDF. We regress the empirical log-SDF on a constant and the CRRA log-SDF.

The regression is run in log-log space because in this space the CRRA pricing kernel is lin-

ear; see (5). Fitted values give the closest, in least square sense, neoclassical CRRA pricing

kernel to the empirical one. In using this procedure, we choose a decomposition that gives

maximum weight to the neoclassical component. We then interpret the residuals from this

regression as an estimate of the sentiment function Λt.

Specifically, for each day t, we obtain a grid of 100 values of gross returns, S (i) T /St,

i = 1, . . . , 100, and regress the empirical log-SDF, log(M (i) t ), on a constant and the log

gross return, log(S (i) T /St).

11 Intercept and slope provide estimates of log(θ0,t) and −θ1,t, 11This procedure is different than Rosenberg and Engle (2002). They calibrated the constrained CRRA-SDF directly to option prices, whereas we fit the constrained CRRA-SDF to the unconstrained empirical SDF. Of course, the two procedures give theoretically the same result when the empirical SDF conforms to the CRRA pricing kernel.

respectively. For each gross return S (i) T /St, we then compute the pointwise difference

d (i) t = log(M

(i) t ) − log(M

(i) t (θ)). (9)

The differences, d (i) t , i = 1, . . . , 100, provide an estimate of the sentiment function Λt over

the support of gross returns, S (i) T /St, i = 1, . . . , 100. We repeat this procedure for each day t

and obtain a time series of θ0,t and θ1,t, as well as a series of sentiment functions Λt.

As a major robustness check of our results, we replace the CRRA SDF by a monotonic

decreasing function of log(ST /St), and then re-estimate sentiment (and other related vari-

ables), for each date t in our sample. Such a monotonic function only presumes a decreasing

marginal utility for the representative investor, without imposing any further restriction

on the utility function. Operationally, for each date t we fit the empirical log-SDF using

monotonic regressions and then take the residuals as the estimate of sentiment. The online

appendix describes the procedure in detail and shows that our results discussed below hold

true when sentiment is simply derived from a monotonic decreasing marginal utility.

5. Empirical Findings: Estimating Sentiment

Output from the estimation procedure in Sections 3–4 consists of a series of estimates for the

objective and risk neutral GARCH parameters, the SDF (Mt), CRRA (θ1,t), time preference

(θ0,t), the objective return pdf (p(ST /St)), the risk neutral pdf (q(ST /St)), the representative

investor’s pdf (pR(ST /St)), and sentiment (Λt). We first describe our dataset and then

summarize the main features of the estimation results.

5.1. Dataset

We use European options on the S&P 500 index (symbol: SPX) to calibrate the risk neutral

GARCH models. SPX options are among the most actively traded index options in the

world, have no wild card features, and can be hedged using S&P 500 futures. Consequently,

these options have been the focus of many empirical studies, e.g., Chernov and Ghysels

(2000) and Heston and Nandi (2000).

We use closing prices of out-of-the-money (OTM) put and call options on Wednesdays

from January 2, 2002 to October 28, 2009. It is known that OTM options are more actively

traded than in-the-money options.12 Option data and all the other necessary data are down-

loaded from OptionMetrics. The average of bid and ask prices are taken as option prices.

Options with time to maturity less than 10 days or more than 360 days, or prices less than

$0.05 are discarded. As in Barone-Adesi, Engle, and Mancini (2008), we also discard options

with implied volatility larger than 70% from January 2, 2002 to December 29, 2004, which

is a relatively low volatility period. For the remaining sample period, which is a relatively

high volatility period, we only discard options with implied volatility larger than 150%. This

procedure yields a sample of 121,243 options, which are roughly split in calls (45.5%) and

puts (54.5%).

Using the term structure of zero coupon risk free rates, the risk free rate for each option

maturity is obtained by linearly interpolating the two interest rates whose maturities straddle

the given maturity. This procedure is repeated for each contract and each day in the sample.

We divide the option data into several categories according to time to maturity and

moneyness, m, which is defined as the ratio of the strike price over the S&P 500 index. A

put option is said to be deep OTM if its moneyness m < 0.85, or OTM if 0.85 ≤ m < 1. A

call option is said to be OTM if 1 ≤ m < 1.15, or deep OTM if m ≥ 1.15. We also classify

option contracts according to the time to maturity: short maturity (< 60 days), medium

maturity (60–160 days), or long maturity (> 160 days).

Table 1 describes the 121,243 option prices, and their implied volatilities. The average

put (call) prices range from $1.31 ($0.67) for short maturity, deep OTM options to $43.95

($44.83) for long maturity, OTM options. OTM put and call options account for 28% and

25%, respectively, of the total sample. Short and long maturity options account for 40%

12For example, daily trading volumes of out-of-the-money put options are usually several times as large as trading volumes of in-the-money puts. This phenomenon started after the October 1987 crash and reflects the strong demand by portfolio managers for protective puts.

and 29%, respectively, of the total sample. The table also shows the familiar volatility smile

and the corresponding term structure. The smile across moneyness is evident for each set

of maturities. When the time to maturity increases, the smile tends to become flatter. The

number of options on each Wednesday is on average 296.4, with a standard deviation of

127.8, a minimum of 142, and a maximum of 726 option contracts. The average moneyness

of OTM put is 0.81, with standard deviation of 0.16, and minimum value of 0.18. The average

moneyness of OTM call is 1.21, with standard deviation of 0.24, and maximum value of 3.51.

Importantly, our estimates of the empirical pricing kernel pertain to a range of gross returns

of about 0.69 to 1.35 and a time horizon of one year. Such a range of gross returns and time

horizon are well within the span of option moneyness and time to maturities, respectively.

During the sample period, the S&P 500 ranges from a minimum of $676.5 to a maximum

of $1,565.2, with an average level of $1,157.7. The average daily log-return is close to zero

(−5.2×10−5), the standard deviation is 22.4% on an annual base, and skewness and kurtosis

are −0.13 and 12, respectively. In particular, the high kurtosis of S&P 500 returns appears

to be due to the large market swings in the fall 2008.

5.2. GARCH Estimation and Calibration

Table 2 shows objective and risk neutral GARCH parameters of the model (2)–(3).

Objective GARCH parameters are estimated quite precisely and exhibit little variation

over time. For each date t in our sample, the online appendix reports Ljung–Box and La-

grange Multiplier ARCH tests for squared daily returns and squared standardized historical

innovations. These tests show that the GARCH model is highly effective in removing the

volatility clustering in S&P 500 returns. This finding is well documented in the GARCH

literature. Thus, the GARCH model (2)–(3) provides a good description of the S&P 500

return dynamic.

Risk neutral GARCH parameters exhibit more time variation, but the persistency and

long-run mean of the GARCH volatility are estimated quite precisely. FHS GARCH param-

eters are generally less volatile than Gauss GARCH parameters, especially for the long-run

mean volatility. Both risk neutral GARCH volatilities appear to be larger and less persistent

on average than objective GARCH volatilities.13 These findings are in line with a recent lit-

erature on variance risk premium, e.g., Carr and Wu (2009), Bollerslev and Todorov (2011),

and Aı̈t-Sahalia, Karaman, and Mancini (2012).

Table 3 shows mean and root mean square error of option price errors of the risk neutral

GARCH model based on the FHS method. The price error is defined as model-based option

price minus actual option price. Average price errors tend to be positive, but root mean

square errors across all moneyness/maturity categories are small and in line with those

reported in Barone-Adesi, Engle, and Mancini (2008). Indeed, they find that the GARCH

model (2)–(3) with FHS innovations outperforms various competing models. The online

appendix shows the fitting of the GARCH model to SPX options and visually confirms the

good fit of the model.

5.3. Pricing Kernel Over Time

Figure 2 displays the empirical SDF estimated on each Wednesday from January 2002 to

October 2009 using the FHS method. At the beginning of the period, the pricing kernel

featured a declining pattern. By December of 2003, the pricing kernel featured a U-shape.

During 2005, the shape of the pricing kernel had changed to an inverted-U. In 2009, the

pricing kernel became steeper, similar to what it had been at the beginning of the sample

period. The empirical SDF based on the Gauss method features a similar evolution over

time, and is reported in the online appendix. However, it is significantly steeper to the left,

which is in line with the findings in Barone-Adesi, Engle, and Mancini (2008). The online

appendix collects additional analysis of the SDF.

13We compared our risk neutral pdf estimates with Birru and Figlewski (2012), who use a shorter time to expiration than we do. Notably, the general patterns we find appear to be similar to those in Birru and Figlewski (2012).

5.4. Representative Investor’s Beliefs, Optimism and Overconfidence

Equations (7) and (8) provide the theoretical basis for estimating the beliefs pR of the

representative investor. Equation (7) shows that dt is a scaled estimate for the sentiment

function Λt. Therefore e dt can be interpreted as being proportional to a change of measure

which transforms the objective density p into the representative investor’s density pR. 14

Figure 3 displays our estimates of optimism and overconfidence. Optimism is defined

as the difference between the expected market return under the representative investor’s

and objective pdfs, i.e., E pR t [ST /St] − E

p t [ST /St], where E

p t is the conditional expectation

under the objective pdf, computed numerically by integrating the gross return against p, and

similarly for E pR t . Overconfidence is defined as the difference between the expected volatility

of the market return under objective and representative investor’s pdfs, i.e., √ Var

p t [ST /St]−√

Var pR t [ST /St]. With the exception of the period following the Lehman bankruptcy in

September 2008, both optimism and overconfidence generally rose and fell with the market,

exhibiting procyclical behavior. The correlation coefficient for the two variables is 0.50. In

the middle of the sample, which is a relatively low volatility period of stable market growth,

the representative investor is excessively optimistic and overconfident, judging the expected

return as too high and the future volatility as too low. Notably, this pattern is reversed at

the beginning and end of the sample period, which are more turbulent periods, when the

representative investor is pessimistic and at times underconfident, especially after fall 2008.

In the discussion above, the objective expected S&P 500 return is based on the inverse

of the price-earnings ratio. The online appendix shows estimates of optimism and overconfi-

dence (as well as other quantities) when the expected return, in excess of the risk free rate,

is set to a constant value of 4%. Optimism is more stable over time, but is still econom-

ically important. Overconfidence is virtually unaffected by the alternative specification of

the expected return. The correlation between optimism and overconfidence is positive and

14As an example, Figure 1 shows the typical shape of the dt function during the middle portion of our sample period, for 21/12/2005. For this date, the dt function is positive between 0.99 and 1.16, and negative outside this interval. This implies that a change of measure based on dt, when applied to p, will shift probability mass to the region [0.99, 1.16] from the tails. The modes of p and pR are about the same, but the mass of p is more spread out than the mass of pR.

high, 0.57. The online appendix also shows that estimates of optimism and overconfidence

are nearly the same as those in Figure 3, when the CRRA SDF is replaced by a monotonic

SDF.

5.5. Impact of Sentiment on Equity and Variance Risk Premiums

We now discuss the impact of excessive optimism and overconfidence on risk premiums.

The equity risk premium is the difference between the expected return and the risk free

rate. There are two equity risk premiums, one associated with the objective pdf and the

other associated with the representative investor’s pdf. The objective equity risk premium is

negatively correlated with both excessive optimism (−0.91) and overconfidence (−0.49). The

signs are consistent with the intuition that increases in excessive optimism and overconfidence

drive up prices, thereby reducing the risk premium. Consider a regression of the objective risk

premium on its most recent lagged value, excessive optimism, and overconfidence, reported

in Table 4. The most recent lag is included as a regressor to control for the autocorrelation

of the risk premium. The coefficient on excessive optimism is negative (t-statistic = −2.48),

but the coefficient on overconfidence is not statistically significant (t-statistic = 1.46). Thus,

the equity risk premium appears to be more affected by optimism than overconfidence.

In a similar regression for the representative investor’s equity risk premium, optimism

and overconfidence have the opposite signs than the regression above and are both statis-

tically significant, see Table 4. When optimism increases and overconfidence declines, the

representative investor perceives (incorrectly) that the equity risk premium increases. From

the representative investor’s viewpoint, this is an obvious consequence, given the perceived

risk and returns.

The variance risk premium is the difference between the return variance under the ob-

jective and risk neutral distributions.15 It naturally arises when investors face a stochas-

15A fast growing literature studies the variance risk premium, e.g., Jiang and Tian (2005), Carr and Wu (2009), Todorov (2010), Bollerslev and Todorov (2011), Drechsler and Yaron (2011), Aı̈t-Sahalia, Karaman, and Mancini (2012) and Buraschi, Trojani, and Vedolin (2013). A main goal of this literature is to study the impact of the variance risk premium on asset prices.

tic investment opportunity set and require a compensation for volatility risk. Bollerslev,

Tauchen, and Zhou (2009), among others, suggest that the variance risk premium is a mea-

sure of uncertainty in the economy. Based on our estimates with the FHS method, the

average variance risk premium is negative and around −1.4% (= 0.198 − 0.212, see Table 2)

in volatility units, which is in line with the literature. Similarly to the equity risk premium,

there are two variance risk premiums, one objective and one perceived by the representative

investor. Table 4 shows the regression results of the variance risk premium on its most recent

lagged value, excessive optimism, and overconfidence. The objective variance risk premium

is significantly affected by overconfidence, but not by excessive optimism. An increase in

overconfidence induces a less negative variance risk premium, reducing the risk premium in

absolute value. This finding suggests that when investors underestimate return volatility,

they also require a lower risk premium for volatility risk. The variance risk premium per-

ceived by the representative investor appears to follow a very persistent dynamic and is not

affected by biases, once we control for its own autocorrelation.

To understand the dynamic of excessive optimism and overconfidence, we regress these

variables on its own lagged values, past one year S&P 500 return and past one year S&P 500

volatility computed using the standard deviation of daily log-returns. Table 5 shows the

regression results. Excessive optimism is positively related to past one year returns (t-

statistic is 2.17). Both excessive optimism and overconfidence are negatively related to past

volatility (t-statistics are −4.22 and −2.95, respectively). We re-run these regressions using

end-of-month rather than weekly observations, using six months rather than one year returns,

as well as other measures of past volatility, such as six months squared returns, standard

deviations of monthly returns and high minus low values of returns during the prior twelve

months. Regression results remain largely unchanged.

Chaining these relationships together, we have the following: High past returns and low

volatility leads to high excessive optimism and high overconfidence. In turn, high optimism,

which is likely to occur when the S&P 500 grows smoothly, leads to a low equity risk premium.

High overconfidence, which is likely to occur in a low volatility period, leads to a low variance

risk premium in absolute value.

6. Sentiment and External Measures

As we mentioned in the introduction, there are two distinguishing features to our analy-

sis. First, our change of measure approach to sentiment is general, formal, and precise. In

contrast, the existing behavioral finance literature has no clear formal definition of senti-

ment. Second, to estimate sentiment we use option prices, which provide the richest source

to identify variations in sentiment. In the remainder of the paper, we demonstrate that

our measure of sentiment is parsimonious, strongly reflects a disparate collection of other

sentiment measures, and yet contains additional information.

To do so, we compare our estimates of sentiment with four independent measures, namely

the Baker–Wurgler series, the Duke/CFO survey responses, the Yale/Shiller crash confidence

index, and the excess bond premium of Gilchrist and Zakraǰsek (2012). We also compare our

results to the analysis of Yu and Yuan (2011) who use the Baker–Wurgler series to study how

risk and return are related over time. In addition, we examine other aspects of sentiment,

besides optimism and overconfidence, such as biases associated with skewness, kurtosis, and

left tail events (crashes).

6.1. Relationship of Biases to Baker–Wurgler Series

We analyze the relationship between the Baker and Wurgler (2006) series (BW) and vari-

ables based on our estimates of sentiment.16 Baker and Wurgler do not provide a precise

interpretation of what their series exactly measures, although they do suggest thinking about

the series as if it measures excessive optimism for stocks. Their suggestion is intuitive, but

untested. Because we obtain estimates for both excessive optimism and overconfidence, we

16Baker and Wurgler develop two series, one which reflects economic fundamentals and a second which removes the effect of economic fundamentals. We analyze both series and the results are quite similar for both. For this reason we only report findings for the first series. An updated version of the Baker–Wurgler monthly series for sentiment is available from July 1965 through December 2010, at Jeff Wurgler’s website, http://people.stern.nyu.edu/jwurgler/.

are able to assess the degree to which the Baker–Wurgler series reflects both biases.

We find that BW heavily reflects excessive optimism. Table 6 shows a regression of BW

on its two most recent lagged values, excessive optimism and overconfidence. The t-statistic

for excessive optimism is 5.32. Including the S&P 500 monthly returns and the VIX volatility

index as regressors, the t-statistic remains high at 3.75. The online appendix shows that

optimism has a significant and strong impact on BW when estimating sentiment using a

monotonic SDF (rather than a CRRA SDF) and when setting the excess expected return

of the S&P 500 to 4% (rather than specifying the expected return as the inverse of the

price-earnings ratio). Also, allowing for an AR(2) error term in the regression, the t-statistic

of optimism remains high at 3.54. Figure 4 (top panel) visually confirms that BW and

optimism comove significantly during our sample period.

Although the coefficient of overconfidence is significant in Table 6, this finding is not

robust. Controlling for S&P 500 returns and VIX index, the t-statistic of overconfidence

drops to −1.78 when sentiment is measured using a monotonic SDF and to −1.07 when the

excess expected return of the S&P 500 is set to 4%. To save space, these regression results

are collected in the online appendix. Also, allowing for an AR(2) error term, the t-statistic

of overconfidence is only −1.48. These findings imply that the statistical significance for

overconfidence is not robust to alternative specifications.

Table 6 also shows that the VIX index has a significant negative impact on the BW series,

when not controlling for optimism and overconfidence. Including these sentiment variables

as regressors makes the impact of the VIX index on the BW series disappear. This suggests

that our sentiment variables subsume the information in the VIX index which is related

to the BW series dynamics. The online appendix shows that this finding holds true also

when sentiment is estimated using a monotonic SDF and the excess expected return of the

S&P 500 is set to 4%.

Although the BW series weakly and negatively reflects overconfidence, our estimated

sentiment functions suggest significant overconfidence in much of our sample period. Recall

that overconfidence is associated with a sentiment function, or log-change of measure, that

has the shape of an inverted U. Figure 5 illustrates several sentiment functions for the first

nine months of 2002. Notice the pronounced inverted U-shapes. We conclude that the

BW series fails to capture an important aspect of sentiment, namely the overconfidence

component that is independent of excessive optimism.

6.2. Risk, Return, and Sentiment

The existence of a positive relationship between risk and return is a cornerstone concept of

academic finance.17 Yu and Yuan (2011) report that the relationship between risk and return,

while positive when the Baker–Wurgler index is low, weakens when the Baker–Wurgler index

is high to the point where it becomes insignificant. Yu and Yuan suggest that overconfidence

plays a role in the risk-return dynamics, but do not include a measure of overconfidence

in their formal analysis. Because our approach to sentiment is quite general, we are able

to extend the discussion to explicitly analyze the role of overconfidence in the risk-return

dynamics.

We regress (ex-ante) expected return on (ex-ante) return standard deviation, and a con-

stant, under the objective pdf p. Table 7 shows the regression results. The slope coefficient is

0.12 and the intercept is 0.02, and both estimates are statistically significant. Using end-of-

month observations, regression estimates are nearly the same as in Table 7, and statistically

significant. These parameter values are generally consistent with neoclassical theory.

In the behavioral approach, prices reflect not the objective pdf p but the representative

investor’s pdf pR. A regression of (ex-ante) expected return on (ex-ante) return standard

deviation under the representative investor’s pdf pR has a slope coefficient of −0.13 and

an intercept of 0.07, and both estimates are statistically significant. Using end-of-month

observations, the slope coefficient is −0.11 and the intercept is 0.06, again both statistically 17A large empirical literature studies the risk-return trade-off. After two decades of empirical research, there is little consensus on the basic properties of the relation between the expected market return and volatility. Studies such as Goyal and Santa-Clara (2003), Ghysels, Santa-Clara, and Valkanov (2005), Guo and Whitelaw (2006), and Ludvigson and Ng (2007) find a positive trade-off, while conversely Nelson (1991), Glosten, Jagannathan, and Runkle (1993), Whitelaw (1994), Brandt and Kang (2004), and Conrad, Dittmar, and Ghysels (2013) find a negative trade-off. Harvey (1989, 2001) shows that the risk-return relation changes over time. Rossi and Timmermann (2010) provide empirical evidence that the relation may be not linear.

significant. The online appendix shows that the perceived negative risk-return trade-off ex-

ists also when the expected excess return of the S&P 500 is set to 4% and when sentiment

is measured using a monotonic SDF, although in the latter case the relationship is weaker.

The negative slope coefficient reflects the perspective that risk and return are negatively

related.18 Shefrin (2008) discusses several studies about the perception that risk and return

are negatively related.19 One key behavioral feature involves excessive optimism and over-

confidence being positively correlated. In our data, the correlation between the two series is

0.50. A positive correlation implies that whenever the representative investor is excessively

optimistic and overestimates expected return, he tends to be overconfident and underesti-

mates future volatility. Associating high returns to low risk is the hallmark of a negative

perceived relationship.

We hasten to add that a positive correlation between excessive optimism and overconfi-

dence does not necessarily imply a negative perceived relationship between risk and return.

This is because if sentiment is small, then it will not override the fundamental component.

For example, during the period September 2008 through the end of our sample period, the

representative investor’s perceived risk and return were positively correlated (with a regres-

sion slope coefficient of 0.07). At the same time, excessive optimism and overconfidence were

still positively related, with a correlation coefficient of 0.8, but sentiment was small during

this period.20

Figure 6 shows the relationship between expected returns and return standard deviations

under the objective pdf and the representative investor’s pdf. The opposite relationships

emerge clearly.

Table 7 reports regression results of (ex-post) realized one year returns on (ex-ante) ex-

18Marfè (2012) develops an equilibrium model to account for a negative risk-return trade-off. 19These studies focus on behavior at the level of the individual, and suggest that excessive optimism and overconfidence are positively correlated across the population. From a dynamic perspective, wealth transfers resulting from trading will induce a time series correlation as well. This occurs as wealth shifts from, say, less optimistic, less confident investors to more optimistic, more confident investors, inducing an increase over time in both the representative investor’s degree of excessive optimism and overconfidence. 20See the graph of the distance measures, RMSE and MAE, between the log empirical SDF and log CRRA SDF, reported in the online appendix.

pected returns, and a constant. As the frequency of observations is weekly, such predictive

regressions need to be interpreted cautiously.21 The relationship between the representa-

tive investor’s expected return and subsequent realized return is negative and statistically

insignificant with a t-statistic −1.84. Using end-of-month observations, the t-statistic even

drops to −0.2. The relationship between objective expected return and subsequent real-

ized return is positive and statistically significant, with a t-statistic of 4.8. Notice that the

R-squared associated with the objective expected return is higher than the R-squared associ-

ated with the representative investor’s expected return. Moreover, maximizing the predictive

power of the representative investor’s expected return requires using a negative regression

slope, in effect to control for the bias.

Like Yu and Yuan (2011), we find that for the objective pdf, the relationship between

risk and return is weaker when the Baker–Wurgler sentiment is positive than when it is

negative.22 Yu and Yuan suggest that this is because when sentiment is high, constraints on

short sales magnify the impact of investor errors.

When we perform the same analysis for the representative investor’s pdf encapsulating the

“market’s perception” of risk and return, which provides the basis for pricing assets, we find

no statistically discernable difference between periods of high sentiment and periods of low

sentiment. Rather, our analysis indicates that the negative relationship stems from excessive

optimism and overconfidence being positively correlated and strong. Yu and Yuan (2011)

base their regression analysis on the Baker–Wurgler index, which is effectively a measure only

of excessive optimism. Yet, their informal explanation is based on both excessive optimism

and overconfidence. Our findings suggest that the perceived risk-return trade-off is driven by

the co-movements of excessive optimism and overconfidence, not just the level of excessive

optimism.

21Boudoukh, Whitelaw, and Richardson (2008) show that even in the absence of any increase in the return predictability, the values of R-squared in regressions involving highly persistent predictor variables and overlapping returns will by construction increase roughly proportionally to the return horizon and the length of the overlap. Baker, Taliaferro, and Wurgler (2006) and Goyal and Welch (2008) provide related studies. 22In going from negative to positive sentiment in our analysis, the coefficient of expected return on standard deviation drops by about half, from 0.12 to 0.07. Both coefficients are statistically significant, implying that we do not find the relationship to become flat when sentiment is high.

6.3. Duke/CFO Survey Responses

To provide another external check on our sentiment estimates, we use the Duke/CFO survey

data. The questions in the Duke/CFO survey that are most relevant to our study pertain

to expected S&P 500 return, volatility, and skewness, for a one year horizon. Graham and

Harvey (2012) describe how the survey is conducted and provide an overview of the survey

results.23

The estimates for expected return, volatility, and skewness that are derived form the

Duke/CFO survey responses provide an interesting contrast to our estimates from the repre-

sentative investor’s pdf. Figure 7 (top panel) shows that the Duke/CFO expected return and

representative investor’s expected return are highly correlated after 2005, with a correlation

coefficient of 0.6.24 For the entire sample period, the correlation coefficient is 0.2.

As discussed in Ben-David, Graham, and Harvey (2013), the Duke/CFO series exhibits

very large overconfidence, with an average one year return volatility around 5%. The repre-

sentative investor’s conditional return volatility is around 20%, and thus more in line with

historical levels. Interestingly, the correlation between the Duke/CFO volatility series and

the representative investor’s return volatility is a very high 0.8; see Figure 7 (bottom panel).

Although the two volatility predictions are an order of magnitude different, the two measures

comove strongly.

As for skewness, the correlation between the Duke/CFO values and the representative

investor values is negative (−0.39). The former features an inverted-U shape over time,

while the latter is U-shaped over time, as shown in the online appendix. This suggests that

when volatility increases at the beginning and end of our sample, the respondents to the

Duke/CFO survey overfocus on volatility associated with negative returns. In contrast, the

representative investor focuses on high positive returns, as well as negative returns, during

periods of heightened volatility.

23An archive of past surveys is available under the “Past Results” tab at http://www.cfosurvey.org. 24The sample of CFOs changed in 2004 when Duke changed survey partners from Financial Executives International to CFO magazine. For this reason, the data from 2005 on appears to be more consistent than the data from the earlier period.

6.4. Yale/Shiller Crash Confidence Indexes

Next we turn our attention to left tail events, meaning crashes. To do so, we compare the

probability of a left tail event under the representative investor’s pdf with two indepen-

dent survey-based counterparts, the Yale/Shiller crash confidence indexes for professional

investors (CP), and for individual investors (CI). Each crash confidence index is the percent

of respondents who attach little probability to a stock market crash in the next six months.25

Therefore, a crash confidence index pertains to the probability of a left tail event.

To compare the crash confidence indexes CP and CI with our representative investor

approach, we consider left tail probabilities under the representative investor’s and objective

pdfs. For each date t, we compute the conditional probabilities of a one year market return

being less than −20%, i.e., pR{ST /St < 0.8} and p{ST /St < 0.8}, slightly abusing the

notation. Then we define the left tail sentiment bias as log(pR{ST /St < 0.8}/p{ST /St <

0.8}), which resembles the sentiment function Λt in (7).

We find that the correlation coefficient between the representative investor’s left tail

probability pR{ST /St < 0.8} and CP is −0.79, and for CI is −0.63. In and of itself, there is

no prior stipulation that CP need reflect investors’ bias. However, the correlation coefficient

between CP and the objective left tail probability p{ST /St < 0.8} is only −0.57. Moreover,

an AR(2) regression of CP on p{ST /St < 0.8} and the left tail sentiment bias has only the

bias term being statistically significant, with a t-statistic of −2.0.

Figure 4 (bottom panel) shows CP and pR{ST /St > 0.8}, where the latter is the probabil-

ity of not having a crash under the representative investors’ pdf. The comovements between

the two series are evident. For example, both series reach lowest levels at the end of 2002

and 2008, i.e., periods of market turmoil. Figure 4 (bottom panel) also suggests that the

fear of a market crash, as measured by pR{ST /St > 0.8} and CP, fell in the middle part of

our sample period. This period is characterized by relatively stable market growth and low

volatility, as well as high excessive optimism and overconfidence, Figure 3.

25A detail description of the index and corresponding data are available at http://icf.som.yale.edu/stock- market-confidence-indices-explanation.

6.5. Corporate Bond Default Premiums

Given that left tail events give rise to corporate bond defaults, there is reason to expect that

left tail probabilities under the representative investor’s distribution impact credit spreads.

We now investigate whether this is the case.

In recent work, Gilchrist and Zakraǰsek (2012) develop an excess bond premium measur-

ing the component of corporate bond spreads that is not related to firm-specific information

on expected defaults. Gilchrist and Zakraǰsek contend that a rise in the excess bond pre-

mium represents a reduction in the effective risk-bearing capacity of the financial sector

and, as a result, a contraction in the supply of credit with significant adverse consequences

for the macroeconomy. Their credit spread index decomposes into a predictable component

that captures the available firm-specific information on expected defaults and a residual

component – the excess bond premium.

We find a strong positive impact of left tail probabilities under the representative in-

vestor’s distribution on the excess bond premium. The correlation between the excess bond

premium and pR{ST /St < 0.8} is 0.9.26 In addition, when regressing the excess bond

premium on pR{ST /St < 0.8}, p{ST /St < 0.8} and a constant, the t-statistics are 5.25,

1.09 and −4.74, respectively, with an R-squared of 82%. Although pR{ST /St < 0.8} and

p{ST /St < 0.8} are highly correlated (85%), the regression result above confirms that the

representative investor’s left tail probability has a large impact on the excess bond premium,

and subsumes the information in the objective left tail probability. Further regression anal-

ysis reveals that the excess bond premium is significantly affected by the representative

investor’s left tail probability (t-statistic = 6.9), when controlling for excessive optimism

and overconfidence. This suggests that a large amount of sentiment, when concentrated on

left tail events, can impair the intermediation capacity of the financial sector with negative

effects on the real economy.

26We thank Simon Gilchrist for providing us with the data for this series.

7. Conclusion

The main contribution of this paper is the estimation of a theoretically-based notion of

sentiment, using option prices, market returns, and risk free rates. We analyze sentiment

using two investors, one (representative investor) who sets prices and one (econometrician)

whose beliefs are objectively correct. The difference between the two provides us with a

theoretically grounded measure of sentiment. In this regard, option prices provide the richest

source to identify variations in sentiment. In contrast to the existing behavioral finance

literature, which has no clear formal definition of sentiment, our definition of sentiment

is general, formal, and precise. Our measure of sentiment is also parsimonious, strongly

reflecting a disparate collection of other sentiment measures, and yet contains additional

information.

Our estimates of sentiment are consistent with independent measures of investor senti-

ment such as the Baker–Wurgler series, the Duke/CFO survey data, the Yale/Shiller crash

confidence indexes, and the corporate bond default premium developed by Gilchrist and

Zakraǰsek. For much of our sample period, the pricing kernel features an upward sloping

portion, which is consistent with overconfidence bias.

Our analysis suggests that the Baker–Wurgler series robustly reflects excessive optimism,

but not the component of overconfidence that is independent of excessive optimism. We also

find that the Yale/Shiller crash confidence indexes are effectively sentiment measures that are

highly correlated with left tail probabilities under the representative investor’s distribution,

and for that matter with overconfidence as well.

Excessive optimism and overconfidence are positively correlated throughout our sample

period. During periods of stable market growth and low volatility, the representative investor

is excessively optimistic and overconfident. During a crisis, he is pessimistic and underconfi-

dent. During periods in which these biases are strong, the representative investor perceives

risk and return to be negatively related, whereas objectively they are positively related.

A. Derivation of the Sentiment Function Λ

The sentiment function encapsulates the representative investor’s biases. In this section, we

briefly describe the structure of the sentiment function, and its manifestation within the

SDF. To simplify notation, we drop the t-subscripts and the argument of the pdf.

Let ξ denote state price. Then the SDF is given by M = ξ/p, which in a representative

investor CRRA-framework has the form ξ = pR θ0 (ST /St) −θ1. This last relationship follows

from the optimizing condition in which marginal rate of substitution (for expected utility)

is set equal to relative state prices, with consumption at t = 0 serving as numeraire.27

Divide both sides of the previous equation for ξ by p θ0,e, where θ0,e corresponds to the

value of θ0 that would prevail if all investors held correct beliefs. Here, the subscript e denotes

efficiency. This last operation leads to the expression ξ/p = (θ0/θ0,e) (pR/p) θ0,e (ST /St) −θ1.

Define eΛ = (θ0/θ0,e) (pR/p), which is a scaled change of measure and corresponds to (7).

The change of measure (pR/p) associated with Λ exactly specifies the transformation of

the objective pdf p into the representative investor’s pdf pR. Therefore, Λ encapsulates the

representative investor’s biases.

Shefrin (2008) establishes that θ1 does not vary as investors’ beliefs change. Then, in the

preceding expression for the SDF, eΛ multiplies the term θ0,e (ST /St) −θ1, and the latter is

the SDF Me that would prevail if all investors held correct beliefs. Therefore M = e ΛMe.

Taking logs, obtain log(M) = Λ + log(Me). This expression stipulates that the log-SDF can

be decomposed into two components, one being the sentiment function and the other being

the neoclassical log-SDF that would prevail if all investors held correct beliefs.

Rearranging the decomposition of the log-SDF yields Λ = log(M) − log(Me). Notably,

the last relationship corresponds to (9) and explains why d serves as our estimate of the

sentiment function Λ.

27For the purpose of this discussion, we assume a representative investor with CRRA-preferences. Shefrin (2008) develops an aggregation theorem for a model involving heterogeneous investors in which the repre- sentative investor’s preferences are approximately CRRA, and whose beliefs are given by a Hölder average. To reduce complexity, we mostly abstract from aggregation issues in this paper.

Maturity Less than 60 60 to 160 More than 160

Moneyness Mean Std. Mean Std. Mean Std. < 0.85 Put price $ 1.31 2.74 3.89 5.61 10.20 11.63

σbs% 49.44 20.16 39.45 14.83 31.26 10.18 Observations 11,849 10,359 10,292

0.85–1.00 Put price $ 9.94 10.67 24.37 16.55 43.95 23.18

σbs% 23.74 10.21 23.52 8.64 22.20 6.86 Observations 15,876 9,510 8,152

1.00–1.15 Call price $ 8.58 10.76 21.07 17.48 44.83 25.49

σbs% 17.76 8.86 18.57 7.99 18.22 6.46 Observations 13,807 9,201 7,538

> 1.15 Call price $ 0.67 1.67 1.74 3.65 6.38 9.33

σbs% 37.71 16.05 26.79 9.62 21.61 6.53 Observations 7,136 7,977 9,530

Table 1. Option dataset. For each moneyness/maturity category, entries show mean and standard deviation (Std.) of out-of-the-money call and put option prices on the S&P 500

index, as well as of Black–Scholes implied volatility (σbs) in percentage. Sample data are options observed on Wednesdays from January 2002 to October 2009. Observations are the number of options for each moneyness/maturity category. Filtering criteria of options are described in Section 5. Moneyness is strike price divided by S&P 500 index. Maturity is in calendar days.

ω × 106 β α × 103 γ Persist. Ann. vol. Objective GARCH parameters

Mean 1.215 0.926 3.473 0.117 0.989 0.198 Std. 0.207 0.005 4.141 0.013 0.002 0.016

Risk Neutral FHS GARCH parameters Mean 4.153 0.789 2.169 0.358 0.970 0.212 Std. 5.600 0.208 9.366 0.360 0.033 0.074

Risk Neutral GAUSS GARCH parameters Mean 3.987 0.756 3.479 0.448 0.983 0.252 Std. 5.575 0.201 12.280 0.371 0.021 0.112

Table 2. Objective and risk neutral GARCH parameters. The GARCH model is log(Su/Su−1) = µu + ϵu, where Su is the S&P 500 index at day u, µu is the drift, and the conditional variance σ2u = ω + βσ

2 u−1 + αϵ

2 u−1 + γIu−1ϵ

2 u−1, where ϵu = σuzu, zu is a

standardized innovation and Iu−1 = 1 when ϵu−1 < 0, and Iu−1 = 0 otherwise. For each Wednesday from January 2002 to October 2009, a GARCH model is estimated using histor- ical daily S&P 500 returns by maximizing a Pseudo Maximum Likelihood, a GARCH model driven by Gaussian innovations is calibrated to out-of-the-money options on the S&P 500 index by minimizing the sum of squared pricing errors, a GARCH model driven by filtered historical innovations is similarly calibrated to options on the S&P 500 index. Persist. is the persistency of the GARCH volatility and given by β + α + γ/2. Ann. vol. is the annualized long-run mean of the GARCH volatility.

Maturity Less than 60 60 to 160 More than 160

Moneyness Mean RMSE Mean RMSE Mean RMSE < 0.85 0.49 1.20 0.55 1.22 0.56 1.46

0.85–1.00 0.41 1.31 −0.41 1.43 −0.95 1.70 1.00–1.15 0.44 1.29 0.03 1.19 0.42 1.55 > 1.15 0.08 0.46 0.12 0.57 0.94 1.75

Table 3. Option price errors. For each moneyness/maturity category, entries show mean and root mean square error (RMSE) of option price errors of the risk neutral FHS GARCH model. Price error is defined as model-based option price minus actual option price. Using the FHS method, each Wednesday from January 2002 to October 2009, the GARCH model is calibrated to out-of-the-money call and put options on the S&P 500 index. Calibration procedure is described in Section 3. Filtering criteria of options are described in Section 5. Moneyness is strike price divided by S&P 500 index. Maturity is in calendar days.

Intercept Lag1 Optimism Overconf. R2

Panel A: Objective Equity Risk Premium 0.18 0.92 −0.09 0.01 0.99 (1.98) (7.40) (−2.48) (1.46) Panel B: Representative Investor’s Equity Risk Premium 0.79 0.71 0.07 −0.02 0.56 (4.79) (13.26) (3.96) (−2.02)

Panel C: Objective Variance Risk Premium −1.57 0.33 0.05 0.38 0.92 (−5.40) (3.15) (0.81) (4.85) Panel D: Representative Investor’s Variance Risk Premium −0.04 0.96 0.01 −0.01 0.94 (−1.03) (5.95) (0.81) (−0.85)

Table 4. Impact of Sentiment on Equity and Variance risk premiums. Panel A: Time series regression of objective equity risk premium on a constant (Intercept), its two most recent lagged value (Lag1), optimism and overconfidence; t-statistics in parentheses. Objective equity risk premium is (E

p t [ST /St]−E

q t [ST /St])×100, where E

p t is the conditional expectation

at date t under the objective pdf p, E q t is the conditional expectation at date t under the

risk neutral pdf q, St is the S&P 500 index at date t, and (T − t) is one year. Optimism is (E

pR t [ST /St]−E

p t [ST /St])×100, where E

pR t is the conditional expectation at date t under the

representative investor’s pdf pR. Overconfidence is ( √ Var

p t [ST /St] −

√ Var

pR t [ST /St]) × 100.

Panel B: Same time series regression as in Panel A for the equity risk premium perceived by the representative investor, defined as (E

pR t [ST /St] − E

q t [ST /St]) × 100. Panel C: Time

series regression of objective variance risk premium on a constant (Intercept), its two most recent lagged value (Lag1), optimism and overconfidence. Objective variance risk premium is (Var

p t [ST /St]−Var

q t [ST /St])×100. Panel D: Same time series regression as in Panel C for the

variance risk premium perceived by the representative investor, defined as (Var pR t [ST /St] −

Var q t [ST /St]) × 100. R2 is the adjusted R-squared. Robust standard errors are computed

using the Newey and West (1987) covariance matrix estimator with the number of lags optimally chosen according to Andrews (1991). Weekly observations from January 2002 to October 2009.

Intercept Lag1 Ret Stdv R2

Panel A: Optimism 0.70 0.67 0.64 −3.50 0.91 (4.55) (10.55) (2.17) (−4.22)

Panel B: Overconfidence 0.87 0.86 −0.22 −2.48 0.83 (3.93) (10.54) (−0.40) (−2.95)

Table 5. Optimism and overconfidence. Panel A: Time series regression of optimism on a constant (Intercept), its most recent lagged value (Lag1), past one year S&P 500 return (Ret) and past one year S&P 500 volatility (Stdv) namely the standard deviation of daily S&P 500 log-returns. Optimism is (E

pR t [ST /St] − E

p t [ST /St]) × 100, where E

pR t is the condi-

tional expectation at date t under the representative investor’s pdf pR, E p t is the conditional

expectation at date t under the objective pdf p, St is the S&P 500 index at date t, and (T − t) is one year. Panel B: Same time series regression for overconfidence, defined as ( √ Var

p t [ST /St] −

√ Var

pR t [ST /St]) × 100. R2 is the adjusted R-squared. Robust standard

errors are computed using the Newey and West (1987) covariance matrix estimator with the number of lags optimally chosen according to Andrews (1991). Weekly observations from January 2002 to October 2009.

Baker–Wurgler Series Intercept Lag1 Lag2 Optimism Overconf. S&P VIX R2

0.01 0.92 −0.06 4.35 −1.00 0.93 (0.49) (10.21) (−0.75) (5.32) (−2.22) 0.00 0.92 −0.06 4.65 −1.09 −0.06 0.03 0.93 (0.11) (10.23) (−0.76) (3.75) (−2.01) (−0.15) (0.16) 0.08 0.94 −0.03 −0.27 −0.45 0.92 (3.45) (8.86) (−0.29) (−0.71) (−4.56)

Table 6. Baker–Wurgler series and sentiment. Time series regression of monthly Baker– Wurgler series on a constant (Intercept), its two most recent lagged values (Lag1, Lag2), optimism, overconfidence, S&P 500 monthly return (S&P), and VIX index; t-statistics in parentheses. Optimism is E

pR t [ST /St]−E

p t [ST /St], where E

pR t is the conditional expectation

at date t under the representative investor’s pdf pR, E p t is the conditional expectation at

date t under the objective pdf p, St is the S&P 500 index at date t, and (T − t) is one year. Overconfidence is

√ Var

p t [ST /St] −

√ Var

pR t [ST /St]. Robust standard errors are computed

using the Newey and West (1987) covariance matrix estimator with the number of lags optimally chosen according to Andrews (1991). R2 is the adjusted R-squared. Observations are end-of-month from January 2002 to October 2009.

Intercept Slope R2

1. Objective expected return vs. volatility 0.02 0.12 0.67 (7.73) (6.26)

2. Rep. investor expected return vs. volatility 0.07 −0.13 0.50 (15.20) (−5.03)

3. Ex-post vs. objective expected return −0.48 11.26 0.19 (−3.56) (4.80)

4. Ex-post vs. rep. investor expected return 0.34 −6.36 0.12 (2.26) (−1.84)

Table 7. Risk and return. Regression 1: Time series regression of objective expected mar- ket return on a constant (Intercept) and expected objective volatility (Slope); t-statistics in parentheses. Objective expected return is E

p t [ST /St − 1], where E

p t is the conditional expec-

tation at date t under the objective pdf p, St is the S&P 500 index at date t, and (T − t) is one year; expected objective volatility is

√ Var

p t [ST /St]. Regression 2: Same regression

as Regression 1 for expected return and volatility under the representative investor’s pdf. Regression 3: Time series regression of ex-post, actual annual return on a constant (Inter- cept) and objective expected return (Slope). Regression 4: Same regression as Regression 3 for representative investor’s expected return. Robust standard errors are computed using the Newey and West (1987) covariance matrix estimator with the number of lags optimally chosen according to Andrews (1991). R2 is the adjusted R-squared. Observations are weekly from January 2002 to October 2009.

0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 0

0.5

1.5

2.5

3.5

Gross Return

Pobj Prep

0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 −3

−2

−1

Gross Return

CRRAKernel BehavKernel LogDiff

Figure 1. Upper graph: Objective (Pobj) and representative investor’s (Prep) probability density functions for the date 21/12/2005. Lower graph: Behavioral unconstrained SDF (BehavKernel), CRRA-constrained SDF (CRRAKernel), and the LogDiff function, i.e., log BehavKernel minus log CRRAKernel. The latter difference is the function dt in (9), for the date 21/12/2005.

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0

0.5

1.5

2.5

3.5

S D

F 2 0 0 2 2

0 0 3

S T / S

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0

0.5

1.5

2.5

3.5

S D

F 2 0 0 4 2

0 0 5

S T / S

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0

0.5

1.5

2.5

3.5

S D

F 2 0 0 6 2

0 0 7

S T / S

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 0

0.5

1.5

2.5

3.5

S D

F 2 0 0 8 2

0 0 9

S T / S

Figure 2. Empirical SDF. For each Wednesday t in our sample, the empirical stochastic discount factor (SDF), Mt, is estimated as Mt = e

−rf (T−t) q(ST /St)/p(ST /St), where q is the conditional risk neutral density of ST /St, p the conditional objective density of ST /St, rf is the risk free rate, St the S&P 500 index at date t, and (T −t) is one year. The densities p and q are conditional on the information available at date t and based on GARCH models with FHS innovations estimated using historical S&P 500 returns and SPX options, respectively. Each graph shows the empirical SDF over the corresponding two years period. Superimposed (solid thick line) is the average SDF.

2002/10 2004/02 2005/07 2006/11 2008/04 2009/08 −6

−4

−2

2002/10 2004/02 2005/07 2006/11 2008/04 2009/08 −5

20 Optimism (left axis) Overconfidence (right axis)

Figure 3. Time series for optimism and overconfidence. Optimism is (E pR t [ST /St] −

E p t [ST /St]) × 100, where E

pR t is the time-t conditional expectation under the representa-

tive investor’s pdf pR, St is the S&P 500 index at date t, (T − t) is one year, and similarly for E

p t . Overconfidence is (

√ Var

p t [ST /St] −

√ Var

pR t [ST /St]) × 100. Density estimates are

obtained using the FHS method.

2002/10 2004/02 2005/07 2006/11 2008/04 2009/08 −6

−4

−2

2002/10 2004/02 2005/07 2006/11 2008/04 2009/08 −1

−0.5

0.5

1.5 Optimism (left axis) Baker−Wurgler (right axis)

2002/10 2004/02 2005/07 2006/11 2008/04 2009/08 0.75

0.8

0.85

0.9

0.95

2002/10 2004/02 2005/07 2006/11 2008/04 2009/08 10

Prob{S T /S

t > 0.8} (left axis)

Yale/Shiller Crash Confidence Index (right axis)

Figure 4. Upper graph: Baker–Wurgler sentiment series and optimism. Baker and Wurgler (2006) monthly series of sentiment extracted using Principal Component Analysis of six specific sentiment proxies, i.e., turnover on the New York Stock Exchange (NYSE), dividend premium, closed-end fund discount, number and first-day returns on IPOs, and the equity share in new issues. Optimism is (E

pR t [ST /St]−E

p t [ST /St])×100, where E

pR t is the conditional

expectation at date t under the representative investor’s pdf pR, St is the S&P 500 index at date t, (T − t) is one year, and similarly Ept is the conditional expectation under the objective pdf p. Lower graph: Yale/Shiller crash confidence index (CP) and “probability of no crash” under the representative investor’s pdf. The latter is Prob{ST /St > 0.8} under the representative investor’s pdf pR. For each Wednesday t, from January 2002 to October 2009, the conditional probability Prob{ST /St > 0.8} is computed numerically integrating the conditional density pR of the gross return ST /St, given the information available at date t.

0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 −1

−0.8

−0.6

−0.4

−0.2

0.2

0.4

0.6

0.8

Gross Return

S e

n tim

e n

t Λ

30.01.2002 27.02.2002 27.03.2002 24.04.2002 29.05.2002 26.06.2002 31.07.2002 28.08.2002 25.09.2002

Figure 5. Sentiment functions plotted for several days in 2002. The sentiment function at date t is Λt = log(Mt) − log(Mt(θ)), where Mt = e−rf (T−t) q(ST /St)/p(ST /St) is the unconstrained SDF and Mt(θ) = θ0,t (ST /St)

−θ1,t is the CRRA-constrained SDF. q is the conditional risk neutral density of ST /St, p is the conditional objective (i.e., historical) density of ST /St, rf is the instantaneous risk free rate, θ0,t is the time discount factor, θ1,t is the coefficient of relative risk aversion, St is the S&P 500 index at date t, and (T − t) is one year. On the x-axis, gross return is ST /St.

10 20 30 40 50 1

Stdv. Return, Objective

E xp

e ct

e d

R e

tu rn

, O

b je

ct iv

10 20 30 40 50 1

Stdv. Return, Rep. Investor

E xp

e ct

e d

R e

tu rn

, R

e p

. In

ve st

o r

Figure 6. Risk and return. For each Wednesday t from January 2002 to October 2009, “Expected Return, Objective” is the time-t conditional expected market return under the objective pdf p, i.e., E

p t [ST /St − 1] × 100, where St is the S&P 500 index at date t, and

(T −t) is one year; “Stdv. Return, Objective” is the time-t conditional expected volatility of market return under the objective pdf p, i.e.,

√ Var

p t [ST /St] × 100. “Expected Return, Rep.

Investor” and “Stdv. Return, Rep. Investor” are representative investor’s expected return and volatility, respectively, computed using time-t conditional representative investor’s pdf, pR. In each graph, superimposed is the regression line.

2002/10 2004/02 2005/07 2006/11 2008/04 2009/08

Expected Ret Duke/CFO Expected Ret RepInv

2002/10 2004/02 2005/07 2006/11 2008/04 2009/08 0

40 Stdv Ret Duke/CFO (left axis) Stdv Ret Rep Inv (right axis)

Figure 7. Upper graph: Time series of one year S&P 500 expected return based on Duke/CFO survey responses and the representative investor’s distribution. Duke/CFO sur- vey data are described in Graham and Harvey (2012), quarterly frequency. The represen- tative investor one year S&P 500 expected return is given by E

pR t [ST /St − 1] × 100, where

E pR t is the conditional expectation at each Wednesday t in our sample under the represen-

tative investor’s pdf pR, St is the S&P 500 index at date t, and (T − t) is one year; weekly frequency. Lower graph: Time series of one year S&P 500 return standard deviation based on Duke/CFO survey responses and the representative investor’s distribution. For each Wednesday t in our sample, return standard deviation under the representative investor’s

pdf is √ Var

pR t [ST /St] × 100.

References

Aı̈t-Sahalia, Y., M. Karaman, and L. Mancini, 2012, “The Term Structure of Variance Swaps, Risk Premia and the Expectation Hypothesis,” working paper, Princeton University and EPFL.

Aı̈t-Sahalia, Y., and A. Lo, 2000, “Nonparametric Risk Management and Implied Risk Aver- sion,” Journal of Econometrics, 94, 9–51.

Andrews, D., 1991, “Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation,” Econometrica, 59, 817–858.

Baker, M., R. Taliaferro, and J. Wurgler, 2006, “Predicting Returns with Managerial Deci- sion Variables: Is There a Small-Sample Bias?,” Journal of Finance, 61, 1711–1730.

Baker, M., and J. Wurgler, 2004, “A Catering Theory of Dividends,” Journal of Finance, 59, 1125–1165.

, 2006, “Investor Sentiment and the Cross-section of Stock Returns,” Journal of Finance, 61, 1645–1680.

, 2007, “Investor Sentiment in the Stock Market,” Journal of Economic Perspectives, 21, 129–151.

Barberis, N., M. Huang, and T. Santos, 2001, “Prospect Theory and Asset Prices,” Quarterly Journal of Economics, 116, 1–53.

Barberis, N., and R. Thaler, 2003, “A Survey of Behavioral Finance,” in Handbook of the Economics of Finance, ed. by G. Constantinides, R. Stulz, and M. Harris. North Holland, Amsterdam.

Barone-Adesi, G., R. Engle, and L. Mancini, 2008, “A GARCH Option Pricing Model in Incomplete Markets,” Review of Financial Studies, 21, 1223–1258.

Bekaert, G., and G. Wu, 2000, “Asymmetric Volatility and Risk in Equity Markets,” Review of Financial Studies, 13, 1–42.

Ben-David, I., J. R. Graham, and C. R. Harvey, 2013, “Managerial Miscalibration,” Quar- terly Journal of Economics, forthcoming.

Birru, J., and S. Figlewski, 2012, “Anatomy of a Meltdown: The Risk Neutral Density for the S&P 500 in the Fall of 2008,” Journal of Financial Markets, 15, 151–180.

Black, F., 1976, “Studies of Stock Price Volatility Changes,” in Proceedings of the 1976 Meetings of the American Statistical Association, pp. 171–181.

Bollerslev, T., G. Tauchen, and H. Zhou, 2009, “Expected Stock Returns and Variance Risk Premia,” Review of Financial Studies, 22, 4463–4492.

Bollerslev, T., and V. Todorov, 2011, “Tails, Fears and Risk Premia,” Journal of Finance, 66, 2165–2211.

Bollerslev, T., and J. M. Wooldridge, 1992, “Quasi-maximum Likelihood Estimation and Inference in Dynamic Models with Time Varying Covariances,” Econometric Reviews, 11, 143–172.

Boudoukh, J., R. Whitelaw, and M. Richardson, 2008, “The Myth of Long-Horizon Pre- dictability,” Review of Financial Studies, 21, 1577–1605.

Brandt, M., and Q. Kang, 2004, “On the Relationship Between the Conditional Mean and Volatility of Stock Returns: A Latent VAR Approach,” Journal of Financial Economics, 72, 217–257.

Buraschi, A., F. Trojani, and A. Vedolin, 2013, “When Uncertainty Blows in the Orchard: Comovement and Equilibrium Volatility Risk Premia,” Journal of Finance, forthcoming.

Campbell, J. Y., and L. Hentschel, 1992, “No News is Good News: An Asymmetric Model of Changing Volatility in Stock Returns,” Journal of Financial Economics, 31, 281–318.

Campbell, J. Y., and R. J. Shiller, 1998, “Valuation Ratios and the Long Run Market Outlook,” Journal of Portfolio Management, 24, 11–26.

Carr, P., and L. Wu, 2009, “Variance Risk Premiums,” Review of Financial Studies, 22, 1311–1341.

Cheng, I.-H., S. Raina, and W. Xiong, 2013, “Wall Street and the Housing Bubble,” working paper, University of Michigan and Princeton University.

Chernov, M., and E. Ghysels, 2000, “A Study Towards a Unified Approach to the Joint Estimation of Objective and Risk Neutral Measures for the Purpose of Options Valuation,” Journal of Financial Economics, 57, 407–458.

Christie, A., 1982, “The Stochastic Behavior of Common Stock Variances: Value, Leverage and Interest Rate Effects,” Journal of Financial Economics, 10, 407–432.

Cochrane, J., 2005, Asset Pricing. Princeton University Press, Princeton, Second edition.

Conrad, J., R. F. Dittmar, and E. Ghysels, 2013, “Ex Ante Skewness and Expected Stock Returns,” Journal of Finance, 68, 85–124.

Daniel, K. D., D. Hirshleifer, and A. Subrahmanyam, 1998, “Investor Psychology and Secu- rity Market Under- and Overreactions,” Journal of Finance, 53, 1839–1885.

, 2001, “Overconfidence, Arbitrage, and Equilibrium Asset Pricing,” Journal of Fi- nance, 56, 921–965.

De Bondt, W., and R. H. Thaler, 1985, “Does the Stock Market Overreact?,” Journal of Finance, 40, 793–805.

, 1987, “Further Evidence on Investor Overreaction and Stock Market Seasonality,” Journal of Finance, 42, 793–805.

Drechsler, I., and A. Yaron, 2011, “What’s Vol Got to Do With It,” Review of Financial Studies, 24, 1–45.

Duan, J.-C., and J. Simonato, 1998, “Empirical Martingale Simulation for Asset Prices,” Management Science, 44, 1218–1233.

Dumas, B., A. Kurshev, and R. Uppal, 2009, “Equilibrium Portfolio Strategies in the Pres- ence of Sentiment Risk and Excess Volatility,” Journal of Finance, 64, 579–629.

Edmans, A., D. Garćıa, and O. Norli, 2007, “Sports Sentiment and Stock Returns,” Journal of Finance, 62, 1967–1998.

Ghysels, E., P. Santa-Clara, and R. Valkanov, 2005, “There is a Risk-Return Trade-off After All,” Journal of Financial Economics, 76, 509–548.

Gilchrist, S., and E. Zakraǰsek, 2012, “Credit Spreads and Business Cycle Fluctuations,” American Economic Review, 102, 1692–1720.

Glosten, L., R. Jagannathan, and D. Runkle, 1993, “On the Relation Between the Expected Value and the Volatility of the Nominal Excess Return on Stocks,” Journal of Finance, 48, 1779–1801.

Gourieroux, C., A. Monfort, and A. Trognon, 1984, “Pseudo Maximum Likelihood Methods: Theory,” Econometrica, 52, 681–700.

Goyal, A., and P. Santa-Clara, 2003, “Idiosyncratic Risk Matters!,” Journal of Finance, 58, 975–1007.

Goyal, A., and I. Welch, 2008, “A Comprehensive Look at the Empirical Performance of Equity Premium Prediction,” Review of Financial Studies, 21, 1455–1508.

Graham, J. R., and C. R. Harvey, 2012, “The Equity Risk Premium in 2012,” working paper, Duke University.

Grith, M., W. K. Härdle, and V. Krätschmer, 2013, “Reference Dependent Preferences and the EPK Puzzle,” working paper, Humboldt University.

Guo, H., and R. Whitelaw, 2006, “Uncovering the Risk-Return Relation in the Stock Mar- ket,” Journal of Finance, 61, 1433–1463.

Harvey, C. R., 1989, “Time-Varying Conditional Covariances in Tests of Asset Pricing Mod- els,” Journal of Financial Economics, 24, 289–317.

, 2001, “The Specification of Conditional Expectations,” Journal of Empirical Fi- nance, 8, 573–637.

Heston, S., and S. Nandi, 2000, “A closed-form GARCH Option Valuation Model,” Review of Financial Studies, 13, 585–625.

Hirshleifer, D., 2001, “Investor Psychology and Asset Pricing,” Journal of Finance, 56, 1533– 1597.

Hong, H., and J. C. Stein, 1999, “A Unified Theory of Underreaction, Momentum Trading, and Overreaction in Asset Markets,” Journal of Finance, 54, 2143–2184.

Jackwerth, J., 2000, “Recovering Risk Aversion from Options Prices and Realized Returns,” Review of Financial Studies, 13, 433–451.

Jegadeesh, N., and S. Titman, 1993, “Returns to Buying Winners and Selling Losers: Im- plications for Stock Market Efficiency,” Journal of Finance, 48, 65–91.

, 2001, “Profitability of Momentum Strategies: An Evaluation of Alternative Strate- gies,” Journal of Finance, 56, 699–720.

Jiang, G. J., and Y. S. Tian, 2005, “The Model-Free Implied Volatility and its Information Content,” Review of Financial Studies, 18, 1305–1342.

Kahneman, D., P. Slovic, and A. Tversky, 1982, Judgment Under Uncertainty: Heuristics and Biases. Cambridge University Press, New York.

Kahneman, D., and A. Tversky, 1979, “Prospect Theory: An Analysis of Decision Making Under Risk,” Econometrica, 47, 263–291.

Kamstra, M. J., L. A. Kramer, and M. D. Levi, 2000, “Losing Sleep at the Market: The Daylight Saving Anomaly,” American Economic Review, 12, 1000–1005.

Keynes, J. M., 1936, The General Theory of Employment, Interest and Money. Palgrave Macmillan, United Kingdom.

Lee, C., A. Shleifer, and R. H. Thaler, 1991, “Investor Sentiment and the Closed-end Fund Puzzle,” Journal of Finance, 46, 75–109.

Loughran, T., and J. Ritter, 1995, “The New Issues Puzzle,” Journal of Finance, 50, 23–51.

Ludvigson, S., and S. Ng, 2007, “The Empirical Risk-Return Relation: A Factor Analysis Approach,” Journal of Financial Economics, 83, 171–222.

Marfè, R., 2012, “Realized Expectations and the Equilibrium Risk-Return Trade Off,” work- ing paper, Swiss Finance Institute.

Nelson, D., 1991, “Conditional Heteroskedasticity in Asset Returns: A New Approach,” Econometrics, 59, 347–370.

Newey, W. K., and K. D. West, 1987, “A Simple, Positive Semi-definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix,” Econometrica, 55, 703–708.

Poteshman, A. M., 2001, “Underreaction, Overreaction, and Increasing Misreaction to In- formation in the Options Market,” Journal of Finance, 56, 851–876.

Rosenberg, J., and R. Engle, 2002, “Empirical Pricing Kernels,” Journal of Financial Eco- nomics, 64, 341–372.

Rossi, A., and A. Timmermann, 2010, “What is the Shape of the Risk-Return Relation?,” working paper, University of California San Diego.

Scheinkman, J., and W. Xiong, 2003, “Overconfidence and Speculative Bubbles,” Journal of Political Economy, 111, 1183–1219.

Shefrin, H., 2008, A Behavioral Approach to Asset Pricing. Elsevier Academic Press, Boston, Second edition.

Shiller, R. J., 1981, “Do Stock Prices Move Too Much to Be Justified by Subsequent Changes in Dividends?,” American Economic Review, 71, 421–436.

Subrahmanyam, A., 2007, “Behavioural Finance: A Review and Synthesis,” European Fi- nancial Management, 14, 12–29.

Todorov, V., 2010, “Variance Risk Premium Dynamics: The Role of Jumps,” Review of Financial Studies, 23, 345–383.

White, H., 1982, “Maximum Likelihood Estimation of Misspecified Models,” Econometrica, 50, 1–25.

Whitelaw, R., 1994, “Time Variations and Covariations in the Expectation and Volatility of Stock Market Returns,” Journal of Finance, 49, 515–541.

Xiong, W., and H. Yan, 2010, “Heterogeneous Expectations and Bond Markets,” Review of Financial Studies, 23, 1433–1466.

Yu, J., and Y. Yuan, 2011, “Investor Sentiment and the Mean-Variance Relation,” Journal of Financial Economics, 100, 367–381.