Optimism Has Been Low This Year, but Not Unusually So

This week notwithstanding, optimism about the short-term direction of the stock market has been below average for 40 out of 50 weeks this year in our AAII Sentiment Survey. Given this trend, I was curious as to how 2017 compares to the 30 previous years tracked by our survey. So, I updated the historical analysis I’ve been keeping on unusually high and low readings (see the table at the end of this commentary) and then conducted calendar-year analysis.

Year to date, bullish sentiment is averaging 34.5% (including this week’s latest reading of 45.0%). Historically, bullish sentiment has averaged 38.5% over the life of our survey. (The average is technically 38.27%, but investing is messy and rounding works well enough.) Relative to the other 30 calendar years we have survey data for, 2017 ranks eighth in terms of having the lowest average bullish sentiment.

As you can see on the table located to the right, 1988 and 1990 had the lowest average levels of optimism at 27.3% each. Notably, in third and fourth place were 2016 and 2015. Not shown in the table are the years with the highest level of average bullish sentiment: 2000 (49.4%), 2004 (49.0%) and 2003 (48.4%).

While average bullish sentiment has not been exceptionally low this year, its weekly readings have been comparatively stable. When I calculated the standard deviation (a measure of variance) on a calendar-year basis, this year ranked as the second lowest. The year with the lowest level of standard deviation was 1995, with a standard deviation of 5.2%. This year may actually be tied for the lowest because prior to 2000 the weekly readings were rounded to whole numbers (we used to conduct the survey by mailing out postcards.) Now, members vote online and the results are automatically tabulated by a database. The report we see internally goes out to four decimal places.

The low level of variance in our survey has occurred at the same time volatility in the stock market has been historically low. The CBOE’s Volatility Index (aka, the “VIX”) has averaged 11.14 year to date. The next least volatile year was 1994, when the VIX’s average reading was more than a full point higher at 12.69. To give some context to these numbers, since 1990, the VIX has had an average daily reading of 19.38 and a median daily reading of 17.52.

For those of you who are curious, the year with the greatest variance in bullish sentiment was 2003. Bullish sentiment swung that year from a low of 21.1% on February 20, to a high of 71.4% on June 26. The big swing occurred as the post-dotcom bear market bottomed and stocks rebounded strongly. [This year, optimism has ranged from 23.9% (May 18) to 46.2% (January 5).]

The calendar-year averages are interesting and provide some color, but where our survey has the strongest link with market returns is when sentiment reaches extremes. Specifically, when bullish sentiment falls to an unusually low level, the S&P 500 has historically outperformed. As the table below shows, the large-cap index has risen by an average of 7.4% over the following six-month period whenever bullish sentiment has been more than one standard deviation below its historical average. (Currently, a reading below 28.0% would be unusually low.)

Conversely, whenever optimism is too high, stocks tend to underperform. The S&P 500 has realized an average six-month return of just 2.8% whenever bullish sentiment has been more than one standard deviation above its historical average. (Readings above 48.5% would be unusually high based on the current range.)

Despite suggestions for buying “when there’s blood on the streets,” analysis of our survey’s history suggests that buying when everybody else is gloomy about future returns is a better strategy. You should not, however, base your portfolio decisions off of our or anybody else’s survey data. Though a link exists, it is not causal. Rather, use unusually high or unusually low bullish sentiment as a sign to check other factors (e.g., valuations) and to see if your portfolio needs to be rebalanced.

Performance of the AAII Sentiment Survey without Hindsight