Feature: Investor Sentiment as a Contrarian Indicator
Sentiment describes the opinions, emotions or views of a group of people. In investing, sentiment can be a powerful determinant of security prices, especially in the short run. Here, emotions—whether rational or irrational—can drive market prices. The price at which an individual security trades is the sum total of the sentiment of all market participants. If you could forecast changes in sentiment, you should have an advantage in determining changes in the market. The problem with sentiment is that it’s really only known after the fact.
Market sentiment—the summation of all expectations for the market as a whole—often directly reflects where the market has been, not where it is going. When market sentiment is low, the majority believes the market will fall, while high market sentiment means that the majority feels the market will rise in value.
However, history shows us that more times than not the market will go against the majority. Extremely bullish levels of sentiment often come after strong market run-ups when investors are fully invested in the market. Even if they are bullish about the future, they have limited additional resources to invest. By following market sentiment indicators, you may be able to pick out market tops and bottoms. In other words, investor sentiment may be used as a contrarian indicator for the overall market.
There are different ways to measure market sentiment—through surveys of individual and professional investors, by examining trading data such as block trades and short interest, or even by monitoring the trend in cover stories of major publications. Computerized Investing covered several such sentiment measures in the May/June 2001 issue [“Gauging Market Sentiment”]. In this article, we focus on one gauge of market opinion—the AAII Sentiment Survey—and its potential use as a contrarian indicator.
The AAII Sentiment Survey measures the percentage of individuals who are bullish, bearish, and neutral about the stock market over the next six months. Originally started in 1987 as a weekly survey sent out via snail mail to a random sample of AAII members, the survey has been conducted on-line since the beginning of 2000. Members of AAII can vote once a week at the Member Surveys area of AAII.com. Results of the survey are compiled on a weekly basis (Thursdays) at the AAII Web site and are also published weekly in Barron’s. The average AAII member is a male in his late-50s with a graduate degree. In addition, over half of AAII members have an investment portfolio of at least $500,000. Taking this into account, the AAII Sentiment Survey is unique among sentiment surveys in that it represents the upper echelon of active, “hands-on” individual investors.
A spreadsheet that contains the complete weekly readings of the AAII Sentiment Survey, along with historical averages and historical extremes, is available at the Member Surveys area of the AAII Web site. Table 1 provides some summary statistics of AAII member sentiment over the period July 24, 1987, to September 2, 2004. Historically, the average bullish sentiment value is 38.8%, while the median (mid-point) value is 38.0%. For neutral sentiment, the average is 33.4% and the median 34.0%. Bearish sentiment has averaged 27.9%, while the median is 27.0%. So, on average over time, almost two-fifths of AAII members believe that the market will rise over the next six months; roughly one-third believe the market will be flat over the next six months; and a little more than one-quarter think the market will decline over the next six months.
As was mentioned earlier, sentiment data is normally used as a contrarian indicator. Week-to-week changes in member sentiment do not reveal any meaningful relationships between sentiment and market performance. However, identifying extremes in member sentiment may provide useful insight into how the market will perform in the succeeding months.
|Table 1. Historical Averages & Extremes of AAII Member Sentiment*|
|*data covers period from July 24, 1987 to September 2, 2004.|
Looking again at Table 1, we see the historical extremes for bullish, bearish, and neutral sentiment. Bullish sentiment reached its highest levels on January 6, 2000—the height of the tech bubble—at 75.0%. Levels of extreme bullishness—if viewed as a contrarian indicator—would lead one to believe that a market decline is in the wings. Figure 1 shows that the S&P 500 had gained 12.5% in the three months prior to the peak. In the meantime, bullish sentiment climbed from 34% to 75%. Following January 6, the market went through a series of peaks and valleys—even managing to rise 8.8% above the January 6 close. A prolonged decline in the market eventually began on September 1, 2000. Twelve months after reaching the bullish high of 75%, the S&P 500 had fallen 7.1% and bullish sentiment was at 36%.
Bullish sentiment reached an all-time low of 12% on November 16, 1990. Iraq had invaded Kuwait and the world was on the verge of the first Iraq war, oil prices were rising, and the U.S. economy was in decline (sound familiar?). In the three months leading up to November 16, 1990, the S&P 500 had slipped 4.6% and bullish sentiment had fallen from 18% to 12% (Figure 2). A contrarian would view such extreme levels of pessimism among individual investors as a sign of good things ahead for the market. As shown in Figure 2, within 12 months, the S&P 500 was up 25.2%.
Extreme levels of bearishness can also be used to gauge potential market direction. Coinciding somewhat with the all-time low in bullishness is the high in bearish sentiment, which occurred on October 19, 1990—the third anniversary of the crash of 1987 and less than a month before bullishness reached its historical low. Again, the S&P 500 had been in decline leading up to October 19, spurred in part by the Iraqi invasion of Kuwait. Bearish sentiment had risen from 30% on July 20, 1990, to the high of 67% on October 19, 1990. The market had already found its bottom the week before, and 12 months after October 19, 1990, the S&P 500 had gained 25.6% (Figure 3).
Finally, looking at low levels of bearish sentiment, AAII members were their most upbeat, or at their least bearish, on August 21, 1987, when bearish sentiment fell to 6%. As we now know, this was less than two months before the market crash of October. The market seemed to know only one direction—up—and the S&P 500 had gained 19.9% in the three months prior to August 21, 1987 (Figure 4). However, only two trading days later the S&P closed at 336.77, a level not attained again until late July of 1989. Between August 21 and October 19, the S&P 500 fell 33.3% and, although the market rebounded slightly, was still down 23.4% 12 months following August 21.
While we have shown you the market’s direction following all-time extreme levels of bearishness and bullishness, we could wait years and never see sentiment reach such levels again. Instead, you can look for points that, while still extreme, happen with greater frequency to use as indicators.
In order to establish the “extreme” levels, we calculated those values that were two or three standard deviations from the mean for both bullish and bearish sentiment. The spreadsheet available for download from the Member Surveys area presents many of the required values already calculated, namely the average values and standard deviations for bullish, neutral, and bearish sentiment. The values are also presented in Table 1.
|Table 2. Market Performance at Sentiment Extremes|
|Bullish > Mean + 2 S.D.||38.8% + 2 * 11.3% = 61.4%||31**||(5.5%)|
|Bullish < Mean - 2 S.D.||38.8% - 2 * 11.3% = 16.2%||12||16.2%|
|Bearish > Mean + 2 S.D.||27.9% + 2 * 9.2% = 46.3%||40||18.0%|
|Bearish > Mean + 3 S.D.||27.9% + 3 * 9.2% = 55.5%||8||23.7%|
|Bearish < Mean - 2 S.D.||27.9% - 2 * 9.2% = 9.5%||5||(17.6%)|
*52 weeks of data required following sentiment extreme to be included.
**9 observations were within last 52 weeks and were not included in S&P return calculation.
Table 2 shows the average market percentage gain or loss when bearish or bullish sentiment reached levels that were two or three standard deviations above or below their respective averages (mean). Again, we see significant market moves in the direction prescribed by contrarian thought.
Looking at bullish sentiment, there are 31 times (out of the 893 total as of September 2, 2004) that sentiment mesured above the “threshold” value of the mean plus two standard deviations, or 61.4%. We then calculated the 52-week percentage change in the S&P 500 following each week in which bullish sentiment exceeded 61.4%. For the 22 observations that had 52 weeks of data following them, the average change in the S&P 500 was a 5.5% decline. There were no instances when bullish sentiment was more than three standard deviations above the mean.
We made 12 observations of bullish sentiment at least two standard deviations below the mean (less than 16.2%), indicating a low level of optimism and, potentially, an indication of a market rise. For these 12 instances, the S&P 500, on average, gained 16.2%.
The results were just as telling when looking at bearishness. We made 40 observations where bearish sentiment was two standard deviations above the mean (greater than 46.3%). This is an indication of high pessimism among investors and, potentially, a sign of a market rally in the coming months. When this occurred, the S&P 500, on average, gained 18.0% in the succeeding year.
Raising the bar even further, we found eight times when bearish sentiment was three standard deviations above this mean (greater than 55.5%). The results were even more remarkable, with an average one-year gain in the S&P 500 of 23.7%.
Lastly, we examined the market’s direction when bearish sentiment was two standard deviations below the mean (less than 9.5%). This happened five times, with an average decline in the S&P 500 of 17.6% over the next 12 months.
Investor sentiment measures have historically been used as contrarian indicators—meaning one would expect the market to do the opposite of what the data was saying. The analysis here appears to support that point.
While little may be gleaned from changes in investor sentiment, identifying extreme levels of positive or negative sentiment appears to offer a glimpse of where the markets may be headed. In looking at the AAII Member Sentiment survey, we found that when sentiment reached overly bullish levels, the markets normally responded negatively in the months that followed. Conversely, the market tended to rise when members became overly bearish.
While our results here seem to lend validity to the notion that investor sentiment may be used as a contrarian indicator, it would not be wise to base all your investment decisions upon it. Indicators such as this are best used in tandem with others so that you receive confirming signals of potential market movements. Sentiment merely serves as an additional tool when making investment decisions.
Lastly, even if you do not use sentiment data as an indicator, it is a good idea to be mindful of it. As investors become overly bullish or overly bearish, it is easy to get caught up in the “herd mentality.” However, as we have shown, if you run with the herd, you might get trampled.