Diversification: A Failure of Fact or Expectation?

by Sam Stovall

Diversification: A Failure Of Fact Or Expectation? Splash image

The bear market of October 9, 2007, through March 9, 2009, witnessed not only a 57% decline in U.S. equity prices, but also the demise of many investors’ faith in the volatility-reducing effects of diversification. Fortunes were wiped away in this 17-month mega-meltdown that was triggered by the simultaneous popping of the commodity, credit, real estate and emerging equity market bubbles.

During this most recent bear market, which was the second-deepest in the past 80 years, but only the ninth longest of 15, nearly all asset classes—be they U.S. or foreign equities, real estate or commodities—exhibited the glide path of a crowbar, slumping in unison, particularly during the final six months of the bear. Today, as a result, many wonder if traditional “buy-and-hold” investing should be replaced with “run and rotate.” They also question if previously uncorrelated asset classes will now forever move in lockstep. Other investors, however, believe that during periods of financial crises, it is typical that equity-oriented or economically sensitive assets will experience positive correlations during these market downturns, only to revert to previous appreciation trajectories once the crisis has passed and they are able to relax once again.

Today, one can unemotionally sift through portfolio embers for clues to seemingly unprecedented asset class performances. We at Standard & Poor’s (S&P) conclude that diversification didn’t fail investors. Rather, allocation did do its job—a typically balanced portfolio’s (60% U.S. large-cap equities and 40% long-term government bonds) 13.1% decline in 2008 was less than the 14.8% fall seen in 1974 and much better than the results of 1931 (–22.2%) and 1937 (–19%).

However, investors’ expectations or memories failed. We also believe that many investors simply went too far out on the risk curve, embracing inappropriate equity exposures for their age groups, risk tolerances, or trading acumen. Hopefully, these errors in memory and expectation have been adjusted.

Investor Misjudgments of History

Why did investors hold on so long?

During this sixth “mega-meltdown,” or bear market in excess of 40%, in the past 80 years, many investors simply closed their eyes to the financial carnage and maintained their equity exposure as the S&P 500 eventually fell 57%. Allocations were never adjusted, just net worth. We at S&P believe one reason investors—as well as their advisors—chose to do nothing is because many advisors were ardent students of stock market history and, as a result, trained their clients too well, encouraging a “buy-on-the-dip” mentality during all types of market declines.

Yet two mistakes were made along the way. First, we assumed that the financial carnage that occurred prior to World War II was an anomaly and would never be repeated in our lifetime. Second, many believed that “black swan” events, like 100-year floods, occurred so infrequently that they wouldn’t happen again. Unfortunately, 100-year clocks don’t get reset annually.

The first mistake investors made was to be swayed by history regarding the time it has taken for equity markets to recover all of their losses following declines of 5% or more since 1945. Prior to the bear market of 2007–2009, the S&P 500 experienced 75 price declines of 5% to 49%. S&P describes price declines of 5% to 9.9% as “pullbacks,” declines of 10% to 19.9% as “corrections,” and declines of 20% or more as “bear markets.”

We further refine bear markets into “regular” and “mega-meltdowns.” Regular or “garden variety” bear markets decline 20%–39.9%; there have been eight since WWII. “Mega-meltdown” bear markets decline 40% or more, and there have been only two from 1945–2007. Figure 1 illustrates the bear market price declines since 1929.

From 1945 through 2007, there were 49 pullbacks, lasting an average of one month and declining an average 7%. The S&P 500 took an average of only two months to get back to breakeven following these pullbacks.

There were 16 corrections, lasting an average six months and declining an average 16%. The market recovered all that was lost in these corrections in an average of only four months. At this point, it is important to note that 65 of the 75 market declines in excess of 5% from 1945–2007 recovered all that they lost in only four months. It is therefore easy to understand why many investors were encouraged not only to “stay the course,” but also to leave sector weightings unchanged, since the market would already likely have been on the mend just as nervous investors were readjusting their portfolios to preserve capital in the event of further decline.

Even when you consider the eight “garden variety” bear markets, which declined an average 27% over 14 months, the S&P 500 took only 12 months to reach breakeven once again. This statistic may have been viewed encouragingly by financial advisors and investors as reason to take advantage of these rare buying opportunities.

Mega-meltdowns, on the other hand, have admittedly wreaked havoc on portfolios. Twice since 1945, we had endured declines in excess of 40%: 1973–1974, when the S&P 500 fell 48%; and 2000–2002, when the S&P 500 slumped 49%. Both bear markets occurred over an average 26 months (two-plus years), and it took an average 63 months (five-plus years) for share prices to get back to their pre-meltdown levels. Maybe this infrequency of mega-meltdowns gave investors an inappropriate sense of security. Not since the 1930s did we experience two mega-meltdowns in the same decade, which likely caused investors to let their guard down by assuming we wouldn’t see another mega-meltdown for 25 more years or so. These assumptions, while easy to understand, were obviously false and detrimental to one’s net worth.

Gain exclusive access to all of, PLUS our market-beating Model Stock Portfolio — currently trouncing the S&P 4-to-1!


Understanding Sector Slump

A lot of investors have confessed that they must have done something wrong during this most recent bear market, since no matter where they employed their capital, they experienced losses. Whether they invested in cyclical sectors or the traditional “defensive” or “safe haven” sectors of consumer staples, health care or utilities, their investments declined in value. “Shouldn’t these sectors have held up, since the demand for these products and services remain fairly static during both good times and bad?” they asked. History says, “no”—not only during this bear market, but during all since WWII.

What history shows is that during the decline of 2007–2009, there was no place to hide within the U.S. equity market, as all sectors within the S&P 500 posted declines ranging from 31% to 83%. What is intriguing, however, is that the three best performers during this most recent bear market were the traditional defensive sectors: consumer staples, health care and utilities. This is also intriguing in that history shows these three defensive sectors recorded identical ranking placements during average bear markets from 1946–2002. In other words, whenever the S&P 500 suffered through a bear market (garden varieties or mega-meltdowns), all sectors posted declines. Yet the defensive sectors of consumer staples, health care, utilities, and to a lesser extent, energy, were the best relative performers, ranking 1, 2, 3 and 4 (see Table 1). So while the term “defensive” may be accurate, “safe haven” is not.

S&P 500 Sectors 10/9/2007– Average 1946–2002
% Change % Change Rank
Consumer Staples -31 -9 1
Health Care -40 -12 2
Utilities -46 -19 3
Energy -47 -21 4
Telecommunications Services -51 -33 na
Information Technology -53 -26 7
S&P 500—All Industries* -57 -24 na
Consumer Discretionary -58 -29 8
Materials -60 -23 5
Industrials -65 -31 9
Financial -83 -24 6

In addition, three of the four sectors that posted the worst performances this time around traditionally ranked among the worst performers during average bear markets since 1945. Therefore, while the magnitude of the declines were outsized during this recent bear market, their performance rankings were similar to the average rankings during bear markets from 1946–2002.

Sectors on the Upside

How do sectors traditionally perform in new bull markets?

Historically, the cyclical sectors that underperformed during bear markets took a leadership role during the first 12 months of new bull markets. Since 1949, the strongest average price performances came from the consumer discretionary, financial, industrial and information technology sectors (shown in Table 2). And while the magnitude of advances differed from one bull market to the next, the leadership tended to remain fairly consistent. Also encouraging was that the old adage “a rising tide lifts all boats” was true for the first year of new bull markets. What’s more, the same leadership is occurring in this embryonic bull market of March 9, 2009, through January 29, 2010, thus indicating that while the magnitudes change, tendencies do not.

  % Change (Excluding Dividends)
Average Mar-09 Oct-02 Dec-87 Aug-82 Oct-74 May-70 Oct-66 Jun-62 Oct-57 Jun-49
S&P 500 32 59 22 19 52 32 37 26 27 25 25
  Growth Group 36 52 25 17 53
  Value Group 40 67 20 23 52
Consumer Discretionary 47 81 21 21 81 51 66 53 32 24 38
Consumer Staples 29 36 2 22 42 34 45 29 15 35 27
Energy 28 32 13 8 42 39 51 33 24 9 24
Financial 42 128 23 11 66 26 35 41 31 24 32
Health Care 32 44 10 8 59 15 17 70 24 60 18
Industrials 43 81 19 16 71 34 56 50 39 30 36
Information Technology 48 70 59 -3 100 26 45 66 30 46 46
Materials 33 68 23 19 77 23 15 16 20 28 40
Telecomm Services 18 18 25 11
Utilities 22 32 17 1 33 36 19 13 23 23 23

Comparing Asset Classes

During the 2007–2009 global bear market, the S&P 500 declined 51% on a total return basis. Despite assurances that international equity markets had “de-coupled” from U.S. equity markets, the international developed and emerging markets declined by similar amounts. Was this the beginning of the “great unwind,” in which correlations between asset classes began to fall apart and merge toward 1.00? Or, like sector performances within the S&P 500, did the total return action during the most recent bear market replicate performances during previous bear markets? If so, will these same equity-oriented asset classes revert to their normal tendencies now that the crisis has passed?

The results appear to indicate that equity-oriented asset classes tend to move in similar fashion during bull markets and bear markets in U.S. stocks. In all observations since 1974—the earliest date that includes most asset classes—when U.S. large-cap (and small-cap) issues advanced or declined, international developed and emerging markets also rose and fell. Only in 1982 did the emerging markets buck the trend by falling in the first 12 months of the embryonic bull market.

The same can be said for U.S. real estate investment trusts (REITS), in that they mirrored the action of large-cap stocks in 11 of 12 observations. Commodities, however, tended to respond to the type of bear market, falling when global economic concerns were center stage, yet rising when bear markets either caused a swift rush to the safety of gold (such as following the 1987 crash), or seemed to be attributed to selected sectors of the market (such as large-cap technology and telecom stocks following the popping of the Internet bubble in 2000). Long-term government bonds and Treasury bills, however, appeared to provide the consistent negative correlation to U.S. and international equities that investors have come to rely on.

From the perspective of asset class performance, therefore, results were similar to that for sectors within the S&P 500. During the crash of 2007–2009, even though the magnitude of performances may have differed, the direction of asset class performances followed the traditional patterns. Equity-oriented asset classes (global equities and REITs) continue to be “coupled,” rising and falling in unison, while commodities follow suit whenever the global economic health is called into question. Indeed, in the eight months since the S&P 500 bottomed on March 9, 2009, the global equity benchmarks posted total return advances of 50% to 92%, while long-term Treasuries were flat.

Correlations & Volatility

As of October 2007, when the S&P 500 last peaked, standard deviations of monthly returns over a 36-month period were surprisingly tame as the S&P 500 suffered its first one-day decline of 2% or more in nearly four years in February of that year. The standard deviation of 36-month returns was 7.5% for the S&P 500 index, 12.3% for the S&P SmallCap 600 index, 15.7% for REITs, 9.5% for the MSCI EAFE (developed international equities), 17.7% for emerging market equities, 22.6% for the S&P-GSCI index (commodities), 6.8% for long-term Treasuries, and 0.4% for three-month T-bills. However, as of the bottom of the 2007–2009 bear market, all asset classes experienced a substantial increase in standard deviations, with the S&P 500, REITs, and the MSCI EAFE each seeing at least a doubling of volatility. In addition, all asset classes’ correlations with the S&P 500 rose from October 2007 versus March 2009. Note, however, that the correlations for long-term Treasuries remained low versus the S&P 500, despite its rise over this period.

Annual Portfolio Returns

The true test of correlations comes when they are applied to overall portfolios. Successful diversification is a function of asset class correlations, not the quantity of assets held. Holding a variety of equity-influenced or economically sensitive benchmarks will not offer the kind of downside protection that a combination of equity and fixed-income asset classes will, as equity-oriented correlations have either been high or have been rising over the past 20 years.

The 2007–2009 bear market’s price decline of 57% for the S&P 500 was the deepest since the 1929–1932 crash. In addition, the S&P 500 fell 38.5% in price during 2008, just slightly below the 1937 decline of 38.6%. As mentioned at the beginning of this article, many investors assumed (incorrectly) that a typically balanced portfolio of 60% U.S. large-cap equities and 40% long-term government bonds also posted a return that rivaled that of the 1930s. Yet the muted 13.1% decline that this portfolio experienced last year was due to the 22.7% rise in long-term government bonds stemming from a flight to safety by global investors. As a result, this portfolio’s return in 2008—while below the average 9.4% rise since 1926, and slightly more than one standard deviation below its long-term mean—was better than the 14.8% decline experienced in 1974 and much better than the results suffered in 1931 (–22.2%) and 1937 (–19.0%).

As a result, one could say that proper diversification did not fail the average investor. The reason investors think diversification failed could be that they decided to alter their interpretation of proper diversification by increasing their exposure to a wider variety of equity types. Investors may have assumed that a high exposure to fixed income was no longer needed and that they would be safe to reach further out on the risk curve. They obviously assumed incorrectly.

Comparing the different asset classes in Table 3, we see quite a variety of performances and volatilities since 1971, which was a common starting date for all (except the MSCI Emerging Markets index, which started in 1976).

Asset Classes 2008
Total Returns
Compound Standard
Total Returns Deviation
(%) (%)
Large Cap U.S Equities -37.0 9.9 18.6
Small Cap U.S. Equities -31.1 11.8 21.4
Int’l Developed Equities -43.1 10.2 23.3
Emerging Mkt Equities (Since 1976) -53.2 11.0 32.0
Real Estate Investment Trusts -37.7 11.6 19.1
Commodities -46.5 7.0 7.3
Long-Term Government Bonds 22.7 8.6 11.8
Treasury Bills 2.2 5.8 3.0
Portfolio 1 -13.1 9.9 12.7
Portfolio 2 -16.1 10.0 10.8

Clearly, these equity-oriented asset classes offered a higher compound annual return over time, but were also accompanied by much higher levels of risk (higher standard deviations of annual returns). The debt instruments, on the other hand, generated less volatility, but also offered lower rates of return. Finally, commodities appeared to offer nothing other than a reduction in portfolio volatility, due to the relatively low correlation with large-cap U.S. equities.

As Table 3 also shows, a simple portfolio of 60% U.S. large-cap equities and 40% long-term government debt (portfolio 1) provided the same return from 1971 through 2009 as U.S. large-cap stocks alone (with substantially less risk). A more diversified portfolio of large and small U.S. equities, developed and emerging foreign stocks, REITs, commodities, long-term bonds and T-bills (portfolio 2) delivered an even higher compound annual total return than the simple 60/40 allocation since 1971 (despite falling by 300 basis points more in 2008) as well as enduring a lower standard deviation of annual returns.

Rising Volatility

The tenets of diversification may have been questioned as a result of this recent mega-meltdown, only to be suggested that they are still alive and well. Yet due to global computerized trading strategies, hedge funds, and the introduction of leveraged long and short equity exchange-traded funds ETFs, an increase in volatility has likely become the new norm. From 1960 through 1999, the S&P 500 fell by 2% or more in a single day an average of five times per year. In 2008, however, it declined by 2% or more 41 times. Also during this decade, the S&P 500 fell by 2% or more in a single day 146 times, versus 42 times in the 1990s, 51 times in the 1980s, 30 in the 1970s and 11 in the 1960s. So we better get used to elevated volatility.

Lessons Learned

The recent mega-meltdown caused many investors to suffer unprecedented declines in their net worth. Was it because diversification failed them? Will this decline in net worth be temporary or permanent? History suggests, but does not guarantee, that diversification did not fail. Mixing asset classes with low correlations (equity-oriented investments with fixed-income instruments) cushioned prior declines, but did not eliminate them. And the same was true during the most recent bear market.

What’s more, while the magnitude of performance results may differ for equity-oriented asset classes, their correlations were, and remain, quite high. In addition, their correlations typically spike during market panics. History also suggests that portfolios will eventually recover and then exceed prior levels. The time it will take to get back to breakeven will also likely depend on how exposed investors were to equities. Those that went too far out on the risk curve by embracing an excessive amount of equities will likely take longer to recover. Obviously, those investors who had higher levels of fixed-income exposure will probably reach breakeven sooner. Other lessons learned include:

  • Do not assume the worst of times will never be repeated—1930s-style declines have returned.
  • Sectors of the S&P 500, as well as equity-oriented asset classes, will likely revert to 1.00 during bear markets, but then return to long-term tendencies after panics have subsided.
  • Do not confuse magnitude of returns with correlations. Equity-oriented assets—whether large- or small-cap U.S. stocks, developed international or emerging market issues, or equity-oriented REITs—will tend to follow similar trajectories during bull and bear markets.
  • Do not confuse quantity with correlation. True diversification comes from a mixture of asset classes with low correlations, not a large variety of asset classes with high correlations.
  • Global and U.S. equity markets have not de-coupled.
  • Do not go too far out on the risk curve. Reconsider the old adage that your age should represent your fixed-income exposure.
  • Heightened volatility will either be here to stay, or could easily be re-awakened.
Sam Stovall , chief equity strategist of S&P Capital IQ, serves as chairman of the S&P Investment Policy Committee. He is the author of the books, "The Seven Rules of Wall Street" (McGraw-Hill, 2009) and “The Standard & Poor’s Guide to Sector Investing” (McGraw-Hill, 1995). He writes a weekly investment piece on S&P’s MarketScope Advisor platform ( focusing on market and sector history, as well as industry momentum..


John from UK citizen/resident posted over 3 years ago:

Sam, thank you for a great article which motivated me to revisit some of my calculations. I take the opportunity to share a couple of associated observations with you, and AAII Members.

UP FRONT, LET ME SAY THAT I AM A UK CITIZEN/RESIDENT, so my investment portfolio is in GBP, however, I believe that my comments are likely to be relevant to any investor, worldwide.

FIRST, your point about due to global computerised trading strategies, etc., an increase in volatility has likely become the new norm. I did some equity volatility sensitivity analysis using my 2008 (the year I retired) Monte Carlo portfolio simulation. Keeping all other inputs constant, multiplying the assumed equity standard deviation X 1.1 decreased the 90% chance portfolio life by 2 years (from 31 yrs); and X 1.2 decreased the life by 3 yrs. CONCLUSION - any new, higher volatility norm will significantly impact my portfolio.

SECOND, I looked at real declines for rolling 10-yr holding periods for a UK 50% equities, 25% bonds, and 25% short-term investments and cash portfolio (my post-retirement asset allocation). I concluded that the 18 yrs, 1964-1981, were the worst for the UK since WW2, and the average real return during that time was -2.2% p.a. (over the 9, rolling 10-yr periods within the 18 yrs). CONCLUSION - there is likely to be more UK investment pain in the next 5 to 10 yrs if post-WW2 history repeats itself.

Thanks again for a great article.

David Vornholt from HI posted 11 months ago:

Very nice article, thank you. I learned several things in reading it.
I beleive the 'elephant in the room' is that investors in general have been told for many years by people like John Bogle and the financial websites is that the average investor has neither the expertise nor time to make any kind of intelligent investing decisions and therefore should only invest in three index funds (S&P500, Large international and a broad bond fund). Well, no wonder there was correlation! Everyone was investing in the same things! I stay away from large company stocks now as well as bonds. I only look at small caps with reasonable liquidity. My backtesting shows that group to have the best draw down minimization. Large caps are terrible from a risk/reward standpoint.

You need to log in as a registered AAII user before commenting.
Create an account

Log In