The Price We Pay for Being Different

There are definite short-term challenges we are likely to face when we pursue a “different” investing approach, which is why understanding and conviction are crucial.


Advocates of investing strategies that favor small, value, momentum, quality or other factors are fond of pointing out the accuracy of their models. Many of them are capable of describing 95% or more of the variation in past returns.

As impressive as this is, it’s a little like saying that the weather where I live in Sunnyvale, California, will often be similar to the weather in the cities surrounding Sunnyvale, and the climate will generally be mild and pleasant, similar to previous years. What it can’t tell me is that on September 9, 2020, we would have a day with no sunshine and such dense wildfire-smoke-filtered orange light that it looked more like we were living on Mars. And, yes, that actually happened.

Neither factor-based investing strategies nor weather forecasts are particularly good at telling us what will happen tomorrow, next week or next year, but that doesn’t make them useless. It just means we may have to tolerate a lot of short-term variation to get the long-term benefits.

So, what kind of short-term challenges are we likely to face when we pursue a “different” investing approach?

Returns Will Be Different

The first, and most obvious, one is that our results won’t track those of our neighbors, our friends and the news of the day. More than once, I’ve been asked by an investor who tilted their portfolio to certain factors why it was underperforming the S&P 500 index. Somehow, they all seemed okay with the idea that they’d someday outperform the market as long as they didn’t underperform it first. Unfortunately, there are no strategies that I’m aware of that can guarantee this outcome. Some people might get lucky and see their returns exceed the market initially and then always stay ahead of it, but they will be few and far between. The vast majority of investors who choose to be different will likely experience years or even decades of underperformance before experiencing the benefits of their different approaches.

To illustrate, we’ll look at two relatively simple all-equity portfolios: one that’s invested 100% in the S&P 500 and another that’s made up of four equal 25% investments in the S&P 500, a U.S. large-cap value fund, a U.S. small-cap blend fund and a U.S. small-cap value fund. The first portfolio is what many consider “the market.” It’s capitalization-weighted and gets exposure to only the market-risk factor. The second portfolio has a larger percentage invested in small and value stocks than the cap-weighted portfolio. This tilt adds exposure to the small and value risk factors.

How long might someone have to wait for the advantages of being different to show up? By analyzing historical returns from 1970 through 2019 using all 600 possible starting months and looping back to the 1970 returns when needed, we can generate a picture of how long investors might have needed to wait in the past. It’s a wide range. Figure 1 is a chart showing the percentage of past scenarios where the Four-Fund solution outperformed the S&P 500 for different investing time frames.

It’s clear from the chart that the Four-Fund portfolio tilts the odds in our favor, but it’s also clear that it may take some time to see those benefits. After five years of investing and rebalancing every year, there’s a 42% chance that we’re underperforming the S&P 500. At 10 years, things are only slightly better, and at 20 years, there’s a roughly one-in-seven chance that we are still underperforming. At 25 years, the Four-Fund portfolio outperformed the S&P 500 100% of the time. As encouraging as that is, it’s important to remember this is based on only 50 years of history, and there are no guarantees that the next 50 years won’t have a 25-year period of time when a Four-Fund portfolio might underperform. Investing differently is a strategy that requires patience.

At this point, some of you are probably asking why anyone would sign up for such a fickle supposed advantage. The biggest attraction is the higher expected returns based on the premiums for smaller and cheaper companies. In this backtest, the median compound annual growth rates were 10.5% for the S&P 500 and 12.2% for the Four-Fund portfolio.

Since those are the compound annual growth rates, the cumulative effects were quite large. Over 25 years, an initial $100,000 lump-sum investment would have grown to $1.23 million in the S&P 500 versus $1.78 million in the Four-Fund portfolio. Once again though, what we get is not the median. We get a chance sequence unique to the time when we invest, so it’s important to think about the ranges of possible outcomes. The low-to-high range for the S&P 500 was $286,000 to $5.3 million versus $384,000 to $8.3 million for the Four-Fund solution. Yes, the odds are still tilted in favor of the factor-weighted portfolio, but you might spend decades patiently waiting and still see significantly less than your expected rate of return. On the low end, the S&P 500 produced a nominal return of 8.3% versus 9.3% for the Four-Fund solution.

Costs Will Be Different Too

Perhaps one of the hardest realities is that there are costs that come from investing in a speculatively advantageous strategy that are certain and immediate.

To get significant factor exposure, we will likely need to pay a higher expense ratio if implementing our strategy with exchange-traded funds (ETFs) or mutual funds. If we’re implementing with stocks, we may have to follow detailed trading guidelines using limit orders and being sensitive to the liquidity of the stocks we’re trading.

Either approach will be more complex to manage than simply purchasing an S&P 500 index fund, and all of these costs will begin on day one and persist whether we see the premium we’re hoping for with our different approach or not. The historical return data I’ve used includes the higher expense ratios for the small and value funds, so it appears that the costs were justified, but that doesn’t mean they won’t weigh on an investor’s mind as they wait for their premium to come in.

More Long-Term Volatility and Risk

Another downside that investors must tolerate in pursuing higher factor-tilted returns is increased volatility and drawdown risk. The maximum peak-to-trough drawdown experienced by the S&P 500 in the 1970 to 2019 period was 51%, versus 57% for the Four-Fund portfolio. Surprisingly, the month-to-month volatility for the Four-Fund portfolio was only 5% compared to 7% for the S&P 500, so short-term volatility was better even though long-term volatility was worse.

Since many AAII members are retired, I also analyzed the maximum safe withdrawal rate for each of these approaches. The rate was found by looking for the worst-case 40-year scenario across all 600 starting months. Circular bootstrapping was used, so we loop from 2019 returns to 1970 for scenarios starting later than 1980. A lump-sum initial investment was followed by annual withdrawals of a cash amount set to a fixed percentage of the original balance and increased by the CPI (consumer price index) inflation rate for every year to maintain a fixed purchasing power. Withdrawal amounts were not adjusted based on increases or decreases in the size of the nest egg as a result of market conditions.

The result is more good news for the investor willing to be different. The Four-Fund portfolio sustained a 4.8% safe withdrawal rate, while the S&P 500 could only sustain a 3.2% safe withdrawal rate. Though there’s no guarantee the future will look like the past, it shows that being diversified across multiple risk factors (market, size and value) has delivered higher returns with sufficient consistency to enable higher safe withdrawal rates over the past 50 years.

I also ran a similar analysis going back to 1928. With the added stress of the Great Depression, the 40-year safe withdrawal rates dropped to 3.0% for the S&P 500 and 3.1% for the Four-Fund portfolio. Portfolios that included fixed-income diversification would have likely had higher safe withdrawal rates, but also lower expected returns.

Large Advantages With Periods of Underperformance

So, what are we to make of all of this, and how can we visualize it? My colleague, Daryl Bahls, at The Merriman Financial Education Foundation has generated a chart that I think explains it well. Figure 2 shows the cumulative performance of the Four-Fund portfolio relative to the S&P 500 for the past 90 years. Anything above one means the Four-Fund portfolio is ahead of the S&P 500. Anything below one means it’s behind. A positive slope means the Four-Fund portfolio is growing faster than the S&P 500, and a negative slope means it’s falling behind.

The good news is that there’s a very large advantage to the small and value tilted Four-Fund approach over the 90 years. The bad news is that the advantage comes in short, seemingly random periods separated by long periods of lackluster underperformance. The longest of these was 20 years, and the shortest was eight years. What’s more, practically all of the premium came in less than 25% of the total time. This is more evidence that investing differently requires conviction and patience.

With such a high degree of randomness, why not invest in many different factors at once? This is the approach suggested by Larry Swedroe and Andrew Berkin in their book “Your Complete Guide to Factor-Based Investing” (Buckingham, 2016). The argument is that by combining different investments that sporadically deliver outsized returns at different times you’ll get a smoother return stream. I try to apply this principle in our selection of best-in-class ETFs by looking for funds that have positive exposure to factors like momentum, quality or low volatility in addition to the market, size and value factors. Unfortunately, at this point, the number of funds available that focus specifically on these additional factors is limited. What’s more, many of them have short track records and are less efficient at delivering the expected factor premiums compared to the market, small and value-focused funds. This isn’t to say it’s not worth trying, but so far, it doesn’t appear as accessible to do-it-yourself investors interested in using low-cost ETFs or mutual funds.

The Price We Pay for Being Different Video

We think you’d like this related webinar! Simple & Effective Balanced Portfolios for Lifetime Investing Success


Deciding Whether to Be Different

So, where does this leave us? Is it just a reminder that things of value don’t come for free?

I think it’s deeper than that.

In this day and age of thousands of mutual funds and ETFs, sophisticated robo-advisers, automated no-fee trading, target-date funds and other innovations, I think it’s become all too easy to think of the market as some magic black box—a place where we put money and then expect it to mysteriously grow. There is no magic.

Whether we’re buying pieces of all of the companies in the S&P 500 proportional to their capitalization or disproportionately purchasing pieces of smaller and cheaper companies, we’re taking on risk in exchange for a share of those companies’ growth and profits. The likelihood that we lose all of our money is small because there are many companies. The likelihood that we get a good return is high because tens of thousands, if not millions, of employees and executives are striving every workday to make their businesses succeed. It’s also likely that we’ll see our investment decline a few times along the way because market valuations are approximate and randomized by the news of the day.

It’s unlikely that the two approaches will deliver the same results because small companies and less-valuable companies thrive in different times and circumstances than large ones. History suggests that smaller and cheaper companies will, on average, have more growth, but more volatility as well. As co-owners in the companies, like their employees, we’ll have to wait for the time when favorable market circumstances let them realize their potential. If we can’t tolerate the increased risk or wait for those favorable circumstances, then we can invest in the cap-weighted market or take even less risk by diversifying with bonds and cash.

Perhaps the most difficult questions we must answer in deciding whether we are suited to invest differently or not are: What do each of us know and believe, and how deep is that foundation? Several studies have shown that even random portfolios have historically outperformed the market, but few people are likely to be able to stick with investing randomly for long. Since not all random portfolios outperformed the market, who’s to say you haven’t chosen a loser when you see the first quarter of underperformance? And, if you switch at that point, you’re starting down the path of performance-chasing, which is never-ending and likely has lower returns.

The pursuit of a different return requires understanding and conviction. One way to get that is to study historical returns. Another way is to read the many books and papers explaining why different factors have delivered their premiums over time. Sometimes, the most valuable financial legacy a parent leaves a child is a deep understanding of an approach to saving and investing. Whatever your source of knowledge and conviction, it’s critical to know how deep it goes before committing to a different investing approach. Otherwise, you’re likely to find out at the wrong time that it’s not deep enough.

There is also a coping technique we can use to guard against the temptation to lose faith in our approach and switch strategies. We can look away. In many ways, our investing experience is like a long bumpy cross-country drive to a better destination. The difference is that we have the option of sending our goods in a moving truck and flying there at the end. We don’t have to sit in the truck watching every bend and feeling every bump. If we set up automatic withdrawals into a 401(k), robo-adviser or an automated brokerage like M1 Finance with the asset allocation we want, it can all happen in the background while we live our lives blissfully unaware of the ups and downs along the way.

What If You’re Not Comfortable Being Different?

Finally, if you lack the conviction to commit to a different investing strategy and are worried you won’t be able to avoid the doubts that come from lagging the market, there’s nothing wrong with being less different. Investing in the whole market is fine. Investing a small portion of your portfolio in a different approach will be easier to stick with.

One of my favorite things about a broadly diversified portfolio is that it’s harder to have regrets. You can still be a little bit different, but also own some of what’s winning at any point in time. The opportunity for outsized returns may be lower, but so is the risk and pain that must be endured to get it. ?

Discussion

STEPHEN R from CA posted over 5 years ago:

I appreciate that this paper considered small cap and value investments in terms of factors, and I also appreciate the observation that investors being different have to be disciplined and patient. I would like to have seen what happens if foreign stocks are included in the 'different investor's" portfolio.


ROBERT L from VA posted over 5 years ago:

"...years or even decades of underperformance..." ?? Then how can you, in good conscience, suggest that such an approach is advisable for an older investor? I would say it is only a viable approach if your goal is to leave more money for your heirs. The older investor is well advised to only use a [relatively] small portion of his or her portfolio to practice the 'different might be better' approach.


Chris P from CA posted over 5 years ago:

Thanks, Stephen R -- Although there will be periods when the US outperforms international or vice versa, academics say under-diversifying geographically is taking a risk we're not compensated for. Having said that, it is slightly cheaper to invest in the US, and there are more options for inexpensively investing in factor tilts like small and value.


Chris P from CA posted over 5 years ago:

Thanks, Robert L -- I hear what you're saying and agree that someone who needs to access and spend their money in the near term shouldn't put much of it at risk. As you point out, some older investors are essentially caretakers for resources that will be passed on to heirs, and that part of their portfolio is something they can consider investing in less-predictable approaches. There's little reason to believe either the S&P 500 or a portfolio tilted to small and value is particularly predictable. For money that's needed in the near-term, it's far better to protect it with an allocation to bonds. I'll be talking about this at my AAII presentation on October 21st.


GREGG S from CA posted over 5 years ago:

Thanks Chris for an informative article. As a resident in California and dealing with being taxed at their highest marginal rates naturally tax consequences of different investment approaches is of real consequence. Does the four equally balanced portfolio carry a significant tax cost penalty versus the singular SP500 holding which requires no rebalancing? Thanks


Chris P from CA posted over 5 years ago:

Hi Gregg -- Yes, rebalancing any portfolio in taxable accounts will create some tax liabilities, and over the long haul, they could be significant. There are, however, several things we can do to minimize the impact. We can apply contributions to the under-represented assets to rebalance as we go, and we can use tax-loss harvesting to offset some of the gains we'll incur when we do need to rebalance. Vanguard created a nice paper on this topic titled "A Guide to Smart Rebalancing." Sorry, I don't have a more quantified answer for you. I'll have to add this topic to my future research list.


GREGG S from CA posted over 5 years ago:

Thanks Chris for the response, much appreciated. I will read Vanguard article. Hopefully you will get a chance to do some research on this topic. I have found the tax subject to be significantly underdeveloped. Most reply "We cannot give any more insight because tax rates vary for individuals". I find that disappointing. When we study any issue we normally need to understand the boundary conditions to grasp correctly the big picture of the range of possibilities on a variable. Therefore, simply applying the top bracket rates would provide important insights as to whether tax es were a major consideration or a minor consideration. This is of growing importance given the chances that future tax rates in the near term may rise dramatically.


BARRY J from TX posted over 5 years ago:

I really enjoyed reading this article. I am a data freak. The Monte Carlo simulation data underlying the data points are impressive. I would have preferred a factor analysis. I currently experimenting with a Four Fund portfolio build on the AAII SSR approach where you pick 4 stocks from 4 funds of 10 stocks each modeled on 4 well-known successful strategies. Although it is only 2 months old as I write this, it is up nicely vs the S&P 500 over the same period. So much for dumb luck. This article reminded me of most fund prospectus I have read; if you highlight every statement that describes an upside and everyone that describes a downside risk, it always comes out very close to a 50%-50% split. I guess several lawyers edited them. I did the same here. I got the same result. I am not motivated to adopt a “flip a coin” investment strategy. The one thing I did notice is the sharp contract in the Peaks and Troughs chart (Figure 2). Over the 90 years, it covers (1930-2020), c. 67% of the time were 5 trough cycles where the 4 Fund Strategy outperformed the S&P 500 strategy consistently as noted in the article but during these periods. I also noticed that c. 30% of the time there were 4 Peaks cycles where both strategies grew very fast at relatively high rates. All of the transitions were bounded by major events – The Great Depression (1930-1941), World War II (1941-1945, the Kennedy Assassination (1963), the Reagan tax cuts (1982-1983) and the Dot Com bubble (2000-2004). I am not recommending anyone ever adopt a PURE stock market timing strategy, but it seems very plausible that someone could have got out in or out of the market around (+/- 1-2 years) these inflection points and fared significantly better than staying invested during the troughs with either portfolio. Please comment on this. I am swinging about my weight-class here.


DAVE G from WA posted over 5 years ago:

Thanks Chris, for a great article. I particularly liked figure 2 generated by Daryl Bahls that creates a line composed of the Four-Fund portfolio minus the S&P 500 index. I have seen others use this technique but wonder if there is a website that would let me build my own charts using that technique? Thanks again for the article.


You need to log in as a registered AAII user before commenting.
Create an account

Log In

Get your free copy of our special report analyzing the tech stocks most likely to outperform the market.

Download the FREE Report Here: