Survivorship Bias and Other Tricks of the Trade

One of the reasons we turn to evidence-based investing is to guide us past the misguided strategies that can otherwise cause an investor’s expected returns to run aground. That said, there is a lot of “evidence” out there. How do we determine which of it comes from sound science and which may steer you wrong?

Survivorship bias is one trick of the trade we must watch for when accepting or rejecting a performance analysis.

What Is Survivorship Bias?

Only the strong survive. This is a familiar adage because it’s often true – especially in our financial markets. That’s why it is important to remember the expression whenever we want to accurately assess a sample of past returns. Examples of a “sample” might be the returns from all actively managed U.S. stock funds during the past decade, or the returns from all global bond funds from 2000–2014.

Survivorship bias occurs when an analysis omits returns from in-sample funds that were closed, merged into other funds, or otherwise died along the way.

How Often Do Funds Go Under?

Some new funds are truly innovative, do well by their investors, and become familiar names. Less-sturdy ones may instead focus on trying to seize and profit from popular trends. For these, the expression “cannon fodder” comes to mind. They may (or may not) soar briefly, only to fizzle fast when popular appeal shifts.

In the competitive capital markets in which we operate, fund managers launch new products and discontinue existing ones all the time. Individual funds probably disappear far more frequently than you might think.

• A recent S&P Dow Jones Indices analysis found that, for the five-year period ending in December 2015, “nearly 23% of domestic equity funds, 22% of global/international equity funds, and 17% of fixed income funds have been merged or liquidated.”[1]
• As might be expected, the longer the timeframe, the higher the death rate. A January 2013 Vanguard analysis of survivorship bias looked at a 15-year, 1997–2011 sample of funds identified by Morningstar. The analysis found that 46 percent “were either liquidated or merged, in some cases more than once.”[2]  
• A May 2015 Pensions & Investments (P&I) article reported that Exchange-Traded Products (including ETFs) weren’t immune from the phenomenon either, having just reached the milestone of 500 products closed.[3] According to the “ETF Deathwatch” cited source, this represented a mortality rate of just under 23 percent.[4]
• The same P&I piece cited Dimensional Fund Advisors and Vanguard analyses that estimated 15-year mortality rates for traditional U.S. mutual funds in the range of a 50/50 coin flip, or worse.[5]
• A November 2015 article by financial columnist Scott Burns found similar survival rates for the 15-year period ending in 2014. “At the beginning of the period, there were 2,711 funds,” he reported. “At the end of the period, there were 1,139. Only 42 percent of the starting funds had survived.”[6]

Why Does Survivorship Bias Matter?

Why should you care about the returns of funds that no longer exist?

The funds that disappear from view are usually the ones that have underperformed their peers. The aforementioned Vanguard analysis found that, whether a fund was liquidated or merged out of existence, underperformance was the common denominator prior to closure.[7]

If these disregarded data points were athletes on a professional sports team, they’d be the ones bringing down their team’s averages. When assessing a team’s overall performance, it’s important to consider both the wins and the losses, right? Same thing with fund performance.

Instead, an analysis marred by survivorship bias is highly likely to report overly optimistic outcomes for the group being considered. While a degree of optimism can be admirable in many walks of life, basing your investment decisions on artificially inflated numbers is more likely to set you up for future disappointment than to position you for realistic, long-term success.

Moreover, survivorship bias is only one of a number of faults that can weaken seemingly solid reports. One way in which we strive to add value to investors’ evidence-based investment experience is to help them separate robust data analysis from misleading data trickery. We hope you’ll be in touch if we can assist you with your own strategies and selections in a market that is too often rigged against the individual investor.