Does excluding extinct funds skew Morningstar's data on fund performance?
By Christine Benz | 03-30-06 | 02:05 PM | Email Article

You may not be familiar with the term "survivorship bias." But if you believe the authors of a recent study by Savant Capital and the Zero Alpha Group, it has had a big impact on how investors select mutual funds.

Christine Benz is Morningstar's director of personal finance and author of 30-Minute Money Solutions: A Step-by-Step Guide to Managing Your Finances and the Morningstar Guide to Mutual Funds: 5-Star Strategies for Success. Follow Christine on Twitter: @christine_benz.

At Morningstar, we agree that survivorship bias--the nitty gritty of which I'll outline below--is important to consider when evaluating mutual fund performance. But does survivorship bias mean that investors will make poor choices and underestimate the potency of index strategies, as the authors suggest? And does Morningstar's presentation of fund data distort historical returns? We'd argue otherwise.

What It Is
Survivorship bias is created when poor-performing funds liquidate or are merged away. When these losing funds are omitted from category-average performance statistics, the averages tend to creep higher than they would be if the losers were still in the mix.

Here's an example: Say there are just three funds in a given category--one with a five-year annualized return of 15%, one with a five-year return of 10%, and the other with a five-year return of 5%. The average return for those three funds--and in turn the category--would be 10%. But if the laggard with the 5% return were to go belly-up and be removed from the category, the average five-year return for the group would become a more impressive 12.5%.

By inflating category-average returns, the study argues, survivorship bias tends to make your chances of selecting an actively managed fund that will beat an inexpensive index fund look better than they actually are. The study also asserts that when you add the extinct funds back into the averages, the average actively managed fund lags its appropriate index in nearly every category. (Never mind that you can't actually buy an index, but instead must opt for an index fund, which has costs associated with it.)

Index Funds Haven't Lacked for Attention
Although the recent study asserts that Morningstar has had blinders on about the issue of survivorship bias, we've discussed the issue openly, as in this recent article. In addition, we're in the process of developing a survivorship-bias-free database, and frequently turn to it when running studies on funds' past performance. (A recent study in Morningstar FundInvestor, for example, used survivorship-bias-free data to prove that--surprise, surprise--the average fund is hard-pressed to beat an appropriate index.)

Although we're in complete agreement that survivorship-bias-free data merits further study, there's absolutely no evidence that survivorship bias has prompted investors to shy away from index funds, as the study's authors assert. In fact, the evidence runs contrary to that argument. The Vanguard Group, whose domestic-equity lineup is anchored by index funds, remains the largest fund shop in the country, and three of the 12 largest mutual funds are Vanguard index offerings. Moreover, by far the fastest-growing pocket of the fund industry for the past several years has been in exchange-traded funds, all of which are index offerings. Recent data from Financial Research Corporation showed that six of the 10 biggest asset gatherers for the year to date were all index funds, either ETFs or traditional mutual funds.

And even if asset flows into index funds weren't robust, it would be a mistake to suggest that index-fund performance was deterring investors. That's because performance data on indexing--even in a database that only includes funds still in existence--is encouraging. Eighty-one percent of index funds with 10-year records land in the top half of their respective peer groups, and most beat their category peers by a huge margin. Indexing has looked somewhat less compelling relative to active strategies in the small-cap arena, but that owes more to the fact that the major small-cap indexes are frequently targets of front-runners (who drive up stock prices when indexes reconstitute their benchmarks) than it does to survivorship bias.

In keeping with the strength of the data on the merits of index strategies, Morningstar has been extremely supportive of indexing. Ten of our  Fund Analyst Picks--our analysts' best ideas across categories--use index or index-based strategies. We've also bulked up our index-fund coverage over the past several years, and currently publish Analyst Reports on 175 distinct index funds, both conventional funds and exchange-traded funds.

The Funds of Lake Wobegone
In addition, it's worth noting that eliminating survivorship bias from fund-performance calculations introduces its own set of complications. Although the Savant/Zero Alpha study argues that survivorship bias amounts to "grade inflation" for the surviving funds, the opposite is true. Including now-defunct funds in our category rankings would likely elevate the still-existing funds in a category to higher percentile rankings than they otherwise would earn. After all, the now-dead funds would have been likely to continue performing poorly, and therefore would occupy the lowest rungs in the return rankings for a given category, so the still-existing funds' rankings would be likely to creep up. In categories where a high percentage of funds are no longer around, survivorship-free data could have the peculiar effect of showing all of the remaining funds to be above average, just like the children in Garrison Keillor's fictional Lake Wobegone.

In a related vein, calculating category averages and return rankings using the existing set of funds makes intuitive sense for investors who are selecting funds. After all, investors are choosing among funds that are in existence today--not among all funds ever created. A fund that rates as a top-half performer in a survivorship-bias-free category ranking might drop to that group's bottom third once it is ranked against only existing (and presumably better-performing) funds. Thus, it's no surprise that all of the major fund-data providers--not just Morningstar--currently do not include defunct funds in their performance rankings.

Finally, we'd assert that another industry convention--giving equal weight to all funds in a given category regardless of size--paints a far more misleading picture of fund returns than does calculating category averages using only the still-existing set of funds. Asset-weighted returns--that is, performance statistics that give greater weight to a category's largest funds and less weight to its small fry--are far more reflective of the typical investor's experience than are category returns calculated using simple averages. Consequently, we have increasingly turned to asset-weighted returns when discussing fund performance.

By looking at survivorship-bias-free data, investors can draw worthwhile conclusions about funds' performance. And increased attention to the issue of survivorship bias also helps shine a light on the industry's unfortunate practice of launching trendy offerings, then sweeping them under the rug once performance heads south. Thus, we'll continue to research and write about survivorship bias on and in Morningstar's other products. However, presenting survivorship-bias-free data carries its own set of caveats, and in any case, it's a stretch to suggest that there's a connection between survivorship bias and investor choices.

Securities mentioned in this article



Morningstar Rating Morningstar Analyst Report
With Morningstar Analyst reports you can get our expert Buy/Sell opinions on over 3,900 Stock and Funds
Christine Benz does not own shares in any of the securities mentioned above. Find out about Morningstar's editorial policies.
Sponsored Links
Sponsor Center
Content Partners