Index Funds vs. Stock Picking — The Data

Why It Matters

The investment industry employs tens of thousands of highly educated, well-resourced professionals whose job is to pick stocks that beat the market. If skill can beat the market consistently, they would show it. The SPIVA data, published semi-annually by S&P Dow Jones Indices since 2002, shows what actually happens.

Over any given 15-year period, approximately 88–92% of US large-cap active fund managers fail to beat the S&P 500 after fees. Over 20 years, the percentage of survivors who also outperformed is even lower — many funds close before the measurement period ends, a bias called survivorship bias.

This matters practically: if you invest in an average actively managed fund instead of an index fund, you will almost certainly end up with less money. The arithmetic is not subtle — it compounds over decades.

Understanding why this happens — not just that it happens — is what separates a person who parrot statistics from one who actually understands markets. The 'why' is in the math of fees, the mechanics of markets, and an honest accounting of what 'beating the market' even means.

A Story

The Bet

In January 2008, Warren Buffett made a public wager through Long Bets, a nonprofit platform for verifiable predictions. His bet: a simple S&P 500 index fund would outperform a portfolio of hedge funds over the next ten years, after fees.

Protege Partners, a New York fund-of-funds firm, accepted. They selected five funds of funds — each of which itself held dozens of underlying hedge funds managed by sophisticated professionals. Buffett chose the Vanguard S&P 500 Admiral fund. Each side put up roughly $318,000 into zero-coupon Treasury bonds, to be worth $1 million at the end of 2017. The winner donated the proceeds to charity.

The competition ran from January 1, 2008, through December 31, 2017. The financial crisis hit almost immediately. By early 2009, the S&P 500 was down nearly 50% from its peak.

Protege's hedge funds fell less than the index in 2008 — they lost about 24% while the index lost 37%. The hedge fund managers felt vindicated. They had protected capital. This is what they were paid to do.

Then markets recovered. The S&P 500 compounded relentlessly upward. The hedge funds, carrying 1% management fees and 20% performance fees, clawed back gains more slowly. By 2012, the index fund had taken the lead.

By 2015, the outcome was no longer in doubt. Buffett said he would have offered to settle early, but the Protege partners were too professional to walk away from a public bet.

Final result at December 31, 2017: the S&P 500 index fund returned 125.8% over the decade. The five hedge funds of funds averaged 36.3%. Buffett won by a margin of nearly 90 percentage points.

The $1 million went to Girls Inc. of Omaha. Buffett noted in his 2017 annual letter that the hedge fund managers had probably earned tens of millions in fees from their investors during the same decade their funds underperformed a passive index.

Protege's Ted Seides, who managed the bet for Protege, later wrote a thoughtful post-mortem. He acknowledged the result but argued his fund selections had been unlucky in timing — the bet started at a market peak and ran through an unusually strong bull market. The index fund structure is most advantaged in strong bull markets.

Buffett's response was to note that no one knows in advance when a bear market will arrive, and the hedge funds had higher fees in every environment. Over long periods, fees win.

Vocabulary

index fund

A fund designed to replicate the returns of a market index (like the S&P 500) by holding all or most of the index's constituent securities in proportion to their weight. Requires no active stock selection.

active management

An investment approach where a fund manager selects securities trying to outperform a benchmark index. Typically involves higher research costs and trading, resulting in higher expense ratios.

survivorship bias

The statistical distortion caused by only analyzing funds that still exist. Funds that closed due to poor performance are excluded from historical averages, making past active fund performance look better than it actually was.

alpha

Return earned above and beyond what a benchmark index produced for the same period, adjusted for risk. A manager with consistent positive alpha is genuinely adding value; most do not.

tracking error

The degree to which a fund's returns deviate from its benchmark index. Low tracking error means the fund closely mirrors the index; high tracking error means it diverges — which can be good or bad.

expense ratio

Annual fund fee as a percentage of assets. Vanguard's S&P 500 index fund charges 0.04%; a typical active fund charges 0.6–1.0%; hedge funds often charge 2% plus 20% of profits.

Guided Teaching

Start with a basic question: if you hire the best money manager in the world, can they beat the market reliably? Ask students to guess what percentage of professional fund managers beat the S&P 500 over 15 years. Most will guess 40–50%. The actual answer is roughly 10–12%.

Ask: If only 10% of professionals beat the index after fees over 15 years, what does that tell you about your own chances of picking winning stocks? This is not a trick — it is a genuine inference problem.

The SPIVA (S&P Indices Versus Active) scorecard is published free at spglobal.com. It is not a blog post or an opinion — it is audited data covering thousands of funds over 20+ years. The consistency of the finding across time periods and geographies is striking.

The core arithmetic: every dollar paid in fees is a dollar not compounding. A fund with a 1% expense ratio that charges $100 on a $10,000 account each year — that $100 would have compounded at 7% for 30 more years. The true cost of a 1% fee is not 1% of your ending balance; it is the compounded value of every fee dollar taken over the full holding period.

Walk through the numbers explicitly. $10,000 at 7% net return for 40 years = $10,000 × (1.07^40) = $149,745. At 5% net return (after a 2% annual fee): $10,000 × (1.05^40) = $70,400. The 2% fee does not cost 2% of $149,745 = $2,995. It costs $79,345 — more than seven times your original investment.

Ask: Why do active funds still attract trillions of dollars if the data is this clear? Possible answers: marketing, optimism, recency bias (recent outperformers attract money), complexity that feels like expertise, and the genuine small possibility of picking a winner.

Survivorship bias deserves its own moment. If 1,000 funds launch in 2000 and 300 close by 2015, any database that only contains the 700 survivors will show average returns that look better than reality. The funds that closed were mostly bad ones. The advertised '10-year track record' of funds in 2015 only includes funds that survived 10 years.

The efficient market hypothesis — in its semi-strong form — says that publicly available information is already reflected in stock prices. If that is roughly true, no amount of research on public data can systematically produce superior returns after costs. Ask: Does this mean markets are perfectly efficient? No — but it means the edge needed to overcome fees is high enough that most professionals cannot clear it.

The honest conclusion: index investing is not exciting. It does not make for good stories. But the data is what the data is, and intellectual honesty means following evidence rather than narrative.

For Discussion

1. 1.The Buffett-Protege bet ran from 2008–2017, a period that included a severe crash and then an unusually strong bull market. Does that make the result less valid? Why or why not?
2. 2.If 90% of active managers underperform the index, why do people keep investing in active funds? What psychological or structural reasons explain this?
3. 3.Survivorship bias affects almost every 'best of' list in any industry. Can you think of three fields besides investing where survivorship bias might distort how we perceive average outcomes?
4. 4.A friend says 'I know this active fund has beaten the S&P 500 for 5 years straight — that proves the manager is good.' What are three reasons this reasoning might be flawed?
5. 5.The SPIVA data shows active managers underperform more in US large-cap than in small-cap or international markets. Why might that be?
6. 6.Jack Bogle argued that in aggregate, active investors cannot outperform the market because they are the market. What does this mean mathematically?
7. 7.Warren Buffett has beaten the S&P 500 over his 60-year career. Does that contradict the SPIVA data? How should you think about genuine outliers in any statistical distribution?

Practice

Run the Fee Compounding Math

1. 1.Take a starting balance of $10,000. Calculate the ending balance after 30 years at four different annual net return rates: 7.0% (low-cost index), 6.0% (modest active fee), 5.0% (typical active fee), and 4.0% (high-fee or fund-of-funds). Use the formula: Ending = Principal × (1 + r)^years.
2. 2.Build a simple table showing: rate, ending balance, and the dollar gap versus the 7% baseline. For each gap, express it as a multiple of the original $10,000 investment.
3. 3.Look up the actual current expense ratio for: (a) Vanguard S&P 500 Index Fund Admiral Shares (VFIAX), (b) a Fidelity actively managed US equity fund of your choice, and (c) any hedge fund you can find data for. Calculate the annual fee on $10,000 for each.
4. 4.Find the most recent SPIVA US Year-End Scorecard at spglobal.com. Record: what percentage of US large-cap active funds underperformed the S&P 500 over 1, 5, 10, and 15 years?
5. 5.Write a two-paragraph summary: given this math and this data, what is the rational default choice for a long-term investor who has no specific informational edge? Acknowledge what an investor would be giving up.

Memory Questions

1. 1.According to the SPIVA scorecard, approximately what percentage of US large-cap active fund managers underperform the S&P 500 after fees over 15 years?
2. 2.Describe the Buffett-Protege bet: who bet what, over what period, and what was the result?
3. 3.What is survivorship bias, and why does it make historical active fund performance look better than the median investor's actual experience?
4. 4.If a fund charges 2% annually versus 0% (index), and both earn 7% gross, what is the dollar difference after 40 years on $10,000?
5. 5.What does alpha mean, and why is consistently positive alpha difficult to achieve after fees?
6. 6.State Jack Bogle's core argument for why active investors in aggregate cannot outperform the market.