Press "Enter" to skip to content

Why is it so hard to beat the market index?

Why is it that almost nobody these days can beat the S&P 500 index by picking single stocks? I keep hearing that it’s impossible to beat an index fund, but why?

The idea that “you can’t beat the index” (whether that index is the DOW or S&P 500 or another index) is one of those pearls of wisdom which is often given as advice. In context, in the right situation, the advice is good and the wisdom holds. But over time the soundbite, the core of the wisdom, gets shared without context. It gets repeated often and without context until it’s just seen as an absolute rule. At which point the wisdom, which in one context was very useful, becomes incorrect.

Putting aside money for a bit, let’s talk about another pearl of wisdom that often works well, but gets repeated a lot and would be dangerous outside of its proper context. For instance, the advice, “Don’t worry about what other people think, just be yourself,” is really supportive advice to give to a young artist, or a nervous teenager about to go on a date, or a person about to meet their future in-laws. However, it’s terrible advice to give to a sociopath struggling to learn appropriate social behaviour. Likewise, “You should be more careful,” is fine advice to give a daredevil about to ride their motorcycle along the edge of a cliff, it’s not great advice to give someone constantly plagued by anxiety.

The advice “Don’t try to beat the index” is one of those pearls of wisdom which works well a lot of the time, but it requires context to be appropriate and meaningful.

The reason why “Don’t try to beat the index” or “You can’t beat the index” is often heard in investment circles is passive investing (putting money into index funds or mutual funds) usually works well for most people. Most of us don’t have the time and energy to properly research companies, stay on top of financial news, analyze stock prices, and shuffle money between stocks to squeeze the best performance out of our investments. Most of us don’t have the time, knowledge, or luck to consistently beat the market’s average performance. For most of us, index funds will get the job done, do it well, and make us a profit. Plus it will take almost no effort on our parts. Index fund investing is pretty much a “set once and forget about it” situation.

Some people can and do “beat the market”, getting returns better than index funds. Some of these people are lucky, some are good at analyzing companies, some are good at timing the market. However, and this is key: It’s not all that hard to beat the market once, especially in an “up” year when stocks are rising.

Someone can buy Apple at the right time or roll the dice on Amazon or buy into the right energy company on the right year. And they can outperform the market, at least that one time, that one year. But there are not many people in the general population who can get better returns than the market consistently, year after year. Especially on a down year. It’s easy to make money when the stock market is rising quickly, it’s tricky on the “down” years to pick the big winners (or the smallest losers).

On a longer time line most people make mistakes, get unlucky, or mess up their timing and it can damage their investing portfolio significantly. People who invest in individual stocks eventually back the wrong stock or move out of investing in a company at the wrong time and they lose a lot of money. Meanwhile the index funds chug along at an average pace.

So this is the context: it’s not impossible to beat the market, on a short time line it might not even be hard to beat the market. This is why there are stories about monkeys throwing darts at lists of stocks and picking winners, or five year olds helping their parents pick a winning investment. It’s not hard to get lucky once, for one quarter or one year. But it’s increasingly difficult to outperform the market over a span of years or decades.

This is why – for most people, most of the time – the wise choice is to ignore individual stocks and invest in mutual funds or index funds. Because these are relatively stable, relatively predictable, and relatively safe. And, over the long-term, they are likely to outperform people who try to pick individual stocks and juggle their own portfolios.

Comments are closed, but trackbacks and pingbacks are open.