Volume 19: The Efficient Market Hypothesis

It's Why You (Probably) Can't Beat the Market

“The stock market is a device for transferring money from the impatient to the patient,” – Warren Buffett

---

As a society, we have decided to invest our time, our currency, and some of our greatest minds towards the task of beating the market. Teams of Ph. D’s sit in New York, Chicago, Greenwich and elsewhere, looking for any tiny edge they can exploit. They combine form a giant machine, taking in information, spitting out predictions for asset prices.

As our knowledge base becomes larger - satellite pictures of Iowa cornfields, micro-data on consumer spending, professional “Fed Watchers” – this machine is able to incorporate more information, faster. With all of this effort, we do a pretty good job; stocks, bonds, commodities and foreign exchange rates must cost just about what they are worth. But this, itself, is a problem. If you can only buy or sell something for what it’s worth, then there is no way to profit. If there were no way to profit, then all of this effort would be wasted. Something doesn’t add up.

This is the Efficient Market Hypothesis, and its paradox.

The Efficient Market Hypothesis is wrong. If you can figure out how and when it’s wrong, you’ll soon find yourself a billionaire.

• What is the Efficient Market Hypothesis?

• Do professional investors beat the market?

• How does anybody make any money?

---

What is the Efficient Market Hypothesis?

Classical economics is the explanation of production, consumption and transfer of wealth. It predicts how much the price and quantity of apples will change after a weak harvest. It gives direction for how changes in tax policy will affect the economy.[1] It tackles the thorny issues involved when governments regulate the crossing of otherwise arbitrary lines, otherwise known as international trade.

As international capital markets became larger and more sophisticated in the early 20th century, we realized that financial assets don’t always follow the same economic rules as physical goods. To study these differences, Finance Theory began to forge its way as a separate discipline. The Efficient Market Hypothesis, or EMH, belongs to Finance Theory.[2]

In 1950, despite formal financial markets having existed for over 350 years, we still had little theoretical understanding of how assets were (or should be) priced.[3] We knew it had something to do with risk; investors demand higher expected returns for a garage-based start-up than for a U.S. Treasury Bond. But we knew that not all risk was worth of reward. Just because the casino game of roulette has massive risk doesn’t mean its return is better than other investments. We also knew that diversification was generally a good idea. As early as 1615, Sancho Panza told us that a wise man does not keep all his eggs in one basket. But that’s just about all we knew.[4]

A large, early step in the development of finance theory was portfolio optimization. Advanced by Harry Markowitz and others in the early 1950s, portfolio optimization uses correlation to increase returns without increasing risk. Holding a portfolio split between two assets generally has less risk than a portfolio holding only one.[5] Adding additional assets allows even better portfolios. Taking into account the entire investment universe, an optimal portfolio can be constructed for any desired amount of risk. Because this is a limit which theoretically can not be exceeded, these portfolios are called the efficient frontier.

Figure 1 - The Efficient Frontier and Capital Allocation Line

The next step towards understanding asset prices was the development of the Capital Asset Pricing Model (CAPM). Figure 1 shows the efficient frontier; however, you’ll see an additional point, representing the return possible from a risk-free portfolio. Investors can put any or all of their funds in this portfolio. A line starting at this point will be tangent to our efficient frontier at a single point, which we call the tangency portfolio. An investment of some of our assets in this tangency portfolio, with the rest in the risk-free asset, will outperform any other investment. Each investor can choose their own risk tolerance. The portfolios so described are on the capital allocation line, or CAL.[6]

But, therein lies the rub. Taken to an illogical extreme, all an investor needs to do is identify their personal tolerance for risk, feed it into an algorithm, and invest in the resulting portfolio. These investments won’t be based on whether a company is well run, or if people actually want its products, but rather market parameters like volatility and correlation.[7] The results of the algorithm will be optimized, in that, for a given level of risk, the portfolio will have an expected return greater than or equal to any other portfolio. This is a serious issue for mutual fund managers, asset allocators, and other who charge millions to advise clients on portfolio construction.

The Capital Asset Pricing Model and Capital Allocation Line are theoretical constructs; there is no perfect algorithm for real-world markets. But, even if they are vast oversimplifications, they still call into question the vast resources we put into security analysis. If these assumptions can eliminate literally every reason for such analysis, then perhaps the real-world value is still far less than often assumed. This, finally, brings us to the EMH.[8]

When we say that markets are efficient, we mean that their prices incorporate all available information (or a specific subset thereof). We can use an example from sports gambling, which is as useful as it was way back in Volume 7. Consider an NFL game: the Patriots are playing the Packers. Let’s say the initial “market” of gamblers assume that each team has a 50% chance of winning.[9] Then, Tom Brady blows out his knee in practice; it is immediately reported to everybody. The odds will move, with the Packers now being heavily favored. The market was efficient with respect to this new information; it was immediately incorporated in the price. Alternatively, let’s say Tom called only me and said that he was going to sit on Sunday.[10] I would be able to bet against the Pats at very favorable odds; the market won’t really move right away, because nobody else knows. In this situation, the market was not efficient.

We can take another example from financial markets proper: a company reporting quarterly earnings. If a company reports good earnings, the stock price will go up before you could buy any shares. The positive information contained within the announcement is therefore incorporated almost immediately, which prevents you from taking advantage of it. Even further, the nothing of “good earnings” will be based on the expectations of the market overall. This means that not only the earnings report, but also the in-depth studies of many professional analysts are also incorporated in the stock price.

These examples demonstrate a major reason why asset prices change: the market receives new information. This is the essence of market efficiency. However, we’ve also seen that markets are not equally efficient with respect to different types of information. For this reason, there are three forms of the EMH and we are, finally, ready to state them. Recall, however, that no version of the EMH actually describes real markets.[11]

EMH Weak Form: Asset prices incorporate all past price and volume data.

If you flip a coin ten times, getting ten heads, it is tempting to think that on the 11th roll, another head is the most likely result. If the coin is fair, this is of course not true. Similarly, some people think that stocks that have been going up will continue to go up and that stocks that have been going down will continue to go down. If the Weak Form of the EMH were true, so-called momentum trading is no better than a coin flip.

Using “charts” of past prices and data to predict future movement is generally called technical analysis. There is a massive amount of literature showing that making money via technical analysis is, really, really difficult. That being said, there are numerous billions of dollars invested in such strategies.[12] For the first time, we’ve found an apparent paradox with in the EMH.

EMH Semi-Strong Form: Asset prices incorporate all publicly available information.

A lot of people buy stocks because they think its underlying company will be successful. There are literally 24-hour television channels, CNBC being the most prominent, theoretically dedicated to informing the public as to which companies are the good ones. Investment banks, mutual fund managers, hedge funds and other investment managers employ armies of equity researchers to understand and predict the future results and stock prices of corporations. If the Semi-Strong Form were true, all of this effort would be in vain.

Those who make investment decisions based on balance sheets, management credibility, or even whether they saw a full parking lot at a given store, are participating in fundamental analysis. There is, again, a lot of research showing that fundamental analysis doesn’t work very well. Specifically, over the long term, few active equity managers outperform broad-based equity indices.[13] This again brings us back to the paradox; if fundamental analysis is pointless, why do people bother doing it?

EMH Strong Form: Asset prices incorporate all available information.

If you are working at an investment bank, and have non-public knowledge that a client is about to be bought by another company for a large premium, you might be tempted to go and buy some of the stock before the news goes public. Do not do this, it is very illegal, and you are very likely to get caught. And if the Strong Form were true, it would be pointless anyway.

Those who trade on material, non-public information are said to be engaging in insider trading. In equity markets, insider trading is illegal because it can be used to earn outsized profits without taking financial risk. This isn’t only unfair, but would also poison faith in the market.[14] In these markets, the Strong Form is definitionally not true; trading on insider information is expected to result in profits.

There are markets, specifically commodities and foreign exchange, that do not prohibit insider trading. They couldn’t operate otherwise; corn growers always have insider knowledge of future corn prices. A prohibition on their use of this knowledge would be a prohibition on producers trading corn, which largely defeats the purpose of having a commodity market. Markets without prohibitions on insider trading are literally “buyer beware.”[15]

Do professional investors beat the market?

Let’s take a step back, and define a term, the investment benchmark. A benchmark is a gauge, usually an index, whose performance will mirror a broader market. There are stock indices, bond indices, foreign exchange indices and more. For each of these, there are sub-categories: equities in France, emerging market currencies, high-yield bonds – just about anything you can think of. There are literally thousands of these indices produced by dozens of index sponsors.

It is easy to see that investors, as a group, can not outperform the benchmark.[16] The benchmark represents the entire market. All investors, put together, represent the exact same thing. The two must therefore be equal. This isn’t finance, it’s just math. For every investor who outperforms their market, there must be one who underperforms.

It seems plausible that there would be good investors and bad investors. The former would be expected to outperform the benchmark, on average and over the long term. If good investors are expected to outperform, this necessarily means that the latter will underperform.

If true, the EMH tells us that this is not an accurate description of the world. There would be no such thing as good or bad investors, simply lucky or unlucky ones. Mutual fund managers are no more likely to be right than a roulette wheel. Everybody’s expected return is the point on the CAL determined by their risk tolerance.[17] As we’ve said above, the EMH is definitively not true across markets, and we’ll come back to show some of the reasons why. But first, we can find a shocking amount of evidence against the existence of good investors.

The first piece of news in favor of the EMH is the poor long-term performance of active equity managers. The mutual funds run by these managers generally have a clearly stated benchmark against which they can fairly be compared. And, again, this specific pocket of the financial industry pockets billions in fees annually. Those who own these funds are saying, with their wallets, that they are managed by good investors.

This is easy enough to check – how do active equity managers perform against their benchmarks? Fortunately, we don’t need to do these calculations ourselves, Standard and Poor’s regularly considers the question for us.[18] In 2016, depending on the category, between 60% and 97% of active managers underperformed their benchmark. Over a 15-year period, between 80% and 99% of active managers underperformed, depending on the category. These managers do not appear to be choosing their investments wisely.

There are a few caveats that should be mentioned. The period in question was one of strong equity returns. Actively managed funds tend to invest in stocks that are less risky than the broad market. They also hold some portion of their funds in cash. Both of these factors will cause a drag on performance during a period of sustained equity market appreciation. In addition, the study doesn’t directly address the EMH; it lends evidence that these investors don’t outperform the market on average. The EMH is stronger; it says that no investor should be able to have consistently superlative returns.

To address the EMH, we must consider the possibility that, among this group of managers, there are some who outperform, offset by some who underperform. However, in order to count as outperformance, good investors should do so via skill rather than luck. We would expect to see the same managers on our “naughty” and “nice” lists each year. If there is such a thing as investment skill, it should persist.

You can probably guess not only the punchline, but also that there are studies backing it up. One of many examples is a recent study that followed 2,862 actively managed funds from 2010 through 2014. It looked to see if specific funds remained in the top quartile from year to year.[19] Only two funds outperformed their peers in this way during each of the five relevant years. If the performance of all the funds had been random, we would have expected three such outperformers.[20]

So, as a group, mutual fund managers don’t outperform the market, and within the group, there are no standouts. This, of course, doesn’t prove the EMH, but it is a strong piece of circumstantial evidence for the Semi-Strong Form. But what about hedge funds? Unlike mutual funds, which are bought by schmucks like you and me, hedge funds cater to the wealthy and sophisticated, both individuals and institutions. Unlike mutual funds, whose managers are multi-millionaires, successful hedge fund managers are some of the world’s wealthiest billionaires.

Clearly, some hedge funds have shown the ability to produce consistently exceptional returns over long periods. Some of these hedge funds, who exhibit consistent expected returns greater than their risk would imply, invest on a purely technical basis, contradicting even the Weak Form EMH. That being said, investing in a successful hedge fund – or even measuring performance of the industry – is far from easy.

Many analyses of hedge fund performance do not reflect the experience of actually investing in these products. One reason why is survivorship bias. If you look at all the extant funds at a single point in time, they will tend to have strong historical returns. There is a good reason for this – funds that lose money tend to shut down quickly, thus falling out of your snapshot. Properly performed studies of hedge fund returns take this into account; after having done so, the value-add of the hedge fund industry is significantly lower.

The hedge fund universe is also replete with stories of once high-flying funds whose returns have since come back down to earth. As funds have grown in size, few of even the successful ones still drop jaws. One reason for this is that many strategies are not scalable to managing more assets. In addition, there is a clear decline in hedge fund performance on average over the last 20 years. This would support a story where markets are becoming more efficient, making outsized profits more difficult to achieve.

Which is not to say that there are no money managers who perform well, over the long-term, via skill rather than luck. As the EMH implies there should literally be no such investors, we know that the Hypothesis does not perfectly describe capital markets. However, precisely because the significant evidence in its favor, it is worth spending some time discussing some of the EMH’s flaws, and how difficult they are to exploit.

How does anybody make any money?

After that likely-depressing trip down efficiency lane, we can finally get to our point: showing where and why the EMH does not always work.

We can start with some big carve-outs; the EMH applies to tradable investments with at least some amount of trading liquidity.[21] Investments like venture capital, private investments in small companies, are outside its scope. Same for real estate although there are signs that this market is becoming more efficient as its liquidity and transparency increase. An operating business is also not going to fall prey to the EMH; go ahead and start that laundromat, restaurant, coffee shop or international shipping company.

Recall that the EMH involves the dissemination of information into market prices. This shows us one way to avoid its wrath. If you are able to legally get information before everybody else, then you may be able to trade before it is fully incorporated, earning a profit.[22] The extreme example of traders who try to incorporate information faster than others are the so-called high-frequency traders. The best of them do, indeed, earn excess returns. Some rely only on price and volume information; these provide counterexamples to even the Weak Form of the EMH.

Successful high frequency traders are very, very good at incorporating information quickly. They hire Ph. D’s in physics and mathematics to work with some of the best computer scientists in the world. They make crazy infrastructure investments, like private microwave networks, to move information between trading centers faster. They put their servers in literally the same building as their relevant exchange, so that they are not slowed down by the speed of light. They talk in terms of microseconds, and not very many of those. If you are not doing any of these things, you are probably subject to the Weak Form of the EMH.

Some investors are also able to get better fundamental information than the market, usually through sheer doggedness. They fly planes over the Gulf of Mexico, counting how many oil rigs are pumping. They station themselves in the parking lot of your local superstore, counting how many people shop there. They scour social media, trying to determine how many users each network has (among other things). They buy credit card purchasing data in bulk. These are real examples, and they are legal.[23]

By creating such a differentiation in information, an investor can reasonably hope to profit, even if no longer operating in microseconds. This breaks the Semi-Strong Form of the EMH. Again, I urge caution before following such a strategy. Information is an arms race; you are fighting against investors with information of the type I’ve described here. Also, having information isn’t good enough – you need to know what to do with it. Maybe you counted the number of bags carried by each customer exiting a grocery store, gaining information that nobody else has. This information does you no good if you can’t use it to predict future asset prices.

If you don’t think you are able to avoid the strictures of the EMH, you should generally be happy to just meet your benchmark. Fortuitously, just as the EMH was gaining popularity, a product set was birthed that permitted just this type of “index investing.” Many exchange-traded funds (and some other structures) manage money passively, mechanically trying to do what the benchmark does. These products have proven popular with investors of all sorts. The best-known passive investment product, the SPDR S&P 500 ETF Trust, manages over $250 billion in total assets.

A world with too many passive investors has its own problems. If literally every dollar invested in the stock market was in index funds, then nobody would be paying any attention to what companies are worth. In this extreme scenario, a stock price would be completely divorced from any value of its underlying company. Such a market would be highly inefficient.[24] If you were the only investor in this market who used information, be it technical or fundamental, you would certainly be able to earn outsized profits. The efficiency that helped popularize this type of investing could well destroy itself.

Passive investments currently account for 29% of all U.S. equity holdings.[25] And growing…

---

As some of you know, I’ve spent 15 years working on Wall Street. During that time, friends from outside the business have often asked how I square what I do for work with my political views, or even my ethics and morality in general. In practice, I’ve never had much issue in doing so. Ruthless, heartless, unregulated financial markets require a libertarianism that is not consistent with my world view. However, I also have little doubt that free, if regulated, capital markets are necessary to improve our standard of living. These markets only work if independent actors participate, motivated by personal gain.

Financial services are a huge, global business. If you are looking for something in it, good or bad, you’ll find it somewhere. You’ll find good people, bad people, good investments, bad investments. You’ll find people who help the junior members of their team, selflessly serving as personal and professional mentors. You’ll find people who think nothing of hurting others to further their own career. You’ll find people who defraud those whose trust they had fraudulently earned. You’ll find people who fight to help their clients, often at their own expense. You’ll find people who fight for their clients past the limits of what is appropriate, and, on rare occasions, what is legal. In this way, I don’t think finance is so different from any other industry.

What is different is the amount of money at stake.

If you sell ice cream, selling two ice cream cones is nearly twice as difficult as selling one. It takes twice as much milk, twice as many ice cream makers, twice as many people working the cash registers, and, eventually, twice as many trucks and storefronts.[26] Making a trade of twice the size requires virtually zero incremental effort. If you trade for yourself, you know this – buying 200 shares instead of 100 earns twice the profit. It takes twice the capital, but exactly the same amount of effort.

This results in a situation where the best performers in finance can hope to earn profits virtually impossible to achieve in other industries. It also means that there can be personal riches available to those who would over-promise and under-deliver.

Therefore, you should be careful when somebody brings you investment ideas. The EMH is a tool you can use when considering investment choices. Many “opportunities” involve their proprietor implying that they know how to avoid market efficiency. But, as we have seen, doing so is not easy; very few are able.

When considering an investment with somebody who claims that the EMH doesn’t apply to them, it is important to determine why. Are they getting their information faster? Are they getting more information?[27] Are they better at using the information they have to trade? Are they involved in illiquid securities, which can be expected to be less efficient? Do they have some business plan, allowing them to profit from actual operations as opposed to trading?

If you aren’t able to answer yes to any of these questions, be wary. And, never insider trade.

---

[1] Even if certain political parties don’t like the answers.

[2] As usually defined, finance theory, also called “financial economics,” concerns how corporations are structured and how assets are valued. This this piece, we are concerned with the latter.

[3] At a simple level, the value of a financial asset is equal to the sum of all the cash you’ll ever receive due to buying it; this is called the discounted cash flow, or DCF, model. DCF analysis, a core component of finance theory, is both a science and an art and beyond the scope of this piece. We’ll simply note that it is both an art and a science.

[4] Disclaimer: nothing in this piece should be interpreted as an offer or sale of any security, as defined by the Securities Act of 1933.

[5] The boundary case is when the assets are perfectly correlated.

[6] With the assumption that we can also freely borrow at the risk-free rate, we can extend the CAL past the tangency portfolio via leverage. You and I don’t have access to this option, but is realistic for many large, institutional investors.

[7] Of course, these might be related to the quality of the company. Or they might not; markets are funny like that.

[8] This telling of the EMH’s story is logical, rather than historical. In fact, the idea of market efficiency pre-dated the publication of the CAPM and CAL by decades, at least. The modern, formal, empirical statement of the EMH comes from work by Eugene Fama and Paul Samuelson in in the late 1960s. Samuelson’s Nobel came in 1970, with the designation noting his work across economics. Fama had to wait until 2013, but his prize was specifically related to his work on asset prices. A nice timeline of all these goings-on is available here.

[9] We’ve magically repaired Aaron Rodgers.

[10] He said he “just needs some me time.”

[11] These are my statements of the EMH. I haven’t found a single, canonical wording that I like.

[12] Again, managers of such investments reap billions in fees annually.

[13] Again, this doesn’t prevent such managers from earning billions in fees.

[14] For what it’s worth, there are managers who have charged billions in fees to engage in de facto insider trading. The authorities continue to get better at catching them. Many have been shut down but some, certainly, still remain.

[15] Do I have to tell you again? Don’t insider trade. Whether it’s Wall Street or Trading Places, your favorite film tells you that that it does not end well for people who do this. These movies are not entirely fictional!

[16] We’re assuming that the benchmarks are well constructed. “Light all my money on fire” is, in theory, an easily beatable benchmark.

[17] This does assume that investors are rational. If you had some investors ignoring or avoiding optimal portfolios, they could be expected to underperform, allowing outperformance by others.

[18] Their 2016 report is available here. See specifically, Report 1 on page 8.

[19] Reported on here.

[20] Rounded up at least: 2,862 * (0.25)^4 = 2.79. For what it’s worth, the two funds in question are not especially prominent.

[21] Liquidity means the ability to buy and sell an asset with relative ease. Large-cap stocks are very liquid; real estate less so.

[22] Again: do not trade on insider information. If you think there is even a possibility you have come into material, non-public information, find the correct resource at your employer and explain the issue. If your knowledge comes from somewhere other than your employment, you may need to speak to an attorney.

[23] All this information is available to the public, if costly to obtain.

[24] Recently, prominent manager Sanford C. Bernstein released the provocatively titled “The Silent Road to Serfdom: Why Passive Investing Is Worse Than Marxism,” expounding on this concept. The idea is that in even in a statist economy, some sentient being is allocating capital, no matter how inefficiently. In a world of only passive investors, nobody is even trying to do so. I’m not able to find a public link to the article, but I encourage you to do so; otherwise, see reporting on it here, as well as many other places.

[25] According to a Moody’s study.

[26] Economies of scale, blah, blah, blah. Just let me have my analogy.

[27] And if so, is such information legal for them to use in their investment process?