Which is to say the traditional more expensive managed funds that actually pay attention to the fundamentals of the companies they invest in should see a comeback. While this style of fund is more expensive (because a human can only examine a few companies in a year in enough detail to decide if they are worth investing in - as a full time job you can maybe do 50) by investing only in companies that will do better than average they can beat the market (or shorting if you want to play companies that will do far worse than average). So far the low costs of index funds have made them a better investment despite them not investing in strong companies, but we should see the day where a managed fund can beat the index funds just because the index funds are leaving the advantages of analysis on the table.

You can argue [meaning this might or might not be correct] that historically managed funds have done worse than index funds because there are so many managers that anytime there is a slight deal someone jumps on it before the deal is large enough to pay for the costs of finding it. However if you don't jump on it someone else will and they make something on the deal while you make nothing. Thus as index funds take over there will be more and more deals for the managers to find, and managers can wait until they are large enough to be worth the price.

It will be interesting to see when/where the line is crossed.

I think the main point is that index funds still rely on traditional market players to effectively allocate risk. And as index funds take up more of the market, they become less able to do that. Right?

The fund has to manage holdings around fund purchases and redemptions, and when the index changes.

In my view, index funds aren’t really passive at all, they are crowdsourcing the best ideas of active management. This is why many indices (like S&P 500) produce pretty good returns.

If you created an index held every US equity in equal proportions, regardless of price movement, that would be like what you’re talking.

If you compared the returns of the S&P 500 against this theoretical index (let’s make it an ETF and call it “DUMB”), you would find that the S&P 500 would have much better returns.

The takeaway from this is that many indices produce stellar returns and aren’t as “passive” as one might think. Think of factor indices or whatever.

Normally markets remain efficient because they provide an incentive for people to actively research & surface all available information on a company's future prospects. If most people aren't doing this, then a.) the market price will be slower to react to bad information about the company and b.) people who do actively react will make larger profits, as they can trade on their information before the majority of the market takes it into account.

There's an equilibrium level of disequilibrium - as more people pursue passive investing, returns to active investors rise, until some of those passive investors realize they can make large profits as active investors, restore market efficiency, and destroy the profit potential of active investing. I'm not sure exactly where we are in that cycle, but there's some evidence that stock prices have become less volatile overall except for major news-related panics, which would be expected if a large proportion of people are passively investing.

Ultimately the problem domain of monitoring every publicly traded company and prognosticating their actions is huge, and the job is so messy that it will never be cheap. There should be an information theory paper on this somewhere.

The main investment risk to index funds growing is that, if everybody is a passive investor, then the passive investors are worse off as there are very few active investors who actually try and value companies appropriately.

On the other hand, if the market is littered with active investors, then the market is likely more efficient and 'correct', and so you're (probably) better off as a passive investor.

I'd rather see slower, predictable gains than bet my nest egg trying to go toe-to-toe with hyperefficient machines -- or hand it off to some Manhattan finance bro making that bet on my behalf.

That being said - you're incorrect about about competition between active investors and HFT. That's a common misconception. HFT primarily occupies a marketing making role, which means they try to play both sides of the spread very quickly for a very, very small profit on each trade. There are elements of valuation here, but what's really much more important is very small holding times and low latency turnaround. The ideal goal of an HFT operation is a trading strategy which earns a profit 51% of the time and trades very frequently.

In contrast, active investors - whether quantitative, fundamental or some mix thereof - care more about being correct on fewer bets, which have more money behind them and which are held for longer periods of time (hours, days, weeks or months). These funds are not competing with HFT: HFT only competes with HFT. This is because HFT activity and active investing activity are completely alien to one another. HFT has a material impact on the profit margins (slippage), volume and liquidity available to active investors, but strictly speaking they don't actually compete (except in the narrow sense that you "compete" with a car salesman to buy a car for a better price).

HFT is a relatively tiny portion of the financial industry which gets outsized attention. It's generally more accurate to think of HFT firms as financial utility providers rather than investing firms.

If you're a day trader trying to flip stocks by holding them for a couple of seconds at a time HFT is why you're bankrupt.

But it's also not true that HFT folks create markets. To create a market you need to sit on shares and offer them for sale. HFT leeches off of existing markets. It's true they offer share for sale, but only ones they bought a few nanoseconds earlier for the original price.

Day traders flipping stocks every few seconds won't lose money because of HFT firms, they'll lose money because of trading fees. Trading every few seconds is a wildly unrealistic strategy for most people to pursue on their own. On an average, per-trade basis the fees associated with buying and selling are several orders of magnitude higher than the profit margins of any HFT strategy. The only way your fees will even come close to the profit margins of an HFT are if your volume is such that you've become a market maker yourself. This is a very basic and fundamental tension that precludes HFT from being a competitive force to other traders engaging in speculation and investing.

Your final paragraph strikes me as ideologically bent, particularly with your use of the word "leech." It's an uncontroversial fact that HFT firms facilitate market making. HFT firms do sit on shares and offer them for sale. Most often they do this quickly, but occasionally they have holding times with longer horizons. What's more important than the turnaround time is the low latency with which they execute orders. Definitionally, HFT is engaging in market making because when someone wants to purchase a share, an HFT is ready to sell it to them. Likewise when someone wants to sell a share, an HFT is ready to buy it from them. This is quite literally, "making a market."

In point of fact, your hypothetical day trader would not be capable of buying shares every few seconds if it weren't for HFT (inadvisable though it may be). How do you propose they'd achieve the same kind of liquidity otherwise? By calling a broker? There are far fewer market makers than there are active investors. Passive investing activity with index funds also dwarfs the scale of HFTs. The straightforward conclusion that follows is that fewer, faster parties must exist to make markets for the many, comparatively slower investors.

This is an extremely well-studied subject; when you peel back the pomp and PR about HFT as an industry, you'll encounter an incontrovertible reality. There is no way to service modern trading activity happening every second without the HFT activity that happens every microsecond. By calling that latter activity "leeching", you read more like someone delivering an opinion rather than a cogent, well-informed and substantive criticism.

Just because they're stuck holding the bag on trades that end up being cancelled sometimes and have to wait for them to unload doesn't means they're making markets. If a market doesn't already exist the HFT firm is not going to create it.

They absolutely do increase the volume figures on markets, and that can be interpreted as making markets, but it's not an accurate representation of the big picture.

This is also incorrect. HFT firms routinely hold positions for days and weeks. It would be inefficient and strange to literally deplete all positions and begin fresh anew each day. Executing orders with extremely low latency is not equivalent to be perfectly symmetric in buy and sell orders.

I encourage you to learn more about the topic you're talking about, because what you're saying is substantially at odds with how the industry actually works. At this point I'm curious how you would define market making, because it's very ironic for me (and anyone else reading) to hear you say these things and talk about what is or is not "an accurate representation of the big picture."

I'd point out, however, that your competition is usually not "HFT algos on servers located as physically close as possible," unless you are, yourself, a HFT trader. Even if you're buying a security for a few cents more because an HFT firm has corrected the price, if you're holding for weeks, months, or years... what's the difference? There's room for both of you to succeed, as long as your investment philosophies and holding periods differ that significantly.

If you want slow, predictable gains then invest in highly rated bonds. Handing investment decisions off to some Manhattan finance bro is unlikely to improve your long-term risk-adjusted returns.

That's usually not true at all. HFT makes up a huge portion of market volume, but for almost all investors is basically negligible to their return, despite what Michael Lewis might scare you into believing. HFT firms make a comparatively small profit in the universe of Wall Street, so they aren't eating your returns.

That's not to say actively managing your money is not difficult. You're mostly competing against sophisticated investors and firms with a far greater capital and knowledge base than you. It's just that in general, the majority of capital being bet against you is not from High Frequency Trading

Here's a pretty good article: https://www.aqr.com/Insights/Research/Alternative-Thinking/A...

Of course there is such a thing as price manipulation which active investors can try - if there are only a few and they work together this can work out. However the investors have incentive to cheat when working together as the cheater wins against his peers, thus this currently is confined to "penny stocks" (for example the company behind the stock doesn't exist anymore but they didn't properly delist their stock so technically it can be traded - you can buy such stocks for say a penny each and then hype them to suckers as the next big thing and sell for 10 cents each and make a killing - since the company doesn't exist no one else pays attention and the scam works.)

I didn't read past the paywall but one good thing if more and more people own index funds then they are participating in the success of those corporations represented in that index. Might tend to tone down some of the shrill agitation that everything "corporations" do is evil and greedy.