The value of high paying jobs in finance doesn't come from activities that can be automated, but from stuff that can't like connections(which university you attended, who your parents are, who you play golf with and also insider trading) and soft skills(charisma, drive, looks, personality, ability to influence people, lying to them or throwing them under the bus).
For example, people like Elon Musk and Steve Jobs could pick up the phone and convince some very wealthy people to part with their cash on some investments that ML algos would consider stupidly risky.
That largely hasn’t proven true historically. Whole swaths of high paying jobs in finance are gone or greatly diminished. From pit trader to analyst they’ve been replaced with computers.
Maybe the machines won’t come for venture/angels but I certainly won’t bet on it.
There’s a relationship between how liquid a market is and how easily it can be automated. There are very few humans on treasury desks, but M&A is still highly relationship driven. No algorithm is going to convince a CEO to sell their company or buy another.
Thats not an accurate view of what automating M & A is going to look like. The future is CRMs, more data transportability, and generally improved tooling will change the 1 deal/month capacity of your rain maker M&A team of 8 to a team of 3 doing 6 deals/month. You don’t need to replace the whole flow with computers to add automation.
Completely agreed, that liquidity spectrum still holds though. Closer to 100% of treasury trading than M&A can be automated, but tools can certainly increase the efficiency of the M&A process and reduce headcount if demand doesn’t grow in kind.
M&A is being increasingly automated too. There are simply too many companies out there to run off of relationships alone. Pre-merger due diligence is mostly automated already.
Exactly. SV likes to brag how their tools are disrupting collaboration, reducing the need for physical presence and increasing automation is making jobs redundant but big money deals are always done in person, with a handshake at a dinner in a fancy restaurant. No amount of tech will ever disrupt this ritual.
Won't the venture funds just own the machines and use them to concentrate wealth even better? Automation inherently strongly favors capital over labor, so whomever has the money when automation revolutionizes an industry is the one that makes the profits. With no value judgment intended, that's just kind of obvious right? Machines are capital and owned by someone who makes money from them instead of paying for labor.
"Automation inherently strongly favors capital over labor"
That's an odd thing to say. If machines and programs are capital, which seems reasonable, the more automation there is, the more capital, which drives down the returns and makes labor relatively more expensive. Isn't this obviously happening in a world where savings accounts pay 0.01% but it costs $120/hr to hire a plumber?
Maybe it would be better to say that the extremely low returns on capital favor automation, which favors the people who make the highest wages. That's obviously? not the same as capital vs. labor.
What I mean is that if we assume automation reduces the number of people required to complete a job, those that remain have a high pay rate. Agreed?
This continues as technology gets more sophisticated until the only people left producing things and making money are those that own the tools that do the job.
It only costs $120/hr to hire a plumber because of artificial barriers to entry and a guaranteed market. A better example might be a typist who previously had a solid middle class job taking notes and dealing with faxes because their boss didn't have a cheaper way to do it.
That is what I mean by favoring. Eventually Bezos just owns a machine that runs everything he controls and needs no employees. Eventually. Don't doubt he would if he could.
Your statement contains the core of it though -- systems that favor the people who make the highest wages are in an unstable equilibrium where eventually very very few people make all the wages.
>"The value of high paying jobs in finance doesn't come from activities that can be automated, but from stuff that can't like connections..."
relationships were expensive (and thus valuable) when people had to cultivate them through temporal and spatial investment. relationships between computer systems are not bound by those constraints and are thus fast and cheap (via markets and networks).
but more to the point, the value of a relationship in finance is really the highly valuable non-public information you can glean from them.
knowing who has (or will have) the resources you want at or below the price you want to pay for them (or vice versa) is the hard part. people (and the algorithms they create) will actively try to hide that information because it maximizes revenue.
i read somewhere when comparing a financial advisor, managing one's portfolio vs an index fund - There was no discernible higher return with the advisor managed portfolio vs the index fund.
While this has been true in the past, because of this advice, huge amounts of wealth are now in index tracking etfs or similar passive index tracking funds. This essentially means that a larger and larger amount of the market is just “betting in the market”. If you want to spot where the next financial crisis will come from a generally good idea is not to ask “what looks crazy right now” (ie VC funding) but “what does essentially nobody question that is also a major change in mainstream investing”.
Because everyone is doubling down on investing on the status quo, don’t be surprised if a disruption to the status quo has far more dramatic impact than it would have 20 years ago.
the wrong assumption is that the large bets on ETFs are "wrong" for extended periods.
If the ETFs are incorrectly pricing assets because of blind buying, then an enterprising managed index fund could bet against it. Or, find some other arbitration mechanism. But of course, this doesn't actually happen, because price discovery _still happens_ even if ETFs account for a large percentage of trades (since price discovery can happen even with small number of trades).
And if ETFs is just betting on the market average, then the worst that can happen is they get average results. ETFs don't use leverage or debt.
An index fund typically performs much better over any meaningful timeframe because it has much lower fees.
But that's not the point here. The big money in investing isn't in beating the market, but charging fees, so fund managers only need to convince people to give them their money, they don't actually have to deliver results, provided they're working within the fund's stated goals.
If the trend continues of index fund growth as a percentage of the market, then the remaining active investors will gain an outsized influence on market direction. This is because index fund managers make very few trades on their own initiative, relying on market stats (trade history) to direct their own trades. Those market stats are built on the recent trades made by active investors.
If (when!) there is a correction, there will be analysis done on the cause (pick one or more of: student loan debt, sub-prime auto loans, trade war, something else) and then new investment strategies will be devised. And maybe passive investing will be shown to leave you vulnerable to a sea-change.
That's not true at all! There are many instances of managed portfolios greatly outperforming index funds. There are also examples of them greatly under performing. The crux is picking the right one - which is similar to the point you are making but not the same.
Yes, see Renaissance Technologies Medallion Fund* which earned their employees a 66% annualized return (39% after fees) over 30 years. It's one of the most well known examples out there.
It makes sense... I mean just look at stock trading. Manual stock traders used to be a high paying profession and now it’s a job that’s almost entirely been replaced by automation.
It’s not a ‘could’ in the future but something that’s already happening in significant amounts. The trading room floors of most the big banks are a tiny fraction of what they once were despite trading volumes skyrocking.
There are still millions of stock traders, if you're meaning brokers, then yes, automated routing and matching has dropped the number of brokers dramatically. It's estimated around 10% of the volume of trades are still the result of people making decisions, which is actually an incredibly large amount because electronic trades create such an enormous volume that to even make up 10% means a lot of traders are still out there picking stocks.
I’m always skeptical of these articles, but this is accurate:
>many of whom will lose their jobs -- not necessarily because they are replaced by machines, but because they are not trained to work alongside algorithms,”
The story with technology is about adaptation, not replacement. Teams, departments, technologies have come and gone out from under me in about 15 years in the software field.
One could say I was replaced by software (or cheaper devs elsewhere) when I was laid off or when teams I was on were eliminated. But if I wanted to keep getting a paycheck, I had to adapt to something else in demand.
That’s life and these sensational articles generate clicks but it happens in almost every private industry.
Agreed, but we now know that as people age it becomes increasingly difficult for them to learn new things. This is a basic function of biology and how the brain ages.
Due to automation, change is happening increasingly quickly, more rapidly than many people can adapt to. People are being left behind, which has pretty serious social and political implications.
That's why it's quite important when you do get that high paying job to bank a large percentage of your salary into income generating assets. It's basic diversification of income streams.
It is the Low and Middle Income that gets squeezed if they are not already squeezed enough. And you will be surprised how many of them wont be able to adopt.
And I am not surprised it would be hard for anyone working in Tech to understand, because I have come to the conclusion most of the so called Middle Class Income group around the world mostly shrank to Technology Sector.
I think you need to take an even bigger step back. I mean:
>Robots in Government could wipe out just as many high paying jobs
I didn't know there were all that many high paying jobs in government. I can't think of many jobs in government where people make as much as investment bankers for instance?
Maybe he was talking about AI taking out consultants that contract with the government? (I know some of them make in the high 6 figures, and many can make 7 to 8 figures depending on what that consulting firm is doing.) Or maybe he has a different idea of what "high-paying" means? High 5 figures may be high-paying to a lot of people.
Robots in reality, we get an expensive dog from Sony called Aibo.
The whole, robots of tomorrow will be doing..... is like the flying car, everybody expects it soon and been that way for generations. People from the 1950's and 60's are still waiting for a robot to do all the housework.
Still you have to wonder how peoples trust in AI/Robots will play out, would an AI think that as humans don't trust me that I should not trust humans! Or would it go, the human reason for distrust is flawed, ego the human is wrong about not trusting me, however if the human has made that mistake, then I can not trust what the human says. It then applies the rules of Asimov and runs away never to be seen again.
It could happen, unlikely, but more likely to happen before all of the above.
And yet.. How many people do you know with a Roomba? Driver assistance in their vehicles (lane keep and adaptive cruise), the list goes on. It's incremental.
"How many people do you know with a Roomba?" None, not even another branded robot hoover. Though I know people who have a cleaner once or twice a week.
But seriously - they are laying the foundations of the platform of mobility that will be built upon to do many things - yet like most things, military comes first.
Automated bureaucracy would as all bureaucracy does, tier down and whilst we may see the 3 day working week, we would also see the 3 day bureaucratic hoops we would have to endure above and beyond the 3 day work.
I imagine many of those personnel won't be redundant until the available technology meets or exceeds their level of output. We used to have people welding cars together by hand; now we have fewer technicians that manage many welding robots.
Depending on how you count, between 10% and 40% of the US is employed by the govt either directly or by proxy.
I understand your sentiment, but the reality of just shedding a large fraction of those jobs is hard to even imagine. Unemployment in 2008-2009 peaked at around 10%.
Between income taxes, corporate taxes, sales tax etc. most people with jobs give up half the money they make to the government anyway, so that makes sense.
I think one thing needs to be stated; I do not have concrete numbers, but it seems as if algorithmic trading technology has actually levelled the playing field and created more jobs.
I am a series 3 (commodities) and quant. Take this example, at one point in time people were literally trading seats on the exchange, and a seat on the NYSE was as high as 3 million !!! While there may have been more people with those jobs than today, you had virtually no choice to be in a large institution.
Today, almost anyone can raise some money, use the simple Interactive Brokers API which sits on top of FIX protocol, and build a trading strategy. The main reason I think you don't see more competition from small new firms - even though there is a lot - is that the regulation is fierce and most funds already offer nice compensation.
Within those firms, yes there may be some decline in those jobs like a floor trader, but someone has to program and manage those algorithms.
It's also important to keep in mind that many firms are only replacing execution traders while directing traders through fundamental analysis, and if all firms ever go completely algorithmic you'll find firms trying to exploit these algos in ways unique to humans.
I would interpret "algorithms" here as referring to entire separate machine learning models trained differently (if not trained on entirely separate data sets). It would make sense to me that you would want to train separate models for identifying specific categories of abuse instead of training one Ultimate Model, because debugging focused models is going to be easier. That would line up with the massive number of parameters as well.
> Nasdaq North America Surveillance team monitors 3 equity markets, 6 options markets and one futures market with real-time surveillance and post-trade surveillance of unusual market activity. The surveillance department is monitoring the markets for Insider Trading, Fraud and Manipulation, including manipulation through trading—pump and dump—and order book manipulation—spoofing and layering, as well as handling events in the market such as clearly erroneous transactions.
> The surveillance program today is using algorithmic coding to detect unusual market behavior running over 40 different algorithms in real-time, looking for market abuse and manipulation. The patterns have sophisticated algorithms that use approximately 35,000 parameters. In addition to real-time surveillance, there are over 150 patterns covering post trade surveillance, which are used to identify a wide range of potential misconduct.
> The activity is monitored across equity and options markets, with some market-specific alerts and some alerts encompassing data from all markets.
So they have 40 different predictive models running in production (a lot!), and 150 hand-written (or single-pattern) rules. They also employ active learning with experts and this reduces their training data needs with 95%. 40 models using 35000 features, is 875 features per model on average. All this bickering is typical.
I think the reason why I laughed is that I can't understand what 40 algorithms actually means. It probably takes more than 40 algorithms to accept a request at this url https://news.ycombinator.com/item?id=21730608 and return the response that you are seeing on your screen. That bullet in the article added no value to me as the reader.
As far as imprecise journalism (let alone blatant inaccuracy goes) this is pretty inoffensive IMHO.
It’s not even that far off. Algorithms can be composed. When we publish a paper on a complete ML application, the whole thing is generally referred to as “the algorithm” because it can be documented as precise set of steps from initial raw inputs to some kind of output. Just because there are literally thousands of applications of sorting and searching, matrix multiplies and what have you buried under the process doesn’t make it unclear what is being referred to.
Almost all algorithms are composition of algorithms. Probably a good way to define a singular algorithm is the point a human has to be the source of its input. A self driving car is a composition of thousands of algorithms by definition -- a process that step by step calculates with an input and output which is every single method in all its code. However, for sake of conversation in the number of complete autonomous systems consider the car to be a single algorithm defined by the point a human has to be the originating input at the interface where the key gets turned to turn it on.
I think this definition will be important in law because it is where we will define responsibility for actions of autonomous things.
I'm guessing you are more of an expert on the technology involved than the journalist writing this article, and thus not the target audience. Maybe look for other areas of value in it, and skip that particular bullet point. I imagine mechanical engineers often find laughable lines in Popular Mechanics as well...
> Nasdaq runs algorithms to look for market abuse and manipulation in real time.
That's how you write honestly for the target audience.
> Nasdaq runs more than 40 different algorithms, using about 35,000 parameters, to look for market abuse and manipulation in real time.
That is how you use big words on an audience not capable of seeing through their meaninglessness, and it's the responsibility of people who do see through it to call it out.
I completely agree with your general assessment, but disagree with the point you're trying to make. Do you believe that NASDAQ is trying to is, in fact, ring to look for market abuse and manipulation in real time? Excepting some particular edge cases, I sure do! It is in the best interest of NASDAQ. You state "this is how you use big words on an audience not capable of seeing through their meaninglessnes". Yes, the audience probably doesn't understand the denotation or parlance to point out that it is entirely obvious that NASDAQ uses more than 40 algorithms. Your cellular phone uses more than 40 algorithms per second. The article is written for an audience that doesn't understand that an algorithm is a set of instructions designed to accomplish a goal. So, add this to your array, b-tree, dictionary, or whatever system you use to keep to retain state: algorithm="problem solving implementation to search for abuse or manipulation": parameter="critical input variable used in market analysis". Apply this heuristic generally to your workplace to quickly resolve communication issues with non programmers and avoid looking like a pedasshole (a pedantic asshole who should understand basic communication translation protocols)
Nasdaq has about 3500 listed companies, so that's only ten parameters per company. Not that many. If you're talking about running the algorithm per market participant then it's an even weirder claim.
I wouldn't say I assumed anything. I would just say that the statement about number of parameters and algorithms is incoherent, or at least so vastly underspecified as to communicate no meaning.
No one who has got a machine learning algorithm into production in an application with substantial capital at risk would have found the 40 number low. Anyone, can demo, POC, prototype... but prod... that is a big persons game.
The algorithms probably aren't that great, and more of them would likely have diminishing returns. Adding substantially more false positives could actually be a bad thing.
As someone else mentioned. The 35k parameters is skeptical. Taleb and Tversky and Kahneman have good evidence that most algorithms are better with less parameters. The more parameters, the more noise.
In my opinion, it's a large part of Thinking, Fast and Slow and The Black Swan.
Basically, as your signals approach infinity, your chance of finding correlation approaches 1. Taleb argues your chance of finding causation is likely to decrease.
Does anyone know why de Prado left AQR so quickly? It seems kind of funny: AQR hates on ML, AQR reconsiders and hires this guy, and pretty much instantly ditches him...bad look for ML.
Btw, I think some of this is correct. It still amazes me that anyone thinks the CFA is relevant (it is largely done by Chinese/Indian students who will never work in finance). But there will always be a place for fundamental investing (i.e. you should be aware that not everything in life can be measured quantitatively).
One important thing to understand about fundamental investing: most fund managers can't outperform the index the way they manage money, and so the business model is lever up on beta, hope to catch heat (close fund down and reopen until you do), take the fees, and then repeat in the next cycle after it blows up...and that is if the manager is actually trying. Skilled management doesn't really work as a business (without luck).
So it is actually very rare. I have probably come across less than 10 managers who are +EV, and the majority don't manage any public money (again, economics of the business).
But it is straightforward: are they doing actual research (the majority of fund managers don't)? Are they turning over their portfolio frequently? Do they say dumb shit (i.e. constantly use buzzwords)? It isn't magic.
My point was that a quant manager can at least point to a strategy that's been backtested. The non-quant manager, who doesn't have a method, can't point to any test of his method. I agree with you that there are talented non-quant managers, I just think it would be hard to measure and identify who they are other than by the eyeball test.
All backtesting proves is that you have discovered a technique that worked in the past. Unless you have a time machine, that isn't very useful. The competitive advantage in quant and non-quant is identical, it is only the tools that are different.
It isn't hard. As I said, the first thing you have to understand is that 99% of managers cannot outperform and are not trying to.
Every year around this time Bloomberg, Economist and the like publishes PR stunt articles to lure highly skilled developers to Wall Street. I can tell you from my experience that ML is not replacing high paying jobs in trading AT ALL!!! Not even close.
He is saying something less controversial that was put through the PR media machine: those without the skills or willingness to work alongside the machines will lose their jobs.
If you look at who is saying this, and follow the money (not disclosed here, but a short search away), you'll notice that this is a butcher grading his own meat. Without blinking he further suggests that the U.S. government hold tournaments on anonymized data to crowdsource market manipulation detection.
The bias in ML sections are all mealy-mouthed and elitist. Just because you hold a PhD in a technical field does not mean you understand economics, AI futurology, public policy, or ethics. It would be career suicide to stray from the path of the popular safe opinion. So you get language-imperialist terms like "Latinx" instead of "South Americans" to describe Brazilians, claims that automation will favor taking jobs from minorities, that the government should promote women in leadership positions at private companies, and that especially women of color have no access to study computer engineering.
Yeah, no. A pressure washer can easily shoot a stream of water fifty feet, but the upright monkey that can stand back the furthest and still hit the urinal gets the corner office.
There are several demos on the internet of robotic arms flipping burgers, many even include machine vision, for roughly $70k per device. I'm ignorant of the limitations, but it may be relatively soon that they become economically viable.
Then again, there are few kitchen jobs that solely consist of flipping burger patties.
It's cheap, especially for 24/7 joints such as many McDonalds or Burger King. Assuming a well built product, the burger flip robot will produce consistent quality at a workload a human cannot physically cope with, with none of the downsides that employees have (wages, vacations, sick days, inability to do the same task repetitive over 8h with no break)... that robot will pay itself off in two or three years, after then it is pure profit (maintenance aside).
The thing is, I (and many other lefties) would welcome a world in which robots do all the work while humans are free to do whatever they want - the problem is that in current capitalism, the burger will cost the same for the customer while the costs implode (most cost in restaurants is staff!)... meaning the extra profit goes to the owner class, not the worker class which has to fight for the few jobs that remain.
> Contrary to traditional assumptions, high-level reasoning requires very little computation, but low-level sensorimotor skills require enormous computational resources.
I think with basic systems you have to give up the notion of controlling what goes where, give it data and "trust" it to put everything in a reasonable place. That's not a [frontend] dev approach; but it's a "i want information showing in a webpage" approach.
I have the same problem with Automator on OS X. Once you understand control flow, it's pretty hard to dial back to a flat list of steps that do something useful.
The more human jobs get replaced by robots the closer we get to money-less world. Sad thing is that lower classes will suffer most. But faster we get there shorter the suffer.
My prediction is that the blockchains are going to eat all of this stuff actually.
Smart ethereum contracts based on chainlink data are going to be the new financial instruments. You are already seeing the fertilization of this with stuff like makerdao.
I know it’s annoyingly to talk about the blockchain all the time, but unfortunately I think I may have become a true believer. I don’t think human “financial advisors” are going to be able to outperform these sorts of funds.
Also: deregulation (or difficult to enforce regulation) is going to allow a LOT of innovation to happen. How long until some 17 year olds create a token which represents the consumer side of a fund? And then how long until the funds strategy is codified into a smart contract and runs on its own?
> I don’t think human “financial advisors” are going to be able to outperform these sorts of funds.
I think this is moot, because human financial advisors are not trying to beat anything, they are trying to spend your money. Read "Where Are the Customer's Yachts?" By Fred Shwed
Also Jack Bogle used to talk about the S&P 500 would be most people trading individual stocks (sorry I need to dig up the source for this).
Point being: you may not need fancy algorithms to outperform humans at the market, if your goal is only to outperform most humans. You may need something fancy to outperform all humans.
> Coinbase is currently valued at almost $10 billion
Coinbase's last round's investors valued the top-of-the-stack preference they were given at a price per share that, if multiplied across the cap table, comes to $10bn.
That valuation printed when crypto hype was near its peak. It's reasonable to conclude it may be stale. Given the preference, enterprise value would fall faster than the value of those most-recent shares.
Valuation is a poor sole measure of success. It's easily manipulated, determined by a few people and often stale.
For example, people like Elon Musk and Steve Jobs could pick up the phone and convince some very wealthy people to part with their cash on some investments that ML algos would consider stupidly risky.