Without finance, there is no biomedical research (private especially). Without finance, there is no aerospace advancement. Without finance, there is no alternative energy. Being able to facilitate a more liquid, more transparent financial sector is, in my opinion, a calling worthy of any programmer who seeks to make the world a better place.

Your assignment of people as 'questionable' is likely an artifact of the 2007 recession, and that kind of thing is a personal choice. I can guarantee you that there are 'questionable' people in every sector and in virtually every company. Sometimes people are great, and sometimes they aren't. Their field of employment has very little to do with what makes them that way, and I think that it's a little ignorant to be so casual (and I say this because you are certainly not alone in your judgment) with how you view people.

Remember how, as children, we are taught to not judge books by their covers? Why do the same with a programmer who works in finance? Sure, some of the work may be a little pedantic, like squeezing an extra millisecond out of an algorithm, but hey, that's computer science in general, and I liken that to an F1 racing team spending hour upon hour sculpting the perfect frame for their car.

Sorry for the rant, I've spent a decent amount of time working on financial algorithms, and what you said touched a nerve.

Have a great day!

I stumbled into finance a few years ago and I’ve had overall the best bosses and work env ofanyone I know. I’m also doing a more audacious program of research engineering/ computer science than I could likely do anywhere else.

I like to sometimes describe my work as some combination of “making the systems that fund aerospace at least as reliable as commercial aircraft”, as transparent as intergalactic vacuum, and easy to use to boot! Or at least that’s the idea :)

On the hft front I recently started at poking at how to get accurate time on a dev computer, which gets fiddly the moment you want really interesting accuracy on commodity hardware :)

I looked you up, and see that you are likely working exclusively in Haskell, but thought you might derive some value from Carl Cook's presentation on optimizing HFT code at the most recent cpp convention. I recognize the languages are distinctly different, but he provides some interesting theoretical points along the way that might be advantageous to you.

Here's a link: https://www.youtube.com/watch?v=NH1Tta7purM&feature=youtu.be

Also, I can sympathize with your plight to run scalable sims. I've done some work in bioinformatics, and am building a small cluster at home so that I can learn to write code that'll scale to more massively parallel systems that are de rigueur in that domain.

Cheers!

Yeah building stuff that works easily in the small and sanely in the large is a fun challenge. Also hard.

Possibly because of your original comment, or perhaps unrelatedly, I’ve been lately saying “finance done right is the lock free wait free scheduling algorithm for moving society’s resources around” —— all the other stuff folks think of as finance is really just icing and fancy wrapping on top of that core truth

>Also, I'm unable to make myself do work if I don't see it contributing to some greater vision or cause

This simply is not something I experience. I've never felt any particular satisfaction from that particular aspect of a job, it's just something that never crosses my mind and incentive me in any way.

It may not be the same as creating a product or service that people love or solves a need, but it is an interesting problem to tackle.

There's also beta, which despite being the next greek letter, isn't the same at all! Beta is a measure of the price correlation between an asset and some benchmark in terms of volatility. Beta of 1 means that the asset moves in lockstep with the benchmark. Low beta means the asset doesn't move as much as the benchmark and high beta means the asset moves a lot compared to the benchmark. Tech IPOs are often high beta compared to the S&P 500.

Anyway, finance has a lot of intellectually challenging problems that happen to overlap well with people trained to build mathematical models that attempt to describe stochastic processes. CS, Physics, Stats, some branches of pure Mathematics -- all have direct applications in the work they do to some aspect of being able to understand and predict markets.

First you have basic lending and borrowing activities. The bank can afford to pay you interest (well not now that interest rates are at zero) only because your money is invested, usually in loans.

Then you have payments, cash management for companies.

Then you have more complex services offered to larger companies, raising capital (equity and debt) on the market, helping them manage their risk with derivatives (interest rate, currency, commodities (oil, metals, livestock, etc prices), inflation, etc) plus some advisory on corporate actions (mergers, IPOs, etc).

You have insurance, reinsurance.

And then you have asset management, which you can call speculation if you want, but which is essentially people managing investment portfolios on behalf of savers, pension funds, insurance premium for long term risks, etc. With a wide variety of approaches from very conservative money market funds which invest in highly safe and liquid assets, to trackers who try to replicate a benchmark by over performing it by a little, to investments in illiquid assets (property, non listed companies) or very aggressive/leveraged strategies (hedge funds). Fundamentally what they are doing is to ensure your money gets invested wisely (at least they try).

And of course all sorts of intermediaries: consultants, brokers, market data providers, research (sort of financial journalists), prime brokers (who provide financing when you leverage liquid financial assets), system providers, auditors, rating agencies, etc.

Now you could call everything speculation: when a bank lends you money, it bets on your financial soundness. When an investor takes the opposite side of a commodity (say oil) transaction, it bets on the price of oil. But that allowed you to get money to run a business or to offset an oil price risk that could have made you lose money if oil increased (say your are an airline company). There is no real dinstinction between risk taking / risk transfer and speculation. But in most instances it does have a social benefit.

This is it for me. Futures trading is a zero sum game (negative sum if you factor in fees), so writing a program to automatically extract value from an ever changing market is a nontrivial and ongoing project. But unlike a game, getting good at it can pay the bills in a real way. I understand that in the big picture long term, I'm not making much of a mark on society, but nevertheless it satisfies my personal and financial needs.

http://www.businessinsider.com/the-stock-market-is-not-a-zer...

> In the financial markets, options and futures are examples of zero-sum games, excluding transaction costs. For every person who gains on a contract, there is a counter-party who loses.

[1] https://www.investopedia.com/terms/z/zero-sumgame.asp

It's almost always going to be about finding the optimal way to funnel money toward whoever is in control of your employer.

Be aware that a large part of the Internet industry consists of persuading people to click ads, in order to enrich the Saudi sovereign wealth fund.

https://www.nytimes.com/2017/11/06/technology/unsavory-sourc...

Does that float your boat?

Financial markets serve a hugely important social function of allocating capital (intermediating between people who have money and those who need money).

The "questionable people" include grandparents, firemen, teachers, etc with pensions and retirement funds.

>I don't see it contributing to some greater vision or cause. [...] I guess it's just a case of lack of empathy on my part.

Your sentiment is common and I think it's caused by people not connecting the dots from grandparents' retirement pension to the hedge fund trader.

As one example, at the micro level, grandparents are depending on a pension.[1] They also don't want their _real_ purchasing power to diminish. The "real" not "nominal" purchasing power is an important distinction because if the price of bread is $2 in 2017 but becomes $4 in 2037, they want to have adequate future money to buy food. In other words, they want their money to grow and keep up with inflation.

Your grandparents stashing money in a savings account paying less than 1% will not protect their purchasing power from inflation. Likewise, the pension manager that oversees a $1 billion retirement fund to pay retired teachers will declare it bankrupt if it only gets a 1% return by buying Treasury Bills. The pension's assumptions to avoid insolvency might require 8% or better annual returns.

Therefore, all these participants in the economy are looking for better returns. You can't get 8% from savings accounts and t-bills. That's where money managers like private equity, hedge funds, and VCs in Silicon Valley come in. They sell their ability to generate higher returns. This expertise attracts capital from entities like pension funds.

So basically, a bunch of programmers are fine tuning algorithms because [...connect a bunch of dots...] your grandmother doesn't want to be a starving destitute and live under a bridge in her retirement years. If you multiply millions of everyday people not wanting to be destitute (macro level), you end up with hedge fund managers with billions to play with.

That said, can we "reconfigure society" so that all this competitive activity of shuffling money from one pile to another isn't done? Maybe. It would involve fundamental changes that we all can't agree on. For example, if you stop inflation you can stop the incentive to generate returns that try to beat inflation; but then some would say that "deflation" would lead to people hoarding money and the economy would crash. Inflation is the current winning ideology for the modern economy. It's hard to see how we could alter society such that Wall Street traders go away.

[1] for example, the $300 billion California Public Employees pension fund provides these benefits: https://en.wikipedia.org/wiki/CalPERS#Benefits

Your proposals boil down to ending deficit spending. Aside from the authority issue (no single entity has authority to do this, not even congress, definitely not the president), that means the US Federal government has to cut $650 billion dollars in spending.

Or to put it more directly: We currently have pensions, social security, an army (meaning having an army at all). Pick 2. The other one gets destroyed.

Still sounds reasonable ?

1. Time - I quote buy and sell prices continuously in Microsoft stock trying to earn the spread between them. You want to buy stock and my offer to sell is the cheapest in the market. We trade. Later, someone else comes along to sell and I have the highest bid, so I buy the stock back from them.

In between I'm exposed to price fluctuations, so sometimes I make money and sometimes I lose, but I make a profit on average by charging a spread between my buy/sell prices and predicting small price movements. The two of you could have met had you waited, but I let you lock in a sure thing by bearing risks you didn't want. Think of this like insurance. Odds are you spend more on car insurance than you're expected to receive in payments, otherwise the insurers would go out of business. But you probably prefer spending $1000 a year vs. spending $0 most years and $30000 once in your life.

2. Place - You're in the US and want to buy Nokia stock. Someone in Finland wants to sell for a cheaper price. You don't have access to the European exchanges and she doesn't have access to US exchanges, but I do. I trade against both your orders and earn a couple cents per share in profit. I helped you meet halfway across the world and also keep prices efficient to reflect global supply and demand.

3. Product - You want to invest in an index ETF that holds 40 different stocks. I quote an offer to sell based on where they're trading in the market plus a small profit margin. You buy my offer. I turn around and buy the stocks to hedge my risk.

I'm better at trading than you are and have very low costs. Even considering my profit margin, it's cheaper and less risky for you to buy the ETF from me than 40 individual stocks. I keep the ETF and stock prices aligned and also helped people who wanted to sell stocks get their orders filled. Win win win.

I think the work I do clearly benefits the market and natural investors. All of these trades are competitive too, so I can only earn razor thin margins per trade, or someone else will undercut me.

I do sometimes question how the market picks winners and losers, typically based on speed. Is someone whose system is hundreds of nanoseconds faster more deserving of arbitrage profits? Do high technology costs naturally lead to consolidation and reduced competition? https://faculty.chicagobooth.edu/eric.budish/research/HFT-Fr... this paper has some interesting thoughts on the topic.

FWIW I do think continuous price-time priority trading is flawed in that it creates an arms race in speed, but it's the least bad option. Other mechanisms like batch auctions are even worse. As a thought experiment, consider the #3 ETF trade I described. If I can't immediately hedge my risk and instead go into a batch auction for each stock, how will that affect the price I offer in the ETF?

PS: I do think it's a fun job. You get to play with cool technology and it's like a game that gets harder every day against very skilled opponents. Even the arms race isn't all bad. HFT firms have effectively bankrolled modern innovations in high speed ethernet, FPGAs, low latency kernel bypass networking, etc. that have knock-on benefits for other applications.

My idea for this is that when you rest an order, you should be able to attach a set of hedge orders to it that will initially be inactive, but will fire when the owning order is filled. This would let everyone get hedges done at better-than-HFT speed, and make things like speed bumps or batches a lot more tolerable for market makers.

You'd want some kind of pro-rata, or multiple orders at fill thresholds, so you could show big size and respond sensibly to partial fills. You might want alternative hedges to cope with price moves in the hedge target. You might want some mechanism whereby the exchange will pull the owning quote if the hedge disappears. There's all sorts of fun you could have.

For this to work really well, you'd want the hedges to be processed synchronously with the fill of the owning order, so there's no HFTable window of vulnerability between fill and hedge. That couldn't be done across different matching engines in the typical architecture current exchanges use. For futures, though, it might be workable - i think all the different expiries of a given underlying, and their spreads, are already handled synchronously, so that the exchange can do implied matching. That might not include some of the more exotic constructs which aren't in the implied chain, though.

You're right that there's precedent for this:

-Implied engines as you mentioned where traders can atomically express a view on the term structure without legging risk. For the benefit of non-finance readers: Say you believe the fair price difference between Dec and March futures is 2 index points, you could quote in the spread between the contracts (long one, short the other) buying 1.75 selling 2.25 without being fastest to respond to every shift in the index itself. Your order in the spread will create implied orders in the underlying contracts themselves and you can only be filled if both your long and short trade.

-IEX's D-Peg effectively runs a high frequency pricing model inside the matching engine to predict when an adverse price tick is likely.

-I think D.E. Shaw has a patent on a domain-specific language for complex orders where they can be priced off any other instrument(s), ratios between instruments, features of the order book, etc.

The real problem with batch auctions is coordination. So long as you have other exchanges trading correlated assets in continuous time, there will still be a race to adjust orders in the batch auction market after a pricing signal occurs. Even with a random delay or crossing time, it's still beneficial in statistical expectation to be faster.

You'd need to trade most similar products worldwide on a single platform. Since exchanges are a network effect business, this would likely lead to monopoly profits for the exchange operator.

I'm not sure "no high-speed arbitrage/monopoly exchange" is net better than "some high speed arbitrage/competing exchanges." I suppose you could have the government run the exchange as a utility, but that would stifle innovation. A lot of things we now take for granted started in experimental ATS and ECN startup markets.

There a lot of challenging problems in the finance. Some people like the problem solving and $$

> I just can't understand why anyone would want to put so much effort and skill into making bots for farming money for a bunch of questionable people

Its making money for yourself as well. Many jobs in Wall Street at % of profits based rather than a fixed salary.

It's actually a recommended route in the effective altruism community for those with the aptitude.