Introduction to Algotrading (2011) [pdf] | Hacker News Reader

Introduction to Algotrading (2011) [pdf]

isomorphisms.sdf.org

393 pointssteventhedev130 comments

Thanks for posting. I wrote this back in college and looking back a lot of it is outdated (Getco doesn't exist anymore), and some parts even cringeworthy...

After writing that I joined Headlands, where I've been working for the past 8 years as we grew from a startup to a relatively large firm (trading is actually a very small industry). If you want a more up to date description of low-latency trading in particular, please see this more recent post: https://blog.headlandstech.com/2017/08/03/quantitative-tradi...

Feel free to email me, my email address in the pdf is still valid.

I remember reading that intro you wrote sometime after you wrote it.. . I was an algotrader at a big bank at the time.

My biggest takeaway was just confirmation that really "many people are doing the same things" or looks at the market in very similar ways.

One good way to get an edge is to think how would someone in a similar position approach the problem. Now iterate again and think of it differently.

Its the game theory of trading.

Game theory itself has been applied to trading ad Infinitum . Indeed short squeezes, liquidity traps, algos that hunt algos, are all a common feature and motivated by a study of the competition.

I’d venture to say that trading might just be the most competitive field in existence, with all the smart and cunning that goes with it.

Also algo-trader since Jan2018. I got IT and FS background, I like to think algo-trading as a computer game, where the 'monsters' have their own AI and different levels of aggression.

I am also closely following the MQL5 market studying who the retail thinks. I see how the waves of people gather one strategy/signal/EA that seems to be having a good streak, and then jump to the next one once the previous one 'just' failed. There is great ignorance in the space, it feel like 80% of the people I talk to in retail haven't even read a book and/or never heard of Kathy Lien or monitor the news on BB or Reuters.

I will definitely absorb those 57 pages and will stary monitoring HT blog.

>some parts even cringeworthy...

Indeed, "work shall set you free" is not a great reference to include.

I just looked that phrase up. That’s some supremely bad taste from the author.

To be fair, you had to look it up. It's possible if you didn't know it was in bad taste you'd have repeated it in the future as well.

Yea but when I did, I couldn’t really find any reference to it devoid of Nazi association.

Since you needed to look it up, how about you give the author the benefit of the doubt rather than act so offended?

Now, if the quote had been "Arbeit macht frei" I would understand the outrage, but to me the English translation sounds more like a reflection of the Calvinist/Protestant work ethic from the time when the saying was minted, at least 100 years before the Nazis rose to power. To some people the quote in itself holds appeal, and why shouldn't it? If read without context it is actually quite inspiring.

I didn't need to look it up to know what it meant. Although I can easily see the author didn't mean to insult, it's offensive.

Anyone with a basic modern education should know exactly what that phrase references. It's like pretending "that's all, folks!" doesn't immediately evoke Looney Tunes

I'm Jewish and I didn't know what that phrase references, and I've also been to many Holocaust museums.

> It's like pretending "that's all, folks!" doesn't immediately evoke Looney Tunes

It doesn't for people who haven't grown up with Looney Tunes. You're on a fairly international site here. Similarly, you can't expect everyone's education about the Holocaust to have covered this specific thing in a way they'd reliably remember and avoid an embarrassing mistake like that.

Correct, you get to decide what the author meant and what his motivations were. It's not that he pulled an ambiguous, five word, phrase out of the back of his head. One which we've all heard and are all only vaguely familiar with. No, the author is actually a closeted racist. He chooses to push his racist agenda, not through directly communicating his ideas, but by using banal, dated, phrases which have a racist context which most people are not familiar with. It's really not possible to use these five, every-day, words in that sequence without summoning all of the racist intensity and hatred which some people attached to this sequence of words some time ago when most of us weren't alive.

You do not understand the distinction between insulting someone and offending someone.

Thank you for your post write up. I'm going to go through that first as my main intro to quant trading and programming for trading, and then will check out your later blog posts. But so far a few pages in and I'm hooked, this should be a great and fun exercise to get into a new field of programming.

Hey, awesome that you're posting here. I actually just asked a question as a post on the algotrading subreddit, but I'm hoping you can maybe help here. I think this would be broadly helpful, so I'm asking here instead of emailing.

Is there any authoritative literature on the execution aspect of algotrading? I'm talking specifically about the things that are hard, if not possible, to analyze via backtesting. As I've moved to shorter and shorter timeframes, I feel like there's a huge execution gap that I'm suffering from. I've got good strategies that work but can definitely be improved with better execution. For example:

    How to optimize limit pricing to get better fill rates or better entries or exits

    Market microstructure analysis and modeling. How to analyze book depth and order flow to predict short term (shorter than 5-10s) behavior

    Applications for and optimizations of various order types like IOC, MIT, LIT, FOK, Iceberg, Basket, etc.

    How to handle the million ways that algotrading can do unexpected things: market gaps, trading halts, network outages or latency spikes, news events and volume explosions, etc.

    Metastrategies: creating strategies that "trade" strategies to turn them off or swap them out based on market conditions.

    Statistical process control, or other ways to detect when an algorithm is no longer working as expected.

    Use of metainformation to dynamically inform strategies, such as implied volatility or any of the option greeks.

Do you know of any resources to point me to?

ITG Research & Microstructure research from brokers electronic desks.

Regarding execution optimization, you can just see how Rentech (Renaissance Technologies) does it:

https://patents.google.com/patent/US20160035027

BTW- not sure why they wanted to patent this, since that just makes it all public information.

I maintain my belief that this patent is either defensive or a prank played by their research team on their CEOs.

Thank you for creating this resource; it is incredibly interesting. Regarding the brain teasers (and, by extension, the higher-level math involved), do you have any resources you would recommend for (re)learning the mechanics involved? I took college-level mathematics through Differential Equations and Statistics for Engineering, but I would like to refresh and enhance my skill.

Prob, stats, and linear algebra

This is full of gems ; and a great compendium I wish I had 10 years ago.

However, honest advise: if you actually really want to trade in this way, start with a solid course in mathematical finance. It’ll teach you how to model the space, think about arbitrage, portfolios, risk, factors and the sheer important of theory and hypothesis in real strategy.

Building a backtester is exceptionally hard to do if you include scenario analysis, slippage, event risk, lag, multi hypothesis testing, data snooping, etc. Testing is a huge source of competitive advantage.

In summary, if you’re starting out, go learn the basics like the back of your hand, and he basics are mathematical finance.

Do you have recommendations on good Mathematical Finance books? I have a background on maths and engineering. Thank you.

As a textbook, I suggest Investment Science by Leunberger. It's concise and has great coverage, and you can just keep referencing it over time.

As an initial focus, I suggest understanding discrete time models well. If you've got some LA and calculus then I'd suggest just hoping into Introduction to Mathematical Finance by Pliska. It's concise, zero filler, but really clear.

I work in HFT, and I'll be contrary and suggest that if you want the most value from your time, focus on learning as much maths and probability as possible. Once you have a strong maths background, learning most financial maths is relatively easy. Any of the financial models you find in textbooks are probably useless because if they work, everybody else who's read the textbook has also used them so there's unlikely to be any edge left, hence it's the ideas underneath them that matter, and understanding and extending these is much easier with a strong maths background. The key skill is developing and assessing new models, and the better one's maths, the easier this is (imagine if a market behaved like a complex aerodynamic system; only somebody with enough mathematical background to model a complex aerodynamic system would be able to create a very accurate model of that market).

One of the most successful hedge funds, Renaissance Technologies, also takes this approach. Regarding its employees: "a third have PhDs, not in finance, but in fields like physics, mathematics and statistics. Renaissance has been called “the best physics and mathematics department in the world” and, according to Weatherall, “avoids hiring anyone with even the slightest whiff of Wall Street bona fides." (https://en.wikipedia.org/wiki/Renaissance_Technologies). Similarly some of the best HFT hires I've been have been people with a maths or science background.

A quote sourced to Renaissance Technologies from https://www.quora.com/What-is-the-secret-behind-Renaissance-...: "We have some Tier One mathematicians and a lot of Tier Two mathematicians. Other quants funds have mostly Tier Three mathematicians, and worse, they don’t even know that these tiers exist! Someone has to create the correlations in the markets." Aim to be one of those Tier one mathematicians!

How would you even become a 'Tier One' mathematician? What does that entail?

I suppose work hard on learning, deeply understanding and practicing as much math as possible, and trying to make novel discoveries / models. I'm certainly not a Tier One mathematician, so I'm probably not the best person to ask. Maybe it requires people to be born with some innate ability, although I'd certainly like to hope it doesn't. Regardless of whether it's actually achievable or not, however, it's definitely a worthy goal to strive towards if you want to be a good quant/trader (and a very intellectually stimulating one).

A nice post on the topic by someone who most certainly is a Tier One mathematician: https://terrytao.wordpress.com/career-advice/does-one-have-t....

Remember reading an article by an ex rentech guy who says that linear regression is about as complex as the modelling gets; that the edge is not in making things hard.

Deep neural nets approximate polynomial regression. That's probably the limit now.

The finance industry loves deep neural networks because it provides them a way to extract money from all of the naive tech engineers who took a deep learning class on Coursera and thought they could apply neural networks to trading. The general consensus is that deep learning for general trading purposes is complete bullshit because it unavoidably overfits.

There are some quality uses for NN, but price or volatility prediction is not one of them.

I'd be surprised if that's the case, as surely somebody else would have replicated it by now. https://www.quora.com/What-are-the-investment-strategies-of-... describes something that sounds much more complex than what could be achieved by linear regression alone.

Tbh I imagine their edge is more likely to be better data and a secret theory.

Then the applied part of it is simple but different: different analysis, different theory, different data, extraordinary results.

Edit: just read your article ... kind of poo poos my theory if what he says is true ; at least at the level of the mega model.

It doesnt refute your theory, it's just that your theory applies very specifically to HFT. Alpha generation in HFT is incredibly simple, and HFT analysis typically relies on very basic math: linear regression and recursive filters. HFT relies on execution far more than modeling to generate returns.

But HFT is inherently low capacity. You can't put a billion dollars on an order book and expect to make the same returns as you would with a thousand. That's the reason HFT firms are almost always proprietary trading firms...they don't need or want more capital.

A hedge fund, like Rentech, is typically on much higher timeframes because their size necessitates higher capacity strategies. This could mean holding periods of minutes for the million dollar funds to days for the billion dollar funds. As you get to higher and higher timeframes, your mathematics are going to need to be dramatically more sophisticated in order to beat the market. The math you are looking at in that quora link is about pairs trading which is where pairs of instruments mean revert over time. I would expect Rentech to be doing this type of trading over long timeframes and holding a trade for days to weeks.

I don't think you will find a generally-accepted definition of "Tier One." For this context, I would say you need to be capable of creating mathematical techniques that are significantly more powerful than what is considered state-of-the-art. That entails a high level of talent and a great deal of education in the mathematics associated with the particular problem you are solving.

I think at least caveating your advise with “learn as much applied maths as possible” may be an improvement .

From there, “learn as much relevant applied maths as possible” would be more time efficient.

And from there, “learn financial maths first” proceeds as an obvious corollary.

So I kinda feel we agree :)

I suppose it comes down to how the "financial maths" is taught. If it thoroughly covers the foundations, giving somebody the tools they need to build new models upon them, I agree. But if it's like "engineering maths", just teaching formulas and models without building a deep understanding of the foundations, then to me that's counterproductive, giving people tools they can't really use but not the ability to build better tools. Maybe I'm just biased however from seeing the results of assigning moderately maths-heavy problems to potentially unrepresentative samples of interns with finance degrees.

Do you have a recommendation for a good probability course/textbook for that matter?

https://www.amazon.com/Probability-Theory-Science-T-Jaynes/d... is pretty decent.

Thanks!

I've always wondered what an algorithmic trader actually does. Does an algorithmic trader get to write algorithms like the job title suggests?

I'd say it depends on the firm. They could write and even implement algorithms, but they might also just spend most of their time monitoring and fine-tuning existing algorithms.

That sounds interesting. Its exactly the kind problem solving I enjoy. You also gave some good advice on what prerequsites are needed to work as a HFT trader. Is HFT the same as algorithmic trading? If not what are the differences?

Algorithmic trading can just mean trying to place orders with minimal slippage, e.g. if you were a bank that only places one order per day, that's definitely not HFT, but you might still want to get the best price for that order, which you won't if you just place it all in one go.

> I work in HFT, and I'll be contrary and suggest that if you want the most value from your time, focus on learning as much maths and probability as possible.

Why not go even further and just not bother with algotrading, dumping money into index funds like VTI instead? It certainly gives a lot more value for your time.

I'm not sure what you mean here by algotrading, but HFT at least can achieve much better returns than index funds, like 2-5x more at least.

I was just going by the post title; thanks for the correction. Also, I was mostly responding to the value/time efficiency part of your comment. By putting in nearly zero effort, you would get the gains of the total market.

While we're here, though, I'm genuinely curious: how accessible is HFT to the average retail investor, let alone those who aren't investing at all? Lastly, can HFT consistently beat a total stock market index like SPTMI year-over-year?

HFT is not accessible to an average retail investor or even one who is far above average. It requires extensive technological expertise, expensive equipment, and a professional infrastructure. Yes, good automated trading systems can consistently beat any index. With that said, there are bad systems that actually look good for a long period of time until an unusual situation occurs and wipes out all past profits or even brings down an entire firm. Risk management is a key component.

Has there been any serious push to get HFT accessible to the average retail investor in the same way that Vanguard did with the S&P 500?

As for beating indexes, it goes back to the age-old question of being able to identify the great automated trading systems.

Not that I am aware of. HFT firms generally don't need outside capital, so they have no motivation to deal with the headaches of running a fund. Funds exist for capital-intensive strategies where receiving fees provides a higher risk-adjusted return than the actual strategy.

I figured HFT firms exist in their own bubble away from most people. So from the perspective of the average joe, they only indirectly exist as a tiny market force that does price discovery.

As for the funds you mentioned, are there any that have a minimum investment of less than $1,000?

The impact of HFT firms is significant, although not obvious to the average person.

Investing directly in a hedge fund requires a much higher initial investment as well as a high net worth. I would not recommend it. It is very difficult to select a portfolio of hedge funds that can outperform a simple low-cost index fund tracking the S&P 500.

Or a more tongue-in-cheek answer: if you're starting out, just pour money into ITOT, SCHB, or VTI and call it a day =P

In all seriousness, though, it's crazy that we're at a point where these hyperactive algorithms can perform price discovery fairly decently.

https://meanderful.blogspot.com/2018/01/the-accidental-hft-f...

My all time favorite article related to this topic, I always try to promote it Guys like the author started this whole rodeo through some really ingenious hardware/software/networking jerryrigging.

Kind of unrelated but I would like to have HN opinion: what is the actual point in working as a trader ? At the end of the day you just made more money or lost money. But you’re not contributing in anything for the world. I really don’t understand why people are happy to say they are trader when at the end it seems that the only do it for money. In 40 years when I look back I don’t personally want to see that I worked for 20 years for a company just to make money, rather I would like to say for example: I built this product which helped some people

Good question. I am asking this myself as I will finish University soon. Maybe someone has an opinion on it

I'm not saying that everyone trading has idealistic, altruistic reasons for doing so, but products are built on the back of capital markets.

At least ideally, successful trading makes markets more efficient, and making markets more efficient ripples out making it easier for many people to make many products that create value for their users.

Even if all of that is true though, it's an unnatural framing for people to work with and is much harder to find existential meaning in abstract changes to the system that make people's lives better in complex and hard to trace ways than it is to find meaning in a direct value proposition to your users.

Because being rich is awesome. And if you're rich, you can do whatever you want, including helping overs if that's your prerogative.

Because you like to know something that other people don't know and won't publish.

Or you don't worry about contributing to the world.

Or you enjoy intellectual entertainment above what you might add to society.

For some reason this type of comment comes up quite often on algo trading threads. Keep in mind most people are not deciding on their careers based on what they contribute to the world.

TLDR, for people like you and me, who want to do useful things that help people, money can be very useful for that.

For an extreme version, look up Effective Altrusim. Some argue that the best way you can help others in this world, is to earn as much as possible (for example at a trading desk) and then give all your surplus earnings to the most charitable causes.

Personally, I would like to earn money so that I can work on projects that I think are worth doing and have a chance of using my particular skills and drives to good effect.

For example, I feel ready to take open hardware to a new level by building a small-scale chip (IC) factory capable of manufacturing open source designs (or any designs) at a realistic cost for customisation. And that's just one step in a series of related things, each of which will contribute something to the world if they are successful. That's just an example: Other people have their own grand projects they think are worthwhile.

But that venture is both challenging and expensive, and I'm still paying off debts from the last failed bootstrap. So my next big project is paused, until I have earned enough to be able to bootstrap all over again.

Something like HFT would be rather useful at providing the money, and its on my list of things to explore in the short term, because it's also a good match for my experience in several areas. But it's not something I'd want to do for a long time for its own sake. It would be for the money, so that I can do other useful things down the line.

I sleep better working in HFT than working in regular tech

HFT, at worst, adds exactly zero value to society. I don't buy into the memes of "we provide liquidity!" or "we make markets efficient!" being some noble mission. But HFT certainly does no harm

Regular tech ultimately just pushes ads or products on people, which I consider poisonous to society

> But you’re not contributing in anything for the world.

The role of capital markets is (among others) to integrate all available information from the real world into a price. A trader does this (for example) by studying company financials, their corporate strategy, expectations for the performance of their products and then buying or selling stock if the current price does not reflect this information. In a capitalist economy capital market serve as the major signal on the performance (another word for efficiency) of a company.

If your work defines who you are, then sure. I don't trade as a purpose for my life...I trade to make money, full stop.

View the markets as a mechanism for prioritizing resources. Better resource prioritization increases prosperity, which benefits a great number of people. Traders who reliably improve pricing increase world prosperity. Traders who are unable to improve pricing tend to not last long. Traders who worsen pricing by trickery are economic parasites who run considerable risk of being punished by an exchange or regulatory agency.

Algo trader here. This document is fantastic.

I think it's hard to show, but coding is of paramount importance. General skills like keeping things in version control, writing simple code, and making things modular. But also quite advanced specific topics like CRTP or cache optimization. Could be a whole book in itself without the financial parts.

Yeah it's always worth remembering how heavily fundamental engineering work processes come into play. Knight Capital went down for a manual deployment error, of all things

Watch out, if you get too good at technology they’ll stick you in the back office and never let you trade, or even see what the pnl is.

Back office's job is trade reconciliation, pnl, and accounting.

This document is amazing. I'm interested in working in quant trading when I graduate, and I can't believe I haven't found this before. Do you think if I wrote my own back tester and test strategy and put it onto GitHub it would be a positive in applications/interviews, or is that too basic to be seen as interesting/substantial?

It depends what kind of job you want. If you want to go into quant research you’re probably just going to be writing inefficient python code. So if you want to do that, just go straight to something like Quantopian where the infrastructure is already set up.

If you want to be the guy who actually knows how to program (the guy translating the Python into C++), then I think a backtester would be a nice project. Look at the python libraries zipline and pyfolio and then try to create something yourself (ideally not in python).

Minor point but the following isn't a Korean proverb:

>When you laugh, the world laughs with you. When you cry, you cry alone.

It was in the film Oldboy but it's actually from an English-language poem. Ella Wheeler's Solitude:

  Laugh, and the world laughs with you;
  Weep, and you weep alone.
  For the sad old earth must borrow its mirth
  But has trouble enough of its own

Here's my question I've always wanted ask re:trading. How do EMH & profitable trading (especially by institutions) coexist? I already understand EMH & 'active management'. Do trading profits consistently exceed a market benchmark like the S&P 500's return for the year? (I'm assuming so). If so- doesn't this completely disprove EMH? And if so, further- why does any institution do active management when they could just actively trade assets for profit instead? Where is really the line between active management and trading?

Sorry for the dumb questions- I have genuinely spent significant time Googling and could never find the answers, thought I'd use the opportunity to ask here. I am obviously not a market professional, just one guy with most of his assets in index funds. I can rephrase questions if necessary

The efficient-market hypothesis is a useful theoretical ideal that is never perfectly achieved in reality. Good trading drives the markets towards efficiency with the work of the traders being compensated by capturing a portion of the inefficiency that they remove. Traders who are above average at removing inefficiency will be compensated above average and the less capable traders will tend to drop out due to losses. This competition allows markets to converge towards efficiency over time. If the markets were perfectly efficient, there would be no trading firms as they would have no way to get compensation for providing their service. If there were no trading firms, then the markets would be less efficient because of the erratic appearance of buyers and sellers as well as the loss of many trading experts evaluating risk and value. So, you see, there is a nice negative feedback loop at the heart of the markets that drives them closer to efficiency over time as traders are motivated to improve and advancements in technology and theory allow better determination of pricing.

Investing in equity markets is very different from trading. Corporations pay you dividends and capital gains in return for the capital you provide them. That's why long term investments in passive index funds are typically the best bet for someone who is not a finance professional. Rather than trying to beat the professionals at pricing assets, you get to sit back and enjoy the compensation from a mix of companies working hard to increase the value of their stock and/or generate dividends. You just need to ensure you don't take on excessive risk. Stay away from margin at all costs.

Thanks for the response man. Not sure if you'll see this, but- is it actually proven that some % of traders are consistently, year over year profitable? Because that would seem to be another blow to the strong form of EMH. I could see a more EMH-friendly world where some traders are profitable some of the time, but it's not consistent who's who year over year (which is what is supposedly said about active fund managers, one good year has little correlation to having another good year)

I am unaware of much publicly available proof, but in my experience, there is definitely a class of trader that is consistently profitable every year or even much better than that. These are the traders who are meticulous about their risk management and have found a way to improve a market. For a public glimpse, you could have a look at Virtu Financial's S-1 filing from 2014 (https://www.sec.gov/Archives/edgar/data/1592386/000104746914...). In it, they disclose that over the previous five year period, they were profitable on every day except one.

I enjoyed this, but I'm sure the strategies have evolved a lot. Anyone know of more recent resources like this?

This excellent cheatsheet describes strategies building blocks, not actual strategies. Everything there is still quite relevant.

There’s a lot of finance papers on arxiv that describe various (academic) strategies. Most of them are pretty bad but they’re good for providing ideas for coming up with your own.

https://www.academic-quant-news.com

Nobody is going to give you the keys to the castle. Trading strategies that are more recent probably still have some edge in them. These things are closely guarded secrets.

Having said that, in addition to the other recommendations in this thread, lookup SEC settlements. The Athena trading one from a few years back regarding their "Gravy" on close strategy is particularly memorable.

Edit: https://www.sec.gov/news/press-release/2014-229 SEC order at the bottom of the press release

A successful trader won't give out the thing that if replicated by 10.000 people will attract attention that may deem this inoperable any more.

Part of my 'hobby' is to find EAs and put them to test in other pairs, timeframes, and try to optimise and get better configurations for the triad EA-pair-timeframe. When I get one, it goes straight to MY 'parking lot', NOT a blog :)

What is EA?

"Expert Advistor", aka the softrware that stays online (running on a virtual-private-server preferrably) and has the capability to trade 24/5.

I'll add since it isn't clear that this term is usually exclusive to Forex.

I'll also add that the commercial market for them is about as scummy as it comes. You want to know how to game backtest metrics? Have a look at what they're doing with EAs.

The issues with any algo trading strategy, is if it becomes known, then the trade gets crowded out and priced away. The logic really holds that if you know a shortcut to work that saves 60 minutes on a 2 hr commute, and then everyone else found about it and did it too, goodbye secret commute. This is why the strategy aspect of this gets kept very quiet by R-tech, JS, etc.

This is pretty good. I had a quick glance and surprisingly did find some parts that are generally missing from 'Intro to Algo Trading' literature especially in the execution part. Anyway, I work at this company where we are a product that makes production systems for trading algorithms into a PaaS. We also have a library LibKLoudTrader(http://docs.kloudtrader.com) that incorporates pretty much every thing one needs to start with algorithmic trading.

Looks great. It's always interesting to learn more about how these things actually work.

This XKCD is always a good reminder not to get to enthusiastic: https://xkcd.com/1570/

Painfully true.

Numbers is just one aspect, then we got 'the News' (and the specualator, the other EAs that are programmed to kill 'your' EAs), and then we got politicians twitting (I won't name names), and semi-democratic governments that have other agendas unrelated to financial prosperity, and so on, and so forth.

Basically unless one is running algos with a target of more than 20% per annum, they should be losing sleep.

> In many countries such as Russia, corruption is a better career path.

ROFL

This is generally pretty great.

I was taken aback by the 'work shall set you free' quote at the top of Section 2. That seemed in very poor taste.

Or ignorant. ("Arbeit macht frei" was posted at the entrances of many Nazi concentration camps.)

Depends on your interpretation. I found it inspiring

It’s a quote from the entrance of Auschwitz, and lots of other Nazi concentration camps.

They did not invent the saying, i just did a little googling and it seems they didn't even use it on all the camps. It was preserved from previous wars and it became a horrible sign in the context of the holocaust.

I could see how that specific saying is tainted in German but I do not think it should be tainted for all of history and in all translations because of some unfortunate coincidence. I believe it has wisdom to share

I mean they didn’t invent it, but the author of the book that did was a member of the German Nationalist movement that eventually (~75 years later) evolved into the Nazi party.

It’s not like it was completely disassociated with Nazi history and then they decided to put it on their camps because it sounded cool. They literally wanted to work people to death, i.e. freedom.

I used to work for one of the HFT companies listed there out of university.

What are the salaries now-days for a BackOffice Technology (C++ and everything else related)?

I cringe when I see so much goes to building the IT infrastructure and so little to modeling and validating algorithms. Someone points out that most quant funds fail. Key reason is their ML models are badly overfitted. By fitting enough models, you'd always find one with a good looking Sharpe Ratio. This is the classic multiple hypothesis testing problem.

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3274354

It is not about algotrading. It is about quantitative automated trading

The distinction between algo trading vs automated trading is referred to as New York vs Chicago "style" algo trading in the document:

> The phrase “algorithmic trading” has a different meaning depending on if you are in Chicago or New York. In Chicago it will mean using a computer to place trades to try to make a profit. In New York it means to use a computer to work client orders to try to minimize impact.

Algorithmic trading means the same in chicago new york tokyo hongkong or london. Even in singapore and sydney. Same thing: slicing the order to minimize the impact

I'm amazed people are downvoting this. It's 100% correct. Algotraders work the book to get a good price; they're modern block shoppers. Quantitative traders look for profit. There are entire books on algorithmic trading as "work the book."

Max was a fresh out of college graduate at the time, so his usage was a bit off. Knowing this distinction is a sign of actual experience on the street.

I think both of you are correct. That’s also what Wikipedia says.

However, loads of people (and, according to Max’ document, Chicago) refer to automated trading as “algo trading”. Cue the descriptivist vs prescriptivist linguist debate.

Please allow me for this topic to quote Investopedia [1], and not Wikipedia (which I respect and appreciate).

"..is a type of trading done with the use of mathematical formulas run by powerful computers.."

"..makes use of much more complex formulas, combined with mathematical models and human oversight, to make decisions to buy or sell financial securities on an exchange.."

[1]: https://www.investopedia.com/terms/a/algorithmictrading.asp

Ps: In simple english, I go to work, that thing makes profit (if done right). I go to sleep, that thing makes profit. Someone important twits, that thing either makes profit, OR goes bust.

Investopedia makes the same mistake. When you sleep and computer makes you some money, it's automated trading, systematic trading, mechanical trading. This kind of trading was in the industry for centuries: you incent some rules and follow them. Algoritmic trading is about executing large orders by slicing them and filling them thru a trading day. These are the terms seriours industry professionals are using. If you apply for algo trading job in a bank you will not be writing a cash making bot.

Yeah, my impression is that professionals distinguish algorithmic trading (minimising price impact of a given order) and automated trading (rule based trading to generate profit), while amateurs (including Investopedia) tend to use the term "algorithmic trading" either for both, or for the latter while being completely unfamiliar with the former.

Correct. And it is not just Investopedia who is misleading. Type Algorithmic trading in amazon search and you will see tens of books on this topic that have nothing to do with algorithmic trading. No wonder some people are claiming they are doing algorithmictrading and even high frequency trading using their laptops from their bedrooms.

I’ve worked in the industry in NY for 10 years and only an investment banker might make that distinction. Everyone else uses the terms synonymously.

I've been working in the industry in Tokyo for 15 years. And why should we care how everyone else uses the terms if we are professionals? Even the document's title says Automated trading.

How would someone who enjoys algorithms but without a trading background get into the Tokyo HFT/Algorithmic trading industry? For example, do I need to be fluent in Japanese?

It depends on your employer. International banks are present in all major cities. So English is mainstream. And you do not need to be in Tokyo for that. The most important thing is where your trading computers are. Plus mind that hft and algorithmic trading are not the same things. Hft is proptrading which is now mostly forbidden for large international banks by Volker rule.

Thanks for mentioning HFT is not the same as algorithmic trading. Actually, I don't mind learning Japanese at all so may be you have some advice on what skills are needed in that part of the world to land a trading job as well as where to look? Also, is it just international banks that hire HFT traders?

Note that the HFT business is most competitive Amstersam, New York, and Chicago. These places are like Silicon Valey in terms of the people that circulate between firms and know each other. There are good firms in Asia but for the most part they’re not considered top tier in terms of technology or competitiveness.

It’s not uncommon to take an HFT trade that is competitive in the US or German markets and go around applying it to markets like Tokyo, London, Singapore, Etc. But the converse is not usually as successful.

So if you start in Tokyo, it might be harder to move up and out just due to more restrictive regulatory frameworks and less efficient markets there.

That's good to know. What kind of companies hire HFT traders in those parts of the world you mentioned?

This is getting downvoted, but webmascon is correct about the terminology. It might be a bit like cracker vs hacker though, in that popular use will eventually do whatever it wants.

Apart from the Investopedia term, and just because language evolved and (unfortuately) semantics/meanings are redefined, nowadays, algo-trading (retail) is when I run an MT4 software on my laptop, it has a EURUSD chart, I attach a bot (EA) on it, I then sync this MT4 installation/instance with a virtual private server (VPS) somewhere on the planet, then I shut down my laptop, and I go to sleep.

For the large organizations, they got far more complicate systems/servers/software, but these practically do the same thing, just on a massive scale.

Just so we're clear: In the financial industry anyways, there's an arguably artificial but fairly hard distinction between making investment decisions and executing trades to implement those decisions. The term "algorithmic trading" has generally referred to trade execution, whereas in what you're describing there's a bot using logic to make investment decisions.

The point of all this is that algorithmic trading in the industry sense of the term is a product that banks and brokers can sell to buy side funds. "You make the decisions, and we'll handle the execution." If a large org is using algorithmic trading, there's a good chance they're paying someone to do it for them.

When did we move away from smart people sending humanity into space to smart people making billions on market inefficiencies (making everyone else collectively poorer)?

When smart people realized that their scientific interests didn't align with those of any government so they'd better start their own projects. And to do that they'll first have to amass a few billions.

You could just as easily say the best minds of our generation are occupied trying to get people to click on an ad.

I could, but clicking on ads usually happens on platforms providing heaps of societal value - sharing ideas, cherishing memories. Algo trading, on the other hand, literally rigs the markets. Money accrued by HF funds has to be sourced somewhere. Where do you think this is?

You're rehashing a very old, very tired argument.

I could respond by explaining that HFT facilitates liquidity and price discovery, making trading cheaper overall. I could also counter that adtech isn't any better than fintech, or that it's a fallacy to suppose every intelligent person needs to maximize their ethical production in a capitalist society.

But I'm not going to engage in any of those rebuttals. Instead, why don't you take a look at the massive charitable work bankrolled and empowered by James Simons and David Shaw? Take a moment to read through the organizational mandates and scope of research for the Simons Foundation and DE Shaw Research (DESRES). These two men have poured billions of dollars from their personal wealth into making the world a better place in a variety of ways. Both of them explicitly cite cancer research as a core focus - what do you consider more ethical than that?

While we're at it, let's circle back to your specific example. You're talking about sending rockets into space - that's great! I bet you're a fan of Elon Musk and SpaceX then? Are you aware that Elon Musk made his first fortune through PayPal? Do you consider payment processing to be a virtuous industry? Was Musk maximizing his ethical potential by working on PayPal?

Smart not to mention Mercer though!

People (both on HN and at large) seem to believe that HFT is somehow unfair, probably because of Flash Boys or some uninformed NYT article.

But back to the point, unless you believe in communism, then the best way to help the world and add value is to make money. Simply put, if you are making money, you are doing good (there are exceptions, of course and they usually involve the government). It is folly to measure social contribution in any other way (charity isn't doing good, it means whoever you are giving it to is doing good).

People, especially with HFT, think the world and the markets are a zero-sum game. I think the problem people have with HFT is that it doesn't seem "fair." Mom and Pop don't have a co-located server with ASICs churning out latency arb trades so they shouldn't be able to do that either! But why would you expect Mom and Pop to be able to compete in the first place? They probably wouldn't be able to compete with dentists, race car drivers, or scientists, so why should they be able to compete with professional investors?

Making markets efficient helps everybody, and historically, has probably been the biggest driver in making earth a better place.

Let’s take HFT for example. HFT firms are the reason that Mom and Pop investors can get efficient execution on the public markets. Compared to the inefficient markets that existed before, they have generated many many billions of dollars to pension funds, mutual funds, and individual investors.

I don’t think it’s possible to overestimate the positive effect efficient markets have had on everyone’s lives, from the richest to the poorest.

The world isn’t a zero-sum game, and markets aren’t either, everyone profits from a good free market.

Ehhhhh. yes. But that glosses over dark pools, some of the shadier exchange infrastructure arbs that go on, etc.

To be candid, I'm fully on the same page as you: what's going on here is no different than what a really good pit trader did from 1800's -> 2000. It's just the digital, semi-invisible version, and because volumes are so large, the money is exponentially larger as well.

But, that's not all the HFTs get up to; a lot of it doesn't hit public markets, or isn't particularly helping w/ efficiency.

Yeah, there are some shady dark pools and order types going around.

In regards to money, I think that decimalization actually really hurt market makers. There has been massive consolidation in the HFT space with most players either going out of business or being acquired. Profits are way down and latency arbitrage is almost impossible nowadays without billions of dollars of trading infrastructure. Profits have dropped by something like 90% in the last 10 years and the super-normal profits now (after consolidation) are just "normal."

The good thing about HFT is that with the right infrastructure, it's (almost) risk-free. The problem with HFT is that the returns don't compound, there's only a fixed amount of pennies that can be vacuumed up. This is in contrast with other non-HFT quant strategies, which can run tens or even sometimes hundreds of billions of dollars in capital and can make over 10% annually.

The story with HFT is the story with any new market. Players get in, make a ton of money. More players get in, now they are making a little less money. Even more get in, and now they aren't making enough money. Then the people not making enough get bought up or go out of business. And now the equilibrium has been reached: everyone is making "normal" money and everybody forgets about them.

And all of this is good, it's the natural market cycle. People like Elizabeth Warren were talking about how unfair HFT was blah blah, but now, no one cares. Fortunately, the entire cycle resolved before the government could get their sticky little fingers on it.

That's a great point. I agree that this will sort of resolve itself, until MIT puts out some new Cat 8 ethernet or some major hardware jumps, the returns start plateauing quickly. When there's such a large barrier to entry, and with only a handful of HFT/DMMs moving the major volumes, that's somewhat of an systemic risk to the tune of a SysAdmin/DevOps guy screwing up a deployment, i.e. what took down Knight Capital

A "dark pool", despite an ominous-sounding name, is just a type of exchange which does not display order prices and sizes. It is something in between of a regular exchange and an auction, better suited for posting and executing large orders. There is nothing shady in it; it helps larger players (like retirement / index funds) to obtain better prices for their large orders.

There's been a lot of lying by multiple banks to customers about the property of their dark pools. Saying they ban HFTs, or categorize them as "aggressive" when actually doing nothing, or having hidden order types for HFTs, etc etc.

Just to be clear, I mostly think that's fine and whatever, who cares, but there has been a lot of deception in the space.

Yes, not shady, but it totally unravels the argument concerning HFTs helping price discovery. I know the "hey retirement and index funds are doing it, think of the pensioners!" argument is common defense of it, but that's not at all the target audience of DPs. It's liquidity going off exchange, which hurts price discovery, simple as that. Considering how competent trade execution/slippage capabilities are for the types of broker-dealers that would be handling index/ret/pension volumes, that argument is nonsense

Well, dark pools are a counter-HFT measure, of course they have limited price discovery. If you want price discovery, you can go to lit markets. If you want matching without showing your orders, you can put orders to the lit book algorithmically, or go to a dark pool. It's not that there is no choice. And people traded off-exchange like forever, OTC market still exists and predates exchanges.

Reply on news.ycombinator.com