A summary of what quantitative trading firms do
blog.headlandstech.com
blog.headlandstech.com
Please allow me to disagree :) Most larger pure-algo HFT prop shops are self-clearing, they are exchange members (possibly with more favorable fees due to market making) and don't use pbs, instead relying on their own co-located execution infrastructure and funding.
Managing margin and leverage between prime brokers is an important job for the backoffice at the fund I work for. These numbers are watched very closely, and we have strong, multiyear relationships with the people who work at these PBs.
[0] https://www.nyse.com/markets/nyse-arca/membership#directory
Being a good prime broker is also a lot of work, and is an elaborate and involved process.
Now for the firm to take on two jobs, and then keep it secret... I wouldn't worry about it. It's like worrying your accountant is going to take your successful startup idea and run with it because they know how much money you make.
And in all markets, that's one reason why many firms invest in transaction cost and best execution analysis, to make sure they're not getting front-run when they see abnormal dips in broker quality.
nice read: https://steemit.com/arbitrage/@kesor/the-math-behind-cross-e...
(I'm not a recruiter or working at a quant firm, just curious)
Of course time is limited, so when deciding if I'm going to invest the time I do a quick check on ROI. How much time do I think I'll have to invest. This depends on how much I know in the general area of the subject and how specialized I assume the subject to be. So basically how much time do I have to spend to learn the required vocabulary. I value that against how useful I assume the knowledge to be. Can I directly apply the new knowledge, or do I assume it to have synergies with my main subject of interest.
But sometimes a subject is so interesting to me that learning about it is its own reward.
I'm trying to read your whole comment, it's interesting especially since most hackers are somewhat self-taught and there are many fields we can become proficient in (such as dozens of resume keywords) if we want to invest the time.
So I'm interested in how people make these choices. Could you elaborate?
but take a month of hourly commutes in learning basics of NLP or bootstrap or PGP or ... is likely to have much higher ROI
my mental model is
- There are few fields that are wide open right now - it's ML (vision 90%, text 50%), process automation, robotics, security.
- It's got to be widely applicable to more than one industry
- It's got to not involve going back to college for a year of maths training.
- It needs some kind of moat. Learning to use bootstrap better is nice cos your stuff looks better and is cleaner. It's kind of table stakes, but it has zero most. Intel secure enclaves looks interesting to dive into and will have some decent moat factor.
- it's really got to be uISV suitable. This means I can come up with at least two businesses that conceivably will work, selling element extraction from text is not a business but "capturing ten kpis from your base of legal contracts" is (and probably a hotly contested area)
I did the whole "walking through lists of industry SICs to generate ideas - even read every ycombinator investment. no dice" Good business opportunities are hard to imagine.
In these types of strategies you care about latency but on the order of milliseconds not microseconds. The challenge is in building multi-day predictive alpha models with low correlation to each other, getting good executions though brokers can do a reasonable job these days, and especially combining alphas into one book which requires sophisticated mathematical programming (conic programming etc)
What about quantitative trading firms. They make a profit themselves. But how do they benefit society?
https://genius.com/Bill-hicks-on-advertisers-and-marketing-a...
Furthermore, if I want to sell a security and one of these companies puts in the best bid, then I just made more money than I otherwise would have if they weren’t there. Likewise if I was buying, they might offer me a lower price. Isn’t that a good thing? They are happy and the counterparty is happy.
Then there’s arbitrage. If I am trading on an exchange, but there’s a better price on another axchange one of these firms trades on, they might offer me a better price than I otherwise get. Effectively I d get a price on the other market, without having the costs of trading on it. Again, that’s better for me, where ‘me’ might be your savings account or pension fund manager, or a manufacturer or producer on a commodities market. So these firms reduce costs for other participants on the market.
Suppose you are a customer with a connection to an exchange. You can't afford to connect to every exchange, because that would cost you a small fortune.
If you stick your order into this exchange, you might have to wait around for someone to meet your price. In fact, there might be someone willing to trade with you at your price, but you don't know it because you're on different exchanges. (Let's not get into NMS.)
An HFT with a view of the whole market would help you distribute your order to all exchanges. They do this by leaning on your order while posting everywhere else. So if someone came in and met the HFT's order in some market, the HFT would turn around and fill you immediately for a small difference, and they'd pull all the other orders when this happened.
That's a service, and you pay for it.
A lot of HFT strategies provide a service similar to this in some way.
If this was such a great service why is IEX becoming so big?
HFT is a nightmare from a CSR view. Those companies provide no real value to society (not by any standard) and even worse: By hiring all those talented young people with degrees in various fields they actually hurt society. Give those people real problems to work on, instead of wasting them on optimizing those HFT systems.
Also IEX is a very small exchange, not one getting huge. Their PR firm is impressive not their size.
Finally there are lots of trading options that allow you multi-exchange order book access. Getting market access is fairly cheap. The expensive part is the counter party risk mitigation. You need someone to prove you are good for the trades you say you are. That has nothing to do with HFT or electronic trading at all.
You'd have to write a feed handler for every exchange, and a real-time composite orderbook so you could see where the best order was at every point in time. For every stock. You'd need to rent the fastest network as well.
This is no different from "I could tile my own roof, what do roofers do that is so useful to society?"
> Those companies provide no real value to society
A lot of businesses provide no real value to society. The only way to do that is to interact with a large portion of society. So restaurants for instance only serve the local market, no real value. I mean who benefits, other than the people who eat there? I could go on with a whole load of other businesses.
Why would I need to rent a fast network for every stock? I didn't say abolish the exchange, I said make the data open, generally available so I can see for myself whats going on. Also smart order routers are there already, so no I don't have to write them again, every other broker already offers them. Now just cut out the HFT frontrunners and I can actually the price I see on the terminal.
> A lot of businesses provide no real value to society. The only way to do that is to interact with a large portion of society. So restaurants for instance only serve the local market, no real value. I mean who benefits, other than the people who eat there? I could go on with a whole load of other businesses.
That may be right, but that doesn't make HFT any better.
High frequency trading is not front-running. High frequency traders 1) do not have a fiduciary duty to other traders in the market, which is a hard (and definitional) requirement for front-running; and 2) cannot see a retail investor’s order before it is executed on an exchange. They can only alter their prices in reaction to orders that have already executed.
> I didn't say abolish the exchange, I said make the data open, generally available so I can see for myself whats going on. Also smart order routers are there already, so no I don't have to write them again, every other broker already offers them.
1. This data is already available for purchase. There’s nothing intrinsically preventing you from acquiring it as an individual. I’ve personally purchased this data as an individual.
2. Even if you can “see for yourself what’s happening”, without the extremely fast market making provided by HFT you’ll be waiting significantly longer just to find a trading counterparty. Your proposal sacrifices the liquidity and price efficiency provided by HFT in exchange for freely available market data, which the vast majority of retail investors will not be able to use (let alone want to).
3. Smart order routing is possible because there is sufficient inter-exchange liquidity. In fact, it is directly facilited by HFT.
I’m not often this blunt on Hacker News, but in this case I think it’s warranted: you do not appear to have even basic familiarity with what you’re criticizing.
This is not an informed thesis. This is just an appeal reducible to the following statement: “A thing cannot be a good service if it has a large and organized opposition.”
>HFT is a nightmare from a CSR view. Those companies provide no real value to society (not by any standard) and even worse: By hiring all those talented young people with degrees in various fields they actually hurt society.
If your thesis is that HFT provides no benefits for society, please provide a substantive rebuttal against the claim that it improves liquidity and price discovery.
HFT increases the availability of bids and asks in the orderbook by rapidly volunteering to be the trading counterparty in otherwise inefficient markets. By seeking the best price for its own strategies and making as many trades as possible with as little latency as possible, the increased liquidity is accompanied by improved price discovery.
> Give those people real problems to work on, instead of wasting them on optimizing those HFT systems.
Please define “real problem”, and please explain how the aforementioned benefits contributed by HFT are incompatible with this definition. Meanwhile keep in mind that the vast majority of companies, including those in tech, likely do not work on “real problems”, so next I’m going to ask why this is a meaningful criticism in the first place.
1. All trading strategies profit by identifying market inefficiencies.
2. Market inefficiencies occur (in general) when the price of a security is inaccurate.
3. By correcting the inefficiency, the strategy profits consistently until there is no longer a counterparty willing to take the trade at that (inefficient) price.
This process is broadly called price discovery.
The greed and complete disregard for society and it's advancement as portrayed through the media and various outlets simply make it a very unattractive career choice to me.
Now this is not meant as a judgement of a whole industry and it's participants, it's simply my naive and uninformed opinion. I would like to read some comments about insiders who might have had my initial aversion, but came to see something different which is positive in some way. Any HFT devs out there with a similar experience?
The benefit to society is that less money and human capital is wasted on finance so more money and human capital can be allocated to more interesting things.
HFT is a tiny industry that generates a tiny amount of money and they've largely replaced what used to be a much larger industry that generated a lot more money. That sounds like a huge benefit to me.
The empty trading pits in the South Loop Chicago and all those new condos downtown NYC are a testament to that.
Tsunami hits Japan and some Japanese computer memory factories are shut down. As soon as this news hits America, the prices of computer memory chips is expected to go up. But there is a delay in how quickly different people hear the news. Some American memory chip sellers might still be selling at the old (lower) price, while some of their better-informed customers seize the opportunity to buy before the prices go up. In this instance, the sooner the news reach America, the American resellers know to increase their prices, and they get the benefit from the increased prices. (And some buyers end up paying more.)
Later, the Japanese have rebuilt and the factories are about to start producing again. Now memory chip prices are expected to go back down. The sooner the new information spreads around, the more people will benefit from buying at the cheaper prices. (And some resellers will disbenefit by having to decrease their prices sooner.)
The well-informed market makers keep the prices as current as possible.
Buyers benefit from new, cheaper prices, but sellers disbenefit. Or the other way round. But we believe that efficient markets are beneficial to society, so the benefit for whoever gains by buying/selling at the most recent price information is larger than whoever lost because they couldn't sell/buy at the old price.
I'm surprised to see that engineers are required to know hard core VLSI stuff like processors and physical affinity to cores, Non Uniform Memory Access, clock synchronization et al. Thought only Integrated Circuit Digital Design/ Chip design engineers are required to know that.
Would anyone kindly shed more light on this subject? Has anyone used VLSI / Digital Design topics as part of his Quant trading / HFT work?
Any time an algorithm must consult a dataset, and that data is changing rapidly, leveraging knowledge of the underlying system will give you an advantage.
The point the guy is making is that when you're coding this type of thing you need to understand how the computer actually calculates things, rather than just have a vague idea of a machine like you might if you code python. In python and those types of languages a lot of the inner workings are abstracted away. You can sort of just imagine an Oompa Loompa (a thread) reading the code, seeing the instructions (x = a + 2 if y==3 else a - 2 ) and going out to find the value of a in a box and putting in the value of x in another box. No idea where a and x boxes are, how close they are to each other. No idea about how that branching instruction works either, he just checks at the time and decides.
By contrast when you're writing for performance you want stuff to be in cache, so you need to have an idea of a machine that includes cache. Specifically the idea that the further away you are from the registers, the longer things take, and that some cache is shared while some isn't. You also want to think about a machine that can speculate and do branch prediction, so you'll need to think about how to write the code so the branch predictor mostly gets it right. I think the top stackoverflow answer is about that.
Affinity and NUMA are talking to this kind of machine (which is still abstract of course) which has a bit more detail than the Oompa Loompa.
I was thinking more in terms of Digital Electronics.
Will definitely go through it.
Did they hire RTL / Digital Design Engineers?
Just out of curiosity, may I ask the name of the HFT firm?
But they did design the boxes that hold the chips and I was surprised by the amount of effort they dedicated to hardware matters.
That's how far into the weeds they go.
https://github.com/melling/MathAndScienceNotes/blob/master/q...
Some include previous HN posts.
https://www.quantopian.com/posts/max-dama-on-automated-tradi...
The pdf link seems broken, but through Google you should easily find the pdf (maxdama.pdf)
The one thing I would say is that it's very hard to make money doing this. The profits accrue to the select few who have cornered the market in terms of technology and speed.
Which is why firms who make money doing this usually have an edge beyond the technology. In one word: flow.
They are able to interact with a stream of orders... Not theirs... That is low in alpha and they are able effectively to get "first look".
Furthermore you can never just "sit" on a strategy. Eventually your edge gets arbed out and you need to be creative enough to data-mine the heck out of the market to find some new edge.
Some thoughts on figuring out your edge: 1. You're not the only smart person looking at something. There are armies of PhDs who have been there and done that. 2. Study regulation and regulatory changes. Keep abreast of new & upcoming changes. 3. Look for markets where barrier to entry is higher or not as studied. eg: Electricity markets are a _bit_ less understood or traded. 4. Always think in pairs or more... Longing or shorting something outright is almost never the answer. But trading the spread between two (3,4,many) related products is more difficult to calculate and backtest. 5. Always do the more difficult thing - but don't make it complicated. Simple math is usually what works.
From your words I understand that I have got to be very creative in order to find an edge. Especially your advice on thinking in pairs or more is valuable.
> Let two players each have a finite number of pennies (say, n_1 for player one and n_2 for player two). Now, flip one of the pennies (from either player), with each player having 50% probability of winning, and transfer a penny from the loser to the winner. Now repeat the process until one player has all the pennies.
> The chances P_1 and P_2 that players one and two, respectively, will be rendered penniless are
> P_1 = n_2 / (n_1 + n_2)
> P_2 = n_1 / (n_1 + n_2)
So if the current order book looks like:
- Sell 5 for $99.00 (best offer)
- Buy 10 for $98.75 (best bid)
And you chomp up orders randomly, flipping a completed order to the opposite position:
- the probability that the price is above $99.00 (all 5 sell got filled first) is 10 / (5 + 10)
- the probability that the price is below $98.75 (all 10 buy got filled first) is 5 / (5 + 10)
So expected value is (99.00 * 10 + 98.75 * 5) / (10 + 5) = 98.9167.
I am not happy with this explanation. It seems like after you execute an buy for $98.75 you need to immediately put a sell for it at slightly above $98.75 (and vice versa) to fit the random walk model described above. And then I took the expected value using the price at two different points in time. Overall I am still confused myself.
The only impossible part is finding tick-by-tick market data
It's likely a fun project and quant trading seems like an interesting job. If you include on your resume your machine learning quant algorithm, your white paper, how you've extend one market's train data to say altcoins/penny stocks, I'm pretty sure you'll peak an interviewers ear.
Hell you may even make a few bucks.
I am really involved in low frequency trading with as I call them stock trading robots and have been doing this for many years with double digits return on average and all is done with python and few VPS around the world for redundancy e.g. very low brier of entry...
Now I get some attempts to financialise other industries (usually having large social costs) but just wondered if we are seeing this creep out elsewhere (online advert auctions perhaps?)
For Market Making the goal is not to form consensus on the state of the market. Instead, Market Making is about creating a liquid market. That is, making sure that when someone wants to buy or sell, there is a willing counterparty at a non-stupid price. Alternatively, you could describe them as 'keeping the spread low'. That is, keeping the difference between the lowest buy order and highest sell order low.