How to Get a Job at a High Frequency Trading Firm | Hacker News Reader

How to Get a Job at a High Frequency Trading Firm

quantstart.com

41 pointsshogunmike92 comments

While i find the technological challenges in HFT very exciting, i would never want to work for the companies benefitting from that work. If that means i will make less money and someone else will come in and do it, then so be it.

There is a thing in most kinds of investing and trading called slippage. This is when you decide you want to buy something at say $100/unit but by the time your order reaches the market, the price has moved, maybe it's $101, maybe it's $99. If you are buying a lot, then it's unlikely there's a single counterparty you can trade with, so you will actually be doing it as many smaller trades. Sometimes slippage moves for you, sometimes against but that's not the kind of trading you want to do because it involves taking on more risk, you want confidence that the price you ask for is the price you're going to get, and you don't want your trades to affect the price. HFT techniques are invaluable here.

As would trading on a clock. If the market only updated once a second or even a far smaller interval, this whole insane business would be mostly irrelevant.

Then proximity to the second tick would be the game. What you are advocating would have people sending orders in the future to arrive at the gateway as timer expires.

Perhaps I'm missing something, but why would it matter? Trades execute at the state in t-1 at time t. Assuming people's orders are secret in that interval, you don't actually care when your trade gets queued.

because if the trade comes in too early it would automatically rejected by the g/w. Also how could the orders in a published orderbook be secret?

You are misunderstanding what he's saying.

He's saying that you could use a periodic-submission order system. Say that the period is (for ease of discussion), 1 hour, ending on the hour. All orders made during the hour are queued up and simultaneously executed at the start of the next hour. If there are too few bids/offers to satisfy all trades, then trades are satisfied using some fixed rubric (i.e., pro rata, max/min price buyer/seller will accept, etc.) Until execution, the orders remain secret (not published to the orderbook). In theory, this allows people to place orders based on a known price.

In practice, in the past (before electronic trading took over), the published orderbook was updated manually as traders submitted bids, offers, and executed sales information to the exchange.

> In theory, this allows people to place orders based on a known price.

This is exactly backwards. Without a published order book there is no "known price" since everything is secret. Your hourly blind cross will go off at whatever price satisfies the rubric but there is no way for a participant to know what that price will be before hand.

Unfortunately there is actually no such thing as "simultaneously", because these are not discrete events; for every buyer, there will be 1-n sellers, all of whom are involved in other trades of their own. You have to impose an order on trade execution, and each trade will take some finite amount of time to process.

So you randomize instead of doing first come first serve. Everyone who got there orders in by the deadline gets an equal shot. You do verifiable randomness to prevent the exchange from messing with trade orders.

If you do a simple two phase system, you don't have to worry about the exchange leaking orders. You send in a cryptographically binding commitment to your order(this hides the order now, but prevents you from changing it later). After the time period is up, the exchange publishes a commitment to their randomized order of trades , then every party reveals their actual order. Then trades execute.

Sorry, but that's ridiculous. You can't know in that situation what inventory anyone will have at the time you actually make the trade!

I don't think you quite understand that these trades aren't simply converting money to something else. They are actually buying and selling things, abstract things to be sure, but things that your counterparty will either already own or be able to acquire on your behalf. You might as well say Amazon should send things to random addresses and everyone has an equal chance of getting what they ordered.

I think your Amazon point is rather inaccurate. With amazon people are ordering different products at fixed prices from one seller. In this case, the question is how do we actually determine the price for one product(say Google's stock) given many buyers and many sellers all with different price points. It's not the same thing at all.

Someone/some algorithm has to match offers for sale with offers for purchases. Buyers are willing to buy at up to some price and seller sell at down to some prince. The question is when the matching is done, how frequently and whether you gain any advantage by being faster (other than having marginally more time to analysis). By contention is that you can accomplish this.

While this may not work for, I don't think amazon or anything like that has much to do with it.

Sorry but that makes no sense whatsoever.

This sounds like an awesome idea.

No, the idea is that orders are batched up and then crossed in an auction every second. You literally gain no advantage from being a millisecond (or several) ahead of somebody else as long as you're within a given second.

It's kind of like an opening cross, happening like a metronome every second of the day. (They also do these or similar crosses when a stock restarts after a halt, or during an IPO.)

Probably there would be some attempt to game the system by measuring open interest on either side up until the time the cross operates (i.e. when one second 'ticks' into the next). If such numbers were made available.

... And then people would worry about being exactly on the second, not a millisecond over.

If there's not difference between the start of an interval and the end, it doesn't matter much. A second is a long enough time to get an order in that at least people won't be doing insane things to reduce latency.

If you want to adopt a pro-HFT position, can you point to a single major market volatility event when the insanely risk-averse HFT algorithms, didn't just shutdown? I've looked, I haven't found any. Would be nice if they did exist as it would invalidate some of the HFT criticism.

The flash crash was one major one. Some of the HFT firms bailed out, some (the market makers) were required by their contracts with the exchanges to stay in.

One reference: http://www.advfn.com/nasdaq/StockNews.asp?stocknews=CME&arti...

I've looked, Where did you look? That is kind of a big one to miss.

Hey, thanks for supporting my point.

We're not talking about major events, we're just talking about everyday normal trading. My entire point is that HFT is valuable for reducing risk for this kind of work, you say "risk-averse" like it's a bad thing...

It's not a case, btw, of being pro- or anti-. HFT is just a tool. You can use any tool in any way, it doesn't care because it has no moral agency of its own.

>My entire point is that HFT is valuable for reducing risk for this kind of work, you say "risk-averse" like it's a bad thing...

Er.. One of the fundamental reasons that HFT exists is increased market liquidity. Re-read what you wrote. You imply HFTs reduce risk and at the same time seem to accept that they are risk-averse - ergo unable to reduce risk as they shutdown when the markets risk profile increases. So which is it?

Oh and btw, since no humans trade securities at such a high frequency, The HFT trading that takes up such a sizeable portion of the total pie, is just an electronic game being played between two competing firms. So, even if they did reduce risk, it would not help human participants.

You are conflating market risk with counterparty/exchange risk there. Wikipaedia is your friend...

OK, we're arguing two different things. My one and only point is HFTs shut-shop at the slightest sign of risk, leaving you to trade at massive losses - precisely because there are no HFT like counterparties ready to buy/sell inventory.

IIUC Your position seems to be that HFTs amongst themselves can allow for an order execution strategy of breaking up the order into chunks to discount for the price change that occurs when you increase supply of a security while selling it. If that is your position then I withdraw my protest because I agree with it.

This is a very strong statement. It means you wouldn't want to work for any of the software or hardware firms that supply them. Eliminate Microsoft, IBM, Oracle, and most networking and big data providers from your list. Eliminate all financial services firms who make either commissions on the other sides of the trades with them, or get better bid-offer spreads as a result.

They're not saints, but it's very hard to remove oneself from where they have impact.

I dont think one has to go all the way into avoiding everything and anyone associated with these businesses for it to have a positive impact. But you could at least try to not work directly for them.

Likewise, I am also very concerned about the sustainability of the human culture in general, still i dont stop eating meat completely or stop driving my car. But i will try to balance things and reflect my consumption patterns on a regular basis.

Sorry, i am not a native english speaker and can not really word this in the way i want, but well, you might get the idea ;)

Your English is more than strong enough to get the point across. I agree with you - we do what we can on the margins. It's tough to take overly absolute positions. That's what I was calling out in my post.

There are quite a lot of companies out there, I don't see how it would be so hard.

Check if your company invests (e.g. your pension pot) into one of the portfolios that sub-allocates to HTF hedge fund as well. You might want to pull out then. So immoral.

Maybe they already have.

That should exclude most hardware vendors, since they benefit from high frequency traders buying up tons of hardware

I have less problems with HFT than ad driven businesses (e.g. Google and Facebook), military stuff, banking in general ...

why?

In my mind, the value that HFT adds is theoretical at best. The Economist could surely rationalize it in terms of "finding the right prices" or "arbitrage" or something. To me it looks like a systematic way of taking advantage of people who aren't using it.

Would it make more sense to look at it as similar to a poker game? Relatively little economic value is created or destroyed in ultra-short time scales, so the players in that game are essentially playing against each other (minus some small cut taken by the casino).

The ‘good’ players thrive until they reach maximum size for their niche. The ‘bad’ players leave the business. Customers such as long term investors, institutional money, etc, benefit through narrowed spreads and reduced transaction costs.

You could call it a vicious, dog-eat-dog business, but the dogs aren’t really a threat to anybody else.

Interesting way of looking at it. I hadn't thought of it that way before. I'm curious how sound this analogy is.

I thought there was a consensus in this community on how the leading traders just squeeze money out of all other wannabe sharks. The skills I find most relevant are low ethics and networking (not the technological type of networking).

Interesting. I thought the consensus on HN was, we should form unshakeable opinions about entire industries based on Matt Taibbi articles in Rolling Stone, or, better, based on comment-driven games of telephone starting from the most colorful sentences in Taibbi articles, and then that we should wield those opinions to cast aspersions on strangers on HN to make us feel better about ourselves.

Oh, wait. Did I just say the same thing you did? I'm sorry. Carry on.

There are people here who work in that field, and they don't strike me as having 'low ethics', offhand.

The consensus I'm more favorable to on HN is avoiding knee-jerk insults to people who you don't even know.

I'm not wild about HFT myself, but these sorts of nasty comments are something I would prefer not to see in this community.

> but these sorts of nasty comments are something I would prefer not to see in this community.

Because I've made comments in the same vein, I had given some thought to the matter before posting them. You are, of course, right. These are gross generalizations, and I think most readers here would recognize them as such. However, I think that another principle trumps here, and that is setting a sort of "baseline" ethics. The HN community is composed mostly of engineers, who for centuries have had the tendency to concentrate on the technical challenge at hand rather than the social and moral ramifications of their products. By establishing a certain crude (I admit) sentiment that HFT is "wrong unless proven otherwise", or, at the very least, that it demands some serious thought before engaging in, we draw a blurry, faint red line that we, as engineers, should pay attention to. Naturally, this applies to all undertakings, but in our community, HFT serves as a suitable exemplar. I think.

Do you see any difference between saying "HFT is scummy" and "the people doing HFT are scummy"?

Of course! People don't generally set out to do wrong. Most "industrial grade" wrongs are done by people with good, or neutral intentions.

Let's put it this way: if the character of the people you work with matters to you, then HFT is not something you'd want to do.

This is nonsense and typical of HN.

My theory is you all hate it because its based out of Chicago/NY/London and not SF and you are like kids excluded from a club you can't get into.

Software people in HFT and other trading related systems are just smart people who want to solve interesting problems. Same as everywhere else.

Sources?

Are HF traders known to be objectively more “evil” people than others in the finance industry, or vicars of country churches, or social workers, or cabinet ministers? If so, I’d love to see the studies.

I base this on the few people I know in this industry. Not scientific by any means.

Now, people are hardly ever "evil", and they certainly never consider themselves evil. But even strangers would hardly call traders evil; they're just... how shall I put this... if you know some people working on (in?) Wall Street then you know the type. Of course, there are plenty of technical people working on the highly technical aspects of HFT with little regard to the domain itself, who don't fit the prototype; still, there are very few idealists in that industry.

> they're just... how shall I put this... if you know some people working in Wall Street then you know the type.

Well, I don't know anyone working on Wall Street and I don't know the "type." Can you give concrete examples or details to substantiate your claim and make it specific? What is it in particular that you object to about these people?

No offense, but as a disclaimer, I don't want to come across as giving your claim too much credence. In the abstract, any form of trading is a life-supporting activity (someone is voluntarily choosing to trade with you to mutual benefit) and thus is noble, other factors being equal (e.g. you're not committing fraud, etc.).

You're right. I'm sorry for taking the ad-hominem path. We should discuss the practice, not the practitioners (what is it that some Christians say? Love the sinner, hate the sin? ;)

I'm actually still curious about your anecdotal experience, though, if there's anything you'd like to share. Just because quants are not evil "in principle" doesn't mean there might not be certain personality traits that pop up often. I don't know anybody who works in that space.

I don't think it would be appropriate at this point in the discussion. There are numerous articles and books about Wall Street culture; some of them are very true to life.

Fair point. There is a certain ambitious Wall Street vibe that isn’t what I would call ‘evil’, exactly, but I wouldn’t consider it to be ‘socially conscious’ or ‘kind’ either. Sort of blandly self-interested.

That said, I’m not sure that HFT is ‘worse’ than other sectors of the finance world, or law, or consulting, for this particular vibe.

If you include ethics into your decisions then suddenly you are not able to work for a lot of companies. Google - severe tax evasion, Facebook - collecting your personal info for dodgy reasons etc.

> If you include ethics into your decisions...

By definition (almost), ethics is the systematic study of what one should or should not do. Are you suggesting ignoring the systematic study of what one should or should not do when one has to decide what one should or should not do?

> then suddenly you are not able to work for a lot of companies...

Well, every person has their own levels of comfort with various practices. Ethics doesn't give you clear-cut answers, it's just a way to think about decisions.

Not to mention the bubble-startup business model, raise a billion from VCs and blow it. That money belongs to pension funds, university endowments, charity trusts...

It's not a zero-sum game like trading, though. You do try to benefit society and the economy, and even if you don't succeed, people make a living off that "blown" money.

The only thing business with shareholders wants is maximizing the profits. Besides, how are you going to IPO your start-up without traders involved? Capitalism without trading sounds like communism without common property.

A stock exchange doesn't need traders. We all buy and sell things all the time without the help of traders. Retailers and importers provide, in their domain, a much more valuable service than traders do in theirs.

We don't even need traders for traditional risk-reducing options trading for farmers and manufacturers.

> The only thing business with shareholders wants is maximizing the profits.

In the US, generally yes. In other countries (like Germany), businesses have other goals, like providing sustenance to workers.

"like providing sustenance to workers." when enforced by government that is, in some countries you don't even have to pay tax.

That government is elected by the people. It's not a foreign occupier. So I guess that the people of Germany have chosen to make businesses not just about maximizing profit.

a) Trading isn't zero-sum. If you meant to say HFT, no, that isn't zero-sum either.

b) Trying isn't enough. A billion blown is a billion blown, it doesn't matter what your intentions are - and if it does, I'm sure most HFT-traders are convinced that they are benefiting society and the economy.

c) People are also making a living of off money spend in HFT-space, regardless of what their intentions are.

It's not clear to me how society benefits from ever more intrusive ways to show people ads, and "people make a living" in that context is the same as Reagan's so-called "trickle down economics".

^^ We guys can spit out a few lines of code here and there too in addition to the above mentioned hard to find traits by you ;)

I'm surprised that the morality of HFT firms keeps coming up even among technically enlightened audiences.

I'll preface this by saying I'm not a HFT or affiliated to one but I strongly believe that HTFs and UHTFs play a valuable role in well regulated markets. We can argue endlessly as to whether the financial markets are well regulared but for this discussion let us assume they are pretty close and that if they aren't there's an HFT out there that's working to discover why and capitalize on it.

HFTs also provide massive liquidity and allow people that trade large positions (Pensions, Insiders e.g start up founders etc) to enter and exit into the market without causing massive waves in the instruments they are trading.

I think we should be delighted that the morality of anything comes up among technically enlightened audiences. For years, us engineers have been accused (often justly) for not caring.

The value HFT, day trading, and stock traders in general provide is far from clear-cut. When considering the merit of some economic enterprise you can't just take into account its local effects. For example, casinos also provide value: they provide entertainment and they feed a lot of people. But it's uncertain whether our society is better off with them or without them.

If we put aside for a second the overall sleaziness of this industry that makes working for it very unattractive, isn't HFT at a dead end? Do people still make a lot of money off it? Hasn't it reached the tail-end of diminishing returns?

As a HFT, I can safely say that you are correct. Volume and volatility (At least in the futures markets) have been drying up. My firm has gone from 40 to 20 employees over the past 18 months. I know a similar thing is happening at other HFT firms, but I am not sure to what extent.

It really depends on how you define HFT. If we define it as the author does, then it's not so much a dead end, rather it is really hard to move in on. I like to think of latency arbitrage (including pairs trades, exchange arb, etc.) as a utility. It takes a massive amount of investment, but if you do that you can get a very steady return. The problem is that in most of those trades there is no money in being the second best. These leads to an environment where the people already invested are in an arms race to keep at the front.

If on the other hand, you are talking about other forms of algorithmic trading, there is still money to be made by being smart, having insight, having better procedures, or faster time to market etc. The arms race in latency arbitrage has actually made a lot of this easier as it has dramatically brought down the cost of entry to the non-bleeding edge of latency use cases.

> there is still money to be made by being smart, having insight, having better procedures

Other than algorithmic speculation (sentiment analysis, etc), is it still possible to come up with better (non-speculative) procedures that yield substantial profit?

Depends on what you mean by "non-speculative" & "substantial". All trading is "speculative" in some sense, but yes you can still make a tidy living in the algorithmic trading space.

I believe the profits have dropped a lot recently, as competition has driven down margins for everyone involved.

http://www.businessweek.com/articles/2013-06-06/how-the-robo...

Dead end? HFT is more like an evolution of open outcry pit trading to the digital age.

I'm happy to see this pop up on HN since HFT is something I've been looking into lately. If you're interested to dig deeper into HFT and algorithmic trading, 'Inside the Black Box' will be a good read. Link: www.amazon.com/Inside-Black-Box-Quantitative-Frequency/dp/1118362411

If you want to take your reading further, there are some great recommendations on the review section. And just to get an insider look into quant traders and the effect HFT can have on the market, I also recommend these two short documentaries made by the dutch national TV:

1. Quants: The Alchemists of Wall Street https://www.youtube.com/watch?v=ed2FWNWwE3I

2. Money & Speed: Inside the Black Box https://www.youtube.com/watch?v=aq1Ln1UCoEU

Keep the following in mind..

HFT is not very profitable anymore.There are a fair number of companies in the field now.

HFT tends to be algorithmically simplistic.This is not because stupid people are working there but its very difficult to have algorithmic complexity and speed,and speed is more important.

It is very much a winner takes all game,profits are not evenly distributed,being fast is VERY important.

However good you are you are not a trader.You are a codemonkey. You will never be rewarded or compensated as a trader in most of these firms(some firms like Rentech are an exception to this,but rentech is not typical high frequency).

Your hourly rate will probably not be worth it. When you factor in variable compensation(ie bonus) which is not guaranteed and horrible working hours you would in most cases be better off with some boring bank java contract for compensation.

> While many UHFT firms have moved towards custom hardware (both for processing and networking), some less latency-sensitive systematic trading firms still make use of multithreaded C, C++ and Java (with custom garbage collection).

Aside from the hiring theme I have a question. I'd think given that network speeds would start being the bigger bottleneck. Is location at all important to these firms? The speed of light is fast, but the closer you are to the exchanges, the better the advantage over competitors must be.

All the major exchanges offer co-location services, where you can put your trading engine (i.e. the "black box" that executed your trading strategies) in the same building as the exchange's trade matching engines. That means that you'll receive market data quicker and the exchange will receive your orders (or cancellations) quicker than someone who's located a few hundred miles away.

For more info, see: NYSE - http://nysetechnologies.nyx.com/en/infrastructure-solutions/...

London Stock Exchange - http://www.londonstockexchange.com/products-and-services/con...

Eurex - http://www.eurexchange.com/exchange-en/technology/co-locatio...

CME - http://www.cmegroup.com/globex/trading-cme-group-products/co...

Such co-location obviously works when you're talking about a single market but if you're looking to execute trading strategies across multiple markets, then you're going to be in the market for low-latency network connectivity between London and New York - http://www.telegraph.co.uk/technology/news/8753784/The-300m-... - or NYC and Chicago - http://gigaom.com/2012/02/10/wall-street-gains-an-edge-by-tr...

However, it doesn't really matter whether you shave 10ms off your network latency if your software is sluggish, so a lot of effort goes into optimising code and creating execution environments that minimise latency caused by abstraction layers. Companies use real-time operating systems and TCP/IP stacks that are implemented on hardware. They're now starting to use FPGAs and I'd be surprised if someone hasn't already splashed out on an ASIC.

I can't find the story now, but I read about an HFT making a play on the bid/ask spreads between new york and chicago, they spent a large sum of money to build their data center somewhere in the appalachians to save themselves something like 3 ms in total transit time.

I think you are referring to an article called "Barbarians at the Gateways" by Jacob Loveless, that recently popped up on HN:

- Web: http://queue.acm.org/detail.cfm?ref=rss&id=2536492

- PDF: http://portal.acm.org/ft_gateway.cfm?id=2536492&type=pdf

Distance between the exchanges is a factor, I believe there is a minimum distances each company can be between the exchanges. http://www.youtube.com/watch?v=aq1Ln1UCoEU This video at 23:27 talks about minimum 1000ft cable.

You're correct, co-location at the exchange data centre is indeed a requirement for most HFT firms.

I think (but I'm not 100% sure) that the length of network cabling is also tightly regulated at many exchanges so that nobody gains an advantage in that manner. If anybody has more insight into this, I'd love to hear about it.

That's correct. All the servers are connected using the same length cable so that the server at one corner won't receive the data a tiny bit faster than a server at the far corner. So there are simply plenty of unused cable next to the machines.

Generally it's possible to colocate servers in the same datacentre as the exchange servers. This isn't really a competitive advantage though, as anyone who needs latency that low will be doing the same, and the costs will be relatively insignificant compared to salaries of people who can build the required systems.

While I completely agree that there is no advantage over other HFT competitors provided by being located within the exchange datacentre, there is a disadvantage to not being there. This gives the exchange an opportunity to make a significant amount of money from each HFT company without hurting other customers.

There's a great Wired article about this from last year: http://www.wired.com/business/2012/08/ff_wallstreet_trading/...

They explain how firms built private fiber networks between Chicago and New York to shave a few milliseconds off latency.

From what I've read on the subject, yes, location is critical. As I understand it, HFTs are co-located in the exchanges' datacenters.

It is. There's a massive colo right next door to the NYSE.

The NYSE matching engines are in Mahwah, NJ, about 20 miles from the NYSE floor. If you rent colo space there's you're actually in the same building.

You may have to leave your CONSCIENCE at your home.

On the topic of the utility of HFT my personal opinion of is it reduces spread costs but adds systemic fragility. I actually doubt stock markets are socially utile,let alone HFT but that is an argument for another day.

The real trading problem when we look at access and information asymmetry is not HFT but dark pools.HFTs typically all compete for the same pie anyway.

Not a helpful guide as the paths listed almost guarantee that qualifying person would be aware of ways to enter. Would be more interested in the guide "Hot to get a job in HF while being a mainstream web dev who graduated from an unknown uni".

HFT is

1. not a growth industry

2. the regulators are keen to clamp down on this sort of activity.

As such, anyone with the intellectual chops to master HFT, I'd suggest they apply it to greener pastures. Wasn't AdMeld started by former HFT people?

That's what I think of, sure, when I think of where programmers could best use their talent for the advancement of mankind. Not curing cancer or AIDS with protein folding, not probing the mysteries of the universe at the LHC, but finding new ways to make people click on ads.

jgalt was intentionally giving selfish reasons for people to not go into HFT. Apparently he doesn't share with you the premise that people ought to work on whatever "advances mankind."

Reply on news.ycombinator.com