How to Get a Job in Algorithmic Trading
financejobs.co
financejobs.co
However, there wasn't really too much actionable advice in it. I don't think I actually saw any advice as to how to become an algorithmic trader in the post.
I think I actually wrote a better answer in a previous post here:
https://news.ycombinator.com/item?id=8699260
The sad fact is that if you want to get into algorithmic trading you really have 3 choices.
1) got to MIT, get an undergrad in math/engineering, apply to TradeBot, Virtu, Getco, Jane Street.
This provably works as this is how these firms hire, sadly its not very applicable for most people.
2) Get into a small and successful fund that was previously a prop shop( traders without any algorithmic tooling and then start to build it out yourself.
This can work, but its a very hard and long slog. You'll be creating everything from scratch and wont' have alot of people to bounce technical ideas off of. This might be the hardest way to break into the industry, as you'll essentially be creating a new company inside of an existing one, but it is possible, as this is how I did it.
3) Get hired in a technical capacity with a major algorithmic trading firm and move up.
The key here is to not be in a strictly technical capacity for more than a few years. The industry has a tendency to box people into their current roles.
You have to make people aware of what your goals are. Shadow the best traders you can find. Be mentored by the technical talent who writes the strategies. Get as close to the money as possible!
I lied there is actually a secret option 4)
You can go it alone and trade your own money. I really don't recommend this to people as you need a minimum of $50,000 to $100,000 to do this well. Its hard as you won't have anyone to bounce ideas off of or to lean on when times are tough.
The biggest problem I've seen with going it on your own is that since 2009 we've been in a huge bull market. Everyone is making money. We haven't had a challenging market for 5 years so if you've been trading for less its hard to know if its you who is making money(alpha) or if its the market(beta).
I really don't try to time the market but I have a feeling that late next year people who have been trading their own strategies will start to find out what its like to trade in a bear market.
Someone privately messaged me about math. For each of these options, I find math, specifically stats, to be very important. The hard part is getting programmers to learn stats. There is an old Simpsons episode where Homer is trying to learn about marketing. He starts with a huge book on marketing reads it for a few minutes and then goes to a beginners guid to marketing before finally looking the definition of marketing up on the internet.
Don't be afraid to learn math this way. I usually recommend people read chapters 2-5 of Introduction to Statistically learning
http://www-bcf.usc.edu/~gareth/ISL/
If you are flying through it then graduate to Elements of statistical learning http://statweb.stanford.edu/~tibs/ElemStatLearn/
If you are working hard to understand Intro to statistical learning then go to Kahn Academy and spend 2 weeks doing all their stats lessons.
I feel like I'm repeating my self but there are no free lunches. You need to work to learn the material. Don't be afraid to go back to basics.
On his book recommendations, Trading and Exchanges is awesome. Michael Lewis' Flash boys is not, read Scott Patterson's DarkPools instead http://www.amazon.com/Dark-Pools-Machine-Traders-Rigging/dp/...
If you are determined to read Flash Boys then atleast read the counter argument by a HFT https://news.ycombinator.com/item?id=8577237
Its a much more enlightening read and is only a few dollars:)
Regarding Tradebot, they are known to hire from local schools out in Kansas. In fact, Dave Cummings once said it's like half-way to India strategy. You don't have to get an MIT kid and pay them loads of money when you can get a local school kid pretty much do the same.
I used to be in this field, so I can say from my experience, at least, such jobs come out of relationship building with the men of influence. Sometimes it means being ruthless about following up until the right moment comes (e.g. they have a headcount).
But I hear you on that. Having gone to MIT may help.
Also, having a Ph.D. helps, and if somebody wants to go this route I would steer them towards Two Sigma over the competition.
Also keep in mind that they have an image to maintain and some of them are really hurting right now. They may have to look like they're hiring when really they aren't.
(looking over your profile) You might be better off at some of the smaller, specialized HFT shops out there (I hope you like Chicago) if you really know your way around linux kernel programming. That's a really small industry though.
The strategies I actually can get to work yield maybe 10% per year with 5% variance. Allowing me to take out around 5% per year.
For it to pay as a fulltime job and cover office expenses I would have to take out 150.000 euro per year; meaning I would have to have a working capital of 3 mio. euro.
So for me this cannot be anything other than a hobby project.
Somehow, in finance, the impression I always got was that the engineers/algorithm guys were always second fiddle to the traders, and the traders were an incredibly aggressive group of people with a mean competitive streak (part of it I'm sure was the subset of people I met).
It's hard to explain, but I very much enjoy being a nerd among nerds. Being a nerd among finance people always made me feel highly alienated.
That's true, but the VC-funded world is no different. There, engineers are third-class citizens, as shown in their equity slices. (Founders and executives are second-class; investors are first-class.)
If you're comparing working at a hedge fund to working in Google's R&D division, then you're right that the hedge-fund engineers are worse off in terms of status and, quite probably, job satisfaction. If you're comparing the hedge fund to 99% of software jobs, the former wins. Hedge funds are selective enough that they don't need "story points" and "backlog retrospective meetings" and all that "Agile" micromanagement that exists when you're not selective and the managers have no faith in the ability of the engineers to direct their own work.
Some traders are assholes; most are pretty nice people. It's no different from the spectrum you'd see among product managers, tech executives, or VCs. Some are dicks, some are nice, some are idiots and some are competent. It depends on which firm you pick.
Having 60% of one startup, and having to do all the work, is inferior to having 20% of ten startups, and having power but no responsibility to any single one of them. Founders have more power within their specific companies (if things are going well) but investors have the power that matters, because they're diversified and involved in a large number of things.
I do agree with you that engineers (especially the first 4 hires) generally get the worst deal of all.
At small, purely algo shops (ala Jane Street), the traders are your nerds. They're programmers, engineers, mathmeticians, physicists, but they just happen to trade.
The incredibly aggressive people that you're talking about is mostly floor traders, brokers, etc, because they need to be that way, and they're generally on their way out, as algos will be smarter and faster.
I've worked in a variety of industries (in software development), and am currently at an HFT shop, and I've found this to be the first place I feel actually values my abilities.
An algo is basically just a trading strategy (sure, there's a basic set of these) but speed of data collection/processing and execution gives you more options. You can design an algo to hunt other HFTs or not. You can also trade against HFTs if you're not.
This is definitely true but they're not always aggressive. Generally quants are the nicest people I've had to work with in the industry. I've had a floor trader kick me hard under his desk because I wasn't replacing his Bloomberg keyboard fast enough (<2 minutes). That's said, I've have a quant or two be a passive-aggressive dickwad too...but in general, way nicer than floor traders.
This is mostly a stereotype of working at a large iBank. Proprietary trading shops and hedge funds have very rarely worked that way, especially after the rise of electronic trading. In fact, most often now, if someone is coming from one of those environments with the title "trader" they were most likely a clerk that was doing a lower paying job that just wasn't worth automating.
My time in finance generally and algo trading specifically was much more "nerd amongst nerds" than any other development job I've ever had.
Good at coloring within the lines, and grabbing the brass ring, we might say. Nerds they generally are not.
I interned with a hedge fund for a year early on in my career building algo trading models (circa 2002). Since then the fund has gone under and I was contacted by an investigator from the OSC (more or less our SEC) about the exact nature of my work. Thankfully no charges came of it but I learned that I did expose myself to personal liability due to the fact that I had never gained regulatory certification.
In short, don't overlook the laws in this space. They can really bite!
I really don't know. The only reasons I've ever been marginally enticed to do it is that it looks like a fun problem and people make a lot of money off it. But it's hard to beat the job satisfaction at my current workplace. Right now I can release free software, which helps diagnosing and treating neurological diseases.
1) Is alg trading overall a good thing? Possibly. It adds liquidity which helps ensure proper movement of money. Whether that is overall a good thing or not can be debated.
2) Is it a good thing to make a ton of money off of this? This is the question that is maybe a more obvious no. But you can apply the same logic to basically all of financial services. How much is too much?
Your job sounds really interesting though! Are you at liberty to give a bit more details?
I don't like to broadcast too publicly the details of where I work. Email me at jordigh@octave.org if you want more personal details.
This assumes that algorithms are able to deal with edge cases well enough, which I doubt strongly (e.g. Black-Scholes is a massive oversimplification and traders don't even understand that fully) and I don't personally agree with the premise that this is valuable for anyone outside the industry - in particular the distribution of wealth causes severe and increasing problems in society. Martijn also makes good points, although I don't think these are specific to trading.
Your job sounds cool! Nice one and thanks for doing it :)
Blaming the finance industry for it's services is like blaming the shipping industry. The shipping industry doesn't produce anything, it merely generates value off your goods. If it didn't exist however, you'd have to haul goods yourself, at lower efficiency, and without immediate availability. It's not absurd to expect this industry to profit for that. Of course, if there is not enough competition, or if there is collusion between shipping companies, you're going to pay an "unfair" price (for most definition of fairness).
The financial market has enough competition that most modes of unfairness are almost fully compensated for, and whether it is fair will really depend on your definition of fairness; but it's safe to say it's almost fair for most definitions, I believe. If you're trading a major stock like Apple you can be sure you'll trade for a value near the market's best estimate of what the true long-term value of that stock.
Now think of a farmer selling grains. He, an ordinary farmer, has access to the global consensus of the value of it's product at any time: he can be sure he won't be ripped off and can sell for no more or no less than the value of his goods; he can choose to sell at any time, he'll be able to do so instantly. That's a really valuable service.
Why do you never see anyone arguing that a bank balance is just a bit pattern, and if you copy a bit pattern from one place to another you're not actually stealing anything?
You regularly see this argument from opponents of DRM who are justifying the copying of artistic work. Why do they never apply it to finance?
What gives money a special pass on this?
If you can't answer that question without thinking it through - i.e. without just responding emotionally with an 'OMG the world would end if everyone did that' - then you can't make a statement like 'most modes of unfairness are almost fully compensated for' and expect to get free pass on it either.
>If you're trading a major stock like Apple you can be sure you'll trade for a value near the market's best estimate of what the true long-term value of that stock.
There is no such thing as a market. There are only people, and politics, and political relationships.
Finance and 'markets' are just quantified politics, which - like all political systems - happen to privilege one particular set of actors.
>he can be sure he won't be ripped off and can sell for no more or no less than the value of his goods; he can choose to sell at any time, he'll be able to do so instantly.
If you knew anything about farming you'd know that this is exactly what doesn't happen. What actually happens - in the UK at least - is that buyers form cartels and dictate prices to sellers.
Smarter countries like Canada regulate prices to keep farms working. (Forcing a farm to close because of one bad financial year is dangerously stupid if you want to keep feeding your population.)
But manipulation couldn't happen in the pure world of finance, surely?
http://en.wikipedia.org/wiki/Forex_scandal
But that couldn't happen to stock prices, could it?
"alternative business models" Publicly traded companies emphasize stock value because it is their duty to do so. The owners of the company have a right to collectively demand this. If the company fails to promote the value of the stock then the owners will sell their stock and the price will adjust to reflect this increase in supply/fall in demand.
"free public access to data/hyper-speed trading platforms" They are publicly available but they are not free. Data is a good, goods cost money. Believe it or not it takes quite a lot of effort to deliver these products.
"bit pattern" What are you talking about? DRM in finance? You aren't making sense here.
"There is no such thing as a market" Yes there is. People buy and sell companies and commodities on various exchanges. These exchanges are called markets.
"buyers form cartels and dictate prices" The market dictates prices. There are market makers who are legally obligated to buy and sell within a certain range of the free market price. This ensures liquidity and it is not some sort of scam.
"manipulation couldn't happen" Your sarcasm is obvious. People commit crimes, it doesn't mean that the whole system is broken. Often they also go to jail for these crimes.
You can call up a bunch of buyers directly if you want. You might even get a better price for your efforts.
> Where are the alternative business models that don't emphasise stock ownership and stock value?
Nonprofits exist. So do corporations with particular requirements in their charter. What exactly are you after?
> Where is free public access to the freshest financial data and hyper-speed trading platforms?
If you want a consolidated feed at the standard 5s latency you can get it on standard terms at a reasonably proportionate price. If you want dedicated fast feeds there are companies who sell them. No-one's giving it away for free and why would they?
> You regularly see this argument from opponents of DRM who are justifying the copying of artistic work. Why do they never apply it to finance?
Because everyone knows the value of money or stock is in its artificial scarcity, whereas the value of art is supposed to be actual value.
> http://en.wikipedia.org/wiki/Forex_scandal
Forex is a jungle, in part because there aren't public markets and exchanges the way there are for stocks.
> https://ideas.repec.org/a/tpr/restat/v73y1991i2p309-17.html
Huh? That's saying that stock prices fall because the companies the stocks were of - ordinary, non-finance companies - had been price-fixing (and can no longer expect to profit from that now that they've been indicted).
Considering that farming has become uneconomic in my home country (with associated structural problems that will play out in the longer-term), you have really failed to convince me that this is a good thing.
Meanwhile, I personally set my own prices and value my own services successfully without there being a commodity market in tech services (just as farmers do with local distributors, in truth). Is the financial industry really providing this service? If so (which I doubt), what percentage of the financial industry is concerned with such practical matters?
Services are not commodities: a service by one provider is not necessarily indistinguishable from another. Service providers, and producers in general, actually vigorously fight commoditization -- it creates an opportunity for a market, and this improves competitiveness and efficiency. The providers are comfortable with their high margin untradable goods, but consumers get high price variability and discover-ability. Since the service providers fully control the service, they are unlikely to ever allow a market to form.
Now imagine the opposite: imagine servers became fully commoditized. The market then would become highly specialized and efficient: you would probably be purchasing server time directly from the server manufacturer (chosen among many) through some contract and a provider (again chosen among many) would actually host the server and electricity for a cost. There's enough competition in the hosting business that this has essentially happened already: the commodity standard has become AWS. Low efficiency providers have been squeezed out for a long time.
It's hard to compare this to trading grains or iron, though.
The financial industry is 100% concerned with practical matters, since every stock is eventually linked to a real world property, value, good or service. It's a different matter if they are fair, though, as I discussed previously.
So will you go be a farmer?
The answer is no, because farmers are getting fucked. Consumers too - food inflation has been off-the-scale (and that money hasn't been going to farmers). Playing dice against PhDs with algorithms is a sure loss.
But at the end of the day it doesn't really matter. People have traded for profit motives alone for hundreds of years without a social motive. Most of finance is conducted for profit alone. There are tons of businesses out there that are for profit alone. There's no need to look down on people for it.
How algorithmic trading is used, however, is another story. Humans are still the ones who bring the intent to technology, using it for good or evil. But it may interest you to know that a lot of malicious trading in the markets is not done by algorithmic traders, but by teams of manual traders working in concert to place manipulative trades that cause the market making algorithms to move the market in certain ways. (Source: I work at an exchange, and this is what our regulatory and compliance dept. says all the time.)
Market state -> price forecast
Price forecast + other factors (risk, liquidity) -> trading decision
You then manipulate the state of the market to look like that (generally this is "spoofing" or "layering"), wait for the algorithm to respond, and then take advantage of the favourable liquidity it offers.
When people calibrate their algorithms, they use real market data. Most of the time, someone isn't actively manipulating the market, so the model is not calibrated to handle those situations.
I have to say, the regulatory dept. at the exchange I work for does an excellent job of monitoring for any funny business. They have some nifty real-time tools that are scriptable and can replay the state of the market at any point. It's really cool stuff!
I'm hoping that this segment of trading will work to push us in more of the (relatively) transparent corporate model that's practiced by the technology leaders like AAPL, GOOG, etc.
I find it too cynical to do evil just because everyone else is doing evil, although human psychology very much favours the herd mentality (broken window syndrome and so on).
And in general, the objective of a market is to allocate capital in the best possible way.
So, for that, I think that eventually an algorithm - or a market of algorithms - can get to be more fair and more efficient than a bunch of meat-based traders and investors. I'm pretty sure that we are not still there, but, assuming a future in which a huge, benevolent, fair capital allocation system exists, having people working on that might get to be morally good.
What a shop will gain you is a larger pool of capital to work with, so you're open to safer strategies that may only net say 1-5%/mo. (the higher the return the higher the risk, so if you're doing a high risk/reward strategy, the only way to not completely bankrupt is to quit at some point before you blow up, which almost everyone cannot do).
There are a number of ways to pull 1-5% mo. out of capital, which doesn't make it worth the time unless you're well capitalized, so it can make sense to get with a firm, but keep in mind if you have the skills, you don't need to go impress anyone. Your record will speak for itself.
I don't agree. If you trade $25k of your personal capital up to $35k, that's not going to impress a fund enough to hire you if it otherwise wouldn't. You could have gotten lucky. It doesn't prove that you understand markets or the effects of large trades ("slippage") or the game theory and microstructure. If you lose that money, well now you're out by an amount that would be a fair chunk of change for most people.
What a shop will gain you is a larger pool of capital to work with
And relationships and experts that make transaction costs and taxes low, and the ability to colocate and shave milliseconds off your execution time (milliseconds matter; even microseconds matter, these days), and smart people to learn from.
Or I could just be wrong. More than just possible.
Don't try to impress people, or even try to get hired on.
Instead, the goal (should be) to make yourself so undeniably talented that they are trying to impress you to work with them. That may not happen, but if you're really talented, you will make money. You can learn on your own. Takes years, but most of what people "have to teach you" in this area is bullshit anyway.
I agree with your statements. Although, relationships mean nothing (except in service of getting more capital). Mentors can be good. Costs are critical and you can get pretty low on your own, although not as good as HFT - buy why try to even compete there? HFT is not the only game in town however.
- "The Evaluation and Optimization of Trading Strategies" by Pardo
- "The Elements of Statistical Learning" by Hastie et al
I haven't really had much use for math in algo trading so far. I get ideas from looking at historical chart data, code up a base algorithm which implements an idea, and use genetic algorithms to find optimal values for range-bound variables (e.g., a float between 0.995 and 0.997) in the "optimization" step of WFA. I then run WFA across my defined in and out of sample periods on historical tick data directly from my broker (a few gb per year per symbol).
The most complex math I've done in trading so far has been writing some R scripts to generate pretty graphs. It's a stretch to call that "math", though.
WFA completely changed the game for me. I was aware of curve fitting and tried to avoid it before, but after reading about WFA it was like the wool covering was removed from my eyes...
Any suggestions on applied math? I suspect I just haven't been exposed to enough of it to really connect the theory to the real world.
With experience and practice with theoretical math, you also learn how to apply it.
Tool makers have more insight into the assumptions behind the tools they create -- they might know how a tool was meant to be used and what its limits are. That being said, there are times when using a screwdriver as a hammer is expedient and does no harm. Of course, there are other times where using Black-Scholes to model a high-volatility market outside of the 'smoothly differentiable' market assumptions it was built on can crash a large part of the economy. It can be a dangerous game to use someone else's tools without knowing their assumptions.
As for applied math suggestions:
1) Watch Feynman's take on applied math in physics: https://www.youtube.com/watch?v=obCjODeoLVw
2) Read this essay "A Mathematician's Lament" which points out the math learned in school is likely not really mathematics and that surprisingly math is not practical but aesthetic -- mathematics is closer to art than we are taught: http://www.maa.org/external_archive/devlin/LockhartsLament.p...
Do these programmers have a short career span. (lose x amount of dollars and you need to leave.)
How much could someone make in this field. Software engineers regularly make 100K so do these guys make more to justify the risk?
Salaries in finance are low compared to comparable jobs. Bonuses can be very high. A senior programmer who'd make $150k in tech will probably get $130k in a hedge fund, but the bonuses... can be anywhere from 0 to 1000%. Median is probably 30%. Market conditions play a role.
Sadly, compensation has been getting worse for quants over the years. It used to be that you could take a $500k all-in for granted just for knowing college-level math and programming. That's no longer true.
Do these programmers have a short career span. (lose x amount of dollars and you need to leave.)
No worse than startups. It's probably better, to be honest. The age discrimination isn't nearly as bad as in the VC-funded world, people are less-often fired for stupid reasons in hedge funds, and the ethics are generally better in finance than in the VC-funded world. Finally, the lower status of programmers may be more visible in finance (i.e. you're clearly second-rate compared to traders) but less onerous. Engineers are third-class citizens in the Valley, with their 0.01% equity slices of post-A startups, and because the VC-funded startups aren't very selective and a third of the programmers are incompetent, they feel a need to use "Agile" micromanagement that you wouldn't see at a hedge fund.
If you lose serious money (meaning hundreds of thousands) doing something stupid, you'll get fired. If you're unluckym good funds won't fire you. A lot of statistical arbitrage is taking "51/49" bets.
How much could someone make in this field.
The upside is very high. That said, typically it's not going to be more than 25-50% more than you'd make elsewhere. You need to manage your career aggressively and become someone's protege if you want to get above $300k... but it can be done, and some people make 10-50 times that.
What percentage of a hedge fund does a typical engineer get?
In fact, when you hear about giant bonuses in finance what you are often really hearing about are groups that traded salary for profit sharing and it paid off. In the context of finance, this comes off as shady or outrageous but in the context of startup hits it doesn't.
Lets just say, having worked under option based equity arrangements and profit sharing arrangements, they both had their positives and negatives, but only ever felt like a system was stacked against me, as an employee, when equity/options were on the table.
The big issue I think with the equity based model, is the golden handcuffs it applies to early employees. Not only do they frequently forego market based compensation for long periods of time, when they make the decision to stop doing that, they are asked to take on more downside in order to keep any part of their deferred compensation. For all it's downsides, the trading environment would never ask someone to do that.
My immediate reaction to this point, though, is:
* The downsides of equity comp are shared by all roles in a tech company, not just engineers; the exceptions are the very most senior management, and the tiny cohort of founders. Both of which are exceptional at all firms.
* Dev jobs in finance are, I am pretty sure, a better deal than those in tech startups.
* But let's not move the goalposts: the argument is that engineers are second-class in tech startups in a way that isn't true in finance companies. Most engineering jobs at finance companies are cost-center roles.
I think this will vary based on the size of the firm. Larger firms that trade in multiple markets and in different asset classes or firms that become registered broker dealers will tend to have larger back office operations. Smaller firms will outsource as much as they can so that the engineers are primarily focused on supporting whatever strategy the firm wants to trade.
As I mentioned, I do not think it is a fair characterization to say that engineers are more/less valued in either situation. I just think that using equity distribution percentage to argue one way or the other is problematic in that compensation is determined in 2 dramatically different ways.
That said, if I were going to argue anything, it would be that the equity options based compensation packages in tech startups seem to have more opportunity for abuse. So characterizing engineers as 3rd vs 2nd class isn't interesting, but characterizing one compensation structure as more exploitative than the other may be, and I think that is at the heart of what the original post was about.
[edit] I'd also add that any characterizations about the methodologies used in finance/trading vs tech startups is completely inaccurate. I've seen great development process in both places and the converse as well.
I don't think programmers in the field have an appreciably shorter lifespan than in other industries. I don't think that working in an algo trading company is intrinsically any riskier than working in any other kind of firm.
The hours tend to be longer and the work tends to be slightly more stressful than if you are working at a generic bigco, however.
Me personally, I think next year is going to be great. Vol should pick up with the Fed actually tapering and VIX returning to its real median of 16-20. So I think strategies that sell vol could be more effective. Here's a study on Quantopian for that: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=225532
I'm curious about how the rising interest rate will affect things such as gold and US treasuries; obviously, it should drive the yield/prices down as US dollar becomes more attractive. But specifically, how to pair trade it; here's some study on pair trading TLT and SPY: http://seekingalpha.com/article/2714185-the-spy-tlt-universa...
Finally, pair trading GDX and GLD; and also oil futures and oil refining/exploration companies. I'm not very familiar at all the dynamics there, I was told that the correlation of producer to spot is like an option being in the money, if oil or gold drops down or up to the threshold of break-even for producers; that changes the pricing dynamics of these companies. Anyways, let me know what you guys think. Thanks.
Some of these funds have a passionate hatred for "job hoppers" are are inaccessible to almost anyone in tech. It's normal to change jobs every 2 years. Not getting promoted? Switch. Your manager left for greener pastures? Follow him. Hedge funds do a better job of internal promotion, so you're more likely to end up on a track that would merit a longer stay, but they tend to view the average tech CV negatively, because they're extremely paranoid about IP and changing jobs every 18-24 months until you get lucky and "click" with someone powerful and are on a protege/leadership track (which is what you have to do, in tech) is frowned upon.
Learning finance is important for showing an interest in the field (and not just the money) but most of the hard effort is going to be in learning statistics, computer science, microeconomics, and technology.