High-Frequency Programmers Revolt Over Pay
forbes.com
forbes.com
When a welder helps to put together an oil rig that then either makes millions of dollars or explodes, sinks and causes billions in damage the fact that he did it as a salaried employee shields him from the damage just as much as it will insulate him from taking a share in the profits.
That's why it's work for hire. You get to decide up-front if that sort of thing is what you want.
So if someone offers you the opportunity to program a computer and make that company millions of dollars you are being compensated for your time, not for how much money your software will make.
The shareholders of the investment bank and the people that thought up the spec for the thing that you are building will have a much bigger claim to the profits than the guy that codes it up, and not surprising, they're the ones that will eventually make more money on it than you.
Why programmers should be different in this way from welders is not clear to me. Everybody that works for 'big-corp' makes the same kind of deal, and if you didn't make more money for the company than you cost you probably wouldn't have a job to begin with.
Chase Manhattan bank, the Netherlands branch, formerly the Nederlandse Crediet bank, about 3 years from when I was 19 to when I was 21, both as a systems level programmer as well as an application programmer.
> However they are conflating programmers and quants here
Yes, but I didn't, I only spoke about the programmers. They do not come up with the models, they just lay the bricks.
The algorithm developer has a different role here and is more than likely not the same guy as the one that codes it up.
a) Understand the model.
b) Come up with a fast algorithm for the specification.
c) Prove the algorithm correct.
d) Make no mistakes implementing the algorithm.
EDIT:jacquesm, do you have a voting ring or why are your posts at 2 points (and mine at 0) the instant you post?
I can do both (lay bricks and code), and I think there is a distinct parallel between laying bricks and writing code. Both use simple, re-usable building blocks to create very complex systems that need to be built to high standards if they are expected to stand the test of time.
The equivalent between the architecture and the algorithm designer is a similar one.
To demand a share of the cake just for laying the bricks tells me that you should not be doing work for hire but that you should be working as an independent contractor or as a starter-upper. You can't expect 'none of the risks and some of the gains' in a position like that.
A 'mere' programmer, just like a 'mere' bricklayer can be replaced by another programmer or bricklayer.
An architect or an algorithm designer contributes something uniquely theirs without which an endeavour might not happen at all.
Programmers are a rare profession in that they seem to think that somehow their contribution to a project is unique enough that they by the simple act of transcribing a specification to a working prototype have become co-owners, no other discipline suffers from this misconception.
If you want a slice of the action put up your money, but don't just put in your time, expect a paycheck and a slice of the cake if all you do is code, you are more than likely to be found 'expendable'.
you are saying coders should be treated the same as bricklayers (salary); but are algorithm designers treated same as architects? I believe the former are paid percentage and later paid salary.
It seems financial sector do not follow the same rule as construction business.
If there is anybody on HN that has an aversion against voting rings it probably is me, and I wouldn't ever stoop so low as to cheat on something as silly as a conversation on a website (or anywhere else for that matter).
Whoever voted for me is free to step forward, I swear I haven't a clue who voted me up and you down.
edit: This sort of ticks me off by the way, within two days I first get this character calling me a troll: http://news.ycombinator.com/item?id=1556756 , now I have a voting ring? Way to go).
edit2: and if I had a voting ring my posting http://news.ycombinator.com/item?id=1560732 would not be stuck at 3 votes ;)
Wealth is all about percentages and the standard profit percentage share for professionals and engineers, "big-corp" or otherwise, is 0%. These programmers left to start their own trading companies, likely because the salary bumps being offered (a measly $45,000 extra on $150,000 base salary, in the example) were insulting compared to the revenue being generated directly from their work. That's demonstrable value. Consider: a 1% profit share from the example in the article would be over $300,000 per year. Not a large bonus at all for a financial managers or executive, but not an outlandish bonus for a highly valuable member of a lucrative enterprise, either.
Your last paragraph sounds a lot like, "keep your head down and be thankful you have a job." I assume you're just defending the status-quo and pointing out the tradeoff between sharing the risk/reward and being comfortably salaried. You can't possibly be implying that programmers or other skilled professionals shouldn't even try to negotiate better contracts, right?
Take A/B testing, analytics, conversion optimization, etc. If you are good at these, you can generate several million dollars of value over the course of a week. If you can credibly offer the prospect of results like that, some companies will pay you very well indeed.
I'm really curious about non-startup work models that would encourage this. So far, the easiest path would seem to be via consulting, where you sold your services as business services that happened to be software-based rather than as a "warm body" to staff some open slot somewhere.
It seems that overall, startups are probably lower expected value to the early employees in pure financial terms than having compensation tied to measured results for work done for some other company.
I remember essays by both PG and Joel that mentioned the "value effect", PG talking about a "monster of productivity" hacker who added a bunch of value to Viaweb in a single day and in Joel's case, a summer intern who (I vaguely recall) suggested and then built the joelonsoftware jobs board.
I think fogcreek offered the intern some type of stock option bonus for joining fulltime. I don't remember if the Viaweb hacker got something or already had equity of some sort.
Many sales positions are like this (eg. Enterprise Software).
The key of such pay model is being easily measured down to the individual (which sales is). It doesn't apply to most cases because it is hard to accurately measure contributions of individual's performance of a product/result that is comprise of a team of peers.
The "body shop" biz model is also easily measured (sit on seat 1 hr = pay for 1 hr).
You can also have situations in large corporations where your performance was incredible, and by every measurement you brought millions into the company. However, some guy in division X lost billions and the stock tanked.
There was an AIG employee who wrote to the NYTimes about how his bonus was retroactively taxed 95%, and he claimed it was because of this exact situation. He had nothing to do with AIG's collapse. He was part of a highly profitable division, and was asked to stay on to help turn AIG around faster. That people were upset that he got his contractual 1.2 million bonus was lost on him.
Suppose the website goes down and the company doesn't make its millions and now they can measure no performance increase.
Then there is politics. Big Boss Man might go to his Big Big Boss Man and tell him that he deserves the pay rise because he hired you or brought you in as a consultant, and so you're just a tool to his genius. Good luck with that one.
My point is simple: if the bosses or clients don't want to pay you performance, they can find a million ways to do so. Tough contracts only go so far and my suggestion is to never enter such a relationship unless you're 100% trusting of the bosses or clients.
Again, for these programmers, who are literally in the business of making money, its a great approach.
Given the above statement, which I think is accurate, and given that you're pretty good at those things, and given the following statement from your interview:
> You know what my revenue was for today? Nothing, because we’re in the dog days of summer, and sales slow to a crawl until school gets back in session.
Doesn't that mean that your time would be more valuable employed doing the "million dollars of value in a week" stuff?
Also, I don't think anyone minds if you mention that you do consulting; your feedback here is very valuable. One of the things I appreciate about this community is that self promotion is ok as long as it's not completely out of context or otherwise annoying.
Sometimes these are the same person, but in those cases that person almost always has a profit sharing contract, not only a base salary. (And if they don't, they're crazy.) The fact that the programmers in the article only had base salaries leads me to believe that they weren't the actual creators of the trading algorithms, so they don't really deserve a slice of the profits anyway, because someone else a) created the profit machine and b) is taking all the risks of running it.
What really happens is that these programmer guys learn the trading secrets after a few years on the job, then depart to a different firm to recreate the machine themselves. There's no oppression or revolts here.
The difficulty of HFT is designing a machine that can trade fast enough. I'm not sure you realize how difficult this is. You just can't take a quant and make him an über C++ programmer overnight.
There's a reason why you need people with different skills to make money, and the reason is that becoming really skilled in whatever field takes years.
Good companies pay everyone making a direct contribution to the profit a fair share, those who don't lose their talents.
The bank's customers are taking the risk of running it not the traders - it's not the trader's money
Nobody likes to be screwed over, but most people take it in exchange for other perks (job security, for one). But people have a screw threshold, and if you cross it, this is what happens.
Now, this is not the main reason not be a jackass -- on the other side of the spectrum, you usually get much in return for being nice (if only for just differentiating yourself from everyone else) -- but that's another story.
The condescending tone of this line really bugs me.
Count your blessings. At least the article didn't start with "Joe Schmoe, a programmer, is setting out to blah blah blah" and then end with "Back at blah blah, Joe Schmoe is still programming away, sure that millions are just around the corner."
This is why I would never take a job as a programmer in the trading industry working for the big firms. I have dignity, and it would never cross my mind to put up with smug bastards taking this attitude with me all day, no matter how high the salary. It's not that the software is making people tons of money, it's that nobody respects that the software is making people tons of money (the side effect of respect, of course, would be fair compensation).
Honestly, I do know several people working in finance (Perl is apparently heavily used in that industry, and we're a Perl shop, so we meet folks in that industry quite often at conferences), and they generally seem happy with their work and their jobs. It's certainly an interesting field, and one in which you'd get to work on a massive scale, which is usually fun. I worked in the oil and gas industry for a while, and they've got big data (terabytes of it for a single well, for example). It's definitely interesting.
But, the big money guys do have a reputation for supreme arrogance, and recent events in our economy have not made them seem any less so.
But listening to management about job security is absurd, at least these days it is.
I'm not saying I don't want the guy to have a higher salary, but there's an implied fallacy here. It seems he should be paid relative not to how much value his code generates, but to how hard it would be to replace him.
He found a partner, and together they began trading on their own. The programmer now pockets more than half of any profits his software generates. The programmer says he's making about the same money he did at the job he left. But at his old job he'd topped out in pay while now he says the sky's the limit
We are market-makers (MM), so we don't care all that much about forecasting / direction. We want to fill order flow at the cheapest price that we can make a profit on. All the competition in our little MM niche of High Frequency (HF) trading revolves around a fight amongst market makers to give the best price possible to customer orders. This leads to very tight markets. That works out very well for customers.
I don't fault previous employers for paying us what they did as the article mentioned. It's a lot more than I ever expected to make coming out of college. The article seemed to have a programmers versus industry slant that I don't quite agree with. In my opinion industry is being taken over by programmers. Companies have a natural upper bound they can pay any employee.
After that, and I have been on both sides of this, either you can accept the comfort of a regular paycheck or you can throw that all away to take a risk and grow in a different way. If you take the risk you're throwing away a sure thing for upside. I don't have a family yet so to me it was the right time to do this.
The `programmers revolt` has been over for years. Programmer's won. Markets are all electronic or will be soon. It is inevitable and good that this happens, in the same sense it is good that we put robots into factories, use statistics to optimize business processes, etc etc.
It's been a longer road to getting to this point than the article mentions, my first bit advice for someone who in the trading industry and wants to branch out on their own, is it's going to be hard, just like any startup. The money you see the company you are working for making is the result of a lot of work, that you just can't appreciate until you have to do it all yourself from scratch. Which we have, twice. The article was a bit off on this, we already had our first `failure` and are trying again. This time we learned to keep our IP.
With a startup, we've had to wear all the hats that as employee we didn't have to think about at all ourselves. It's a combination of awesome, daunting, miserable and satisfying, like any challenging endeavor. Personally I find creating something from the bottom up a lot more rewarding than grinding out a paycheck.
Is it worth taking "MFE" style classes, like the ones offered at Baruch and NYU? Are the systems Windows or Unix? How much "advanced" math do you need to know? How much high performance infrastructure do you need to know? (networking, specialized storage & I/O, etc.) Is there a way to go directly to a startup firm, rather than first working at a bank or larger hedge fund? Are languages other than C++ used? Is it easier to do this in New York or Chicago? What bars should I hang out at in order to bullshit my way into an interview or partnership?
I never took an "MFE" class in college. I graduated with a CompE degree, taking most of my electives in CS. Math is important, primarily statistics. Being a good coder and loving to program more so. Having a natural analytical bent, and being able to wade through data / formulate then test conjectures, and appreciating how markets will never stop surprising you, even more.
A lot of companies use C++. I don't like it myself, & we use Scala, as functional programming + oo works really well for trading specific coding. Also our last place was Java based, so we have a lot of experience with that. However there are plenty of opportunities no matter what language you use. I know of firms using Python, C++, Java, C#, OCaml. So it's pretty wide open.
Speed is important, but usually not machine level instruction important as people often think. Usually it's more about understand big O and not overwhelming your critical paths / pushing things off to other threads.
As for finding a job doing this stuff, google "Chicago Prop Trading firms". The first result lists a ton. Goto their websites and apply. Or you can use a head-hunter.
I have no experience with the New York part of things, but it's probably similar. Chicago does seem to be a good incubator for starting up a trading company though as the CME is located here and it's a great place for big and small companies to trade (IMHO/YMMV).
As for going directly to a startup, it's tricky, we don't have the budget to pay for more employees at the moment, and I know a lot of start ups are in similar situations. If we do hire someone it is usually below what a prop firm would pay them and they have to be extremely experienced in the area we hire them for.
So, I would suggest prop firm first route, it worked for me and I wouldn't be where I am now had I not gone that route.
I have an interview at one of the NYC prop shops, but it's for something more back office-y. Would it be a mistake to take this job, thinking I could move into something closer to the trading later? I've got mixed advice. Some people say it isn't too hard to shift, others say it's impossible.
As a Direct Market Access developer myself, I wonder what is your environment (target exchange(s), api/transport/codec stack, market data provider) and how time-costly it is for you to 'just connect' to the market (aka, having a platform ready to trade, without the algo stuff).
Could you elaborate a little please?
The current structure for valuating securities does absolutely no good for our society. Not that it's overly evil or anything, it's just pointless, a massive waste of time and money, and is a cancer on our economic system. It's got to be a thrilling thing to code for though.
Your thinking represents a common fallacy: "I cannot immediately see any benefit to X, therefore X is pointless / should be abolished".
You are right about the feedback loops though.
Prescribing what someone you've never met knows or doesn't know would be, if not foolish, presumptuous.
One definition of liquidity is when you can sell something without affecting the price much. Most people on Wall Street will tell you their job somehow increases liquidity -- connecting buyers and sellers in more and more efficient ways.
HFT seems different. It is comparable to front-running other people's orders. Someone tries to buy an item for $1.00, and the HFT algorithm tries to grab the item first and resell it to our original buyer (and other people in the market) for just a tiny bit more.
From my perspective it's effectively a sort of tax, like a bridge toll. It seems to me like this has to make every transaction affect the price more, not less. How does this increase efficiency or liquidity?
In the stock market, just like the real estate/auto market, there are never just true buyers and sellers; there are also brokers and dealers who keep a inventory of goods, so that they can sell to non-discriminating retail customers who just want the goods right now. Their expertise and self-interests in turns, affects the price of the good in the whole entire market.
The stock market once have had traditional broker-dealers that controlled the spread of a stock. That is, dealers are willing to buy stocks from impatient traders who are willing to sell at a market order at a price that is set by the dealer; the dealers then in turn, later sell their inventory at an artificially inflated price to impatient traders who are willing buy the stock at a market at the price named by the dealer.
Traditional dealers, in order to tack on the risk of carrying their inventory of stocks (after all, a stock could theoretically drop to zero before they could sell the whole lot), keep the spread of the stock big (they are willing to buy low and sell high) at the expense of retail investors.
HFT, due to their high-tech platform and high execution speed tightens this spread because they don't carry as high a risk of inventory; because their execution speed is in timescale of micro-seconds; so essentially, in the timeframe of seconds, they buy a position and then sell that position subsequently and not really carry that position through for the market to affect the value of that position.
This in theory is good for retail investors as they actually end up paying less for their stocks. But with good technology, you could also use it for bad. With fast computers and sophisticated algorithms, HFT traders could learn how to game the traders and big funds that they are suppose to server - just like a regular car or real estate dealer.
If you know where the consumer demand is, you could buy up all of the supply ahead of time and artificially inflate the price to make people pay more. If you know that your competitors are replicating your every move, you could deceive them by making a small trade against your true intention, have them jump on the bandwagon and swiftly punish them by executing your true big trade afterwords. The possibilities are endless.
No. The matching engine will match first the highest priced order, and in the case of orders at the same price, whichever order was placed first. You can't jump ahead in the queue, no matter how fast your algorithm is [1].
[1] This statement only applies to US equities/futures/derivatives markets. I think it might be possible in Canadian markets under some limited circumstances.
(There is a "don't route me" flag if you don't want to be routed. In that case, your order will simply go unfilled - exchanges can't match you at a price worse than the best price available on all exchanges.)
HFT seems different. It is comparable to front-running other people's orders
If INET were to flash your order, they give some HFT firm the opportunity to sell you shares at 99. The HFT firm can accept or reject - if they accept, you pay 99/share + commission and INET gets the commission.
That's a completely different game than front running. It has other issues, like being potentially unfair to ARCA and traders who can't afford flash orders, but it isn't front running.
Both negative AND positive feedback loops independent of underlying value are bad.
When using analogies, some explanation is helpful.
The difference with HFC is they don't provide any actual liquidity that matters. They don't hold the apartments across different move in dates.
And what, exactly, is the "tank's holding capacity" for liquidity?
Metaphors are rarely a useful approach to understanding something.
[0] An amusing position for a programmer to take, considering we make completely intangible stuff while spending hours a day sitting on our asses, but one that a lot of people on this site seem to hold.
What is the definition of "real market participant"?
For example, are day-traders real market participants? If not, why not?
How about me? (I rebalance every few months, maybe.)
Note that day traders and I have exactly the same goal - make money. How many people trade for other reasons?
If your job is to record or settle trades, be a day trader with no overnight position, track share holdings, etc. then you are a part of the machinery of this capital market, but you are not one of the participants who provides, or consumes capital. You are part of the cost structure.
The real participants are those who actions reflect beyond the casino itself, and out into the economy beyond.
The purpose of share markets is to provide capital. Long term capital for companies to invest, and in return to provide long term profits for those providing that capital.
If you are providing or consuming capital, even for one day, you are to some extent a real participant.
I think it is fine for people to make money, brokers to charge a fee for a trade, day trading, advising on diversification, etc.. But we should understand that these are costs associated with this system of providing capital/investing.
You sound like you provide capital and participate as an investor.
I'm not buying the argument that 'HFT' provides any meaningful additional liquidity.
Let's say all subsecond trading is abolished tomorrow, and rather than a smooth curve, all asset prices step at 1-second intervals.
What value was destroyed there?
If none, what value are these guys creating? None?
That will tend to expand the bid-ask spread. That hurts both buyer and seller.
Scenario 1:
So someone offers (is willing to sell) a limit order at $100. Buyer bids at $110. They wait until the other order appears and cross according to some market rules (perhaps at $100, $105 or $110), no-one else gets anything.
Scenario 2:
HFC works out the trading is in range $100 to $110, puts bids in at $104, offers at $106, and makes $2. The sellers gets $104, the buyer pays $106 and both are better off than scenario 1?
(I'll note that the bid/ask spread becomes effectively zero every time an actual trade happens -- what if it actually reflected market prices rather than a bunch of video games competing against each other?).
Does it hurt the buyer and seller more than the little nibbles these guys are taking to smooth out that curve?
I see a lot of paychecks being cut in this industry and I don't see the value being created. Looks like highway robbery to me.
Why are you assuming that the buyer and seller are hurt?
> I see a lot of paychecks being cut in this industry and I don't see the value being created. Looks like highway robbery to me.
I don't see the value in smurfs, but don't think that they're "highway robbery".
Liquidity, on the other hand, has huge value. Disagree? Try to sell an illiquid asset.
If they overlap very briefly (say, across different markets) and nobody else has noticed yet, a trading program jumps in and arbs the difference. That increases the spread.
Utter nonsense. In the case of crossed markets it is illegal to trade. Trading under these circumstances would violate RegNMS and most matching engines will reject orders that would cross the markets.
HFT firms reduce the spread due to competition. If the spread is $0.03 (say $9.97 and $10.00), I can do one of two things to be at the top of the order book and be the first to trade. I can either place my order first, or I can place an order at a higher price. If I'm the fastest, I have the earliest order at price $9.97 and I get filled first in the event of a trade. In this case, I make $0.03/share. If I'm not the fastest, I can still get to the top of the book by placing an order at $9.98. This reduces the spread to $0.02, and I can only make $0.02/share.
Investors providing capital should not care what time of day their trade goes through because it will settle at the same time anyway. So this 'liquidity' is worthless.
The purpose of shares is to raise capital for productive investment, in return for income to long term investors who provide the capital.
Both non-computerized traders, and computerized HFC traders have an obvious cost, they extract return from the markets that would otherwise go to investors. This reduces the returns for investors.
What if we create an exchange where market participants submit orders which are crossed once a day? You establish a fair matching system and clearing price algorithm. In that environment there is less money going to minute traders and so there would be better returns for investors.
If these guys want to spend money and time on start-ups that will likely be out of business before they come online, that's no skin off my nose. In fact, please excuse me while I laugh. These guys are basically the last players in on the ponzi scheme.
While I agree with you that HFT is a scam, I disagree with you that it's a ponzi scheme. It's more like ticket-scalping, so the scheme is going to go on forever, as long as SEC allows it (which they will because the sell-side lobby groups will label themselves as liquidity providers that tighten the spread) and normal people are trading.
What amuses me about Main Street's outrage on HFT is that they suddenly take this expose as a new revelation that Wall Street is screwing with retail investors when the big prop shops/broker-dealers have been raping retail investors and pension plans/retirement mutual funds for decades. HFT is just the latest instrument of exploitations.
Anti-scalping laws are a good example of the government intervening in a market to prevent natural price discovery, and to hand advantage to the sell-side.
Perhaps you were thinking along this line: that it discourages venues from offering coupon clipping discounts. It won't though: a venue has a fixed number of seats available, and will attempt to maximise its profit for this available volume of seats. If they're going to have leftovers, they'll find ways to discount out what's left.
Scalping works against a venue practice where they charge higher prices for last-minute tickets with an intent of selling less-than-all the available tickets but netting more profit due to the high charges. With scalping, they compete with the guys out the front gates and last-minute sites on the internet. It now becomes more attractive for them to try and sell all the seats, rather than charging a lot for just a few. Hence, scalping is helping reduce prices, and creates more places at the venue.
Separately, if you got into a situation where you had to choose between seeing Madonna on stage and eating you can sell your ticket and buy food with what you get.
Interesting: scalping becomes less lucrative when it's legal because venues know they are unlike to get away with things that they will do well on when it's banned. As a result, the market won't exist.
Perhaps you have other scenarios I haven't thought of though, if so I'm interested to hear.
I'm suggesting fans will be turned off of performers / entertainers / sports teams / etc. by the perception that they (the performers) are overcharging for their performance, and that will hurt the longevity of the brand, and the sales performance of related products.
Similarly, if you drive out youth and cater for wealthier older people in the interests of maximizing short-term revenue, you endanger brand engagement 10 or 20 years down the line - this is particularly important for sports clubs.
- honest price discovery. consumers don't have to submit to price segmentation.
- the venue are more likely to get rid of risk on the sale of tickets quicker (a bit like forward)
Can anyone recommend a source of data on the cheap so I can papertrade? Clearly realtime data feeds are going to cost... but isn't there some public repository of historical data someplace?
We just released a public API: http://apidocs.s3-external-3.amazonaws.com/index.html
6085.51 | 2010-2
6375.40 | 2010-3
51072.15 | 2010-4
194208.93 | 2010-5
111775.68 | 2010-6
65577.25 | 2010-7
The spike is from the world cup. We will probably see a similar spike in August when the premier league starts.
Their website also has a description of the format: http://www.nasdaqtrader.com/content/technicalSupport/specifi...
For example, if I notice that there are several large orders buying a stock, I can assume that a large institution is trying to purchase a position. If you reach this conclusion quickly enough, you can buy up enough stock, temporarily hiking the price, and selling the stock back to the large institution as they are trying to fulfill their purchase order. Do this enough times, and for large enough orders, and you can make money.
Deleted comment
The exchanges are not allowed to fill a sell order at $9.99 with a buy at $9.99 if there is a public buy order for $10.00 on another network - that would violate RegNMS. This is called "crossed markets" and no trades can occur in this situation.
Not of what they made in the past, but they can certainly terminate the agreement and change the rules from now on (assuming that the boss approves, obviously).
What you should be worried about is your pension fund actively trading, which it is doing because it is being managed by someone who takes ~2% off the top every year and who has to justify this by showing they can beat all these damnfangled machines at stock selection. They'll almost certainly underperform the index over time and overcharge index funds by about, oh, 180 basis points or so, precisely because they insist on conveying your wealth to marketmakers every time they trade.
Example, I'm Fidelity Bob Fund Manager; after extensive research, I decide to make a decision to buy 200,000 shares of AAPL. So I enter my buy order into my OMS screen, which slices and dices these 200,000 shares into blocks of 100 shares (as for traders not to front-run me, because a naked buy order of 200,000 shares will drive the price up and I'll overpay long after my transaction is completed).
HFT program meanwhile is actively scanning the quote book on all market centers for patterns of such a huge "whale order". Bob Fund may leave discriminating traces of evidence, such as executing the same blocks of buy order under my unique MPID multiple times. Or the HFT program is hooked up to a dark pool where it's constantly sending out small sell orders for 1 share to try to sniff out and match Bob Fund's huge buy order. Or it just might be that the HFT program has reverse-engineered the Bob Fund's VWAP algorithm for slicing/dicing orders and can detect how the orders are being sliced/diced under any given market conditions.
Once the HFT algorithm is certain of a "whale order." It'll actively go out to every single market center, buy up all of the remaining AAPL liquidity on those markets and then present it to Bob Fund (voila! I'm providing you with liquidity at artificially inflated price!). Mr.Bob has no choice but to buy AAPL at a maybe a penny or two higher than what he could have paid for without HFT.
The flip-side of the argument for HFT traders are that perhaps, they are better dealers than traditional dealers. For all we know, there might not be any interested retail investors who are willing to trade with Bob Fund; so Mr.Bob ends up trading with traditional dealers which have traditionally higher spread on the price of their stock, as they take on more risk as a dealer because they don't have the same execution speed and high-turnover rate as a HFT dealer. Also, HFT increases nominally execution speed and brings all market center's prices in line, as if there is a NBBO on ARCA and Bob Fund's order is on BATS, HFT will essentially act as the middle-man to match the true best bid and offer together, in the quickest time.
However, the other counter argument against HFT is that it actually doesn't really provide liquidity to the market, as evident by the June flash-crash. During a situation such as this, broker-dealers are suppose to step during irrational exuberance and depression and trade with irrational investors and put a stop to the crash/rise. But HFT dealers stopped trading that day, triggering a lot of people's stop-market orders and hence we saw ridiculous sell orders of Accenture at $0.01.
So I'd say, HFT is a mix-bag.
The sophisticated Bob fund is trying to fool the markets into thinking that demand for AAPL has not increased. I.e., Bob is trying to keep inside information hidden and trade in such a way that he captures all the profits from this information himself. A predatory trader (algorithm or human) will bid up the prices, thereby capturing part of the gains of trade for themselves and part of it for the other market participants.
So basically, while Bob pays more, the retail investor Bob is buying from receives more (and of course, the predator takes a cut).
Who should SEC and public interests side with? Bob, the retail investors Bob trades with, or the HFT/professional broker-dealers that Bob trade with?
Bob would argue that public interests should ultimately place the highest priority with him, as he represents the "pension fund and 401(k) accounts of the hard working American public." (Never mind the fund fees, soft dollars & insider trading)
The retail investors argue that public interests should ultimately rest with them. What how do you define retail investors? The long-term investors that trade blue chips and the day-traders with IB accounts? How are the day-traders with IB accounts different from professional broker-dealer except that they are typically less sophisticated but both out for blood? Should you really let the inmates run the asylums.
The broker-dealers argue that public interests should side with them. Because they are the psychiatrists of the asylum that could orderly organize the inmates, make that the mental hospital is well-run and efficient place. But can you trust people when they have so much more leverage and power at a place? And but if you do away with them, wouldn't mental asylum collapse?
If the dealers don't want to trade, they just make their buy and sell limit orders to be way out of the normal price band. Hence, I want to buy Accenture at $0.01 and sell Accenture at $1000. Yea, I'm meeting my exchange contractual obligation, but who's going to trade with me now?
What happened during the flash-crash is that people's market stop-orders were getting triggered after the initial crash, when the market went down 20%; when all other human traders stepped out of the market, but there were still tons of market stop-orders to sell; guess whose limit buy orders were matched? Accenture buy order at $0.01.
To be fair though, a lot of those ridiculous trades were later busted by the exchanges. But the irony of the whole crash is that the HFT traders actually fanned the fire on the crash and then later rode the wave all the way back up (for the trades that weren't busted anyways).
You are conflating two separate groups:
A) Conservative HFT funds who left the markets due to a fear of broken trades.
B) Risk taking HFT funds who stayed in the markets, hoping to make money off volume/higher spreads.
Group A "fanned the fire" by pulling liquidity out of the market. Group B "rode the wave", and in the process mitigated the flash crash [1].
It's a fallacy to lump all HFT firms together.
[1] The market crashed and corrected itself in 10-15 minutes. Without HFT firms, it's likely that it would have corrected itself much later (if at all).
Playing games with prices is not providing liquidity.
If you look at http://batstrading.com/ and click through to the fee schedule, you see that Bats pays liquidity providers $0.0024 rebate per share traded.
BATS is the latest new-comer to the game and it attracts volume to its exchange by attracting HFT firms who have the highest trade volume with rebates. In fact a lot of these firms don't even try to make money anymore, as long as you have a strategy that breaks even and you collect rebates and trade enough volume. You make a profit.
The definition of liquidity becomes pretty fuzzy here when you have HFT take away liquidity from one market center to offer it to the next depending on how various rebate structure will make them the most money; and exchanges themselves in collusion to pump up their trade volume.
As far as I can tell, this is the only argument presented against HFT.
> But HFT dealers stopped trading that day, triggering a lot of people's stop-market orders
Are there other factors involved in this outcome, or is this a canonical HFT failure?
Arguing that HFT is bad because people sometimes stop doing it abruptly kind of suggests that HFT is good when people are doing it.
The argument is that HFT provides liquidity precisely when traders don't need it. You need broker/dealer to step in and take the other side of the trade when someone wants to trade and there isn't anyone else willing to trade. The issue with HFT is that, as the predatory example provided above, that HFT is actually buying up your liquidity in the market and selling it back to you at a higher price. Specifically, most HFT prop shops deals in ETB (easy to borrow) stocks such as GOOG, AAPL, & BAC where tens of millions of shares are traded daily. These securities don't need liquidity broker/dealers, as there are already tons of true buyers and sellers out there. Are there any HFT dealers in penny stocks or small caps where some liquidity would be much needed? Nope.
> Are there other factors involved in this outcome, or is this a canonical HFT failure?
Nope. The reason a lot of dealers stopped trading that day were tactical. A lot of stat-arb prop shops got burned during when Bear collapsed, one black box decided to sell everything which cascaded another black box to sell everything, which cascaded to everyone wanting to dump everything. So from that experience, HFT shops decided to not hold any positions beyond seconds and shut down everything when market crashes seriously. So they provide liquidity when the market is doing well, but when the shit hits the fan, self-interests also hits in and the "liquidity-providers" head for the hills.
False. There are long tail funds, I believe tradebot works on the long tail (among many others).
The thinly traded stocks have better spreads and less competition, which means there is money to be made.
Incidentally, most HFT firms stopped trading on may 6 due to a fear of broken orders. If you buy at 5, sell at 10, and the market goes up to 15, you could wind up losing $5 on a short position if your buy order is broken. Unfortunately, broken orders are impossible to predict algorithmically, since humans came up with the criteria for order breaks hours later.
The only thing HFT changes is who the marketmaker
that pockets the $200 is going to be [..]
If you don't want to pay the spread, you can place ALO orders and you will NEVER pay the spread. Your order might never be filled, however.
The $0.02 spread you pay is the cost of getting filled right now. Pay it or not, it's your choice, but remember that immediacy isn't free.
The $0.02 spread you pay is the cost of getting filled right now.
If HFT's are actually creating value (or preventing value from being lost), they are immediately cashing all of it. I'm not sure that counts as something good. Inevitable perhaps.