Quant turned whistle blower explains how the market is rigged [video]
tech.fortune.cnn.com
tech.fortune.cnn.com
Reg NMS prohibits exchanges from displaying quotes that would lock (bid==ask) or cross (bid>ask) another marketplace.
As an example, say BATS is 10.01 bid 10.02 offered and the 10.02 offer trades out completely. Some HFTs want to be the first to form the new 10.02 bid to earn the spread + liquidity rebate, so they send post-only bids at 10.02. BATS sees stale 10.02 offers on ARCA, so they re-price the trader's order to 10.01 bid, or reject it (behavior depends on the exchange). In response to this, HFT algos would repeatedly re-submit their orders until the exchange let them in. This led to enormous stress on their matching engines during price moves.
In response, exchanges created order types that would let the trader bid 10.02, but only display it once the away markets faded from their point of view. It's basically like having a callback API rather than a polling one. The end result is basically the same, except less load is placed on the exchange's matching engine.
There's nothing "rigged" about this behavior. Traders forming new price levels on public markets still incur substantial risk that the price will move against them when filled. There aren't any order types that let you get execution priority in front of a large queue like he describes.
These order types and their behavior were well-documented on public websites and anyone concerned with the microstructure of how their trades executed could easily use them. The game he's describing is of little concern to anyone other than HFT MMs and execution algos that depend on favorable queue position. The entire process of a level ticking away and a new price being formed in liquid tick-wide names plays out in microseconds these days.
I have read Patterson's book which scratches the surface in "Dark Pools", Haim's book "The problem of HFT" and his regular pieces on Zerohedge. I cannot help but feel that Bodek has an ulterior motive "for exposing the system"... I don't buy his narrative largely because all this so called adverse behaviour is widely described in literature especially in exchange/venue published documentation, support and educational material.
A prime example is on page 93 of his book [5] which is a scalping scenario based on inter-market sweep orders (he attributes to a significant loss of capital at Trading Machines) to this strategy... Come on Haim really? you never considered the adverse affects of placing vanilla limit orders? The books [1], [2] and [4] touch on these exact scenarios.
I don't buy for a second that he couldn't have picked up a phone and called his account manager/support technician at X venue, or his broker to ask why he was getting an abysmally low rate of fills, or seek colour on orderbook oddities. Despite his lofty background it appears he was out of his depth at "trading machines" and now is attempting to sell his ideas to anyone that will listen. I do commend him though on shedding light over the field and on getting me interested in HFT/execution.
For anyone interested in learning more, some books I would recommend:
[1] Dark Pools - Erik Banks (2010).
[2] Empirical Market Microstructure - Joel Hasbrock (2007).
[3] Dark Pools - Scott Patterson (2012).
[4] HFT - Irene Alridge (2010).
Also if anyone is interested here is Haim Bodek's recently self-published book: [5] http://www.amazon.com/Problem-HFT-Collected-Frequency-Struct...
The video isn't only interesting because of Haim Bodek's allegations by the way, there's quite a few fairly high-profile people (formerly) active in the HFT world commenting on how markets are effectively broken and unstable because of opaque HFT algorithms, stock exchanges actively promoting HFT to drive up the volume of trades (and hence their profits), and regulators not really doing anything to reduce the risk of flash crashes or penalize parties that completely break the market every once in a while.
Admittedly, just like Haim Bodek, DirectEdge uses an annoying voiceover format rather than just writing an easily skimmable document.
Myth? The finance industry, where the HFT folks are near the top, pays epic salaries for the smartest people. They do this to hire the best and the brightest, and would not continue to do so if it did not work.
In the documentary, Haim relates a story where the team that he was on included PhDs in bioinformatics, physics, and mathematics. This does not look like a myth to me. The mathematical knowledge required to understand and be competitive in the HFT environment, coupled with the software development knowledge is very rare. This industry consumes the time of geniuses in order to function and pays them well for it - that this attracts dumb people and bad software developers that are occasionally hired hardly makes the existence of these geniuses a myth.
> 2) We could harness all these people to something that provides more utility. It would be great if that were true, but how do you do that?
Distributed self-organized common pool resources (http://en.wikipedia.org/wiki/Common-pool_resource).
I have found very little correlation between advanced degrees and ability to determine valuable trading strategies. In fact, the strategies that most people bemoan the most with HFT require the least amount of mathematics because they are very simple.
I've also worked outside of trading in both big institutions & start ups. The ratio of genius to average to bad seems about the same in all of these environments.
As far as the common pool resource approach, the problem with that is that it will most likely tilt the balance of power in the markets even further into the hands of a few giant institutions. Why it is popular to protect the poor investment bankers from the ravages of HFT baffles me.
Your second point is trivially answered: you let the market decide where they work.
That's not obvious to you because you conflate "provide utility" with "make money". There are lots of things that make money that provide no utility. A large part of the history of business regulation is basically restricting those so people focus on providing utility rather than making money at things that provide zero or negative societal value.
The argument you respond to is basically suggesting that HFT is not generating any value for society. If you want to demonstrate that it's false, you have to show that HFT creates value in line with its costs.
It's pretty obvious to me that social networks have some pretty dangerous attributes (decrease in privacy & personal securty, etc) to them. Do these dangerous attributes outweigh the value that twitter provides? Who decides and why?
Because most new industries aren't within the financial industry, messing with the fundamental operation of markets. Those that are have to make that argument.
http://www.chrisstucchio.com/blog/2012/hft_whats_broken.html
http://www.chrisstucchio.com/blog/2012/subpenny_rule_respons...
I'm just suggesting that using a callback API rather than polling is hardly a great example of HFT shenanigans.
1) Orders that help deal with the asynchronous nature of the market. Removing the restriction on locked markets can reduce the need for these orders, but with that restriction in place these orders are a net benefit IMO.
2) Orders that help a participant control how an order is filled. Post only, etc, to ensure one gets a passive fill vs taking liquidity.
There are no secrets here. These orders are all public. Whether you can use them or not is up to how you get to the market.
For example, Thomas Peterffy (founder of Interactive Brokers) talking about how they built a robot typist during the 80's to get around NASDAQ rules that required all orders to be entered via the keyboard (14:25).
From another interview [1]:
"Q: What did that thing sound like when it was running?"
"A: Like a machine gun. Brrrrr. Because it was typing so fast."
[1] http://m.npr.org/story/159992076?url=/blogs/money/2012/08/27...
edit: not sure why this was downvoted.
The small amount of text present suggests someone is being allowed to jump the queue. Is that the allegation, or does it go beyond that?
The only equities exchange I know of that lets you jump the queue is NYSE where DMMs and floor brokers can join a level and participate along with the front of the electronic book. DMMs have negative obligations though.
Put another way, hes so smart but losing money in the market thus the market is "wrong" and someone is cheating.
That seems the exchange's fault. Why do non-HFTs accept that?
If HFT is indeed lowering the cost, how come they are making the money and not the retail traders?
This last quarter I may have bought/sold on the order of 10K shares max. If there's a $0.0001 tax per share traded, it would've cost me $1. That's ambient noise level.
This sounds like an odd question if you transpose it to any other market where automated systems have lowered costs:
"If Amazon is indeed lowering the cost of buying books, how come they are making the money and not the consumer?"
"If Expedia is indeed lowering the cost of booking a plane ticket, how come they are making money and not the traveler?"
"If Toyota is indeed lowering the cost of manufacturing a car, how come they are making the money and not the auto buyer?"
Retail traders benefit from the cheaper trades facilitated by high liquidity in automated trading. There wouldn't be anywhere near as high liquidity in many of these stocks if there weren't automated systems trying to make money on penny price shifts.
Other things:
- Prices around the world across all exchanges are kept in line. If you need to hedge your Yen exposure and can only trade in Europe, you will get the closest possible price as someone in Asia or the US. This is true across all asset classes. This lack of massive arbitrage may get a lot of investors/hedgers/speculators significantly better prices.
- Derivatives on almost any asset are similarly tight as their underlying assets. No longer are you crossing wide/illiquid markets to hedge back month products or options. In a lot of markets, you can trade tight/liquid markets at almost any tenure; this is potentially a LARGE cost savings particularly compared to what some of the markets looked like a decade ago.
- Stat Arb + Relative value spreads are kept in line with their variance structures. Stocks move in line with their eigen components, options move smoothly with their volatility structures, futures at various tenures are priced in line with their synthetic interest rate exposure. Without being able to accurately price something, most participants are able to get a 'good' price on almost anything.
While a lot of the HFT is indeed zero sum between firms, the net economic effect of their activity is certainly not. The amount of $ that most firms make from the HFT business is really not that large; most of the big trading firms have made their fortunes from large speculative/fortunate events, not from being the best at constantly collecting a 1/10th of a tick of edge over and over.
[1] http://neighborhoodhacker.com/why-the-fuck-raganwalds-poster...
The question that needs to be answered is where trading stocks crosses over between moral to neutral to immoral.
If we assume:
nanoseconds = worse than Hitler
decades = saintly Warren Buffett
then there must be some timescale at which trading becomes good or evil.
The challenge is that the faster traders operate as counterparties for the slower ones (and for each other). They absorb risk, provide liquidity, and establish definitive prices.
One way to enforce a speed limit would be to have a crossing auction every second (similar to what they do prior to the open, or when a stock is halted, or for an IPO). If you wanted to be an active trader, you would participate in many such crossing auctions during the day.
It makes no sense at all to pretend that humans are too stupid to understand what is and is not good for our planet. The whole point of having language and intelligence is that it should be used, not just discarded with some discredited hand wave that "if we just do nothing, everything will work out for the best!"
Centrally planned economies have been tried. The Soviet Union discouraged a lot of research into areas like psychology that they saw as pointless; artists had to conform to traditional styles. It didn't turn out very well. Letting the market allocate resources has a lot of inefficiencies, but it works better than any alternative that has been tried.
I don't think we've ever tried having the market solely allocate resources. I think pretty much every government has regulated trade, collected taxes, that sort of thing. What seems to have worked best would be letting the market allocate resources to some extent, but collecting taxes and investing in long term goals (such as research and infrastructure). But maybe this isn't necessary, maybe in a truly free market there would always be genius entrepreneurs like Elon Musk appearing every now and again to progress humanity. Still, there's a massive difference between the kinds of long term planning that modern western democracies already do and a communist dictatorship.
I'm not saying it's untrue because it's taught early, but I am saying that it seems you haven't done much due diligence beyond that.
(Indeed, in a certain sense, neoliberalism can no longer be called a "pure" market system, since the government becomes property of the financial and landowning rentier classes, and is used to fight a class war by actively suppressing all other profits and wages in order to increase rent and debt.)
Most modern democratic governments already regulate their financial markets, and already invest in research grants, infrastructure projects and support for start ups. The fact is, providing incentives to work towards long term goals is exactly the kind of thing that free markets aren't good at.
To claim that markets are not "good" at this kind of optimization entails the belief that they should be.
I'd argue that capital allocation in the face of risk and uncertainty is a hard problem, and that while governments (in their role as infrastructure providers) make the problem easier, they (governments in their role as social planners) enact policies to alter the risk landscape to achieve desirable social/economic ends and sometimes rents for specific interest groups.
Yes. And this is a GOOD thing. Because society as a whole (not to mention the human fucking species) is better off when certain special interest groups are advantaged: Children. The Elderly. Parents. Altruistic people. Social services. Healthcare. Investment in basic science research. Space exploration. Protecting the environment.
...and when certain people are disadvantaged: Killers. Psychopaths. Rapists. Thieves. Bullies. People who exploit market inefficiencies. Monopolists. Oligarchs that buy elections and bribe politicians. Aristocrats who throw their weight around to abuse and bully.
If you think certain types of people ("special interest groups") should NOT receive advantages or disadvantages based on their value to the human species... well... then you are f u c k i n g d u m b.
Who decides these things? WE DO. SOCIETY. YES, LIBERTARIANS, SOCIETY IS REAL.
The worst atrocities of history are generally Government power run amok, where governments use propaganda mechanisms to rally the public into a frenzy and then do horrible things (like the Iraq war, Holocaust, etc.)
There is a big difference between the constructive activities of governments (solving coordination problems, building roads and other infrastructure, basic criminal justice) and the perverse social engineering that corrupt governmental organizations seem to universally gravitate toward.
Most of these events had some measure of popular support. Many people wanted to put the Japanese in camps. Many people wanted to rid Europe of Jews. A lot of Americans wanted the Iraq war, and a lot of Americans want prisons to be horrible, believing we're a Christian nation and that a Christian nation should punish the wicked and smite the infidels. It is, I think, a mistake to presume that government even in the act of tyranny necessarily separates people from the better angels of their nature through deception or coercion.
However government is viewed by many as a legitimate purveyor of lethal force, both locally (everything from cops w/ guns to the death penalty) and remotely (intercontinental missile strikes, rendition, and foreign occupation).
Since governments typically have significant control over the media (by being able to classify information or more direct measures) there exists a significant self-perpetuating propaganda regime, which I think calls into question the basic legitimacy government is thought to enjoy, and along with it the mandate to use lethal force.
Most of the things we consider atrocities are the abuse of lethal force. When a militia member in Africa forces a child to murder his parents, we consider that unequivocally wrong, yet when a government murders his parents we consider that a legitimate projection of lethal force, perhaps only b/c we don't know the details.
So while governments do have some legitimacy and do a lot of good, the basic structures (consent, monopoly on coercion, and propaganda) are ripe for abuse and (I'd argue) transition into covert/improper use of force as they stabilize and their purpose becomes widely viewed as oriented toward peacetime activities.
I agree with you. But I also feel that cynicism, while justifiable and oftentimes necessary, can itself be a blinder to the degree to which the faults of government can tend to take root in any power structure of any significance. Government can fall prey to abuse, and often does, but this doesn't necessarily mean that a lack of government in and of itself will limit abuse where any one group of people has any power over another. If government isn't a legitimate purveyor of lethal force or significant social planning then who or what is?
Absolutely.
>cynicism, while justifiable and oftentimes necessary, can itself be a blinder
I agree with this as well, and think that the proper course of action for the individual citizen is to uphold his civic duty and actively dissent, rather than merely resorting to cynicism.
Dissent can take many forms, but generally ought to provide a check to the tendency of government (or any kind of institution) to abuse its' power.
Parents decide how to raise their kids.
Bosses decide how their employees will act.
Elected leaders decide on what norms the country will have.
You are promoting a kind of ridiculous individualism that is completely disconnected from reality. No one is an island. We all exist in a community and a society and the rules of our society are determined--ideally--through mutual discussion, thinking, negotiation, agreement.
"Who should decide for someone else?" WTF are you talking about? Society should decide social issues. That's who.
"Who should decide?" The appropriate authorities that have been selected by social processes to have that authority. Your question is so fucking dumb because it pretends to be oblivious to the fact that legitimate authorities are put into power precisely because the community has delegated the job of deciding to them.
Your question amounts to saying "It's wrong that the legitimate authorities who have been elected, selected, chosen to decide questions within their sphere are making decisions within their sphere! What right do those legitimate community chosen leaders have to make decisions on behalf of the community!?"
Fucking idiot. The community leaders have the legitimate authority to make the community decisions precisely because the entire purpose of their office, and the reason they were put in that office, is so that they would and could make those decisions.
You're begging the question. "Who should decide?" is a retarded libertarian talking point that just emphasizes how little you understand co-operation and the entire fucking purpose of government.
"Who should decide?" The fucking GOVERNMENT. THAT IS WHY THEY EXIST. TO BE THE ONES WHO DECIDE.
Why do you consider yourself someone who understands people?
Also, why do you use profanity in your HN comments. Profanity can certainly add some rhetorical impact at times, but your use of it comes off a bit more like trolling than rhetoric.
I'm educated because, factually, I am fucking educated. Computer nerds--even the ones with computer science degrees who are fewer and fewer these days--always receive extremely narrow technical educations and are complete idiots when it comes to anything that isn't code. This is why they are libertarians. Libertarianism is the dumbest mass movement since puritanism. It's utterly bankrupt of culture and can only exist in the minds of people who severely overestimate their knowledge.
I come here to vent my hatred for libertarians. There are lots of libertarians here. They think the best way to solve major social issues is literally to do nothing and have no government, no community, no society. It's the dumbest ideology ever but it's very well funded by Koch, Thiel, etc. The Tea Party and austerity politics are the achievements of libertarians.
In my perfect world libertarian billionaires would be convicted of crimes against humanity and sentenced to death.
First, the profanity does not add any kind of flourish to your writing. I'm not the least bit offended by profanity. I'm just mentioning to you that your use of it is awkward, like toilet paper stuck to a shoe. Whether I agree with your point or not it just makes it worse than it would have otherwise been and does not add the zing or oomph you apparently think it does. It also feels to me a bit like the way a non-native speaker of English might mimic "everyman" use of profanity he'd seen depicted in a movie.
Second, for someone who claims to value community and society you are not engaging in a constructive discussion or making any kind of argument. If you have any intelligence or education it is certainly not evident from your remarks. No disrespect if you are actually educated or intelligent, but my guess at this point is that you are a 10 year old kid trolling for laughs and trying to learn Ruby on Rails.
It's not about offending people, it's about venting my feelings. I'm aware that I'm just yelling into a void. But venting is it's own reward.
Also I'm definitely not going to put any thought or work into a hacker news comment. Don't throw your pearls before swine and all that.
What HN never fails to offer up to me is libertarians such as yourself who indulge my rants and raves. So thank you. Like a 50 year old fat man showing his penis on ChatRoulette, your talking to me makes me feel like I have an audience.
Give HN a chance and please desist from deliberately polluting it.
It's difficult to have one without the other. Day traders and speculators supply the liquidity that enables long-term investors to buy in when they choose to do so. This does improve the efficiency of the markets, as there has to be another side to every trade.
To take it to the extreme: If everyone was a buy-and-hold investor, what would happen? IPOs and infrequent sells from buy-and-holds existing would be the only supply of stock. Would this be efficient? (Note: I'm not asking this as rhetorical question as I actually don't know the answer)
I am not arguing for or against HFT, as I've read both sides (from "they provide liquidity and lower spreads" to "they are akin to parasitic front-runners") and am not sure of where the truth lies. Instead, I'm just arguing against this concept that "day trading" provides no value.
Yes, did you see the angst when that one trader got stuck with a stock "overnight" ? He acted like having a stock overnight was the stupidest possible thing. Imagine holding a stock for 12 months! Goodness!
These HFT guys are complaining about fractions of a penny. Meanwhile, illiquid markets such as the Bond Market have real issues going on, where bid/ask spreads are completely opaque, and pricing on the same bonds are vastly different from broker to broker. (with differences measured in hundreds of dollars... not fractions of a penny)
More disgruntled trader than whistleblower?
Does anyone have a link/vid for this speech? I've googled and youtubed without luck.
Take it with a grain of salt. He claims one can just use these order types and be fast to basically print money, but glosses over the important parts like deciding how to price a market, when to add or remove liquidity, etc. When he says things like "HFTs use resting orders as insurance to exit at zero-risk", ask yourself how they make those decisions without having some kind of predictive abilities.
Trading != investing
But in 10 years of 252 trading days of 6.5 trading hours, there are 58968000 seconds. If you'd traded AAPL a million times every second, you'd have made $30132648000000000. That's a lot more money. See how unfair it is?
This makes no sense whatsoever.
Hacker news is essentially the dailywtf for traders. Most of the people there have an ax to grind and/or no experience in real markets.
Welcome to the site.
It's a rather diverse industry in terms of the type of developers being sought. People with skills ranged from low level hardware design all the way up to Haskell/Scala are pretty constantly in demand.