Is the U.S. stock market rigged? [video] | Hacker News Reader

Is the U.S. stock market rigged? [video]

cbsnews.com

51 pointsphsource80 comments

I thought this was going to be interesting but it turns out it was just an advertisement for IEX. They brought out a) guy selling a book about IEX (Michael Lewis) b) CEO of IEX (Brad Katsuyama) c) Chief Strategy Officer of IEX (Ronan Ryan) and d) IEX investor (David Einhorn). No shit they are going to make it sound like the markets are rigged and IEX is the answer.

As far as I know, IEX is of no use for the average joe investor. We aren't buying and selling in blocks of 1000 or 10,000.

Believe what you want, but there is a reason companies pay big money for super low latency.

Instead of being angry about hyperbole and ignorance, my new policy for HN story's touching HFT is to just answer questions. I build HFT systems for a living. My experience is only in commodities futures but I can try to explain equities markets as well. Ask away instead of passing on bad info.

Uhh ignorant HNer here. What is your tick to trade latency? What are your fill rates like? Are you doing primarily add or take? Which contracts do you trade? :-)

*I'm not going to answer these questions for the system I currently work on as it would violate IP agreements.

Tick to Trade Latency: Are you asking about my average times, my median times, my 2 9s or my 4 9s;)? Basically call it tens of mikes.

Fill rates: I've worked on systems that fill on nearly every order they place and other systems that skirt with getting banned.

Adding/Taking liquidity: Mostly adding liquidity in my experience, but I have worked on strategies that only take.

Contracts: All the ones that make money, none of the ones that don't.

> other systems that skirt with getting banned.

That sounds a lot like the "price discovery" flavor of securities fraud.

Not to be flippant, but price discovery is not a fraudulent issue in the markets I trade. That said, the exchanges I trade on, are much, much, much more diligent than the relevant governmental agencies.

I won't pass a value judgement on that.

What language do you write your trading platform in? What FIX engine, if any do you use?

Do you colo with a broker or go directly to the exchanges?

Do you arbitrage the roll of ETFs that track commodity futures (e.g., VXX, USO)?

Do you roll your own backtester or use an open-source or off the shelf one?

In general, what is the win/loss and Kelly's expected payoff of your various strategies? Do you aim for out-of-the-ball-park home run's or consistent singles? Thanks.

My experiences have been in C++, JVM languages and FPGA. I've used proprietary FIX solutions, Onixs, and QuickFIX.

I've always colo'd with the exchanges I've traded on.

I've never traded a US equity.

I've used both off the shelf analytics solutions and built my own. (I don't design trades typically, rather I write systems for folks that do).

I don't usually go in for deals that involve expected payouts of PnL. I build technical solutions for a fixed price for people that have a particular technical requirements set. I'm not sure I even know what a Kelly's expected payoff is.

Thanks, kasey_junk for your detailed answers. Greatly appreciate it. Not trying to get any inside or competitive info as I do more options algorithmic trading on a time frame of weeks and months. But I'm always curious how the other half live.

Are you using Scala specifically as your JVM language, CUDA for your FPGA? Also, have you dealt with Complex Event Processing (CEP) as way to combat stale quotes and feeds that require long time to process? These are area's that I haven't explored, so curious about your $0.02 if you have any experience/opinions, thanks.

I prefer to work in Scala on the JVM if I can, but Java is still very prominent. CUDA is most common on FPGA though I haven't worked in it for awhile. Have not dealt with CPE in trading.

how stressful is this line of work? what's the payoff? and how to get started?

The stressful nature of the work varies dramatically from group to group. I don't tend to work at places that resemble the movies so don't find trading any more or less stressful than any other job. One thing that is different is that in trading you are always aware that 1 bad day can end your job, but I think that's true in most places it's just more explicit in trading.

The pay off is getting to work on interesting technical challenges and those technical challenges can be trivially linked to the success or failure of the group. As far as compensation, I make about the same as what I made when I was writing software for a software company.

I got started by being recruited to join a group, but it isn't some secret society or something. Trading groups are advertising positions like anyone else and they hire at lots of different experience levels.

What exactly did you find hyperbolic and ignorant about the article?

Agreed, the article was pushing an agenda of IEX, but to a casual reader this stance seems aligned to the needs of regular traders big and small alike as opposed to those seeking to exploit the system no offense meant.

The built in 320ms latency of IEX in particular seems reasonable.

Does the world really need nanosecond order execution times which give those with the closest boxes an ability to front-run?

The article was an advertisement for IEX and I'm very disappointed that Michael Lewis is shilling for them. That said, the hyperbole and ignorance I was worried about had to do with the standard response in the HN forums to any article touching on HFT.

This community has a severe bias against HFT and are for the most part completely ignorant about how markets work (or how they used to work). This is very frustrating and often makes me angry. Instead of having that response I was hoping to eliminate some of the ignorance with my responses.

So, even though the article was an advertisement for IEX, you still didn't answer the question of whether they really are offering a great service to level the playing field. You just answered with more hyperbole.

I have never used their service or even traded equities as an HFT, so I won't comment on their specific implementation/value add.

That said, A) I reject the entire premise that the current markets are tilted against large institutional investors. or B) that a large purchase order from such an investor shouldn't move the market price.

Large block transactions move the markets not because of nefarious poorly understood bogey man algorithms, but because of basic market forces. This has been the case long before HFT came on the seen. That an execution trader at a major financial institution getting paid a million dollars of client fee money didn't know this is pretty appalling.

I will say I think their marketing is brilliant. Most other dark pools get painted with the same murky brush as HFT. That a dark pool backed by billionaire investors and Goldman Sachs can position themselves as defenders of the little guy is really impressive. Their long fiber cable gimic is also really nice showmanship.

Two big problems with the market:

- Shutting down overnight, lots of schemes happen after hours or pre-market, why don't we have an always on market?

- HFT has run amok and they aren't that smart yet or they are too smart: http://buzz.money.cnn.com/2014/03/26/whoops-shares-named-ocu... + http://kotaku.com/people-are-accidentally-buying-stocks-with... (people aren't buying these, machines did)

It is no longer a market a small investor can make money in except for buying after bad news and long term. Investors need HFT to compete now.

>It is no longer a market a small investor can make money in except for buying after bad news and long term. Investors need HFT to compete now.

How is this a bad thing?

A person is still quite able to invest, in the sense of buying an index fund. Why does it matter if a small investor can't profit by trading on news? Isn't this a sign that the market is efficient?

Yes, but most of the fat is trimmed by the HFT hedge funds, even long term now, the last vestige of growing wealth.

It is a very good thing that there is buying on the dip as a natural force of the market. The good news market is almost totally owned by HFT and already pre-baked.

But the speed of the algos trims much of what would go to smaller investors that really are needed just like consumers. More volume and movement is always good but being so efficient it takes most of the gains from the support structure of the market harms confidence.

Lots of the gaming happens pre-market and post-market where small investors can't always play and where the HFTs own by scraping 80-90% off the top, over time the delta is big. HFT does have benefits being so efficient though, may even help prevent bubbles because it can trade without emotion.

Hahaha. Yes. When big brokers can peek at trade data before the public and do HFT before you can even look at a chart and click "buy," it's pretty damn rigged.

Ok, but this shouldn't really affect anyone but other HFTs, right? Maybe you get a very slightly different price but that shouldn't matter if you are holding investments long enough. If you are trying to outtrade HFTs and losing, I'm not sure that shows the stock market is rigged.

Are you saying you're okay with the price at which you're purchasing a security being driven up (if infinitesimally), at your expense, so that someone else can pocket the delta?

Regardless of what's fair or not, wouldn't a buy limit at the maximum you're willing to spend solve this problem?

I guess specifically I didn't like the way he phrased it, as if they are stealing data somehow. In fact they bought the access. Other than a clock rate (or 60km limiting box, as shown in the video) I'm not sure how you would create rules against HFT. I'm also not convinced if HFT is a net win or loss for economy, I'm not even sure how to consider it.

How does buying access matter? It's like giving access to see other player's cards in poker (because you're a high roller) and then making bets based on it. The rest of the players only see their cards and the river. That's not a fair game.

Effectively it's a tax on you and on the whole economy. Fractions of a penny which end up turning into billions of dollars siphoned out of the economy with little benefit to the rest of society. There are quite a few really smart people earning bundles doing something which is fundamentally mostly useless. If you plug that hole, those really smart people might just drive their attention somewhere else and accomplish something valuable.

What causes you to conclude that there is no benefit to the economy?

HF trading provides liquidity and there is some debatable value to that purpose, but using the speed of light to profit from traders who didn't pay the toll to be closer to the marked pretty clearly doesn't benefit anyone but those in on the scam.

So should we get rid of trading pits as well? Thats another form of colocation.

Literally how do you compare the debatable liquidity value to the debatable cost to those who didn't pay the toll? Why are those who didn't pay the toll more deserving and necessarily more benefitting to the economy? If there is increased liquidity aren't the non-HFTs starting at a lower price than they would've, even if part of their order is more expensive? I'm asking honestly, I don't know either way.

High frequency trading does lots of things and only a few of them are being highlighted as bad. There are a specific set of legal advantages which can be had by being physically close to an exchange which give unfair advantage.

If the speed of light was infinite, these advantages would be clearly and unbeatably illegal: in essence the high frequency traders are paying huge sums of money to impose a delay on every other trader resulting in foreknowledge of events.

As a hypothetical high frequency trader, I could see that you have placed an order for a million shares of Apple, because of my positioning I can execute my own order _before_ yours goes through raising the price a tick and then selling those stocks to you. If exchanges allowed this and profited by it by erecting artificial delays it would be extraordinarily immoral, but because it's a natural delay it is still illegal.

The problem is that laws didn't catch up with technology, and as stated before the whole thing is complicated and not so easily understood and there are a _lot_ of money in it.

This has nothing at all to do with the separate issue of high frequency traders providing liquidity... they can do that without a time delay advantage over everyone else.

Thanks, that's interesting.

> If exchanges allowed this and profited by it by erecting artificial delays it would be extraordinarily immoral, but because it's a natural delay it is still illegal.

This sounds a lot like "if casinos used XYZ poker strategy clearly it would be illegal, but for some reason we let other players use it!" Well, of course, they're players, no?

> This has nothing at all to do with the separate issue of high frequency traders providing liquidity... they can do that without a time delay advantage over everyone else.

Can you explain this more? Not sure I fully understand.

What causes you to conclude that there is benefit to the economy?

Siphoned out of what? No one is making you give HFTs money or buy from them.

True, if you hold the investments long enough, a small difference in price would even out. Investor are only trying to "out trade" HFT in same sense that are trying to make a fair trade without the secret tax that front runners extract.

Nope. Read the Mark Lewis article also highlighted on HN right now:

http://www.reuters.com/article/2014/03/31/us-markets-hft-fla...

With the help of new hire Ronan Ryan, [Brad] Katsuyama[, former head trader in New York for the Royal Bank of Canada], realized that his orders traveled along fiber optic lines and hit the closest exchange first, where high frequency traders would get a glimpse, and then use their speed advantage to beat him to the other 12 U.S. public exchanges and 45 private trading venues. HFT algorithms could then buy the shares Katsuyama wanted, and then sell them to him at a slightly higher price.

That's the definition of front-running, and it was hitting large institutional traders, of whom investment and retirement funds are a significant portion -- that's stealing from you and me, friend.

Their solution at IEX was to coil a bunch of fiber so that high speed traders' orders get slowed down...I'd have to know more about why they thought that was necessary, but on its face this seems like complete overkill. Obviously there are probably reasons that I don't understand but it seems like this problem would be really easy to solve in software. For instance, I don't see why they don't just trade on a step interval. Basically everyone who arrives within 1 second of each other should have their trades executed simultaneously. Thus you have step of 1 second - nobody can get ahead of you because the travelling time between any two exchanges for even the slowest person is much, much less than 1 second. Does this make the market too slow? Just make the step interval whatever the slowest latency is for any given round trip at any given moment. That would still be on the order of milliseconds but nobody would be able to front-run you because any trade you make will be ahead of theirs or at the very worst at the same time as theirs.

Discrete time matching always forgets about the case when there are more buyers than sellers (or vice versa). If 2 people are buying at price X and 3 people are selling at price X, which 2 sellers get "filled"?

In most markets currently that is done in FIFO order. Discrete time values do not eliminate the race into/out of a level.

A fair lottery would solve that.

Yeah - Why does it have to be FIFO? Wouldn't random be just as fair?

There is a pretty serious bias in the markets against non-repeatable matching algorithms. Explaining to the rest of the market participants why a given trader (or group) had better than average luck is not something that most exchanges are interested in doing.

I suppose by repeatable you mean that if you replayed the sequence of orders - both buy and sell - the matching algorithm would match them the exact same way as it did originally?

I understand the desire to have deterministic trading algorithms, but they're already in the unenviable position of having to explain why some traders get information faster than others, which seems like it's even harder to explain.

Plus, the algorithm could be repeatable if you just replay the sequence of random numbers that led to that order - i.e. if the sequence of numbers were made public along with the executed transactions.

You can measure the randomness of a random number generator, so it would be easy to demonstrate the the orders were picked in a perfectly or near-perfectly random way. You could even hook the random generator up to a quantum event like a decaying atom to get even more evenly distributed randomness. I'm sure for something as important as fair market conditions the institutions in question would spare no expense - neither to have the trades executed randomly but to prove to the public that they are generated randomly.

So, A) you are making assumptions about fairness that no trader would ever make. There is literally no argument, mathematical or otherwise that you can create that would explain to them why the idiot in the desk next to them gets a bigger bonus than they do based on trade execution.

B) You are making assumptions about market information "fairness" that no trader would ever make. The exchanges have no problem explaining why some folks have faster information than others because they always have.

Pit traders had an advantage over phone brokers. Phone brokers had an advantage over which ever poor slobs they could get on the hook. No one on the exchange side of things thinks people all have the same information at the same time, if that was a requirement how would you time coffee breaks?

if that was a requirement how would you time coffee breaks?

A fair lottery would solve that.

You're using an interesting sense of bias, where a fair lottery is in fact perfectly unbiased.

There are lots of things which can cause an arbitrary assignment of winnings to a given trader. Go back and re-read A Random Walk Down Wall Street if you've forgotten. The problem isn't the randomness, it's the bias, particularly to those with an inside advantage and edge.

It sounds like that's exactly what the market needs though. That there is a "serious bias" against it, that exchanges don't want to have to explain the math to traders --surely the traders would get it?-- is at once understandable and irrelevant.

You don't have to have this random stuff. You can just do pro rata as they do in certain certain contracts on the CME. Though maybe it turns out that market participants actually prefer price-time priority. There's a shocking thought.

So if multiple opposite positions come through, each is assigned a slice of the transaction?

Yes. You can read about the details in the specification:

http://www.cmegroup.com/confluence/display/EPICSANDBOX/Match...

Thanks.

The algorithms can be repeatable, though at a cost: a defined set of seeds can be used to create a random, but reproduceable, lottery sequence. I'm going beyond my pay grade, but if you've got issues of trust, then seeds based on pre-selected values supplied by independent agents and a random hashing algorith can assure that 1) the sequence is entirely predetermined, 2) random, and 3) not obtainable by any one party or subset of parties without access to all the seed streams.

A deposit of the stream with an escrow holder, under encryption, can ensure that the seeds weren't modified by the seed providers after generation. The encryption key would be provided after the seeds are played.

Those with questions about the fairness of the random sequences can re-assemble the seeds after they've been played, and see that there was, in fact, no gaming of the sequence during the trading period. And if there were discrepencies between the pre-selected values and those actually played, they'd show up.

Feel free to poke holes in this.

Earlier bids match, the other bids are force-frozen for reuse in the next trading step.

I'm missing something. How does this mitigate the race for first in on the level?

Question for kasey_junk, seeing as s/he is taking questions :D

Lewis: "This form of front running is legal. It's legalized front running. It is crazy that it's legal for some people to get advance news on prices and other --[information on] what other investors are doing. It's just nuts. It shouldn't happen."

Do you agree or disagree with Lewis' assessment of the state of HFT?

I completely disagree. First as a caveat, I've never traded US equities. That said, the idea that all participants in the markets have the exact same access to information that every other participant has is ludicrous. That has never been the case, and can never be the case.

Trading costs as a whole are lower now than they have ever been. Is that entirely due to HFT? No, of course not, but any "fix" you could put in place would most likely have huge negative unintended consequences. Let's all take our cheap investments to the bank and be happy that we don't have to pay 6'8" gorilla's to make our trades.

tl;dr If you are buying stocks as an individual for less than 10 years, don't: you are going to get killed.