Mysterious Algorithm Was 4% of Trading Activity Last Week
cnbc.com
cnbc.com
Historically buying into an index fund during a crisis is a perfectly reasonable way to achieve relatively easy alpha above the mutual fund/bond rates (~12% per year).
This type of investing (aka diversified value) is completely separate from HFT. As far as many retail investors should be concerned HFTs can duke it all out between themselves as much as they want - it really doesn't matter and is irrelevant to your long term investment decision.
Indeed HFTs provide the benefit that during times of crisis they provide plenty of buy side liquidity for you to get in at a lower spread and a lower spot price relatively easily.
On the page[1] first documenting this algo NANEX says the following:
"We believe that this algo will continue to grow and if left unchecked, could very well contribute to the next flash crash because it removes precious network capacity and provides zero economic value such as price discovery."
Do you disagree?
The stock price is just the opinion of 0.2% of people at any one time - and hence means essentially nothing to an investor. Does the spot price opinion/first impression of a person matter - or does the long term attributes of their character matter? Just because there are more opinions - being given at a faster rate - doesn't mean that a) they're right or b) they should be acted upon as fact and c) that they shouldn't be exploited for those of a more stable nature.
For example: I bought a huge amount of TSLA stock when it fell 12% in one day a week or so ago for no particular reason. I subsequently realised a 6% gain. I'm happy - thank you HFTs and short term traders - your vol makes my alpha.
Just keep in mind, in the stock market, you haven't made or lost money until you close out your position.
But then, blowing out people's stops so you can buy up the stock cheaper is a time-honored trick.
If you're investing for long term - you shouldn't be investing in things that require stop losses.
If you're investing on margin - take a good hard look at yourself before you blow up.
If you're trading options - unless you're pushing liquidity - watch yourself before you blow up.
Sell side liquidity is there - until it ain't. Stop losses don't protect you.
People who lose out by being on the selling-too-low side of arbitrage have nothing to complain about. If you had a stop-loss order, you cede your position to the possibility of being sold too low. Moral being that like you said, if you're investing in something that might warrant a stop-loss order, you should be sophisticated enough to use something better instead.
Except that in the long term good companies sometimes go bad quite suddenly. Frequently, the harbinger (e.g. CFO suddenly quits) will erase a lot of wealth quite rapidly. Stops are a good way to not have your portfolio blow up while not also having to obsessively follow the news.
>> Buy and hold index funds.
To each his own, but this strategy has been pretty easy to beat over the last 20 years (even easier over the last 10), for those willing to study companies at all.
Any proof to that claim? Liquidity evaporated during the flash crash. The SLPs have no obligation to provide liquidity and can simply pull the plug when the market is in crisis. [1][2]
[1]http://finance.yahoo.com/mbview/threadview/?m=tm&bn=228&... [2]http://compoundingmyinterests.com/compounding-the-blog/2012/...
So if you are a buyer in a crisis - you get a better deal with HFTs quickly matching supply/demand of large stock orders. If you are a seller - you're screwed either way - HFT or not (see history).
If you're following a very simple strategy based on past returns that seems to make money all the time, you should wonder what the rest of the market is afraid of that you can't see.
Imagine how well a "buy in the dips" strategy would pan out if you executed it during the Great Depression: http://stockcharts.com/freecharts/historical/djia19201940.ht...
If you bought in at 200 after the market had plummeted from 380 down to 200, you'd probably be thinking your strategy is working pretty well - especially once it rallied back to 290 or so. But after that point you'd be waitinga LONG TIME to get your money back. It wasn't until 20 years later - the 1950's!! - that the DJIA finally sustained a level above 240 (and not before falling to 40 - good luck staying solvent through that!!).
And this is only because the US economy did eventually recover (thanks to World War II). Argentina's stock market never did recover. There's no such thing as "time diversification". In the long run, the variance of your annualized return increases: http://www.norstad.org/finance/risk-and-time.html
This is why the DJIA fell to 6500 in Oct 2009. If you bought then, you are probably feeling pretty smug now - but it's simply that the rest of the market was afraid of Great Depression II and you may not have even realised that it was a possibility.
To say that you can obtain a positive expected return by following any strategy that is solely based on what the price has done recently is just as "naive" as a retail investor who thinks they can beat HFT algorithms.
As the SEC says: "Past returns are never an indicator of future performance"
2: Risk is risk - TANSTAAFL. Great Depression risk is there just as there is nuclear war risk. I take it because I can. I try and make sure I pay the right rates though.
3: DJIA is not the entire market - it's a highly constrained subset.
4: Following past strategies does have positive value - it's what investing (and everything thing you know) is all about.
You live and die by induction.
Thinking that you are high on your black swan horse by stating otherwise is pointless.
Decisions need to be made and money needs to be correctly invested under uncertainty. Taleb guys bore me.
That's great that your TSLA investment is doing so well! My mum bought me some BHP shares in the early 80s that are doing great too. Thanks Mum!
But I still think it's not really being honest to say that the key to investing is as simple as not selling in a crisis.
Not so much when they front-run you...LOL
For example: You want to off load a metric ton of stock in a company - you tell your broker - he puts out a VWAP sell call to an algorithm - HFTs realise - they short sell to anticipate your liquidity premium (aka you want to sell NOW and you are willing to pay for it).
If you didn't want to get out so bad - you cannot be front run - because you wouldn't demand a liquidity premium.
Maybe there is a terminology glitch. Buy side and Sell side mean something else, generally. Buy side is commonly referred to people holding asset on book, and sell side are capital raisers or intermdiaries.
If I'm buy side, I want liquidity -- period. HFT does not provide liquidity, it provides decreased "viscosity". As you note, (observed) liquidity evaporates under high Vol. Which, if it were true liquidity, or if markets participants met the threshold assumpyions of EMH, would not be the case.
If you're trying to differentiate two sides to a trade on an exchange, that's usually referred to as Bid/Ask. Again, observed lack of liquidity on one side or the other (or: massive spreads), signify commonly held assumptions about the markets are askew.
If we throw away EMH behavioural assumpyions, and we throw away liquidity, what we are left with is the following:
(1) Opportunistic market participants;and
(2) Ultra-low transaction viscosity.
These are a shitty combination, from the perspective of public policy. The lack of viscosity actually increases the returns to increasingly obscure and opaque methods of market maniplation.
We allow those who don't harm others to continue doing what benefits them - until such a time as it is shown to be harmful - then we stop it. HFTs have not been shown to be harmful.
Just add a "in financial markets" after "No one" and you get my stamp of approval.
HFT harms society because it cuts off the number of investors that can do active trading. It's net effect is the same as regulations the keep out entrepreneurs from innovating in things like health care and drones and car manufacturing.
By this same logic society was better off having more people employed over turning dirt by hand in fields..
Concentrating trading in a few hands is the problem we had - too big to fail. HFT will lead to barriers to new entrants (because of increasing startup costs).
No. You're confusing things. High frequency trading had nothing to do with financial bailouts. To my knowledge, no high frequency trading shop has ever been bailed out or deemed too big to fail.
>HFT will lead to barriers to new entrants (because of increasing startup costs).
Please explain this, how does HFT increase startup costs?
I'm not sure who you think benefits the most out of high frequency trading, but it's not huge banks like Goldman Sachs. My understanding is that the best high frequency shops are relatively small. They're made up of a mix of programmer and quants, not traditional investment bankers.
There's a reason these industries are hard and should be hard - these are serious industries with serious consequences. Health care - screw it up and you kill someone. Drones - screw it up and you kill someone. Cars - screw it up and you kill someone. Finance - screw it up and you lose the retirement savings of your investors.
These are not games to be played by unsophisticated people or green entrepreneurs. This does not mean that the extant incumbents are any good - it merely means that new players does not automatically confer innovation goodness (see natural monopolies/booms).
As to regulation on healthcare/drones/cars etc. things aren't perfect, that's all I'm saying. I think things are too restrictive right now, I'm not arguing for abolition of all regulations.
edit: Yes, retail investors should be trading more. That's the idea behind the recent rise in crowdfunding. Innovation happens a lot faster in small c capitalism than in big C capitalism.
But I'll gladly take part and take people's money while the times are good.
I don't particularly care either way - I make money no matter what the market.
And I'm just an individual. What about when I send to an email list and it goes to 100 people? Should Google Groups pay a dollar to send out those 100 emails? What about MailChimp, do they have pay $200 when they send out a newsletter to 20,000 people?
I'm also concerned about how this money will be collected and how, from a technical perspective, all emails will be monitored. Every SMTP server will now have to be registered with the central oversight organization? VPNs and rogue email installations will be the enemy or blocked from communicating with anyone "on the grid"?
The point is that abusive email -- brain-dead bulk spam, and even annoying recruiter pitches and "real" business adverts -- are sent in far higher volume than even the all but the very largest of mailing lists. The standard figure bandied about for most of the past decade is that some high-90% of all email is spam. Let's say 97% (as Microsoft reports: http://news.bbc.co.uk/2/hi/technology/7988579.stm).
Spammers are generating returns based on a small fraction of a percent response rate on an overwhelming volume of mail, which is close to free. Sender costs are estimated at $0.00001 (http://www.clickz.com/clickz/column/2138759/make-spammers-pa...). That's 1/1000th of a peny per mail, or a cent per mil (thousand mails).
I suspect that raising that cost by even just a few cents per mil would be sufficient to put the brakes on most spam. For personal email, that would literally be a few pennies a year for even a high-volume correspondent. Protocols for mailing lists and other legitimate noncommercial use would probably rely on offsetting costs to users.
In reality, proposals such as hash-cash, greymilter, targrubing, and the like extract computational work from an untrusted SMTP sender as a requirement for accepting mail. It's a pretty effective way to put the brakes on high-volume delivery. Computational power is a pretty reasonable proxy for coin of the realm.
And if that's not enough ... there's plenty that's going on now with DKIM and other header validation that would serve to identify a sender. Extend that such that you don't accept non-signed mail (except at a very, very, very slow rate) and that you can associate a designated signer with a verified escrow account, and you've got much of the nuts and bolts of putting a system into place. Not that I see this as entirely desirable, but we may have to go there.
Or the post office for that matter. There's actually one single organization that all those stamped envelopes get sent through, it's not decentralized.
In network terms, it's a SYN attack -- you're opening a port but not connecting to it, or dropping it.
A tax on uncompleted transactions might also help here. I'd say Soros's suggestion also has merits, though perhaps the incomplete transaction tax could be set somewhat higher.
Neither would have to be large to make HFT grossly unprofitable.
I agree. My understanding is that regulations in my own country (Canada) are being put in place to mitigate such problems. I just think it is important to maintain the distinction between high frequency trading and high frequency quoting. They should not be condemned as one entity.
[0] http://www.nasdaqtrader.com/TraderNews.aspx?id=ETA2012-13
Also, in the video they say:
"What you're saying is the individual investor has no shot. Mary and Joe sitting out there have no quants on their staff"
My assumption was always that the average trader holds their shares over a longer period of time, and thus aren't really affected by these HFT algorithms. Is that assumption incorrect?
Either way, Mary and Joe were never competing with market makers and active trading strategies, and still aren't now.
2. If HFTraders are collectively making money, and presumably they must be, that money must be coming from somewhere. If, for example, the stock market rises 50%, via some sequence of gyrations, then there is only a fixed gain to be divided up. Maybe HFT increases the size of the pie, but that feels like mostly a higher-order effect.
Am I wrong?
The money HFTs collect tends to come from the bid/ask spread. It's money that was already being taxed out of trades, but now robots are competing for it.
Closing off a large number of people because they lack "sophistication" is called facism. We should be striving for small c capitalism.
The stock market exists not because of some idealized goal of serving society, but because companies want to sell partial ownership and transfer risk from themselves to the public in exchange for potential future returns on that investment. Anyone member of the public who purchases stock in a company should understand that agreement. It also has secondary benefits like providing liquidity for employees, etc., but the New York Stock Exchange wasn't founded with that purpose in mind. In other words, the stock market doesn't owe you anything.
That point aside, could you clarify how HFT hurts you? If your goal is to use stock markets as an investment vehicle, then your time horizon should be on the order of months and years, not seconds. If that's the case, then HFT has no impact on you whatsoever. If you do want to play in the second time range, then my analogy with F1 racing is perfectly applicable and there's no justification for your complaint.
What they are certainly not, is some higher order of intelligence or ideal that we humans just have to learn to live with, or else. The stock market may not owe me anything, but it certainly owes us something, or we wouldn't use it.
My goal is to trade daily/weekly (in addition to monthly/yearly). I don't want to seconds/minutes/hours (and so can't many others - so the "competitiveness" of markets is actually reduced).
Ultimately, stock markets exist to raise funds for projects (exits for entrepreneurs, financing projects in large corps) - they occur on the daily-years timeframe, not seconds/minutes/hours.
But more importantly htf funds do not stop anyone from participating in the markets, it just harder and riskier to do so. Accounting firms had massive staffs with hundreds of binders and files for each client. Accounting software eliminated most of that. Were accountants talking about the unfair advantage that Intuit had, and how normal accountants couldn't make a living anymore? Probably. Welcome to the future.
No Fascism is an authoritative, nationalistic, militaristic, socially conservative political ideaology.
It's not that expensive to get historical data. You can get daily for free. You can get minute data for years for ~$65.00/month and tick data for an extra $25.00/month.
The expensive part is direct market access, and even that is reasonable if you've got a successful trading strategy.
Secondly, if $100/month is stopping your business from being profitable, that's a fault with your business model, nothing more. It would cost more to get a medium size Windows instance on Amacon EC2 for the month.
>Multiply that by the millions who could be trading but don't (a few hundred dollars a month is a barrier to a lot of people, that might have been the profit of a small strategy that worked).
I don't think you can claim that millions of people are being locked out of the market because of data costs.
What if there's a little appliance that you plug in which makes predictive models (you can think of it as installing software on your laptop). What if millions of people want this appliance but it costs a $100 a month but could have made about $100 dollars a month? They will choose not to invest.
This might seem contrived, but it's also the scenario behind the web (lot less blogs when it cost $100 per month).
The net effect is that less predictive models of the economy are created. This is bad because that's how capitalism allocates resources.
The larger framework is that competition increases quality. Any barrier to business decreases competition.
You can still trade on daily bars. Anyone can. Whether you sum up a days worth of data into a daily bar, or you have an auction once a day you're still going to have the same sized data set.
>The net effect is that less predictive models of the economy are created.
No it's not. $100/month is a reasonable cost. I don't know how to make this clearer to you. Data costs are among the CHEAPEST part of the equation when you're building a financial model. Quantitative analysts are paid six figures. Skilled programmers are paid on the order of six figures. Getting data costs down to $100 is not going to make someone go "Oh you know what? I'm ready to put in 100 hour weeks developing financial models because I can now afford a bus pass".
The examples you're giving sound ridiculous because they are ridiculous, and so is the premise you're basing them upon. It's like me telling you that I want to become a programmer but a $100 laptop cuts into my expenses too much.
>The larger framework is that competition increases quality. Any barrier to business decreases competition.
$100 will not increase competition. I can't put it any more plainly than that.
Trade on opens. Occasionally there are gaps, but they are traditionally due to big news, almost always related to fundamentals. If you're modelling the stock market as a random walk, then you're just as likely to have volatility go for you as against you under normal market conditions.
Issuing a sham mortgage is not a high frequency trade.
"On October 1, 2012, we detected a new form of quote stuffing that tries to hide under the radar. It involves hundreds of stocks and millions of bogus quotes during the trading day. The algo that generates them is very careful to spread out the stocks targeted and limits each test to a blast of exactly 200 quotes in 25 milliseconds or less. The quotes all come from Nasdaq, and sometimes affect the Best Bid or Offer: the only change appears to be a fluttering of the bid or offer size."[2]
Doing my best tptacek impression "the nut graf is":
"We believe that this algo will continue to grow and if left unchecked, could very well contribute to the next flash crash because it removes precious network capacity and provides zero economic value such as price discovery."
They posted an update on the 5th with a neat visualization of the trading activity[3]. If you just want to see a neat animated gif the direct link is:
http://www.nxcoreapi.com/aqck2/Algo200.gif
They have a list of the activity from October 1st[4], but it is not clear to me what the second and third fields are:
Example:
09:01:03|P|9|UMDD
09:01:03|P|6|MVV
09:01:03|P|6|MIDU
09:01:03|P|4|IVOO
09:26:20|P|9|UMDD
09:28:44|P|7|SGOL
13 4 N P Q
0 1 2 3 4 5 6 7 8 9 a A b B C D E F
[1] http://www.nanex.net/aqck/aqckIndex.html
[2] New Quote Stuffing Algo http://www.nanex.net/aqck2/3610.html
[3] Quote Spammer Spotted http://www.nanex.net/aqck2/3614.html
[4] http://www.nanex.net/aqck2/3610%5C20121001.200.txt
[5] http://www.nanex.net/apidocs.html
EDIT:
I am delighted and rather surprised to say that someone from NANEX responded to my inquiry about the second and third fields.
"P = ARCA (we've since determined that the algo only ran on NYSE and Nasdaq listed issues)
N = NYSE
Q = Nasdaq
The 3rd column is the SIP multicast line that would carry the quote. For Nasdaq listed, those are carried on UQDF which uses 6 lines and we label as A, B, C, D, E, F. For NYSE/ARCA listed those are carried on CQS which uses 12 lines, we label as 1-9,a,b."
There are some great charts illustrating how every time exchanges increase quote processing capacity the HF algos gobble up the new bandwidth, but ultimately the number of executed trades has remained relatively consistent over the past five years.
Deleted comment
LOL. You can't do that. It is illegal. And if you'd do that intentionally, SEC would be calling you the same day.
Put a .5 second minimum TTL on every order and you'd see all the 'liquidity' provided by the HFT world dry up instantly.
The orders not meant to be hit is a separate issue from the messaging. These orders are there because of
1) quoting requirements designed to mitigate volatility,
2) to give traders the feeling of depth (illusory or not) because market participants tend to interact with exchanges that look like they have thicker books,
3) to gain queue spot due to any FIFO component of the exchange's matching algorithm in case the price moves to the level where these orders could get executed,
4) to gain order allocation due to any pro rata component of the exchange's matching algorithm.
All of these reasons are controlled by the exchange and to some extent the SEC. The first 2 reasons are the result of the exchange trying to make money by attracting participants. The last 2 reasons are the result of participants rationally reacting to their incentives as dictated by the matching algorithms designed by the exchange.
Creating the illusion of volume where none exists has long been illegal - people used to paint the tape long before HFT exists to achieve the same thing.
You've provided a good description of why HFT do what they do, and one could argue that laws need to be changed to allow this market behavior. (I would disagree) Much of what they do is illegal by present law, but none of the big market players want the law enforced, so the SEC looks the other way.
There is a lot of shady stuff that goes on in trading due to conflicts of interest. Anyone intelligent or informed enough to know what's going on is financially incentivized to be secretive about it. I haven't gotten the impression the SEC is intentionally being incompetent- they really ARE just incompetent because anyone smart enough to realize what's happening does not join or remain in the SEC. Incompetence is the simplest explanation for the crazy rules and fines they have enforced, and the rules obvious to actual traders that they have ignored.
If the exchanges accepted quotes at fractions of a penny (for instance float values), then the speed/latency of quotes would take a backseat to price. It would make pointless a lot of the current shenanigans.
It would be simple for the exchanges to reign in trading by adjusting their pricing model. Make it more expensive to make each type of request. Adding a few cents to the request would squash HFT while not drastically effecting other trading systems/people.
The exchanges have a per-trade charge, and for the data, they charge fees on a per data stream basis, and charge for colocation.
And they already do make each tier more expensive than the previous.
Mm good old days of dialup. All yours for $900/month.
I suppose that's technically correct-- we don't know how the algorithm works. But they make it sound like they have no idea to whom the algorithm belongs. If your logging can't get you this information (the bot's owner), I have no hope for the security of the exchanges.
If this kind of thing is a problem, then read your logs, visit the 'offending' party, and Do Something About It.
if(condition = true){...} // notice the single '='
:-D
Meanwhile, on a options exchange, a different exchange, where trading happens throughout the day because there is no such regulation, other investors in the same stock get to hedge their position by selling call options. They get to act ahead of you because their exchange has no such regulation. Unfortunately for whatever reason, you have no access to this venue. Should regulators regulate these exchanges as well? If they don't, someone will complain that it's unfair to people who have no access to such exotic venues. So, let's say they regulate all exchanges in the country.
Then, people who really want to trade in the middle of the day have to do it the 'good' old-fashion way, away from the exchange, making phone calls to each other, through brokers, where there's less transparency and more chance for corruption and unfair deals because the fair price of an instrument is unclear due to the absence of exchange activity. This would be a step backward.
Additionally, in terms of raising and reallocating capital, companies whose stocks are thus regulated will be at a disadvantage relative to their international counterparts whose stocks are more liquid in the absence of similar regulation, as capital tends to flow where trading is more convenient. So, this effect would impact the country's competitiveness as well.