Hacker News Reader

Comment by 1e-9 | Hacker News Reader

Markets around the world are determining prices on a massive variety of instruments that derive value from the current and future value of products such as currencies, interest rates, equities, grains, livestock, metals, oil, gasoline, natural gas, and electricity. These prices allow us to prioritize resources, make fair transactions, and manage risk (i.e. buy insurance on the value of critical products so that we can plan and invest more effectively). The value of all these instruments are related to one another, as well as to realtime world events, in complex ways that no one entity has a perfect view of. Consequently, markets work by polling the expertise of many different parties who all understand a piece of how things should be valued. This results in millions, if not billions, of interconnected price-discovery feedback loops. The markets are kind of like a massive, distributed, realtime, ensemble, recursive predictor that performs much better than any one of its individual component algorithms could. The reason why shaving a few milliseconds (or even microseconds) can be beneficial is because the price discovery feedback loops get faster, which allows the system to determine a giant pricing vector that is more self-consistent, stable, and beneficial to the economy. It's similar to how increasing the sample rate of a feedback control system improves performance and stability. Providers of such benefits to the markets get rewarded through profit.

View on HN

Is there any empirical evidence that these perceived benefits to society actually ever materialize? It's clear that there is a benefit to a trader from knowing something milliseconds before the rest of the market (otherwise Goldman wouldn't be doing this), but it's not clear at all to me that it helps the rest of us.

We know, empirically, that a lack of liquidity increases trading costs, which in turn is directly channeled to the prices of goods and services that rely on this liquidity (more or less everything in the world, even more indirectly ones like education).

It's difficult to say 'things would be X% more expensive' because of the interconnected complexity the GP was talking about, but there is definitely a very apparent benefit.

I think the question is at what point does liquidity have diminishing returns?

If a security could only be traded once per 10 years then it's obvious that its lack of liquidity would make it less valuable. Holding it would tie up your capital quite significantly.

However, if you had a turn-based market where every trade got cleared at the top of the minute it's not clear to me at all whether that would effectively be less liquid than what we have now.

It seems to me that a model where traders all compete for how many nanoseconds away their HFT servers are from the action doesn't really benefit the market as a whole. If anything it just makes things like flash crashes more likely.

> I think the question is at what point does liquidity have diminishing returns?

The general consensus at this point is - no one actually knows - and it's up for serious debate. We know lack of liquidity absolutely has negative effects (because we've experienced it), but we don't how much liquidity is "too much".

Why don't we install a knob that we can turn that effectively limits trading to X seconds precision?

One day we may decide to turn that knob from the millisecond range to 1 second and see what happens. If it's bad, we can always turn the knob back.

> If it's bad, we can always turn the knob back. Sure, but by then maybe some people or organizations have made/lost millions. So we can't realistically experiment with that (even though I would absolutely love to)

If a businessmodel is questionable from a societal viewpoint, then they shouldn't be surprised that some regulation hits them. See for instance Airbnb, where the businessmodel has become impossible in many cities already.

We kind of already are doing this: https://www.bloomberg.com/opinion/articles/2019-08-01/why-ex...

it would make more sense to first get rid of the penny rule.

I feel like we do know how much liquidity is too much, in the sense that the HFT traders are siphoning profits from value-seekers to rent-seekers. The total of their profits is the amount by which there is too much liquidity.

It's not necessarily clear whether it could be reduced to seconds of liquidity, or minutes, but it's clear that milliseconds is too short. The money isn't remaining in the market long enough to provide value -- especially when the market has already been made and they're just trying to figure out who gets the surplus between ask and bid.

> However, if you had a turn-based market where every trade got cleared at the top of the minute it's not clear to me at all whether that would effectively be less liquid than what we have now.

The primary problem with that is that it pools order flow into a homogenous, undistinguished pool.

HFT heavily rely on profiling order flow into the informed and uninformed. A typical uninformed trader is Joe Sixpack who's rebalancing his 401k. A typical informed trader is a hotshot hedge fund manager, who's invested enormous resources in gaining an informational edge. If Joe's buying a stock that doesn't tell you anything about the value of the stock. If hotshot hedge fund manager is buying a stock, that in and of itself is a credible signal that the stock's worth more than you thought it was.

Liquidity providers love being the counterparts to Joe, and hate being on the other side of the hotshot hedge fund managers. The more you can profile the order flow, the better prices and more liquidity you can offer to Joe, by charging the hotshots more. Think of how life insurance companies can offer better premiums, particularly to the healthy, if they require a physical exam before underwriting a policy.

Even on a millisecond by millisecond basis, there's a ton of distinguishing characteristics regarding the informational content of order flow. Uninformed flow basically looks like a bunch of small, randomly spaced trades. Informed flow is more likely to cluster together in small time windows, move sequentially in the same direction, try to sweep liquidity with huge trades, and immediately follow similar moves in other securities among other things.

If you pool all orders into a homogenous one minute pool, HFTs would lose much of their ability to segment order flow. The end result would mean that the hotshot hedge fund manager would see his trading costs reduced, and Joe Sixpack would see his trading costs increase.

Nice explanation.

However, you postulate that the introduction of a turn-based system would effectively transfer money from Joe Sixpack (who'd face higher costs) to hotshot hedge fund manager (who'd face even lower costs).

Maybe, though, we'd see HFT shops go out of business (and not building micro wave towers between Chicago and NY anymore), and see a transfer from HFT shops to hotshot hedge fund manager and Joe Sixpack, both facing lower costs.

How do you know it's not this second scenario?

I think part of the problem is that "liquidity" is an imprecise term. It implies both velocity and flexibility. HFT definitely increases velocity, but it can make the market either more or less rigid depending on the circumstances.

A turn based system seems like a horrible idea. First the major players would have a legitimate excuse to make sure their trades were first in the queue. Second they would immediately game this so that everyone in the line behind them had to watch as they triggered market changes one minute they were then in a perfect position to take advantage of the next minute.

In a turn based market your place in the queue is determined only by the price you bid. Everyone trades at the same price, except for those who bid too high/low, they don't trade.

These systems are already running in many markets, typically at a rate of one trade round per day, but using them is optional.

Wouldn't that be blatant market manipulation? The SEC would fine the hell out of anyone who did that.

In reality the way that markets are cleared wasn't necessarily so much more different than a turn-based one, even in one where the clearing happens by the minute. The issue is, indeed, mostly with HFT. However, you shouldn't consider HFTs to be market participants in the traditional sense. Most of them focus solely on moving stuff around very fast instead of actually trying to purchase or sell things for a separate economic goal (e.g. production, hedging, short- and long-term investments).

Once you disregard the rapid transactions that do not have a significant effect on the price, your average human investor is probably putting down some fill-or-kills or limit orders and actually benefits from HF liquidity trading.

It's hard for me to understand, let alone explain, why milliseconds would matter for the human investor, but I what I can tell you is that GS is not investing 100M USD just for buying derivatives every other minute.

Not all liquidity is the same. I've seen calculations that stated that HFT doesn't actually add valuable liquidity to markets because the moments you need that extra liquidity are price uncertainty moments where HFT traders disappear. I don't have the reference on hand though.

And yet Vanguard say that HFT helps lower their costs by adding market liquidity and therefore reducing spreads:

https://www.forbes.com/sites/timworstall/2014/04/28/bill-mcn...

> lack of liquidity increases trading costs

Sure, that's almost tautologically true, but that does not mean that the benefits to society materialise:

* Traders getting faster access to an exchange have an advantage against other traders (and might make more money), but that does not imply at all that the market will be more liquid.

* The volume of actual utilitarian trades (investing, borrowing, asset exchanging, hedging) is relatively small compared to overall trading activity. A pension fund investing, someone taking out a mortgage, people exchanging currency for the holidays, the often cited farmer hedging his crop - they trade infrequently, and certainly don't care about some milliseconds.

* The dynamics are those of an arms race. Arms races are wasteful, by and large. Building a "straighter" fibre glass connection between NY and Chicago, and then building a series of micro wave towers (because the speed of light in the air is greater than in the fibre) - how does that benefit society? Just postulating higher liquidity and lower costs is not enough to justify it, I think.

(I am reminded of the earlier discussion about advertising - once Cola does it, Pepsi has to do it, too. Are we thus better off, or would we get cheaper sodas if both didn't?)

> a lack of liquidity increases trading costs

Sure but, does a couple of milliseconds to or from affect that?

Yes, and I'll tell you exactly why. Markets consist of informed and uninformed traders.

For example your typical Joe Sixpack rebalancing his 401k is uninformed. His trading does not tell you anything about the underlying value of the stocks. The typical informed trader is a hedge fund manager, who's investing tons of resources in gaining an informational edge. If he's buying a stock at a particular time that is in and of itself a signal that said stock is worth more than you thought it was otherwise.

Liquidity providers love trading with uninformed traders. The problem with informed traders is that you don't want to be on the other side of their trades. Since liquidity providers are the immediate counterparties to most order flow, they're the ones that primarily eat this cost.

To counter this, liquidity providers invest enormous resources in profiling order flow to try to identify when to what degree its informed. This allows them to provide lower costs and more liquidity to uninformed traders, like Joe Sixpack. It's analogous to how requiring a checkup allows life insurance companies to provide lower premiums, particularly to those who are healthy.

One of the most important ways to profile order flow is to quickly adjust quotes as market conditions evolve. For example said hedge fund manager may be trading a thesis that the chipmakers are all undervalued. He may come in and buy Intel, AMD and Nvidia in one swoop. If an HFT sees a huge buy hit Intel, it can bump its quotes on AMD for a few milliseconds.

If its a cigar-chomping hedge fund manager executing a basket algorithm, it's quite likely that he'll try to hit AMD and thus pay the higher price. But if its Joe Sixpack the probability that his trade just coincidentally lands in a 5 millisecond time window is vanishingly small.

You have to consider some form of multiplier effect that comes into play. Because the prices of assets are derivatives of others (incl. many, many other variables, of course) these millisecond savings end up turning into seconds, even minutes en masse.

Not a hostile question, I'm genuinely trying to understand this: How is liquidity provided by, for example, someone interjecting themselves into a trade that was already going to happen?

If you want to move a large amount of money into an asset, you either have to place bids and wait for people to take your order at a particular price, or you have to pay a slight premium to dig into the ask side of the book and pay more money as you consume orders and liquidity.

If you place an ask and the price moves down it might not fill, so you have to move it over time while the price slips.

People provide liquidity by seeing your new sell order and filling it quickly, or by leaving a large number sell orders that people can take immediately, which results in money moving more quickly and consistently.

Willingness to accept a smaller spread, but more volume?

Forgive me for asking but what does GP here stand for?

Grandparent comment, i.e., two links up in the comment tree.

(It's a common term on internet forums -- I learned it from Slashdot years ago)

Thank you very much. I thought of searching the web but GP is too generic a term.

GP grand parent https://news.ycombinator.com/item?id=8363625

GrandParent comment.

The comment directly above is the parent, and the comment above the parent is the grandparent—in this case the GP they’re referring to was 1e-9’s comment.

Business produces goods and services.

Finance is an online multiplayer game.

They are two weakly-connected systems.

>Business produces goods and services. Finance is an online multiplayer game.

But finance is also products & services.

Think of a corn farmer. It's easier to think of him adding tangible value to society because "corn == food".

In contrast, finance just seems like useless office workers copy pasting numbers around in Excel spreadsheets. (This is probably true in many cases.)

But the farmer often wants to sell "futures" which is a product & service provided by the financial industry. Instead of using the jargon of "futures", we can just say the farmer wants a product/service to give him a "guaranteed-selling-price-regardless-of-future-volatility-of-corn-prices-so-I-can-sleep-at-night".

Other examples of desirable finance products that farmers want include crop insurance and equipment loans/leases.

>They are two weakly-connected systems.

Farmers' food crops and the financial products/services of of futures is an example of how businesses making tangible products and the finance industry are strongly connected.

Those are the weak connections I referred to. Last I looked, the volume of financial transactions was about two orders of magnitude larger than the volume of purchases of goods and services. So maybe a couple percent of futures transactions involve actual buyers and sellers of the commodity, while most trades are among financial system players.

But for example the crypto market has no goods or services - only the market trading aspect. Yet it's value swings wildly anyway.

finance is the meta-game. "All problems in computer science can be solved by another level of indirection"

Nope, it's pure loss to everyone else. Every dollar that an HFT makes by front-running your trade is a dollar you don't make as an investor. The money isn't coming out of nowhere.

HFT's are purely parasitic unless you're another HFT benefitting from the faster trades. But what's a solution that doesn't add so much friction the house of cards falls down?

If a market can't be cleared using a spreadsheet with 2^16 rows and 2^8 columns then the market is not efficient.

Largely because most of the trade strategies live in ancient xls files no one dares edit.

That gives these mechanisms way too much credit. Markets can benefit from a more accurate valuation but ms-response times are not needed for this.

The real reason are competitors. You have an advantage if you have the faster line. In the name of fairness there are lines of the exact same length* in many trade centers precisely for this reason.

There are bots that feign transaction so that others react in a specific way. In the last moment these are canceled again, too late for competitors to still react. This is an example for when you need a faster line.

To suggest this arms race in high frequency trade has a serious economic benefit is ridiculous in my opinion.

* I do literally mean cable length. Yes, they have become that crazy

> There are bots that feign transaction so that others react in a specific way. In the last moment these are canceled again, too late for competitors to still react.

In stock exchanges at least, what you describe is called spoofing. The regulators are not friendly towards it, because spoofing enables to skew price discovery.

It's generally hard to detect and hard to prove, but there have been cases where the regulators have proved sufficiently well that in certain cases, there were orders never intended to be executed. And yes, if you get caught, there are sanctions.

https://en.wikipedia.org/wiki/Spoofing_(finance)

I very seriously doubt you'd lose any of those benefits by just trading on 10 second or even 1 minute increments. Everyone has a bunch of time to submit orders, the auction is ran, and then everyone gets the result back. I'd really like to see a use case in any of those real-world markets for pricing with sub-second granularity. It seems much more likely that trading on sub-second granularity is just extracting money from the actual participants in those markets into the pockets of financial firms with no value returned.

All that'd happen if you did that would be to increase the risk of any market maker having the wrong price at execution time (they have less data to make an informed price discovery), so you'd get wider spreads to compensate.

Wider spreads just makes it more expensive for everyone. Personally, I'd like my pension money going towards the actual investment rather than paying for a wider spread, but I'm just strange.

I don't know. Building and running fast trading systems sure creates more work and requires additional infrastructure and energy. So many smart people must be busy doing that, instead of something else. Do I understand this correctly: the alternative is that somebody else would randomly pocket this money, without working for it? Doesn't sound so bad to me. A bit less fair, but more human lifetime would be available to work on other problems.

"If only all these smart people were curing cancer instead of designing trading systems!"

Presumably you also object to academics working on entirely abstract problems? After all, they could be doing something else much more useful.

Who gets to decide what the most useful allocation of resources is?

> Do I understand this correctly: the alternative is that somebody else would randomly pocket this money, without working for it? Doesn't sound so bad to me

If that doesn't sound so bad to you, I suggest you haven't thought it through enough. We've been there, in the past. It was worse then.

> Presumably you also object to academics working on entirely abstract problems?

Yes I do. But at least there isn't a large monetary incentive to push people into it. They push themselves.

> After all, they could be doing something else much more useful.

I doubt it. Maybe.

> Who gets to decide what the most useful allocation of resources is?

Interesting question, but unrelated. We were discussing about the overhead versus benefits of high frequency trading. It's about the efficiency of the system itself, not about how to act within this system or where to direct the resources you get out of it. There is no conflict of values, I think.

My feeling is: it's an arms race. The profits that used to be randomly allocated are now either lost to you (if you decide not to participate in the race) or they are predictably spent on the race itself. Nobody wins, except those who enjoy the race for its own sake. Before that, someone was just winning randomly, and got to decide what to do with the profit.

Do those fast trades actually increase the overall efficiency of the system by more than their cost?

If you're holding for a pension, wouldn't a wider spread affect you less than any single other market participant? Like a 1 cent different due to spread isn't going to matter in 30 years?

If I'm holding a pension, I'm paying in regularly to a fund that's managed over a long period of time. If they manage it actively and trade it a lot over 30 years, 1 cent a time will add up.

That's not what they're doing, though. The costs will likely be minimal.

Spreads used to be much higher, sure, but that was not because there were no HFT shops around back then.

The upshot of this argument is that this is valuable activity. We need markets to price tradable assets and provide liquidity.

The counterargument is that there are diminishing and/or negative returns to increased liquidity and velocity.

Take just stocks. Liquidity is not a problem. You have liquidity whether trades take minutes or milliseconds. Pricing? I'd say we have pricing covered too, at least the pricing that more/faster algorithmic trading will contribute.

Meanwhile, all this stuff costs money, people, resources that aren't available for actual productive work instead of overhead.

> Meanwhile, all this stuff costs money, people, resources that aren't available for actual productive work instead of overhead.

That's a very high standard. What's productive? What's productive enough, in your book, to be worth the effort used here?

Median productiveness is plenty, assuming this is not productive activity.

But what's "productive?". Farming? Manufacturing? What if the farmed food is unhealthy? If the manufactured good is wasteful?

My view is that current liquidity and pricing are far from perfect, particularly when it matters most (e.g. a market panic) and particularly in the global context of the vast universe of interrelated instruments that need better relative pricing. Given that we benefit from realtime pricing, milliseconds matter when you must determine a large vector of prices with complex dependencies using the ensemble recursive system that is the global market.

> My view is that current liquidity and pricing are far from perfect, particularly when it matters most (e.g. a market panic)

And in a market panic, your friendly HFT shop next door is there and offering to buy and sell to stabilise the price?

> Given that we benefit from realtime pricing

Yeah, if you assume the conclusion that we benefit from it, then we do. But have you shown this?

In which market do we benefit from milliseconds pricing, compared to, say, an auction every minute?

> And in a market panic, your friendly HFT shop next door is there and offering to buy and sell to stabilise the price?

Some are. It's not that they are friendly, it's just that doing so can be highly profitable if one can estimate the mispricing with a sufficient degree of certainty.

> In which market do we benefit from milliseconds pricing, compared to, say, an auction every minute?

I argue that all of the significant ones benefit. The global market is huge and interrelated in complex ways. There are many trading entities with a variety of specialties. They communicate their expertise through the markets by placing orders. It's an iterative process and a lot of information must be conveyed. The faster the entities can communicate back and forth, the more accurately the prices can represent a weighted consensus. Constraining the trading to 1 minute auctions would reduce the communication bandwidth.

>that aren't available for actual productive work instead of overhead.

That’s a very biased view. Another view would be that improving the efficiency of the largest markets in the world have a much larger positive impact on society than the vast majority of the “productive” work you refer to.

It's biased to the premise, which is that "day trading's" contribution to price efficiency & liquidity are of diminishing value, and that the marginal value is essentially 0 or negative.

Meh, the short term price fluctuations are almost pure noise. Faster trading just incase the frequency of the noise. Increasing frequency only works if the feedback loops are stable.

Over very short time scales, the price fluctuations vary from almost pure noise to almost pure signal. The almost pure noise situation occurs far more often, but even then, with good estimation techniques, you can extract a signal component that can improve pricing a small amount. Improving many interrelated prices by small amounts can lead to a significant overall improvement to the markets. The almost pure signal situation occurs far less often, but it is almost always extremely important to overall stability. Handling it well can minimize market overreactions and extreme events such as flash crashes and market panics.

The faster you can trade, the faster you can flash crash. The current state is that bots can crash the market faster than humans can react, how does that help stability?

The faster you trade, the faster you can detect and help limit or prevent a flash crash caused by others trading poorly.

This is such a confident statement, and I don't mean that as a compliment.

For starters, is the evidence behind this Hayekian market efficiency really so strong as to warrant this kind of absolute confidence in the wisdom of markets?

> markets work by polling the expertise of many different parties who all understand a piece of how things should be valued.

…as well as orders of magnitude more people who do not understand how things should be valued. → noise, which is fine ("excess volatility"), but which can also become highly persistent in the presence of correlated expectations ("bubbles")

> This results in millions, if not billions, of interconnected price-discovery feedback loops.

Well, there are negative and positive feedback loops, only one of which is stabilizing!

> beneficial […] because the price discovery feedback loops get faster

This can also backfire. In fact, this is why a number of stock markets have instituted a trading stop if an asset moves "too fast". Slowing things down / reducing liquidity can stabilize a situation. Actually, this reminds me of

[1] W. A. Brock, C. H. Hommes, and F. O. Wagener. More hedging instruments may destabilize markets. Journal of Economic Dynamics and Control, 33:1912–1928, 2009.

where you have a similar counterintuitive argument.

The history of the idea of market efficiency is long and the idea remains controversial or contested. See e.g. Philip Mirowski's writings.

You misread my post. I never said the markets were efficient. On the contrary, they are generally far from it. What I said was that the markets combine the efforts of many different entities in order to determine prices in a way that is superior to what any one entity could do.

> Well, there are negative and positive feedback loops, only one of which is stabilizing!

Absolutely. Entities that consistently contribute positive feedback cause harm to markets and they are generally doing something that is either prohibited or foolish. I don't consider either a good long term profit strategy. The market regulation departments work to remove one and large losses tend to remove the other.

> This can also backfire. In fact, this is why a number of stock markets have instituted a trading stop if an asset moves "too fast". Slowing things down / reducing liquidity can stabilize a situation.

Sure, exchanges use a variety of market integrity controls, including limits on rapid and/or large price changes that can trigger order rejections or trading halts. These controls can be beneficial when the price fluctuation was due to poor trading, but can be damaging to a market when the fluctuation was due to significant new information or because there is a natural high volatility situation such as a derivative that is about to expire or is rarely traded. Consequently, the exchanges have to be careful about how and when halts are invoked. Some exchanges often get it wrong.

The main point I was making is that lowering the latency of the multitude of price discovery feedback loops making up the global market can be very beneficial because it allows the pricing dependencies to be more fully determined.

The theory sounds great. But why then, our streets are lined with homeless, and our nations are stricken with poverty? Could it be that the only real aim and motivation of market traders is to earn money? One day, maybe.... when these are replaced with DAOs on the blockchain. But until then it's the Wolf of Wall Street.

Because having homeless people lining our streets on our commutes to/from our jobs is a daily reminder that if we don't work hard enough to increase corporate profits, then our bosses might lay us off and we'll end up like them.

That or moral apathy. At some point in the 80s we decided that markets driven by business profits should dictate every aspect of society.

I imagine 100 years from now they'll look back at today in disgust.

I find it funny that people scorn the pursuit of economic profits, but complain that the other "better" activities aren't given economic rewards.

Ultimately "Corporate Profits" produce the economic value that people desperately want, and participating in the creation of something people want _should_ be a prerequisite for getting economic value in return.

I'm not really sure what you're arguing here. All I'm saying is that corporate profits shouldn't dictate every aspect of society. That's why sane countries have implemented things like universal healthcare and free primary/secondary education.

I'm also proposing that in a wealthy first world country, perhaps nobody should have to go homeless. Crazy idea, I know.

What have these "corporate profits" you worship ever done for homeless people?

100 years after Communism burst on the scene, we currently look at that development with disgust.

Right now most people have access to abundant food, cellphone in every pocket, access to a wealth of information, access to transportation, incredible medical advances.

I can't imagine that the progress we've made would be scorned. Like other market driven forces, bad players will not be rewarded as information about them increases.

However, if information is not increased because of something like a company buying a newspaper so investigative threats can be used against politicians to avoid information gathering, then we have problems. This is more crony capitalism than just capitalism.

You are right that the world has gotten immensely better in recent decades, pulling hundreds of millions out of poverty.

If you compare real median wage growth in the West in the 60s and 70s with the last two decades though, it doesn't look so great. Maybe we can do better.

"What we have now is the bad kind of capitalism! There's a different, good kind of capitalism which in theory does all these great things!" is essentially the "communism works great in theory" argument.

100%, both systems are great theories. However, the last few hundred years of actually trying to implement capitalism has "most people have access to abundant food, cellphone in every pocket, access to a wealth of information, access to transportation, incredible medical advances.", if by "most people" you mean "possible a majority of people in the richest countries in the world". Unfortunately, the cost of that is that we've done irreparable damage to our environment, are causing the worst Great Extinction ever, and have caused a climate crisis that may cause us to go extinct.

Hundreds of millions of people have been brought out of poverty, outside the United States.

That's due to technology; not capitalism, they aren't the same thing.

A huge amount of R&D happens in academia which isn't capitalism; but then is monetised by capitalism (but doesn't reinvest it back into the academia)

Converting that R&D into actual products and services, I'd wager, is quite difficult without a profit incentive. It's like the difference between having a great idea for a startup and actually turning it into a functioning company.

Without invoking too many absolutes, there's so much bullshit involved in the latter that people doing the former aren't willing to put up with. It's mostly two different kinds of people with two different skillsets.

Doesn't capitalism either directly or indirectly fund that academic research? When I was in a university lab, all of my work was funded by private companies

Nearly every country on Earth that has bought into the Capitalist system has become far better off than they were 100 years ago, not just the richest countries in the world.

I agree that "No true Capitalism" is just as bad a fallacy as "No true Communism". But our real life imperfect Capitalism has still had incredible results whereas real life imperfect communism has lagged significantly and failed more brutally.

Woa who said anything about communism? All I suggested is that we should take care of our citizens. We could do this for example via Universal Basic Income, which is very much capitalist (unless you abide by the common American false notion that "helping people" = communism).

> most people have access to...

Why is "most" a good enough metric? If most people have homes but my commute to work is littered with tents of homeless people, is that adequate? Should politicians just throw in the towel then and call it a day because "most" people have houses?

I think we can do better.

The markets on which financial instruments are traded are effectively separate from the ones where policy decisions are being made on, such as labour, vocational education, and healthcare. It's not the derivatives trading that is problematic, it's the market inefficiencies not being resolved by governments.

Yes, the real aim and motivation of traders is to earn money and respect. What else should it be?

Why pick on traders? I know a lot of developers making well into six figures. I hear them talking about getting 3 new graphics cards for their gaming rigs instead of how they worked at a soup kitchen.

What is your point? That GS should be donating 100m to charity instead of reinvesting into their business?

Where do you think that money goes? Workers will be paid to implement their plan and taxes will be paid on those wages. In fact about 40% of that 100m will eventually end up being paid in taxes.

I don't mean to pick on traders. Nominally it's a hard and honest job and money made is money deserved.

The problem is that if I have a company [and this is a systemic example, no exceptions, see #1], and I allow (central-)bank "friendly" people on its board, so that we can receive as many low- or zero-interest loans (with open due date or refinancing at will, i.e. free money, printed freshly from thin air) from the bank, so that we can under-price, destroy and acquire all our competition, become a monopoly AND finance a massive lobbying power in the DC so that we can get laws passed which increase our profits (at the disadvantage of the citizen), you can bet all your savings that such system's demise is written in the fabric of space and time, because the most essential feedback loops (and the ones that you mention, the ones in the market, work in exactly the opposite way) in that system have been disabled and its just a runaway train without brakes.

Buying laws starts and finances wars, relaxes food, water and environmental toxicity limits, enables false advertising, eventually raises taxes, enables trading of derivatives so detached from reality that a computer game pales in comparison, you name it. The days of this system are numbered and we should really speed up the development of trustless alternatives based on blockchain, or we're going to hit the wall really hard.

#1 https://www.newscientist.com/article/mg21228354-500-revealed...

I just have a nitpick with your analogy to feedback loops.

Assuming the market can be said to have a Nyquist rate, then once you hit that you have all relevant information. Increasing the sample rate past Nyquist does not make a system more stable unless you have a very specific system designed specifically to take advantage of that. More typically, it just increases your noise-bandwidth product and can decrease total system stability and accuracy.

I agree with your comment but am curious on how far this analogy goes. What do you suppose defines the Nyquist frequency in a market?

To stretch the analogy a little further, to take an observation of the market, you have to buy or sell which in turn affects the price. It's very much like the effect of measuring a quantum particle, the impact of a photon is enough to change to observation so you have an inherent uncertainty to everything. Because of this, I don't think you can ever distinguish between noise and meaningful trading with respect to a Nyquist limit in the market. If you tried to for instance, look at the price of all stocks over time and find the 2-D frequency function required to represent that, it is going to creep up based on your measurement rate. Another way to describe it might be that your observability and controllability vectors are not orthogonal.

This also skates around the issue of defining what a Nyquist rate of the market even means in real terms. But, if you want to use control theory to model the market, it's important to know that faster does not inherently mean more accurate. In the simple analogy, increasing trading frequency will improve measurement results, up to a point, after which it will actually likely result in decreasing accuracy.

I've also ignored the whole conflation of frequency and group delay in these analogies to keep things simpler as well.

I don't want to oversimplify your stance, I'm just trying to distill it to something I understand. In the analogy above, would you think this tipping point where the signal is no longer improved is related to the volume of the buyer compared to the overall volume?

In other words, Warren Buffet buying a huge portion of a penny stock will drive past that frequency tipping point easily while lil ol' me buying a few shares of an index fund will have effectively little to no impact?

The (global) market is the whole world itself. I very much doubt you can identify an useful Nyquist frequency.

I think if you could, you would have one of those fancy gold medals.

>> The markets are kind of like a massive, distributed, realtime, ensemble, recursive predictor that performs much better than any one of its individual component algorithms could.

That's really interesting, I never thought of it that way.

>> markets work by polling the expertise of many different parties who all understand a piece of how things should be valued.

Does the whole picture ever become apparent to all of the interested parties after the fact? Or do market movements remain subject to a high degree of interpretation even after they happen?

That argument is basically Hayek's (and Mises's) answer to the economic calculation problem. Hayek argued (in "The Use of Knowledge in Society") that free markets are better than centralised planning, because of dispersed knowledge.

https://en.wikipedia.org/wiki/Friedrich_Hayek#The_economic_c...

https://en.wikipedia.org/wiki/The_Use_of_Knowledge_in_Societ...

Nobody has the whole picture. I would say that the more extreme an individual event is, the easier it tends to be to understand in retrospect. This depends greatly upon one's analytical sophistication, level of market data, fundamental product understanding, and professional network (to know what happened in other firms).

If nobody has the whole picture, how can we be sure if it's beneficial?

edit: Or asked differently: What do we have now with high frequency trade established compared to the situation before?

It's enormously cheaper to do business on markets nowadays. Back in the old days, spreads were sometimes multiple dollars on human-made markets with a lot of inventory.

With HFT, people compete to offer the best market, and spreads are in the pennies.

Is there any evidence defining where the speed of information reaches a diminishing return?

To dilate on your feedback loop comment, physical systems may benefit from a higher sampling rate but usually only to a point. This point is often related to the physical dynamics of the system (e.g., natural frequencies). For example, a small thruster may benefit much more from increasing sampling rates from 1000Hz to 10kHz than a large rocket engine. I assume/wonder if there's a similar analogy to diminished returns in stock information systems, like more volatile markets benefiting more from higher frequency of data. It would be interesting to see where the diminishing returns are.

Yes, that's the model. So, tell me. Would any profit be possible in an ideally functioning market? If so, how is any item worth more than the total cost of all inputs including externalities? If not, how is such a dynamic system attracted to an ideal state, given that those who would make the market ideally functional are best placed to gain from market inefficiency and dysfunction? Given an answer, do you have a sound game theory explanation for this?

Additionally, what is the nyquist limit for such an ideally realised market, if it is indeed to be modelled as a recursive sampled approximator and how is this derived? Given an infinitely recursive network of arbitrarily connected market agents, is any such calculation convergent? If so, why? If not, how does the market ever converge to any appropriate price - a price which accurately reflects the market conditions excluding pricing operations and market costs which aren't directly related to the production of the instrument in question?

Keep in mind that, if the market is functioning ideally no market participant will exceed the nyquist rate as all participants knowledge of market conditions converges to zero. How is any sampling rate, excluding zero, convergent? If not, how is any such market realisable? If so, what is the loss function between ideal model and realisable, perfectly imperfect real world implementation? What is the minimum profit, if not zero, and why?

However, it does seem that arbitrage opportunities decrease when such high-speed trading is occurring and, does so even more quickly the faster trading speed and market sampling are increased. How can we account for this, if not by increased market efficiency?

I conjecture that, by ever increasing the sampling rate and the speed at which transactions complete, markets are not being made more efficient. Instead, I hypothesise that, as markets directly effect the price of the instrument reflexively, the feedback latency produced creates relative local pockets of perceived value - which are only profitable trades in relation to local information asymmetry. As the vast majority of high-speed trading holds market positions on extremely short time scales, shifting exposure constantly, this profit is immediately realised locally resulting in the gradual diffuision of this inefficiency as the increase in price of all instruments. This is a direct result of the cost of trading being factored directly into the agent's local acceptable sale price of held instruments. Every local agent trading action is ideal, but the global market is a divergently inefficient one. Indeed, it is a market in which its pricing inefficiency is maximally concealed from all market participants.

In a sense, I conjecture that the estimator is not functioning to increase market efficiency but is, instead amplifying local inefficiency globally, in effect, much like a charge pump would operate in a voltage multiplier circuit. In essence such a scheme acts to conceal increased market cost and overhead (including the profit of market participants) into market instrument pricing. However, it does so in an extremely small and diffuse way so as to make the rise in price of a single instrument, as a result of this activity, extremely difficult to detect as all instruments increase similarly on the same time scale.

This behaviour appears to be similar in nature to 'salami slicing', an often effective embezzlement technique - except that, instead of exploiting an information asymmetry created by lack of interest in small quantities in the part of auditing accountants, it exploits the information asymmetry created by the speed of light itself.

Of course, the faster the sampling rate, the more efficient the described amplification process would take place. Does this effect correlate between markets with differing but estimable information asymmetry? If there is no correlation, this hypothesis is invalid. It would seem to be an area ripe for research and analysis of market data.

Do you see any technical issue with this conjecture by which we may discount it immediately?

A perfect market where all participants get information at the same time does not exist.

>This profit is immediately realised by the increase in price of all commodities globally.

This would only hold if there weren’t profitable short trades. The profit can also be realized by the decrease in global commodities that would have been slower before.

I'm sorry for the confusion, I should've reviewed my post before submitting - I have edited that section extensively for, hopefully, greater clarity of thought.

In a perfect market, there would be no economic profit. Your entire comment is very interesting; sorry for the short response!

No worries at all. I know it's an extremely long comment on a comment on an article - thanks for reading it.

On further thought, the conjecture's behavioural outcome is actually not quite so analogous to 'salami slicing' as it is analogous to monetary policy caused inflation. In effect, the amplification effect would serve to create profit by creating an apparent valuable trade where none actually exists - such trades essentially print money. This activity would function much like the "profit" realised by a central bank when it chooses to print additional currency for redistribution at government prerogative.

However, monetary policy induced inflation is merely limited in effect to those exposed to any one central bank's monetary policy domain - and generally only occurs when the money supply is permitted to rise for all participants. The type of inflation produced by the activity outlined by the conjecture is inherently global - and would exert a pressure on all existing markets which permit this type of trading; and it is not governments, which are ideally responsible to those they represent, which benefit from this inflation - it is private market participants, in the profit they realise from each trade.

This would certainly seem to account for the new behaviour of central banks having to cut their interest rates to near or at zero to compensate for this asymmetric inflation to drive slowing market activity outside of the financial sector... if the conjecture holds - they appear to have entirely lost control of monetary policy to the global market - and those who are best placed to capture value in those markets as a gestalt - via this mechanism.

If the conjecture holds - and central banks and regulators are unable to reign in the behaviour globally - the economy will experience hyperinflation of Weimarian proportions. Unfortunately, such inflation will have vastly asymmetric effect - benefiting only those best positioned to participate in and drive the amplification behaviour itself.

Indeed, it appears to be a naturally occurring divergent state in a market permitting ever higher sampling and clearing rates. Such behaviours are increasingly profitable - seemingly without end - and so it will attract a geometrically accelerating amount of market activity until such activity is no longer profitable due to market collapse.

The analogy is a fascinating, and scary, thing. It's a bit like considering someone nucleating the economic equivalent of a false-vacuum collapse - or someone already having done so. I need to think about it more and find some way of formally stating and ideally disproving the conjecture.

We might disprove the conjecture by looking for anti-correlations in the growth, availability and capacity of high-speed trading and clearing in markets controlling for the returns of financial institutions instruments and portfolios and the changing monetary policies of various central banks under whose jurisdiction they fall. Simulation of economic systems with and without these elements might also yield some insights, when compared to market conditions at large.

Is anyone aware of any other similar research, work, and/or thought in regards to this concept?

I'm an Econ grad student currently not paying attention in a Maths class (something about the Implicit Function Theorem) and I'm absolutely intrigued by your two comments. Kudos.

Also, to sidetrack a little bit, may I ask how long it took you to gather these thoughts and post them? I'm trying to get a sense of how far along I am about gaining a holistic understanding of markets and trading.

While I don't think any time taken is necessarily a great indicator of general understanding, I was struck by a great feeling of unease while reading 1e-9's comment. Initially I sought to understand the apparent contradiction in the measurable reduction in arbitrage opportunity as a generally accepted proxy measure for increasing market efficiencies with my general understanding that HFT generally massively increases volatility and does not generally appear to result in the reduction of instrument pricing due to lower trading overheads and losses. I also sought to explain the seemingly unending profit stream made possible by such strategies - when, paradoxically the more efficient the market becomes, the less profitable all HFT stratigies should become globally - and yet, it does not seem they do.

All in all, about 10 minutes or so of consideration, followed by about an half an hour of editing.

That said, I'll likely spend much more time looking for existing models of financial markets under the information relativistic conditions created by HFT activity - it occurs to me that as trading moves closer to speed of causality in the market the models underlying market understanding may need to be adapted, perhaps using relativity as a prototype. With any luck they already have and I can elaborate from those to solve for conditions of such markets with information asymmetry and agents capturing value. If such markets are inherently volatile and that volatility increases geometrically nearing the speed of causality - presumably, the speed of light, then this may provide the mechanism for the apparent global inflation of instrument prices via distributed profitable high-speed trading activity while preserving lessened arbitrage opportunity and other visible market behaviours.

I really must formalise this so that it may be thoroughly and logically evaluated - both symbolically and under simulation. However, I'm at a disadvantage in that I am merely a dabbler in the field of economics and game theory. I also have no formal background in stochastic finance or physics. I am but a Systems Engineer. So, fun challenges ahead.

I don't know, I see many problems with this comment (akin to the writing of postmodern continental philosophers). Maybe you can restate your conjecture again?

> In effect, the amplification effect would serve to create profit by creating an apparent valuable trade where none actually exists - such trades essentially print money.

Trading is a zero sum game (notwithstanding the allocative function enabled by proper price signals), so I don't see how it would engender inflation.

> the new behaviour of central banks having to cut their interest rates to near or at zero to compensate for this asymmetric inflation

There are many theories about the persistent low rates ("secular stagnation") etc., but I've _never_ heard that particular problem linked to HFT.

> I conjecture that, by ever increasing the sampling rate and the speed at which transactions complete, markets are not being made more efficient.

That's fairly clear, and I can agree with that.

> Instead, I hypothesise that, as markets directly effect the price of the instrument reflexively, the feedback latency produced creates relative local pockets of perceived value - which are only profitable trades in relation to local information asymmetry.

What?

> As the vast majority of high-speed trading holds market positions on extremely short time scales, shifting exposure constantly, this profit is immediately realised locally resulting in the gradual diffuision of this inefficiency as the increase in price of all instruments.

Not sure what you're saying there, but of course the idea is that traders with superior information can realise trading profits, and via such trading, information spreads through the market, until no such trading opportunities persist. However, HFT does not necessarily follow this kind of Hayekian vision, but is maybe more insightfully analysed in a game-theoretic framework.

> This is a direct result of the cost of trading being factored directly into the agent's local acceptable sale price of held instruments. Every local agent trading action is ideal, but the global market is a divergently inefficient one.

> Indeed, it is a market in which its pricing inefficiency is maximally concealed from all market participants.

What?

> In a sense, I conjecture that the estimator is not functioning to increase market efficiency ...

Possibly, yes.

> ... but is, instead amplifying local inefficiency globally, in effect, much like a charge pump would operate in a voltage multiplier circuit.

How is it amplifying it? Yes, we have seen flash crashes, sure, resulting in some transfer of wealth. But this does not explain or predict inflation, geometrically accelerating market activity, nucleated false-vacuum collapse, or any such things.

All excellent points and questions. I'll consider appropriate answers and restatements and respond soon. I'm rarely satisfied with my ability to communicate ideas like these. I really must express this conjecture formally - in the language of mathematics - so that it may be unambiguously communicated in the appropriate contexts.

This is an accurate description of the game theoretic argument.

It is not very precise about the premisses, "the market", "the reward" or "the critical products"--variables in a non-linear equation, so to speak, that do not necessarily have a unique solution, or no solution.

> The reason why shaving a few milliseconds (or even microseconds) can be beneficial is because the price discovery feedback loops get faster

Berkshire Hathway has couple of trades every minute and difference in bid/ask prices is huge around $1000+, yet you dont see people complaining about that.

The bid/ask spread is a separate issue from latency (Edit: although it should tend to be less with better price discovery). To appreciate the benefit of lower latency, I think you have to consider the bigger picture of many interrelated price discovery feedback loops involving many instruments. Many small speedups can result in a much more stable and beneficial system. In the case of Berkshire Hathaway, the value of their stock is dependent on the value of many other equities, interest rates, energies, raw materials, etc.

> the value of their stock is dependent on the value of many other equities, interest rates, energies, raw materials, etc.

Exactly, its the same with other stocks.

> The bid/ask spread is a separate issue from latency.

Yes but it also the second point made by HFT's that they reduce the spread.

Any market maker's bid-ask spread will be dictated by a combination of factors, including aspects like latency (relative to competitors), expectations on holding time, historical volatility and spread (that one is somewhat circular), availability of information from correlated markets, ability to hedge into correlated markets (and what are _those_ spreads, as the market maker will be crossing that spread to hedge), trading fees, etc.

In markets where many of those aspects can be minimized, market makers will end up very tight and the dominant factor becomes latency. a fast market maker can offer a tighter spread to counterparties and will see a virtuous cycle of increased trading opportunities leading to revenue to stay fast.

If the other factors remain significant risks, then the benefits of latency optimization fall off. So, spread and latency are related, but other market fundamentals do play a part.

> will see a virtuous cycle of increased trading opportunities leading to revenue to stay fast.

Not sure whose revenue you assume to be fast.

Anyway I think you did not understand the point I was making, Berkshire's spread is huge and has been huge for decades, yet you dont see lot of people complaining.

This system is the only one unbiased estimator / decision maker humans ever found. Though it has quite high variance, in the long-term its results are just astounding

If it's unbiased, that's the same as saying it's random?

If it leads towards efficiency, productivity, most beneficial allocation, as suggested, those are all biases.

An unbiased estimator is a technical term. It means that its errors in estimation are equally distributed above and below the true value.

Edit to add: I am not sure GP was using the term accurately, either.

An unbiased estimator is one whose expected value equals the true value. Its sampling distribution can be asymmetrical with median not equal to the true value. You seem to be referring to a median-unbiased estimator.

You are correct, of course. It’s too late for me to correct my comment; I slipped up while trying to de-technicalize the definition. Thanks for the correction.

I second this.

An issue I have found with technical minutiae having,for lack of a better term, general names is that they are prone to being used wrong. And once a critical mass of persons start using it wrongly, there's no going back.

> If it's unbiased, that's the same as saying it's random?

No, its deviations from the true value are essentially random.

Is this sarcasm?

Reply on news.ycombinator.com