What Business is Wall Street In?

Like how twenty years ago you could buy a stock you believed in for like $4 by using a computer system, paying a fraction-of-a-penny spread on average, to have a trade executed in milliseconds to seconds, but now you have to talk to a human on the phone and pay a $400 commission to pay a fraction-of-an-eighth spread and have the trade execute in minutes or hours? That fact pattern would make this critique make sense.

No, it is fantastically easier to trade the stocks of companies you believe in. If there is a problem with the market, the "problem", and one uses that term loosely, is that people are talking about macroeconomic trends more than individual companies because the observable evidence is overwhelmingly in favor of a conclusion we've pretty much known for decades: seeking alpha is a sucker's bet.

Can he explain exactly how this is supposed to happen? Let's think it through. You see a stock priced at B, and you decide you want to buy it. Cuban is saying that a high frequency trader will see that you want to buy the stock, and he'll buy it for B and then sell it back to you at B+X, making X in the process (any you pay more for the stock). To do that, he'd have to know that you were about to submit an order. But he only finds out about your order after you've submitted the order to the exchange, and the exchange has relayed it to him. He doesn't get the opportunity to "jump in front of you", and I have no idea why Cuban thinks he does.

In actuality, what happens is this. The high frequency trader sees that the stock is priced at B if you want to buy it, and S if you want to tell it (with B > S). He then expresses his willingness to sell you the stock at B-X (by sending a limit order to the exchange). When you come along to buy it, you get it at B-X instead of B, which saves you money. The trader gets a rebate from the exchange for supplying liqudity (which he did, since you got the stock for less than you would have otherwise) and hopefully he is able to buy it back for some price around S (with another limit order), thereby making the difference between B-X and S.

There are dangers with high frequency trading [1] but they are not of the kind that Cuban is describing.

[1] http://en.wikipedia.org/wiki/Knight_capital#2012_stock_tradi...

The reality is that the vast majority of the damage Wall Street inflicted on the US economy had nothing to do with electronic trading. Until someone invents High Frequency Lawyering, the bulk of the work involved in trancheing collateralized debt instruments is going to be paperwork. Those instruments are traded OTC. While there are electronic markets for some of them, like swaps for blue chip companies, those markets have nothing resembling the volume of the NASDAQ. The '07-'08 crash was caused by evil phone calls more than evil computers.

The other thing you'd want to understand is that prior to electronic trading, Wall Street was crooked as a bucket of fish hooks. Before retail electronic trading, if you (you meaning anyone who didn't work for a trading firm) wanted to buy or sell a stock, you had to find an agent to execute the trade for you. As you can imagine, most people have call to engage an agent very few times a year, but agents work with each other all the time. You got worse prices because your orders would quietly be routed through the channels that secured the most grift for your broker.

There are bad things about HFT; for instance, they create an incentive to route very strong CS talent to Wall Street instead of Google. But before you decide that HFT must clearly be evil because it allows the Goldman Sachs of the world to get better deals than mom and pop stock traders (which, note to mom and pop: don't trade stocks), you should probably have a very good idea what a continuous double auction is, and what a market maker does, and have a good idea of who the "big fish" in HFT are actually preying on. It probably isn't your pension fund.

-High frequency traders provide liquidity enabling me to transact with slightly lower spreads. While narrow spreads may only provide a marginal benefit to the markets, how is it harming you or I?

-High frequency traders hardly impact my investing decisions. Has anyone here been burned by a high frequency trader? If so, please share your experience.

-I am unaware of any academic research concluding high frequency trading has caused harm to individual investors.

-The flash crashes that have occured, appear to have been temporary in nature, and therefore have had zero impact on real investors.

-Don't use market orders if you are afraid of getting a trade executed at an irrational price.

-if PG, KO, or similar crashes for a few milliseconds as a result of high frequency trading (which may be a dubious claim) how does that hurt you? You may have the opportunity to temporarily exploit a mispriced security and buy shares of a company at a lower price.

Wall Street has a lot of problems. Computers making markets instead of humans isn't one of them.

His argument is like saying that someone that buys and sells used cars has a big advantage over me the consumer, therefore buying a car is rigged. I know that implicitly, it's the friction of low transacting.

So, let's say that Mark's right and the combination of HFT and emphasis on macro trends has caused some securities to be mis-priced compared to his analysis of their long-term prognoses. As a long-term investor, shouldn't he be delighted for the opportunity to bargain shop? If anyone can stay solvent longer than markets stay irrational, it's him.

An analogy (weak but serviceable) is that a home town bank is for "saving money", you save your money, you deposit, you write drafts against your deposits that other people can use to make value limited withdrawals. Oh and the people in the bank? They also have a business where they take your deposits and they loan that to people who need money now and can repay it in the future. But to do that they have to do 'magic math' and figure out how likely it is that this borrower will pay it back and they do that with an interest rate. Now the bank, like it has from the beginning of time, is "making money using your money and dealing with people who will pay to borrow it."

Interestingly this exact same works on Wall street and with 'investment banks'. You "invest" (equivlaent to making a deposit) and instead of a chit that says you have $1000 on deposit you get one that says you have ownership of 50 shares of stock in AT&T. Guess what, the bank doesn't just sit on a bunch of stock certificates, they have another business where they let other people use those shares on the agreement they can always get back 50 shares to give to you if you decide you want the certificates to put in a safe deposit box or something. One customer is 'investing' and one (possibly different) customer is 'trading'.

Guess what you can do this with anything of value, and if its something someone consumes you call it "commodities", if its currency you call it "banking", if its stocks and bonds you call it "investing", and if its gossip you call it "journalism."

Mark is smart enough to know this, so why the blog post?

My guess is that a lot of people don't like high-frequency-trading because they can't afford to play. What has happened is that a large holding company, fund, or bank with programmers and computers and data center space near the exchange can easily out trade someone trying to do this through the e-trade Web API. I get how that could be annoying to someone who used to make money that way, tell it to all the folks who used to be able to make a class A game title with a couple of programmers and an artistically inclined person or two.

A complex tax structure ignores the bottom (who pay very little tax on income) and only harms the middle. The top earners in any system with a complex tax structure have the capital and the assets to make it worthwhile to design vehicles that shield them by exploiting these complex tax rules.

"A hacker wants to jump in front of your shopping cart and grab your credit card and then sell it."

On the other hand, looks like we've already lost this fight.

Before computers, Wall St. was still largely about trading and standing between people who have money and people are willing to trade ownership for money. That's it.

The problem is that HFT is about profiting on even the smallest trades, but cranking up the volume to 11. It's kind of like what Wal-Mart did to retail, they make less money per item, but they literally make it up in volume.

Algorithmic trading can be slow or fast, but it plays off the fact that machines can compute the data and make a decision faster than humans can, especially on a digital marketplace.

It is unfair to human traders, sure. But, you have a digital trading platform, so at some point it's impossible to stop algorithmic trading.

If you want a market where the purpose is to create capital for businesses without dealing with machine trading, you need to start a new market that is not run by machines and is only operated by human, person to person trades.

Maybe it's because of my background in machine learning, but I view computers as a way to reduce the amounts of arbitrage opportunities and insider knowledge that can be exploited by traders to enrich themselves on the back of others.

Well programmed computers are able to use NLP techniques to read thousands of news articles, reports, financial statements, government data, demographic data , analyse millions of data points, sort which ones are important, build predictive models, and do it all in a few seconds of time.

A few computers can do the job of thousands of traders, do it with less bias and with better mathematically proven decisions.

Nowadays every time a human trader tries to beat the market by predicting short terms swings in prices, there is a computer on the other side of his internet connection that is relying on much more information and even models of the human's own personality and biases to trade against him. This means that unless the human has inside information that is not accessible to the computer, (and I'm sure those who operate these computers try to feed them or make them infer the greatest amount of direct or indirect insider knowledge as possible ) it will tend to be impossible to beat the market.

With the greater amount of information, the computers will keep the prices closer to real world fundamentals, bust bubbles before they inflate and reduce all the price swings that traders used to be able to exploit to make a profit on the back of less sophisticated investors.

By making obsolete the jobs of all these traders, they are making the market much more efficient. They are automating a kind of job that didn't produced any real goods. Hopefully, pushing these people into jobs that make something real instead of skimming the top of everybody's retirement savings (I can see why those who want to be traders would be upset about this).

Ah but you say, all we have done is replace humans from skimming our retirement savings by machines that do the same. This would be true if there was only one computer system competing against the human traders but there is a whole industry and they also compete against each other. Two sophisticated machines that trade against each other, will not only beat the human trader and put him out of business but reduce the arbitrage opportunity and margin each of these computer systems can exploit.

Contrary to what the article states, computers that compete against each other will tend to eliminate all short term unjustified price swings and leave only accessible prices based on real company fundamentals. Only long term traders that bet on fundamentals will be able to make money and they will make more money than when they were subject to be exploited through market distortions and bubbles.

This can all be proven by something called the efficient market hypothesis:

There is a great explanation by Glen Whitman of Agoraphilia, that uses grocery line wait time predictions as a metaphor: http://agoraphilia.blogspot.com/2005/03/doing-lines.html

See also:

http://lesswrong.com/lw/yv/markets_are_antiinductive/

http://en.wikipedia.org/wiki/Efficient-market_hypothesis

Given that the rejection of HFT has to be overbroad. If they liquidity they provide is the sort of that supports the functions we like, who cares what their holding periods are?

It certainly seems true that Cuban is right about "hacking" the market insofar as yeah, a good deal of HFT sounds like exploitation of rules and procedures to avoid creating the fair market intended. But that means there is "bad" liquidity, which doesn't serve our purposes. Okay, so let's write rules that better serve our purposes for liquidity. (And if we're worried about market distortions from bad trading liquidity, what about the market distortions from Fed provision of currency liquidity? The wash of cash has lifted all asset prices, it's hard to see economic signals when price correlations are moving to one.)

Cuban seems to think we can do just that by manipulating investor incentives -- give them tax breaks for holding periods, tax transactions, etc. But all of these just make the market more complex, and increase the differential between investment categories -- all of which creates more incentive for financial hacking and further detaches pursuit of financial return from realization of economic return. It makes the rules less comprehensible. And it gives the politicians and regulators more to do in the markets, which makes them more attractive targets for lobbyists. He's increasing the incentives for soft corruption while making it harder to spot, the consequences are easily predicted.

Cuban is a smart and pragmatic guy with billions of reasons to think himself smarter than the average bear. Part of his smarts are an ability to keep it simple, and an instinct when things are out of whack. But here that instinct have lead him through simple on to simplistic.

The argument was that in a normal industry, companies in competition with each other benefit when a competitor goes out of business. Other firms step in and snap up the opportunities. But with Wall Street, the whole network of companies needs a government bail out to stop their wholesale collapse when one company gets into trouble (eg. Bear Stearns, Lehman Brothers). Thus they are not really an industry of competing companies, but should be seen as a single monopolistic entity.

(of course they are filled with competitive people, but think of how departments and employees within the same company compete with each other)

I think part of the point (maybe Cuban's point) is that gambling isn't helping anyone, either. Financial markets were designed in theory to get capital to the right companies. HFT is a lot like a Native American casino...there was something much more profitable to do with the real estate than it's intended use.

I disagree with this. How many people believe in Apple as a company and invested in it's growth over the last 8 years. Or Target over the last 3 and there are many others.

The challenge as I see it is that it is very difficult for the companies to actually leverage the growth in their stock prices. The companies need to buy and sell their own stocks in order to use the money as investment. Unless I am missing something huge here. Most of the time it seems it is only an incentive to management that the stock price increases. But really, the capital isn't invested in the company, it is gambled on the company.

   However we need to do it, we need to get the smart money
   on Wall Street back to thinking about ways to use their  
   capital to help start and grow companies. 

I still look at Prof. Sadoway and think how will he get the investment he wants from US markets. Its either VCs, Green Funds or soverign funds making all the running - the business of creating new businesses seems to have fallen out of favour.

Proof - none that you can speak of, except for my bones.

Also, Wall Street != algo/program/high frequency trading. That area of the capital markets, which requires sophisticated market infrastructure to exist in the first place, is exceedingly small by headcount and profit share.

>"The best analogy for traders? They are hackers."

Traders are hackers, and so are entrepreneurs. The kind that likes to tinker with data, play with mathematical models, and find nuances previously un-noticed or empirical anomalies irrational. The dream is to, in the process, find a way of modelling phenomena presently deemed unpredictable, i.e. to discover something new.

>"Discussion in the market place is not about the performance of specific companies and their returns. Discussion is about macro issues that impact all stocks."

Adam Smith didn't write about economics, he wrote about the political economy. The post-War trend of having a secular divide between politics and markets is historically unprecedented. Discussion has shifted because the salient points have shifted; beta is dominating alpha. A visual could be a slick of oil on top of an ocean - the oil is company-specific factors and the ocean the general market. In calm seas just watching the oil is fine. If you're in a Hellenic thunderstorm, however, the dominating factor is the rolling waves. Don't blame someone else because your model is obsolete.

>"I would be curious if anyone out there knows what percentage of transactions actually return money to a company for any reason"

I'm not sure what this means, so I'll take it as what percent of revenues came directly from capital raises (IPOs and debt offerings). This supposes that all other functions at the bank, e.g. asset management, making markets in stocks, providing brokerage and execution services, etc. are useless. From their latest GS 10-Q [1] we see that they earned $1.4 billion in underwriting for the first half of 2012. Not including net interest income their revenues for the same period were $14.5 billion; thus, underwriting represented 10% of Goldman Sachs's revenues for the first half of 2012. As a percentage of transactions I'm not sure how you'd work this out (is an IPO one transaction? Multiple?) nor what use it would be. For reference, underwriting represents nothing of Blackrock's or Bridgewater Capital's revenues.

>"Wall Street as a whole needs to be in the business of creating capital for companies and selling shares to investors who believe they are shareholders."

A healthy primary market cannot exist without a healthy secondary market, i.e. the success of an IPO, and whether it happens in the first place, is strongly related to how investors feel about being able to sell at some point down the road without incurring losses.

Let's examine the connection between holding term and governance mentality. The FT recently had a piece documenting that institutional investors, who own roughly 70 percent of U.S. stocks, despite having long-term positions and near total voting control, tend to by default vote with management and not participate in corporate governance [2]. Resolving our public capital markets is more complicated than fuck the facilitators; ape-handedly throwing taxes around before thinking through the consequences, intended and otherwise, isn't smart (though it will evidently get your article views).

--

High frequency traders, in a race to the bottom, are in the short-term being run into the red by firms that trade time-consuming safety checks for speed. In the end the bad drives out the good and the market de-stabilises. The solution to this is having a series of pre, inter, and post trade checks that have to be run, e.g. monitoring the total dollar volume of trading over a time interval or the total dollar amount lost mark-to-market (yes, there are firms that took these functions out of pre-trade verification to save a few milliseconds). This puts a natural and non-arbitrary speed limit on the markets which accomplishing something meaningful in that time beyond feeling good about having conducted a witch-hunt.

[1] http://www.sec.gov/Archives/edgar/data/886982/00011931251234...

[2] http://www.ft.com/intl/cms/s/0/e19b6a54-fbf8-11e1-aef9-00144...

Technology is the last thing Wall Street sells.

A talk by the founder at Stanford: http://www.youtube.com/watch?v=jH-wyaS2Rn0

That's why we live in a free society.

There is no valid reason or excuse for putting a gun to someone's head and telling them, "stop trading or else," which is what you keep insisting on.

"Individual investors and the funds that just invest in stocks and bonds are not going to crash the market."

So what happened in 1929 then?

http://imgur.com/DxWer

http://www.economist.com/node/21547988

So say you trade against an HFT algorithm. If you are talking about the narrow context of the trade, it is zero sum. But that is not how the world works. Bill Gates can sell a share of Microsoft to a market maker trader/HFT. Say he does this for $26.00. Tomorrow, it goes to $30.00. In the context of the trade, Bill Gates "loses" $4 to the HFT algorithm. But that is not the whole story. He took that $26 and did something with it. Maybe he helped fund a startup which has created value and doubled his money. So now the HFT trader is up $4, and Bill Gates is up $26. Zero sum, huh? Or maybe he invested it in a Malaria vaccination program that has no easy dollar valuation but is clearly positive to the world. This is why in the context of the system, everyone can win, and it can be a positive sum game.

If you invested your savings 18 years ago for your kid to go to college, at some point you will sell that stock to raise cash and pay for college. You are investing in your child's education, so that he can have a good life, invent things, cure cancer, solve P vs NP, improve Shor's algorithm, and more. That seems like a good return, even if 4 years later the HFT algorithm you sold your stock to made money because the stock went up.


When the world changes, prices change as well. As an HFT firm, you are always sending prices that are competitive for customers. HFT being fast is the effort to improve prices as quickly as possible, or admittedly, pull prices if they are no longer fair. Often, this is a typical scenario for an HFT firm…news comes out on a stock that is positive. The old bid for the stock was 20.00. That means if you are sending an order to sell your shares in the stock, to say, raise money to send your kids to a good college, you will hit the bid at 20.00. If HFT is not fast, you will get $20 for your share, even though the news hit 2 seconds ago. HFT algorithms compete for your business in the sense that they try to be fastest to improve that bid price to 20.05. If they are not fast, you get $20. If they are fast, then you get $20.05. What is the problem here?


The bottom line of all this misunderstanding is, I believe, a distrust of where all this money is coming from that they make. It's simple--it's from them cannibalizing the old human, specialist trader business (do you know how many billions they made from retail before HFT?). It's no different from a startup upending an old industry veteran by undercutting their prices by 90% and taking away all their business.

All of Wall Street exists to do one thing: connect those with capital to those who want it.

The primary market exists to do what Wall Street is meant to do: a company or other entity wants money, an investment bank connects that company with investors, and investors hand over the money. Wall Street acts as a classic broker, executing the function it was meant to perform (match those with capital to those who want capital). For its services, it takes a cut.[1]

This part of Wall Street - the primary market - works reasonably well and there aren't many complaints about it.[2] In fact, people sometimes complain about the IPO market getting too hot, which really just means that more companies in the real economy are getting money. The biggest ongoing complaints about the primary market are that the big banks charge too much for the capital raises and that they hype up the securities. Neither is a particularly cutting complaint though, nor is either issue crippling in any way to the capital markets.[3]

This brings us to the secondary market - the stock market as most people know it. Most people never participate in the primary market (i.e. in the first sale of securities), but rather in the secondary market. Here's the core question: WHY DOES THE SECONDARY MARKET EXIST?

The secondary market serves a support function to the primary markets. It provides "liquidity" to the primary investors - that is, it gives them a reasonably easy and cheap way to offload their shares should they choose to do so. The idea is that if there's a ready secondary market for the shares, primary market investors will be more willing to participate in deals because they know they can get out quickly and cheaply if they want to, and they'll be willing to pay a higher price for the shares for the same reason. In more technical terms, the secondary market serves to increase the flow of capital to companies and lower the cost of capital for companies.

And this is where all the problems Mark Cuban is citing come in, plus all the problems that financial regulators were trying to deal with in the last regulatory push (Volcker rule, Glass-Steagall, etc.).

The main issue here is that you can't really draw much of a line between "market-making" and short-term trading. Market-making is something most people agree is a good thing - you want a healthy number of market makers competing transaction costs down and providing sufficient liquidity (again, all to serve the health of the primary markets). And short-term trading is something that most people feel is a bad thing - it creates short-term thinking in the markets, which usually flows over into the companies, so you have everybody thinking about the next quarter, which leads people to ignore longer-term and deeper issues. But both market-makers and short-term traders are just buying and selling securities - it looks exactly the same. This is why the Volcker rule is so ineffective. The rule stated that a financial firm could only have a few percent of its capital in "proprietary trading," but every trader at any financial firm knows that most of the trading (and most of the lucrative trading) happens on the market-making desks. Buy a portfolio of illiquid emerging market bonds at 70 cents on the dollar from an investor looking to offload quickly, warehouse it for a few days, and offload at 85 cents. That's market making. And it's also short-term trading. There is no difference. The trader had to make a judgment about whether he would profit on the trade - whether the price of the bonds would hold up until he could offload it, or whether he got it at enough of a discount that even a move against him wouldn't hurt him. He probably thought about whether he could hedge it while he held it or if he could somehow line up a buyer before he even bought it. A high frequency trader is technically doing the exact same thing - just buying and selling; they just get very fancy about figuring out whether they'll profit on the trade: fractional penny arbitrage opportunities, information about where the price is headed in the next half-second, etc.

The best idea I've heard in terms of tackling this specific issue is to alter the tax structure. Mark Cuban advocates this in the form of a 10 cent tax on trades held under 1 hour. Another version I've heard is to levy a similar penalty tax on any capital gains reaped on a trade held less than 3 months (i.e. taxed as income plus a penalty; right now it's just taxed as income), and to move the lowered long-term capital gains tax rate to gains on investments held for more than 2 years (right now, long-term is 1 year). This would certainly discourage short-term trading, but that would mean that it would also make trading slightly more expensive for everybody, which some people think would be a good thing in that it would make people think twice before they traded something, while others argue that it would be a terrible thing because it would hurt the smallest players (individual investors) the hardest - after all, they're the ones who feel trading costs the most in percentage terms (trading costs as a percentage of the amount they're investing).

As a final thought, while all this trading and short-term thinking seems like it hurts us in the long-term, I don't think this is where our energies should be focused in terms of regulation. Trying to get people to stop short-term trading in the market would be like trying to get people to stop going to see movies for all the violence. Sure, it'd be nice if everybody thought like Warren Buffett in the market, and it'd be great if everybody just wanted to watch Stanley Kubrick films. But the important thing is not to get people to be "better," but to ensure they can't cause much damage as they're acting on their impulses. People like violence, but we keep guns away from them. People like short-term trading, so we need to keep LEVERAGE away from them. If you limit leverage, you limit bubbles and busts. It's that simple and that difficult. Bubbles and busts will still happen because people will chase up prices of some securities and then run for the hills once prices falter, but you need to make sure they're just running and not rocketing. Leverage is that rocket - limit it, and you've got the most elegant solution to the major problems of the financial markets. Don't try to enforce good behavior; just limit the power of bad behavior.

[1] Some people complain about the size of the cut that Wall Street takes for these services. The cut is stable and large for 3 reasons: 1. There's an oligopoly at the top. 2. The risks to a failed capital raise are huge, financially and reputationally for the company raising capital, so they usually opt to go for one of the few top players (protecting the oligopoly). 3. Like most large negotiated transactions, there are higher costs of doing business (think cars and houses).

[2] In the primary markets, the area that probably poses the biggest danger to the economy and society as a whole is when it gets into non-plain-vanilla securities, i.e. stocks & bonds work just fine, but derivatives and other instruments (like some asset-backed securities in the last crisis) get a bit more tricky. But I'm going to leave those aside for now since Mark Cuban is mostly addressing trading in the stock market and plain vanilla capital raising.

[3] The costs have been pretty stable for long stretches of time without seemingly barring companies from raising capital or making the capital raise so prohibitively expensive that people don't participate. And on hyping the securities, investors know that there's a financial relationship between the company and the bank, and they're for the most part pretty aware of this and therefore do much of their own research. All the major mutual funds and hedge funds do their own research and know not to rely on bankers (to the point that many portfolio managers ask that the bankers remain silent during meetings with companies that are raising capital until the discussion gets to specific deal terms, and if the company is not raising capital but just meeting with portfolio managers or analysts, the PMs or analysts often don't even allow bankers in the meeting room but ask them to wait out in the lobby or waiting area).

so, i believe information asymmetry is possible. i believe that many good ideas are in such a relationship with the world that the people who have them have an asymmetric advantage over those who don't: that possession of the idea equals wealth, as long as you are attempting to execute. (wealth in the net-present-value sense, not liquidity sense.)

this gives you the interesting situation that you can be walking along the road and, if you are the right person and respond properly, you can immediately in a bolt out of the blue become richer by the net-present-value of the idea you are just struck with: provided it is one of those asymmetric ones and you proceed to execute on it.

now. now, for the larger ideas (like Google), the net-present-value was in the millions. but if the people doing it had actually had millions of liquidity, they would not have been coding: they would have hired a coder.

so please assume p, where p is "a person with a net-present-value of $n million is currently coding something which will make money and then allow him to hire coders. assume that with probability 1 he will succeed."

as he is coding, before he actually has succeeded, what business is he in under p?

i would say he is in the information-arbitrage business. he is coding at $0/hour against the net-present-value of the idea he has. it seems to me kind of an arbitrage thing.

this is assuming p, which means that this is an assymetric condition where with probability 1 he pays off. obviously it becomes more complex as we get into probabilities other than 1 - but is this a fair conclusion about the startup hacker?

that he is arbitraging information asymmetry?

as i mentioned at the start of this comment, this is a technical argument and i'm not sure of its validity. any feedback is appreciated.

if you hate my assumptions (specifically, p) i still would appreciate to go with them as well as any other thoughts you may have, which you can address separately.