1,740 karma · joined April 30, 2013
You can e-mail me at: echo "a2VsbHkubGl0dGxlcGFnZUBnbWFpbC5jb20K" | base64 --decode
[ my public key: https://keybase.io/klittlepage; my proof: https://keybase.io/klittlepage/sigs/rwf5jZM7Wj6iCWAfmRwBOcpf6yhp9-doldqP31CVOiA ]
YC Badge: 0x749c80964Ba6Ed43dC2c9ae4f14cD597795E72c3
unsafe impl<T: Send, const N: usize> Sync for WFQueue<T, N> {}
unsafe impl<T: Send, const N: usize> Send for WFQueue<T, N> {}Classical auctions aren't a good fit because they can't handle non-price factors. However, a combinatorial clock auction could handle non-price factors while also improving price discovery. But as Milgrom says (paraphrased), the best auction is the one people use, so the more realistic outcome is what OP describes: running your own (implicit) auction.
Raft is a simple algorithm, but as others have noted, the original paper includes many correctness details often brushed over in toy implementations. Furthermore, the fallibility of real-world hardware (handling memory/disk corruption and grey failures), the requirements of real-world systems with tight latency SLAs, and a need for things like flexible quorum/dynamic cluster membership make implementing it for production a long and daunting task. The commit history of etcd and hashicorp/raft, likely the two most battle-tested open source implementations of raft that still surface correctness bugs on the regular tell you all you need to know.
The tigerbeetle team talks in detail about the real-world aspects of distributed systems on imperfect hardware/non-abstracted system models, and why they chose viewstamp replication, which predates Paxos but looks more like Raft.
[1]: https://github.com/jepsen-io/maelstrom/
[2]: https://github.com/tigerbeetle/tigerbeetle/blob/main/docs/DE...
Vickery Auctions are one of many in the auction format bestiary, and like all classical formats, they predate both computers and modern auction theory. What's really cool these days (and I'm admittedly biased since it's my space) sits at the intersection of computer science and economics. Economic mechanisms and auction formats that use everything from SAT solving and combinatorial optimization to machine learning and function approximation to drive better real-world economic outcomes.
[1]: https://ai.googleblog.com/2006/06/extra-extra-read-all-about...
[1] https://docs.imandra.ai/imandra-docs/notebooks/verifying-mer...
Some combinatorial auctions make this tradeoff, packaging goods either to deal with computational limitations or to concentrate bidding on a few packages. It can work if there's near total consensus on what the economically relevant packages are, but it doesn't work otherwise. US equities is an "otherwise" case given a huge diversity of needs. The chance of someone wanting the opposite side of even a pairs trade at any given point in time is vanishingly rare. It's far more likely that the person doing the pair would interact with two or more counterparties independently actively interested in or willing to for the right price sell FB and buy SNAP (perhaps conditional on hedging). The mechanism design game is more about giving every party the tools they need to communicate their value function to the auctioneer and creating the incentives to bid (close to) truthfully.
> However, if there are many expressive bids with various conditions (e.g. minimum quantities, conditional on execution of another leg, etc), they may not get "implied" into creating a reasonable market, creating exponentially more arbitrage opportunities if they become locked/crossed.
And this is why combinatorial auctions are a global optimization (as opposed to implied, which are effectively an iterative and greedy approximation) and, in our case, one that seeks to find uniform clearing prices. There are formats with price discrimination and others in which mechanical arbitrage within an auction is possible, but ours is not one of them. There isn't a separate price for {A}, {B}, and {A, B} — within each auction there's a uniform price for p_a and p_b, and p_{a,b} = p_a + p_b (and so on for any arbitrary linear combination). Theoretical point: linear prices don't always exist (in practice, they do for interdependent value goods that aren't strongly sub or superadditive, e.g., capital market goods), and they're not inherently desirable. We chose linear pricing largely because it's a natural fit for how capital markets work now, and perceived fairness/simplicity is itself a valid mechanism design consideration.
Thanks! There are two main differences. For one, combos are (as the name suggests) predefined. That works reasonably well for products like futures and options where the 80/20 approach of making combos for somewhat structural ones like different expiries in the crude and eurodollar complex or packs and bundles designed as standalone financial instruments/hedges. An implied generational liquidity mechanism can knock out some basic structural price arbs between combos, resulting in a combinatorial auction approximation.
This approach falls apart when the combinations are very general as they are in the markets for equities, credit, and many of the assets that trade in the screens.
The CLOB/predefined bundle approach also doesn't address substitutability and non-price factors, and dealing with those is key to unlocking Pareto efficiencies.
> There are already a lot of mechanisms in traditional markets that deal with revealing or concealing true demand (e.g. block trades, icebergs, etc)
The problem with block trading venues and other approaches, e.g., conditionals, boils down to incentives. Initiators of block trades are usually going in the same direction, so opportunities for direct interaction/coincidence of wants are rare. And market makers don't want to take large deltas unless they can hedge and/or know the counterparty. The net effect is not much size getting done. Conditionals are a similar story to blocks. They don't have the opportunity cost that a firm block resting on a venue does, but there's information leakage, and the surface area for interaction is still small. Market makers aren't incentivized to provide liquidity, and directional traders are worried about/behave strategically due to concerns over information leakage.
> what's to stop exchanges from (1) creating more common bundles that people want to trade
I'd say that the market has already done this in the form of ETFs and index products and an entire ecosystem of ETF market making emerged around it.
> (2) matching them with price-time priority so everyone gets a fair price? Wouldn't the auction model just create wider or locked/crossed markets?
I'm not sure that I follow this part entirely. The uniform price combinatorial auction that we're running results in everyone getting the same price on a symbol-by-symbol basis. And, we view time priority as a bad thing (the arms race dynamic of time priority was known to practitioners since markets first started going electronic but Budish et al. were the first to write about it in detail). Periodic auctions have better fairness and post-trade mark outs theoretically and in practice. Some of the European venues where batch auctions have made limited inroads demonstrated this.
We don't segment the market at all or exclude subscribers (beyond requiring that they're FINRA registered BDs), but the way that PFOF works means that we likely won't see retail order flow from the brokers that wholesale it. We do view Smart Markets as a win/win/win/win for retail customers, brokers, market makers, and regulators alike (cleaner routing, better transparency and price formation, better allocative outcomes, lower technology costs).
> How does the expressive bidding interact with NBBO held orders?
https://news.ycombinator.com/item?id=30257586
Expanding on that a bit—we have the standard Rule 611 requirements, meaning that we clear within the NBBO on a symbol by symbol basis. We don't route.
A surprising (to us) takeaway was that making a product in this space sound "vanilla"/undifferentiated is a good thing. Once folks aren't concerned about an initial integration being a lift, they're happy to onboard us as "just another trading venue" (but with great story about unique liquidity and match quality). Many then get excited about adopting the lowest hanging fruit incrementally for their specific use cases. And after peeling away a few layers of the onion, they get excited about the future state in which others do the same, and what initially seems like incremental change becomes a market structure transformation.
We studied the history of adoption in other markets where it's gone well (FCC spectrum, display advertising, procurement) and poorly (OptiMark, POSIT4—great attempts, ahead of their time, killed by complexity and subtle mismatches between the mechanism and market participant needs).
Excellent question. Each leg of the trade comes in as an individual limit order with a price/quantity bound on it. This makes it easy for brokers to satisfy their 15c3-5 requirements since Expressive Bidding can only further restrict these limits. We clear the auction with a constraint that bounds the clearing region to the NBBO, snapshotted at the start of each auction. Rule 611 affords up to 1s if the market moves while we're optimizing, and away market movement has no average effect on execution quality, as auction start times are random and orders are firm.
In the future, we'll do ISO sweeps, but their usefulness is somewhat limited since getting fills from a sweep can change what's feasible for the auction. As we gather data on how folks use us, we'll hone on other regulatory options.
> What market data do you publish about your order book?
We report fills to the TRF to satisfy regulatory reporting requirements, but we're entirely dark otherwise.
On the macro front, we're focused on finding Pareto (more accurately Kaldor–Hicks) efficient outcomes missed by simpler auction formats. Imagine that a buyer expects to move the market 5bps while executing, and a seller expects the same. If agency issues prevent them from finding each other outside of OneChronos (perhaps they only have 2bps of statistical expectation) and they trade on OneChronos at the bid/mid/offer, the reader can decide if that constitutes getting a better price than other markets or not. Multiunit auctions make market impact/execution risk known pre-trade and reduce uncertainty for counterparties, potentially reducing the cost of liquidity.
> Conditional and contigent orders have been around for a decade+.
Conditional orders are a real testament to priorities shifting away from concerns over information leakage/fairness and towards concerns about how to get liquidity in an increasingly fragmented landscape. They're a nasty bandaid solution for routing opportunity cost.
> Options exchanges have complex order books
CLOBs for options and implied order books for futures dealt with some limited forms of exposure risk and proved a big boon for both markets, but they're also a bandaid solution (pre-defined packages and in some cases HFT scale legging risk and no ability to deal with substitutability or side constraints). We're excited to see what gains the general approach (combinatorial auctions unlock).
> Supply/Demand curves have been modeled in cryptocurrency smart contracts like Uniswap.
> Clearly this idea is different and novel but borrows somewhat from all these concepts.
While I agree with you that these are not new ideas, I'd phrase that a little differently and credit the actual inventors. Milgrom, Wilson, McAfee, Cramton, Ausubel, and too many others to name pioneered the theory and practice of multiunit and combinatorial auctions and, by extension, all of these concepts. Except for conditional orders, which make mechanism designers cry. There's truly nothing that better highlights the tragedy of the commons that is market fragmentation.
(FWIW and not that it's the market that we're going after per se—our strategy is mostly blue ocean—the market for US equities electronic execution services across the whole stack of technology, market data, broker algos, etc., is $18B/yr.)
For OneChronos, we're a uniform price clearing mechanism that will eventually support submarkets, which can enable (among other things) what's effectively a mutual opt-in repeated play game of reputation and the ability to bid based on reputation. So we don't support any form of price discrimination at present, and what we will eventually support has the nuance to it that both parties chose to opt-in (and Expressive Bidding means that they can bid in multiple submarkets simultaneously without exposure risk). This induces a meta-game of sorts — a market for reputation. We're including it as a cleaner and more transparent version of existing behaviors with the efficiency gains to go with it.
Proxy Bidders are pure functions that map inputs (market conditions on other venues, metadata) to Expressive Bids in our bidding language (a bounded fragment of linear mixed real integer arithmetic logic—LIRA); certain EBs are CFMMs, and our most general solvers are SMT LIRA.
1. There are two places where deep learning and prior-based approaches can come into play for combinatorial auctions. One is pretty analogous to AlphaZero, but substitute placing a piece on a Go board with accepting a bid, hoping that upon reaching a terminal state the set of bids accepted is feasible and close to optimal. The second is perhaps more in line with what you mentioned—using ML for hyperparameter selection in an algorithm portfolio. When we go live and have production data, our meta optimizer will measure how different approaches are doing and allocate computational resources accordingly in an online fashion. We always use a vanilla unit double auction as a baseline to measure relative performance within an auction cycle, and if the baseline is better, we use it instead.
2. There's a fun and serendipitous story here. I wrote an extremely early prototype as a tiny lisp and evaluator to go with it. We needed a very restricted and functionally pure language that we could control the execution context of, symbolically execute, and do basic formal methods on. The approach worked for a POC, but it was a far cry from real-world adoptable. We proceeded to prototype a DSL with an HM inspired type system and a more pythonic syntax, arriving at a poor man's ML. Better, but a DSL, and something limited/bespoke that would ultimately be annoying for developers. Then we met the guys at Imandra [1], who convinced us that we could have our cake and eat it too using vanilla OCaml/ReasonML and an ultra-high level theorem prover to keep code in an acceptable logic fragment. As an aside, rust is our systems PL and where we do most of the heavy lifting. Evaluating Expressive Bids isn't computationally expensive relative to the optimization problem.
It sounds like you have a very relevant background, so please email us if you're interested in discussing further!
[1] https://www.researchgate.net/profile/Alexander-Leo-Hansen/pu...
Folks also plan on using Expressive Bidding to enter other business lines that high startup costs or low margin (ex ongoing technology costs) previously kept them out of.
Information theory tells us that no universal bidding language (there's a representation of any package of interest) is uniformly more compact than the power set representation. Nonetheless, a good bidding language makes "common" bids compact and easy to communicate. We thought about this problem deeply and realized that functionally pure computer programs mapping proposals (packages of goods) to valuations (how much the bidder will pay or would want to receive) are about as natural as it gets. There's a direct analog in asking a human or a pricing algo for a price in a bilateral trade setting. However, our optimizer doesn't know what to do with an arbitrary computer algorithm, and exhaustively querying one to get the power set of prices out is computationally infeasible. However, using formal methods, we can (in the right setting) convert a computer program into an equivalent representation in a logic fragment called mixed integer real arithmetic. And that (via SMT solving) is something that an optimizer can work with.
You can see what Proxy Bidders (the pure functions that create expressive bids) look like here [1].
[1]: https://www.onechronos.com/docs/expressive/bidding-guide/#in...
Our value add is mainly a team that understands these fields and the extreme nuance of capital markets. That's allowed us to generate novel IP and a purpose-built solution.
[1] https://www.nobelprize.org/prizes/economic-sciences/2020/sum...