Unfortunately thoughtful design and engineering doesn't get recognised
11,494 karma · joined April 14, 2013
Unfortunately thoughtful design and engineering doesn't get recognised
A friend of mine works for one of the major crypto firms and they're starting to deploy algorithmic trading bots on their own exchange.
The spreads on these markets can be diabolical
The competition is also stiff with decades of experience and network effects
The truth is these crypto shops have a pretty poor reputation in the traditional finance industry. Nobody in trading tech goes to work for them unless they offer insane salaries, because they (we) know it's an unstable place to be.
For 4 byte keys and 4 byte child pointers (or indexes in to an array) your inner nodes would have 7 keys, 8 child pointers and 1 next pointer, completely filling a 64 byte cache-line and your tree depth for 1 million entries would go down from ~20 to ~7, the top few levels of which are likely to remain cache resident.
With some thought, it's possible to use SIMD on B-tree nodes to speed up the search within the node, but it's all very data dependent.
It's the same for any other high performance decoding of TLV formats (FIX in finance for instance).
This means your consumer isn't getting a lot of benefit from caching the producers position. The queue appears empty the majority of the time and it has to re-load the counter (causing it to claim the cacheline).
Meanwhile the producer goes to write message N+1 and update the counter again, and has to claim it back (S to M in MESI), when it could have just set a completion flag in the message header that the consumer hasn't touched in ages (since the ring buffer last lapped). And it's just written data to this line anyway so already has it exclusively.
So when your queue is almost always empty, this counter is just another cache line being ping ponged between cores.
This gets back to Aeron. In Aerons design the reader can get ahead of the writer and it's safe.
> MPSC (multiple-producer single-consumer) requires a compare-and-swap loop on the head pointer so that two producers can each reserve a contiguous slot without overlap.
Martin Thompsons designs, as used in Aerons logbuffer implementation, don't require a CAS retry loop. Multiple producers can reserve and commit concurrently without blocking one another.
The trade off is you must carefully decide on an upper bound for message size and the number of producer threads (in the hundreds typically). A caretaker thread also needs to run periodically to reclaim/zero memory off the hot path. Typically though, this isn't a problem.
Aeron itself, which you compare at ~250ns, I think (not entirely sure) is paying the price for being multi consumer as well as multi producer, and perhaps implementing flow control to pace producers. You can turn off multi producer by using an exclusive publication, which eliminates one atomic RMW operation on reserve. I'm not sure where the other nanos are lost.
For SPSC, doing away with 2 atomic shared counters and moving to a single counter + inline headers is a win for thread to thread latency. The writer only needs to read the readers new position from a shared cache line when it believes the queue is full. The reader can batch writes to this counter, so it doesn't have to write to memory at all most of the time. The writer has one fewer contended cache line to write to in general since the header lives in the first cacheline of the message, which it's writing anyway. This is where the win comes from under contention (when the queue is ~empty)
Sure we used AIM and MSN Messenger but we also used IRC, visited forums and looked at newsgroups.
So many people these days don't even own laptops. Their entire digital footprint comes through apps on restrictive mobile platforms.
Clearly intended to be the direct competitor though, since "Have a break, have a KitKat" is the KitKat slogan, and timeout is also a break.
I'd probably measure maturity in terms of how one navigates relationships.
When it comes to my partner, being vulnerable, knowing when it's ok to share that I don't feel like an adult, that i'm scared or lack confidence, and when to put on a strong front and say it's all going to be ok, to make her feel safe, is the essence of what I consider to be a "grown ass man".
But we're also planning a trip to the Lego House, Denmark together and we don't have kids. So there's that.
Mutual funds are cheap and have no tax disadvantages for us. In fact, outside of tax sheltered accounts, mutual funds are a lot easier to manage for tax purposes.
No, Vanguard just think it's fine to charge us 4x as much
VWRP, which I mentioned, is also an ETF
Investors in the UK are not partners in Vanguards mutual structure, and Vanguards UK platform ("Vanguard Investor") is not run by Vanguard but by a third party (FNZ, a New Zealand fintech).
OCF for VT, a global equity index ETF in the US, is 0.06%
UK equivalent (the Global All Cap Index Fund, or perhaps the VWRP All World ETF) is 0.23% and 0.19% respectively, and the latter excludes small caps and both have fewer holdings than VT
Invesco's All World ETF in the UK, tracking the same index is 0.15% and HSBC have an index fund tracking the same index also at 0.13%
Vanguard UK have a 0.15% platform fee whereas the best UK alternatives are completely free.
Vanguard UK recently introduced a minimum nominal platform fee on top which screwed over small investors.
Vanguard are no longer cheap and not on our side.
They aren't the cheapest anymore in almost every category, but their brand recognition has exploded here in recent years.
There are plenty of other funds out there that track other indices from other providers.
I've just written the fifth from-scratch version of a component at work. The requirements have never changed (it's a client library for a proprietary server, which has barely ever changed). I'm the 5th developer at the company to write a version of it.
All because nobody gave engineers the breathing room to factor the solution in to well thought out, testable, reusable components. Every version before is a spaghetti soup of code, mixing up unrelated functionality in to a handful of files.
No well thought out interfaces. No automated end-to-end testing, and no automated regression testing. The whole thing is dire and no managers give a fuck.
AI cannot solve for a lack of engineering culture. It can however produce trash faster than ever at these toxic shops.
Actually they are, but it's also true that you need to put solutions in to production.
I can have fresh bread anytime I want from a handful of nearby stores.