C++ Asynchronous Framework | Hacker News Reader

C++ Asynchronous Framework

userver.tech

232 pointsshcheklein106 comments

Based solely on the one example function on the homepage, this seems to be pre-empting asynchronous functions, presumably a fiber yielding and resuming internally, so that the code looks synchronous. libuv's API is based on callbacks. So both the implementation and the usage are very different.

This uses coroutines so there is no preemption. It is much higher level than libuv. I would say it is higher level than Seastar, which uses callbacks. After about 10 minutes of skimming the docs, I don't see a mention of what the actual coroutine mechanism is, and I don't see any mention of Boost or C++20 coroutines. There are a bunch of IO primitives that operate with their coroutine system but I didn't check to see how they work.

I'd like to see versions of all the usual Linux system calls exposed through io_uring instead of having to use different async options or threadpools for everything. I don't know how close we are to that now. I don't know whether userver uses io_uring at all, but it would be nice if it does.

There is a comparison chart with several other such frameworks in various languages, Seastar is not in the chart, but it would be nice if it was.

DPDK is not mentioned unless I missed it (which is possible). I don't know if it is still supposed to be faster than Linux kernel networking. Seastar has good support for it.

The coroutines are stackful and it looks like you have to allocate space for them on instantiation. I don't know if there is any attempt to catch stack overflows. The obvious approaches using mmap might help though they aren't really sound. There is a hardware extension (CHERI) for pointer ownership on ARM and RISC-V that might be cool for this, so I hope it gets traction.

I've never gotten into the deep weeds of cooperative multitasking, but people who have say it gets ugly as the program gets more complicated, and that it tends to give way to OS's with preemption and processes for that reason. C++ itself is also famously fraught with hazards.

Overall userver looks useful and worth looking into so I'm glad to hear about this release.

>This uses coroutines so there is no preemption.

Right, I meant cooperative yielding (which fibers are), but accidentally called it the exact opposite. Whoops.

>I don't see a mention of what the actual coroutine mechanism is, and I don't see any mention of Boost or C++20 coroutines

It's boost coroutines. The /uboost_coro directory is a vendored copy. It's used in /core/src/engine/coro/pool.hpp

Thanks. I have been wondering for a while if Seastar might someday switch to this approach. I also wonder how the overall performance compares with current versions of Go and Goroutines. I don't like Go much from a pure language perspective, but I do like having a garbage collected runtime. I last heard you pay around 2x overhead for it, which is not that bad for most purposes.

> through io_uring instead of having to use different async options or threadpools for everything

io_uring is a threadpool, and to get performance above epoll, it probably needs to expose the pools and the scheduling choices. eBPF for sceduling? All the simplicity of io_uring basically goes out the window wit that. I added ZEROCOPY to an io_uring instance and it wasn't a fun experience.

It is a very strange interface for networking code. Sending is already mostly async for TCP, and for UDP almost useless (esp without send/recv mmsg calls - you have to make all the packet ordering explicit when create the requests - complete pain in the ass. On the receiving end you have to have an outstanding read request even though you have no idea how much you are going to read (reminds me of the HTTP async hack). The Mmap Packet interface on that is better, just dropping packets off in your address space - for the most part read is a useless call for high perforamnce stuff - you're always reading and buffering. Reading UDP sucks also since no mmsg call and you feel like you're pickign up thousands of pennies off the ground because the asshole wouldn't give you a $10 bill.

It doesn't compose well, hard to debug, etc. There are other interfaces (besides the old, decrepit 200+ verb RDMA/RoCe/iWarp crapfest coming out of the latest standards running jokes). Do something besides BSD sockets on the high end. Anybody rememebr SysV IPC - that might be intereting to try again. BSD netowrking is not the end all of networking APIs.

If you are serious about network throughput, you are doing kernel bypass. Packets show up in your process memory in a ring buffer via DMA, asynchronously, with your code just polling for updates to a head index also updated via DMA from the network hardware. Packets are sent by appending them to another ring buffer and then writing to a device register also mapped.

There isn't a significant difference between this and io_uring, both in terms of performance and API.

With io_uring the kernel is doing that for you (you have control of the affinity and how aggressively the kernel thread polls), and copying it to another ring buffer, which is then picked up by userland.

You still get sub-microsecond latency even with this two-level approach, and the advantage is that it works on every NIC that linux has a driver for.

There is a significant amount of difference for UDP. If you go ahead and profile any UDP based high pps or bps application, you will notice that a lot of time is spent in the kernels network stack - resolving routes, applying iptable rules, evaluating bpf scripts, copying data, and ultimately handing it over to the driver. On something like a QUIC server - this can easily account for 30% time - or even > 50% if optimizations like GSO (generic segmentation offload) are not used.

io_uring does pretty much nothing to help reducing this number. It reduces the syscall overhead, but that one actually isn't that high in those applications. What it also will do is moving the actual cost and latency of system calls from when the actual IO is attempted to the time when `io_uring_enter` is called, which essentially batches all IO. For some applications that might be useful to reduce overhead - for others it will just mean `io_uring_enter` becomes an extremely high latency operation which stalls the eventloop. This symptom can be avoided by using kernel-side polling for IO operations which doesn't require `io_uring_enter` anymore. But due to polling overhead this will only be a viable way for a certain set of operations too.

Kernel bypass (AF_XDP/dpdk/etc) will directly avoid the 30% overhead, at the cost of a reduced amount of tooling and observability.

For TCP the story might be slightly different, since the kernel overhead there is usually lower due to more offloads being available. But I think even there, there hasn't been a lot of proof that this actually provides a much different performance profile than existing nonblocking APIs.

first, if you're using io_uring_enter, you're probably not using io_uring in any way that truly minimizes latency. Kernel-side polling is required not just for that, see the latest work on how it integrates with the NAPI driver infrastructure.

second, there should be no spurious copies if you set it up right; it is designed with zero-copy capabilities in mind.

third, whether or not it evalutes ebpf or wastes other time in the kernel stack is a matter of configuration and tuning.

fourth, there are several articles that already demonstrate that it achieves performance close to that of DPDK, with a much easier and more flexible setup.

Yandex (and ClickHouse) does some really, really cool technical stuff.

I like that a lot of the stuff they write about and open-source has a fairly strong focus on efficiency + performance. I also find the supporting docs (those that detail the problem space, how it solves the problem, etc) quite well written and often a lot more comprehensible than docs from other mainstream tech companies (MS docs are the worst, followed by Google).

I’m not sure why this is the case though, is it a different development culture? Different project management culture (provisions time for documentation?) something else?

Unfortunately, larger tech companies in the US have become increasingly (in the last 5-10 years) political inside and a desire to ship things fast has ultimately meant building extremely high performance scalable infra that's foundational enough to be OSS-able is just not a priority since compute is cheap enough and it's not worth the cost of generalizing it to make it OSS. It's just very hard to pitch these kinds of investments. Documentation follows this pattern-- engineers just don't get promoted or rewarded for writing documentation and with IC's constantly being pushed to deliver faster and faster it just won't take priority. And that's a big problem.

In short, it's my impression it's an engineering culture problem in large part caused by bad leadership at the types of companies where these projects could be possible.

Playing devil's advocate, but why is that a big problem? large tech companies aren't altruistic and leadership isn't stupid. If shipping fast gives them a better competitive advantage than writing OSS, why should they invest in OSS?

Not GP but attempting an answer.

It really depends on the perspective you're looking at the situation from. If you're concerned with better competitive advantages and the final bottom line for company X or Y, in this current market, in the short term, sure, that sounds amazing, sign me up on that moon rocket!

If your concern is a bit more hard to quantify, for example maximizing human potential, allowing individuals access to high quality technological infrastructure so that we may develop as a species and reach new heights, then you might feel like this approach is a bit short sighted.

Or even long term investment in the company. Look at where IBM is now vs where they were 50 years ago; Microsoft would be in the same place except for Satya Nadella completely reinvigorating the company. Who knows where some of the big tech companies now will end up in a decade or two given the shortsightedness that seems endemic.

> so that we may develop as a species and reach new heights

unless the company doing this can reap exclusively those new heights, no one will be altruistic enough to make such an investment to the betterment of mankind!

This role is left up to the publicly funded institutions such as gov'ts.

All else (strategy, technical skill etc. ) being equal, the latter group get out-competed by the former group.

Amorphous long term ideals are great but if you go broke tomorrow you aren’t doing much OSS development next month.

Ehm, which company is working to maximize human potential as an higher objective? Seriously mention to me one! I'll send my CV there immediately!

"maximizing human potential" is not terribly specific and can mean different things to different people.

I'm not looking to be a shill in this thread but we might be what you're looking for, and even if it isn't, I'd love to keep in touch as my own life goals involve leveling humanity up in some capacity.

Happy to share details and answer questions if you want to shoot me an email (info is on my user page).

Heaven’s Gate. Unfortunately they got acquired in 1997. Also, some say they were cult-like.

I recently had the displeasure of consulting a start-up whose pitch was a heavy technical infrastructure solution for other developers. I can't really give details, but their pitch felt to me like hearing about AWS in 2010... then I go into the nitty gritty of how they work. 0 engineering. Just rushed out non-fleshed ideas turned into one-line tickets and sent off to devs around the world without any instruction or guidance. Their employees were producing massive amounts of code, infrastructure, deployments etc.

Their concern was the low velocity which felt insane to me because velocity was huge but in an unfocused direction. I am scared of this new world where paper CEO think truly grand visions come to life by half-assing it and playing the part only to the surface level; like I get it if that's your thing, mooning a company and exiting, it's just so hypocritical to act like you believe in the mission statement when you can't even verbalize it, let alone push it forward.

I am hopeful for a new wave of Venture Capitalism with a focus on fundamentals and truly hard problems that are not going to bring in a 3000% ROI in 2 - 3 week-long sprints.

I can assure you that everything you write is just as true for Yandex as it is for any other big tech company.

Disclosure: used to work there.

This is /extremely/ uncharitable. It depends more on the project.

For example, both Go and .NET are very well written engineering achievements with thorough documentation. Many, many other deeply influential projects have also been absorbed by the Apache Foundation or other FOSS initiatives.

Furthermore, every software megacorp has boat loads of teams working on hundreds even thousands (!) of different open source projects. The quality naturally varies.

Let me pick two more esoteric projects as a point of comparison.

Yandex Odyssey [0] an advanced multi-threaded PostgreSQL connection pooler and request router. Figuring out how exactly and when to use this is not quite clear. There is no "getting started" guide for this package. There is barely any explanation for how it works or what it does.

pg_auto_failover [1] run by Citus (owned by Microsoft) monitors and manages automated failover for a Postgres cluster. This repo even has diagrams explaining the workflow and complete instructions.

Frankly, it's unreal just how much code is open nowadays especially from the giants.

Visual Studio, for example, had been under lockdown for 20 years. Now Visual Studio Code is being developed in the open and has extremely thorough documentation that walks developers through the entire build process in painstaking detail.[2] This is a blazing fast editor written in JavaScript of all things! Or consider TypeScript, an incredible engineering feat. There is an entire repo dedicated to engineering notes on how the compiler works with links to interesting feature contributions and a video! [3]

I'm only picking Microsoft here as the primary example since you dumped on them the hardest.

*No other industry does this. It is absolutely wild how much copyleft influenced software culture.*

[0]: https://github.com/yandex/odyssey

[1]: https://github.com/citusdata/pg_auto_failover

[2]: https://github.com/microsoft/vscode/wiki/How-to-Contribute

[3]: https://github.com/microsoft/TypeScript-Compiler-Notes/

I guess the more new and business facing (cloud and clickops), the worse?

> There is no "getting started" guide for this package. There is barely any explanation for how it works or what it doees.

Welcome to Russian software or hardware. Nginx is an exception with okayish documentation.

Is there a reason or point for calling out Russia?

The global situation, for starters.

… is unrelated to software as far as I am aware

Well, let’s increase your awareness, then.

https://en.wikipedia.org/wiki/Cyberwarfare_by_Russia

Okay, you’re just being obtuse. The original comment was about the quality of Russian software documentation. That is unrelated to cyberwarfare, a subject I wrote about extensively, and earned recognition for. [0]

[0] https://spia.news.chass.ncsu.edu/2022/05/13/john-allison-rec...

You are assuming two things:

1) everyone knows you and what you have written and that makes you the authority (not true, obviously, and kind of makes you look like a top hat)

2) what you have written applies to the situation (not true; Russia is an authoritarian dictatorship where every business entity is potentially a state actor)

My point wasn’t to claim that I’m some sort of famous writer, it was simply to prove I have background knowledge in the subject and still disagree with you. The fundamental flaw of this conversation is you assuming that my disagreement with you is out of a lack of information, rather than simply reaching a different conclusion.

Yes, we have previous experience with products from the former USSR. Good product with basically no docs other than a 10 page PDF, just ask Andrey if you have any questions. We're integrators. One could reasonably expect to sell low volumes in Africa, because any European customer would basically laugh in your face.

Thanks for the actual answer!

I concur that Odyssey docs are insufficient to build a big picture. But I'm trying to answer every question in the Internets about Odyssey :) Documenting is hard - everything seems obvious to me, but understand that may things are not that obvious. // Odyssey maintainer

there is nothing worse than google’s documentation

Google's product naming. Ever tried to search for Google Assistant, Home, Auto, Sheets, etc...? You would think Google of all places would be very aware of useful it is to be searchable.

Certain MS documentation is actually incredible. For instance, the docs for the Win32 API are some of the best out there, rich with notes, examples, warnings, links to relevant topics. For instance, here's the page for CreateWindowEx: https://docs.microsoft.com/en-us/windows/win32/api/winuser/n.... Having never written Win32 code before, I was able to get a window open complete with simple rendering from a bitmap backbuffer in about two hours using nothing but those docs.

Most of the more recent library documentation they have is not great though -- seems like autogenerated junk, mostly.

In my experience, documentation quality for closed Yandex projects ranged from excellent to absent. It is definitely not project management culture. Perhaps, Yandex tries to use its open source projects as kind of advertisement targeted to developers, and puts extra efforts into its quality.

I just briefly skimmed the docs, so my observations may be inaccurate, but I got the impression that "stackful coroutines" are basically green threads (not C++20 coroutines, as the name might hint). Green threads need a dedicated stack per thread, with all that implies, and also I don't find the cute "magical" implicit yielding helpful readability- and comprehension-wise, but that's more a matter of taste.

It is either telling or a glaring oversight (let's assume the latter) that there is no comparison with Seastar, which supports both C++20 coroutines and callbacks, with helpful syntax sugar for latter (coroutines are still nicer though!), has green threads too (which it calls "threads"), and can use io_uring, dpdk, etc. Seastar is free, actively developed, and is used by real software with demanding performance and scaling requirements, so in light of Seastar existing I'd expect any new C++ async framework to approach the task of "selling" itself more seriously than "it's like Go or Python but in C++!".

Seastar is absolutely different approach, based on shared nothing architecture. It's not bad, but you cannot easy compare some async multi threaded architecture with it.

Seastar it's something like multi process architecture, where every process doesn't have synchronization, except n spsc queue per process (used to communicate between cores) and have only cooperative multitasking

So it good scales, if you don't have a lot of data to share, and have a very good work load balancer

But commonly you haven't, so go-like approach more and more easy

> Speed of C++, simplicity of Python, coroutine model of Go.

That sounds great. Can someone with more experience with C++ tell how good this framework actually is?

From the docs it looks nice, and it is taking advantage of several modern features.

Apparently it isn't using C++ 20 co-routines to avoid limiting it as existing support it is still pretty much WIP.

Excellent job, I love Yandex products, and this framework seems to be very useful.

Looks good, I usually roll my own async stuff with C++20 coroutines but I'll have to try this out; seems promising.

“Speed of C++, simplicity of Python, coroutine model of Go.”

Is there a speed comparison to Go?

Seems like same principles used in https://github.com/yyzybb537/libgo

Using libgo for 3 year, still cool. Most important: you add it, add some convenience methods on top of it, and forget about it.

Somewhere in that library is code to yield()... and all I see is a forest of .hpp files (which I assume are the equivalent of .h files for c++)

Where does it actually get implemented?

It internally uses ucontext via Boost

Thank you for explaining that to me. I suspected it was something else and was trying to find the code that saved state and forked in C++ userland.

I once did cooperative multi-tasking, which I thought this library did as well, in the 1980s in Turbo Pascal under MS-DOS, and wondered how things have changed.

No Windows support apparently: https://userver.tech/d9/d08/md_en_userver_supported_platform...

"the" C++ asynchronous framework? How presumptuous.

I don't think that's intentional. Russian doesn't have articles [1], so misuse (or omission) of the/a is not uncommon when Russian speakers write English.

[1] "Definite and indefinite articles (corresponding to 'the', 'a', 'an' in English) do not exist in the Russian language. The sense conveyed by such articles can be determined in Russian by context." https://en.wikipedia.org/wiki/Russian_grammar

Yandex top managers are running away from Russia as their government forcing them to spread hate. They alter their news, search results as government asks. Their tech including DB's is used for military use and is one of backbone companies that keep this bloody regime to run.

No matter how good this product is , Yandex as company is extremely toxic. Contributing to this product you are indirectly contribute in killing people.

They are advertising ( online ) recruitment process of an extremely poor and depressed regions to go and fight in ukraine.

They are one of the major propaganda engine of a regime who combined 18th century imperialist where they are excited to capture new territories and praise it will be good for their isolated economy.

80% of Russians Support war. Hence yandex as company is in charge of something even more frightful actions. They are actually hate ukrainains getting to the level of nazzis that considered some the west european countries a broken version of Germans. To compare. Soviet Union had thousands of Schools in ukrainian, printed good quality books in Ukrainian, they only didn't tolerate politics. Russians don't recognize ukraine as a nation with any rights and opened educational camps. The first thing they do on captured territories they destroy Ukrainain books, schools materials and any relations with culture. They import brainwashed teachers from Russia and forcefully educate kids.

School Teachers are backbone of their regime. Guess who provide them software? Yandex .

Yandex software is definitely used to run filtration camps, forcefully re-educate kids from their native language into Russian and other things no one could imagine would exist in 21st century.

So no, this is not yet another proxy war between two big powers. This a war of reborn 18th century imperialism combined with fascist way of mobilizing russian people.

Is up to you to support their project or not, but one day this new world order can come into your house.

Here's a nice article by the Moscow times about how Putin's regime relies on Russian engineers to build out it's surveillance and censorship apparatus: https://www.themoscowtimes.com/2022/07/29/the-good-engineer-...

Yandex is very much a part of this strategy. It started out as a domestic competitor to Google and as Russia has been banning and censoring Google and other foreign services domestically, it became more important.

So, this parent comment while obviously very political given current circumstances, is actually very on topic. Yandex is part of the information war on the ground in the Ukraine right now. And not just the Ukraine. Russia's sphere of influence spreads quite wide.

A few practical considerations:

- Would you use native libraries like this as part of your architecture that are controlled by people with a history of infiltrating e.g. voting machines, infrastructure used by media companies, etc. Believe what you will but that would be solid no for me just from a security point of view. There's just no way to know what piggybacks along with that code base.

- Given the above, would you contribute to such a project and help the people behind it and further their goals?

I'm sure it's technically great stuff, but no thanks.

> There's just no way to know what piggybacks along with that code base.

Thanks for openly admitting your technical incompetence in the domain.

Well, there seem to be a lot of fools that end up with potential C/C++ based exploits in their infrastructure. Sourcing your C++ libraries from sources infamous for making use of such exploits seems foolish, and indeed incompetent.

This doesn't mean much without benchmarks

Do you know about cython ?

Yandex has better long tail results than google or bing.

If you are trying to find something obscure, yandex is always my final option, before I give up.

Google is basically a Amazon pipeline at this point, I save every useful URL because I know I won't be able to find it again.

If you are trying to buy something google is easily the best, or anything local, or news related.

Bing is a better general search engine, for quick searches.

I'm not sure if you noticed, but the Yandex comment was not about the search, but the software engineering and open source projects coming from them.

We may have found the GPT-3

People have low opinions of yandex, why would they trust their software development efforts if they have low opinions of the company.

As my comment notes yandex has its place in the western search world, and is very useful for long tail searches.

Hardly off topic, while your snide response is.

I wasn't trying to be snide; I didn't understand what the search had to do with the company. everyone I've met (including on this thread) have high opinions of Yandex, so to each their own.

I may have conflated your response to the child response that suggested I was a bot.

I apologise

Reply on news.ycombinator.com