Hacker News Reader

Comment by ncmncm | Hacker News Reader

I wonder if the advent of async/await in all the popular and upcoming programming languages will be seen, after some (buffering) interval, as a disaster of major proportions. And, I wonder how the programming world will respond to the disaster.

It seems advisable to begin that response now. The linked article might be the beginning of such a response, but it seems too tentative. We may need an Iron Law of Flow Control, visibly acknowledged and observed in each system that uses an async/await facility, at the point of use, or a note explaining where it is handled farther back up the chain.

TCP vs IP is an excellent example of such an alternative: IP does not bother with buffering, except as a completely local performance optimization, and happily drops packets at the first hint of trouble, assured that somebody closer to the source has buffered copies of whatever they actually care about.

View on HN

The fundamental problem isn't async code, it's buffering. Async-await patterns simply shift scheduling from the system (kernel or low-level runtime) into the application, but that's it. Languages like Python and JavaScript provide broken async-await APIs that do too much hidden buffering, including unbounded buffering. But that's a stupid design decision on their part, a reflection of poor interface design arguably stemming from a) bolting async I/O onto pre-existing architectures and runtimes that can't accommodate it well, and b) the implementers not having sufficient real-world experience designing and implementing such systems.

But even in threaded code you can have buffering problems. For example, Linux' default socket buffer size sysctls are arguably far too large, optimized for high throughput of a few connections (i.e. benchmarks), not for good responsiveness. That makes it too easy for any massively concurrent service daemon, threaded or not, to flood the outbound network queues.

The answer for any I/O model is to never, ever implement an in-memory input or output sink with an unlimited size, whether for buffering data, connections, or any other resource (e.g. stack frames if an event loop recursively reenters itself). That solves most of the problems right there. The remaining problems are more nuanced, and may require reducing buffer sizes as much as possible (think single-digit Kb instead of Mb[1]), and considering how various resource buffers and APIs interact (additively, multiplicatively) to create hidden buffer bloat.

[1] Especially if you're not doing any heavy processing of the buffer contents, such as audio-video processing where you may need moderately larger buffer sizes to benefit from pipelined and SIMD processing.

What langs have correct (not broken) async-await implementations?

Thank you, this is a better analysis than I hoped for.

> I wonder if the advent of async/await in all the popular and upcoming programming languages will be seen, after some (buffering) interval, as a disaster of major proportions. And, I wonder how the programming world will respond to the disaster.

I think that there is value in async/await as a concept, but that some language's specific implementations will be eventually seen as a mistake. My primary experience is in .Net, and I've grown disenchanted with async/await in that realm. I feel like it makes the easy things easier and the hard things harder[1], which, IMO, is the wrong trade-off for a concurrency primitive[2]. By comparison, what I've read about async/await in Rust seems very promising.

[1] Specifically, I think that the Task API is warty and over-complicated, with abominations such as .ConfigureAwait (I know that there are situations where it can be useful, but those don't make the method any less terrible). While await doesn't strictly depend on Task[1a], in practice you're going to be using Task.

[1a] http://blog.i3arnon.com/2018/01/02/task-enumerable-awaiter/

[2] Before anyone chimes in with "async/await has nothing to do with concurrency/threads---it's just state machines": yes, that is technically true in the narrow sense of how the compiler desugars the async/await keywords. However, as soon as you start talking about actually scheduling and executing your tasks, concurrency/threads become very important.

Threads are state machines. Packaging up task state as a closure and packaging up task state as a stack do roughly the same thing. In Go, the two are closer together than in other languages where async was a retrofit.

Sure, but if you're going to go that route, why not just say "the entire computing system at a whole is just a state machine, so what's the difference"?

Threads provide a very privileged form of state machine that are tightly coupled with the underlying platform. This means you have to be thinking about things like cache coherency and context switches. Without careful use of other synchronization primitives (or stronger language guarantees[1]) it's easy to find yourself writing incorrect or less-than-performant[2] code.

[1] Like in Rust

[2] e.g. creating bottlenecks by blocking on access to a shared resource (Edit: perhaps a better way to word that would be "accidentally reintroducing blocking behavior by using synchronization primitives to access a resource")

async/await in .NET is actually good because they added async equivalent APIs to nearly all blocking operations. There's also always (from what I've seen) an optional cancellation token parameter that allows an easy mechanism to alleviate backpressure.

Need an easy way to cancel 100 async requests? Easy, just pass in a cancellation token and cancel it when needed. All the requests will throw an OperationCanceledException which can be cleanly handled.

I haven't seen anything this nice in javascript or python.

> async/await in .NET is actually good because they added async equivalent APIs to nearly all blocking operations

I mean... I get it---it's great how Microsoft put in an effort to add async equivalents to most of your standard operations. But my complaint is about how almost all of those async methods are built upon Task/Task<T> (which, IMO, is a flawed foundation).

> There's also always (from what I've seen) an optional cancellation token parameter that allows an easy mechanism to alleviate backpressure.

> Need an easy way to cancel 100 async requests? Easy, just pass in a cancellation token and cancel it when needed. All the requests will throw an OperationCanceledException which can be cleanly handled.

No significant disagreement from me there. It is nice that most core async methods support (or try to support) cancellation.

> almost all of those async methods are built upon Task/Task<T> (which, IMO, is a flawed foundation)

I like them, very flexible. It’s easy to implement these tasks on top of pretty much anything.

Here’s how I have implemented back pressure when I needed something similar: https://github.com/Const-me/pCloud/blob/master/pCloudCore/Ut... That class is very generic, knows nothing about the nature of the tasks, yet it’s able to do the right thing. Specifically, it limits count of pending tasks to a small number (8 by default), when exceeded the `post` method will start to “block” in the async way, that’s why it returns `Task<Task<TResult>>` instead of just `Task<TResult>`.

> But my complaint is about how almost all of those async methods are built upon Task/Task<T> (which, IMO, is a flawed foundation).

Do you know any not-flawed implementation?

Implementations as in .Net implementations? Or implementations as in other languages' implementations?

As far as .Net goes, I've not looked around. I feel like it would be somewhat of a fool's errand to develop an alternative async/await implementation because almost all of the core APIs use Task/Task<T>, so you'd either be using Task/Task<T> anyway or re-producing a lot of APIs.

async/await itself is "subtly duck typed" in most of the .NET compilers along with other bits of compiler work (among other examples IEnumerable/IEnumerable<T> in C#'s foreach is also subtly duck typed and not dependent on the specific BCL interfaces, though almost no one would use bespoke interfaces), which is how .NET Core was relatively easily able to add ValueTask/ValueTask<T> and a lot of .NET Core APIs have been slowly migrating to ValueTask/ValueTask<T> without much compatibility breaking trouble.

I actually liked .NET async/await because it came with good usable libraries for async RDBMS operations tied into NT IO completion ports, easing a super common bottleneck; compare to other common RDBMS clients that might offer a facade of asynchrony but are really just blocking and queuing behind the curtain.

The task library (TPL) is a smidgen wordy, as was MSFT's way about ten years back, most folks will never get deep into it though.

> I wonder if the advent of async/await in all the popular and upcoming programming languages...

Not all of them. Go has Goroutines. And Java is settling on virtual threads (fibers) with Project Loom instead of async/await.

I trust you have seen https://vorpus.org/blog/notes-on-structured-concurrency-or-g... ? That seems like the response you are asking for, with plenty of control over how asynchronicity is handled.

> IP does not bother with buffering

Did you mean UDP? But even if so, this description overall seems way off in the weeds.

TCP is not based on UDP, so that would not make sense at all.

The parent describes it quite accurately: IP makes no guarantee on delivery and allows dropping packets, TCP promises to fix that on a higher level for applications that need it.

Nobody's claiming TCP/IP to be based on UDP/IP.

I mean to suggest that buffering isn't a proper feature distinction between TCP and unreliable protocols.

Separately, it seems a tad obscure to use IP as the exemplar here--doesn't most everyone want UDP features on top?

Hardly anybody relies on any of the extras UDP provides, so they are typically just wasted overhead. But user-level libraries offer UDP, so that is what people use.

I've never seen a UDP application that didn't depend on port numbers, which are an extra UDP provides.

> the extras UDP provides

Like what? UDP is lightweight and bare-bones, no?

It adds source and destination ports and a checksum. Computing the checksum is a nuisance, and it is usually inadequate, so it is allowed to be zero in IPV4, but not IPV6.

Normally, if you need a checksum, you need one that cannot be forged, and you need more than 16 bits. So the L3 one is just overhead.

I have never seen an actual use of UDP that looked at the port numbers, although firewalls often use them for filtering.

Name a few UDP-based protocols whose implementations do not listen to only a specific port number, or a rely on port numbers established as part of the protocol flow, then?

This is a weird thread. I too would be interested to know about these uses of UDP that don't use ports to post data to processes (or even traverse NAT I guess?).

Yes, I was completely wrong. PBKAC.

> Computing the checksum is a nuisance, and it is usually inadequate, so it is allowed to be zero in IPV4, but not IPV6.

If IPv6 disallows simpy zeroing the field, that sounds like it's not just a nuisance.

> I have never seen an actual use of UDP that looked at the port numbers, although firewalls often use them for filtering.

I don't know what you're saying here. UDP's ports model isn't optional in any sense, it's just like TCP's.

If you run several UDP-based services on your server, port numbers are how the server knows which packets are destined for which processes.

OK. Too much time spent in kernel-bypass coding.

But the checksum is very anemic, inadequate when correctness matters.

UDP copies IP behavior up the chain.

To be clear: every intermediate router is expected to implement only IP, and completely ignore UDP or TCP. Each discards packets freely.

(Router buyers have taken up violating the design, and meddling with TCP along the route. Yet, the point stands.)

It's the word "buffer" that threw me. Like, the buffer is a concept on the sending or receiving system and a separate concern from the protocol. That said I am certainly open to learning something new about TCP/IP today.

The buffer you're thinking of is a packet-level queue to allow mismatches between physical transmission rates and processing rates, and is separate from the protocol.

The buffer the original comment is talking about is likely the TCP receive buffer, which is used both to allow the application layer to process bytes at its leisure rather than as they arrive, and to allow out-of-order data to be reassembled correctly rather than retransmitted in-order every time.

The receive buffer is documented at the protocol level. Well, honestly I don't recall if the buffer is ever documented in the spec or just common implementation practice. There is a documented "receive window" though, which is how many bytes the receiver is comfortable having "in flight" - sent but not acknowledged. The existence of this window implies the existence of a buffer at least as large as that window on the receiver.

Reply on news.ycombinator.com