Dalin – A C++ non-blocking network library on Linux

Talk about not getting it.

Some technical review for the presented library:

- It doesn't seem to support any kind of backpressure, which is basically a nogo if you want to build something reliable on top. On the receiving side you will always get data pushed via callbacks without the possibility to stop it. On the sending side it will always return void and queue the data internally if it can't be send immediatly. A slow, malfunctioning or attacking remote can DOS you through that. Ways to implement backpressure are pull-style operations like in boost asio (you start a read and get a single callback when it's done) or some pause/unpause functions and buffer treshold indicators (like in node.js). I personally prefer the first model now, especially if the single operations return something like a promise.

- That brings me to the next point: The framework does not seem to have a good way to build composable operations. E.g. build a function that reads a websocket frame from the socket. Or another function that performs the websocket handshake by reading the HTTP handshake request and sending the associated response, but leaving the stream intact for any following user to to be able to use if for sending websocket frames. And ideally all operations should be trivially boundable by timeouts. With having a single receive callback for available data there's mostly a need for complex callback-driven state machines.

- Imho a new good framework should also provide a sophisticated and universal story for cancellation of composed operations. Like backpressure that's a thing which can be avoided for demo applications, but for battleproof production environments it is necessary.

- Thread safety and data races: There seem to be a few issues, e.g. TcpConnection::state isn't synchronized and used from multiple threads. loop_->runInLoop([&](){ this->shutdownInLoop(); }); will segfault or cause undefined behavior if the TcpConnection object was deleted befor the queued functor runs. There also might be some reentrancy issues: In each callback to the user (like MessageCallback) the user might fiddle around with the object and change it's internal state. If that isn't expected and guarded against then code that runs after the invocation of the callback might not work as intended. By the way: That's the situation where js-style promises shine: As the callbacks are not immediatly invoked but in the next eventloop iteration there is less rooom for errors. But of course it costs additional performance.

If these things now sound as a harsh critique let me try to bring it back into relation again. First of all: Network programming is super hard because of "concurrency everywhere". There are always some execution paths that one hadn't thought about before, and it takes a lot of time to learn all the gotchas. I do that stuff now for 8 years, probably had worse assumptions and code for quite some time, and still learn now things every day. It's for sure not a bad idea to write an own library in order to learn these things. For users the well-known libraries (asio, libuv, QT Network, etc.) might be a more solid choice.

Also in my opinion even the big and well-known libraries have some dark corners, there hasn't been a golden solution to network programming yet:

- asio is hard to use, callback that are happening on invalidated objects or operations that are still running and use invalidated buffers are are common problem for non-experts. It get's worse if you use asio with multiple threads (especially one io_service and multiple threads). Composed operations are possible but hard to write efficiently (best put all shared-state between operations in shared_ptr's and write a state machine. Maybe it's better with coroutine support.

- libuv is generally well engineered, but is still doesn't offer lots of support for higher level composed operations and cancellation.

- Netty has some dark corners to watch out for. E.g. if handlers are exchanged during runtime (e.g. for websocket upgrades) or handlers are running in multiple threads there's quite some room for errors on the user side. Otherwise it's a well engineered framework in many areas.

- node.js stream abstractions with their multiple modes can be tricky to understand and to use for higher-level abstractions. I guess if networking would be based on promises with async/await support it would be easier now - but those weren't available back then.

Imho the most promising concepts for network programming are the coroutine based ones. Go's model works well, is on the easier side to use (even though goroutines/threads introduce more possibilities for race conditions). It ticks most boxes, e.g. backpressure is naturally available through sync APIs. Cancellation could probably be improved.

Martin Sustriks experiments with libdill also look interesting. As well as Kotlins coroutine model.

The promise based models with async/await are also quite promising, e.g. in C# or node.js. However I find those models fall a little bit down as soon as they go to support both multiple threads and an event loop. Having to remember which callbacks/continuations run on which thread and what is allowed there is cumbersome. Imho either multiple threads with synchronous code or a single thread with an eventloop is ok.

Ooops, that was now a little bit more text than I intended to write. But maybe it helps someone.

On the otherhand, what the heck is wrong with using the syscalls? Every time I write a new high performance networking application, sure, i lose a day or two building primitives out of send recvmsg listen accept and splice, but once i'm done, i'm done, and things work.

I'm confused why people feel the need to make paper thin abstraction layers covering the berkeley sockets API. In the time it takes one to learn how to use the flavor of the month, one probably has gotten halfway there to understanding how to Really do networking.

It's not like networking on Linux is even a shred harder than easy...mumbles off into the distance unlike BLE on Windows10 (and linux)....

Tangentially, I positively can't WAIT for Networking support in c++20 -- that and Concepts I've been waiting for with baited breath.

[EDIT: apologies, i think my tone got away with me.. stupid aspie tendencies... what i MEANT to say somewhere in there, directed to the author, was: "Good Job!!" It's hard work to put yourself out there and enjoy accomplishment. Please do let the rest of the community know if you'd welcome help in porting it to other platforms and adding features and improving performance. Best!]