Comment by neonsunset | Hacker News Reader

Comment by neonsunset | Hacker News Reader

Parent Full thread

.NET deserves a good, separate non-ASP.NET Core+Kestrel web server.

The reason for this is the first-party solution has to be both fast and support a lot of different features, and be compatible with all the ways to process request input/output, parameter bindings, rich telemetry (particularly OTLP) integration and so on and so forth.

Which is why a lightweight UTF-8-based zero-copy pipeline (via span/memory) that tries to reduce call stack depth and context switching, and that moves as much of the feature composition to compilation time as possible could be indispensable.

Such server could be built on top of either raw `Socket` and `SslStream` (of which a toy attempt, really, can be thrown together in under an hour) and its async engine or via a custom one - we have all the tools to build an io-uring network stack with async integration.

.NET's compiler is way better than any other GC-based platform except OpenJDK/Graal but JVM has few features to optimize this further and bridge the gap with C++ and Rust based applications, unlike .NET.

There is a lot of raw runtime performance left on the table and an alternate implementation that gets back to the top of the chart on Techempower would be a welcome change :)

After all, this already has been proven to be possible by Garnet: https://microsoft.github.io/garnet/docs

Last time i looked Kestrel already uses most of the techniques above ( sans an IOUring backend for Socket ) Almost all allocations are pooled, and zero copy as well. Header parsing is even done with System.Runtime.Intrinsics using SIMD where possible.

The higher level ASP.NET Core stack is also quite efficient and optimized.

BUT: as soon as you gove above the basic middleware pipeline its tends to get bloated and slow. ASP.NET COre MVC is particulary bad.

System.Text.Json is also quite nice, and often is allocation free.

We bascially just us the middleware pipeline and nothing else, and can get millions of requests per second on basic hardware.

This is absolutely true, there is a lot of performance work invested at all layers or features that benefit from improvements in runtime itself.

As you noted, the problems happen later in the handling pipeline. There are also choices that ASP.NET Core has to make as an extremely general-purpose web framework like that.

System.Text.Json is pretty good for a default serializer, but it's far from how fast json serialization can be done in .NET too.

Both of these end up reallocating and transcoding data, and STJ also might take a hit if deserialized classes/structs have converters in them as it can disable a fast-path.

My idea here is that a new implementation can benefit from the hindsight of strength/weaknesses of asp.net core and how its design influences the implementation choices of the user (and whether they end up being optimal or not).

It would also not be constrained by backwards compatibility or certain organizational restrictions - for example you do not need to explicitly use out-of-box QuicConnection and QuicStream that rely on mquic and opt to statically link to parts of the stack implemented in Rust, or bring over more logic to C# instead. There is a certain conventional way you are expected to approach this in dotnet/* repositories, and it might be a bit restrictive in achieving end goals of such a design.

It would be able to approach the problem as a library that expects a more advanced user, closer to how e.g. Axum or back in the day actix-web did (and by advanced I mean effective usage of (ref) structs and generics, not that it would need boilerplate).

p.s.: does this organization with millions of RPS have a name?

Reply on news.ycombinator.com