Hacker News Reader

Comment by sp33der89 | Hacker News Reader

I've seen the posts about all the speedups each new version of .NET gets and I'm just wondering, was .NET just alright in performance before all this? Is that the reason they can get all these speedups? :P

I'd be interested in some JVM vs .NET 6 benchmarks too, which platform to chose when.

EDIT: I know about the benchmarks and I also feel like sometimes these benchmarks are really optimized in a non-idiomatic way. I would love to know how performance idiomatic Java/.NET code is and if one is to start a new project today why one would choose the JVM over .NET or when someone would chose .NET over JVM.

View on HN

C# was never really slow in the same way as python, etc. And anyway 99% of the time you're gonna be slow because the sack of meat writing the program screwed up some aspect of the system design or the code. Unless you're using something really shit-tier for perf.

The gap closed significantly with .NET core, which is why everyone was quite surprised when .NET 5 (the next iteration) had a fairly significant speedup in many scenarios.

For reference, Stack Overflow was running on .NET MVC on like 2 servers pretty recently (with some auxiliary infrastructure for CDN and search) and using MS SQL.. I think it might still be running on this setup but not 100%. Honestly I have no idea how they do it on a .NET monolith but there you go.

> Honestly I have no idea how they do it on a .NET monolith but there you go.

Low latency in a single rack can work wonders for performance. All those cloud services talk to each other over miles of cables and if you can slash latency to submillisecond regions, you get less wait time and free resources quicker. If you don't distribute your state across multiple Microservices, you can also save quite a bit of overhead.

Plus hardware is just wicked fast nowadays and SO has a model of millions of reads for a single write.

> Stack Overflow was running on .NET MVC on like 2 servers pretty recently (with some auxiliary infrastructure for CDN and search) and using MS SQL.

They are absolute beasts of machines. But yes, it’s incredible what you can do with just a few colocated physical servers.

It suggests to me lots of dev shops pay the IaaS or PaaS cloud tax, long before they would have hit any scaleabilty walls with a non-cloud setup.

A poster above claimed the servers were “48 logical cores (2 x 12-core Xeons) and 64GB RAM”, which really isn’t what I would consider such a “beast” of a machine when the RAM is in laptop territory, and a modest number of cores for a server.

Nowdays you can purchase that machine on the second hand market for something like $200-$300.

They are measurably faster than even contemporary laptops though plus you often get ECC ram and raid disk setups and the good old Xeon didn't used to ramp up and down in speed, it just ran fast all the time. I'd still characterise that as a beast, especially on $/performance terms (although the power consumption is a worry).

Xeon covers a pretty wide variety of chips. Of course, the Pentium II Xeons didn't have speedstep either. 12-core tells us either fairly high end but older or kind of medium-low but newer. Dual socket tells us not the really low end Xeon that shares a socket (and probably a lot more) with the high end enthusiast desktop chips.

Don’t we have server machines with multi-TBs of RAM nowadays?

Late reply, but you make a good point. Perhaps I am thinking five years ago, and back then the specs seemed more impressive.

Also, I hadn't twigged that “logical” cores are distinct from physical cores, which made the core count seem more impressive.

> And anyway 99% of the time you're gonna be slow because the sack of meat writing the program screwed up some aspect of the system design or the code. Unless

Or in my experience, most of the time a service's latency is dominated by out-of-process calls, e.g. the time taken to talk to other services over http, or to retrieve data from a data store. Speeding up the runtime is welcome, but even a massive 40% speedup of something that constitutes 10% of your total latency is ... closer to a 4% reduction in latency. Design matters more.

If your program is slow because it's waiting for external service response, you're doing programming wrong. Your program should do other work in the meantime. I guess you're already doing that, but if so, doesn't that invalidate your reply here?

> you're doing programming wrong. Your program should do other work in the meantime.

It's not always true that there is other work to do in the meantime. In fact in my experience, it seldom is. "you're doing programming wrong" is a very strong statement, and not one that I take seriously in this context.

Typically you "await" the external service response, so that it is not using a thread to do that, and "other work" in the form of starting to deal with other requests can happen in the meantime, thereby increasing service throughput.

But that won't speed up a given request - you can wait for an external service more efficiently, but you can't wait faster.

Services that do not depend on any other http services or any data store do happen, but they are rare in my experience (calculation engines, I suppose). So for almost every service, when thinking about response time, you have to, first and foremost, think about the latency of the data stores or upstream services.

That is all true, but in the context of .NET performance, it's not relevant.

What then is, in your opinion, most relevant?

Time taken by processing done by the application itself. Faster processing is good even if your backend is mostly waiting for other services - now it can wait for more services at the same time and deal with their output faster, allowing your app to use less resources or handle more traffic. That is especially pronounced on scale - if you're spending X million for computing time a month, reducing it by few percent is very interesting to you.

Both can be relevant. In the real world, designing how to deal with dependencies and data stores is clearly relevant, as it can be the largest large part of the time taken to respond to a given request. It would be a design error to ignore it.

Yes, but external services are irrelevant to performance of . NET.

If your job is to make performant services, in .NET (or in any similar language) external services are 100% relevant to your job.

If You want to be pedantic - and you most certainly do - then only .NET performance itself is relevant to performance of . NET itself, that's a truism, as defined.

But this narrow focus is not useful - if you want to do the job, then you have to think a bit more widely, and understand what the real problem is.

Sorry but this is not pedantic at all. You're saying performance speedup is irrelevant because most time is spent waiting for external services... But that is simply not true if you're beyond just a few servers. Output of external services needs to be processed and having faster software means less resource is spent per request, thus more requests can be served for the same resources. That is very important.

> You're saying performance speedup is irrelevant

I am not saying that; you are oversimplifying into a straw man. great-grand-parent comment is where I literally put a non-zero number to how relevant language perf improvement was: https://news.ycombinator.com/item?id=29295950 and later on "Both can be relevant"; which all contradict your characterisation.

The only person who said above "it's not relevant" is you, and you also said "external services are irrelevant" - You're projecting the "it's irrelevant" statement onto me here.

But the design considerations of how and when to use external services very definitely are relevant to service latency, contrary to what you say, for reasons given multiple times above. Your current odd comments are not fact-based or interesting, so I don't think that you have anything more to add to this discussion at all.

>If your program is slow because it's waiting for external service response, you're doing programming wrong. Your program should do other work in the meantime.

What work? Mine bitcoins while you wait the result for an API call?

Usually processing other requests

I think any sane person here assumes any IO, especially in .NET is already async and allows other requests to use that time… Most web apps are still IO constrained.

Sure, but how is waiting for external requests relevant in any way to .NET performance then?

If you're asking "how is waiting for external requests relevant to my service's response time" the answer is "because it's usually the largest part". I really don't know how to explain it more simply than that.

If you need to improve this, and that code is .NET, then the solution is a different design, also in .NET Code.

I am asking how is waiting for external services relevant to .NET processing time.

I refer you to my answer above. https://news.ycombinator.com/item?id=29306220

Looks like SO's architecture is still pretty simple, yeah. https://stackexchange.com/performance

Stack Overflow runs on 9 web servers with (iirc) 48 logical cores (2 x 12-core Xeons) and 64GB RAM. Those servers are shared by a few apps (Talent/Job, Ads, Chat, Stack Exchange/Overflow itself) but the main app uses, on average, ~5% CPU. Those machines handle roughly 5000 requests/sec and were running .NET 5 as of September 2021 (when I moved on). That’s backed by 2 v. large SQL clusters (each consisting of a primary read/write, a secondary read-only in the primary DC and a secondary in the failover DC). Most traffic to a question page directly hits SQL - cache hit ratio tends to be low so caching in Redis for those hits tends to be not useful. As somebody mentioned below, being just a single network hop away yields really low latency (~0.016ms in this case) - that certainly helps being able to scale on little hardware - typically only 10 - 20 concurrent requests would be running in any instance at any one time because the overall request end-to-end would take < 10ms to run.

Back in full framework days we had to do a fair bit of optimisation to get great performance out of .NET, but as of .NET Core 3.1 the framework _just gets out the way_ - most memory dumps and profiling subsequent to that clearly pinpoint problem areas in your own app rather than being muddied by framework shennanigans.

Source: I used to work on the Platform Engineering team at Stack Overflow :)

That's some great info, thank you!

> cache hit ratio tends to be low

That's surprising to read. Is that because of the sheer volume of question pages? I don't think I've ever been on an SO page that couldn't have been served straight from cache.

Is it? Most people come to SO from Googling their random tech problems/questions. Not sure how much value there is in caching my random Rails questions, etc

I would expect SO usage to follow a distribution like Zipfs — most visits hit a small subset of common Q/A, and there’s a ridiculously long tail of random questions getting a few visits where caching would do next to nothing. I’m fairly positive I’ve seen some post showing this was true for atleast answer-point distributions.

Though I guess it’s possible for a power distribution for page-likely-to-be-hit to still be useless for caching, because I think you could still get that distribution if 99% of hits are on nearly-unique pages; with a long enough tail, you’d still have only relatively few pages worth bothering to cache, but by far most visits are in the tail

.NET has been faster than Java on most of the benchmarkgame benchmarks for a while, since .net core 3 or so.

More specifically though the JVM has tended to be better about optimizing naive code than .net while .net has tended to offer more tools to do your own optimizing (unsafe, simd, value types, etc). So it would be interesting to see if the performance of naive code has improved relative to Java lately

Yeah, I would love it if I could just write idiomatic code for the platform and it'd be just fast enough!

> .NET has been faster than Java on most of the benchmarkgame benchmarks for a while, since .net core 3 or so.

And which benchmarks games are those? If I go to to the Techempower benchmark and select only C# + Java. Java comes on top in every individual category of all the benchmarks.

I'm not claiming that Java is faster than .NET. Just that I don't believe one platform is significantly faster than the other.

[1] https://www.techempower.com/benchmarks/#section=data-r18&hw=...

Such programs are often specially and painstakingly constructed to avoid all the commonly used language features that are inefficient. For example, in Java, user-defined data types are heap allocated and generic code boxes everything, even primitive types (an ArrayList of ints becomes unfortuately an array of pointers).

Are these programs benchmarking typical idiomatic Java, or just some subset of the language?

>> Are these programs benchmarking typical idiomatic Java

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

c# regex redux - 1.42 seconds

java regex redux - 5.31 seconds

Ok... but looking at the code:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

    import java.io.*;
    import java.util.*;
    import java.util.concurrent.CompletableFuture;
    import java.util.Map.Entry;
    import java.util.function.*;
    import java.util.regex.*;
    import static java.util.stream.Collectors.*;
    ...

It's only using vanilla Java features.

c# ?

    ...
    using System.Runtime.InteropServices;
    ...

Interesting, why does it need that?

        [DllImport("pcre2-8", EntryPoint = "pcre2_compile_8", CharSet = CharSet.Ansi)]
        extern static IntPtr PcreCompile(string pattern, long length, uint options,
            out int errorcode, out long erroroffset, IntPtr ccontext);

        [DllImport("pcre2-8", EntryPoint = "pcre2_jit_compile_8", CharSet = CharSet.Ansi)]
        extern static int PcreJitCompile(IntPtr code, uint options);

        [DllImport("pcre2-8", EntryPoint = "pcre2_jit_match_8", CharSet = CharSet.Ansi)]
        extern unsafe static int PcreJitMatch(IntPtr code, byte* subject,
            long length, long startoffset, int options, IntPtr match_data, IntPtr mcontext);

        [DllImport("pcre2-8", EntryPoint = "pcre2_match_data_create_8", CharSet = CharSet.Ansi)]
        extern unsafe static IntPtr PcreMatchDataCreate(uint ovecsize, IntPtr mcontext);

        [DllImport("pcre2-8", EntryPoint = "pcre2_get_error_message_8", CharSet = CharSet.Ansi)]
        extern unsafe static int PcreGetErrorMessage(int errorcode, StringBuilder buffer, long bufflen);

        [DllImport("pcre2-8", EntryPoint = "pcre2_get_ovector_pointer_8", CharSet = CharSet.Ansi)]
        extern unsafe static IntPtr PcreGetOvectorPointer(IntPtr match_data);

        [DllImport("pcre2-8", EntryPoint = "pcre2_substitute_8", CharSet = CharSet.Ansi)]
        extern unsafe static int PcreSubstitute(IntPtr code, byte* subject,
            long length, long startoffset, int options, IntPtr match_data, IntPtr mcontext,
            byte* replacement, long rlength, byte* outputbuffer, out long outlength);

Aha! It's because the c# impl is really just a wrapper round a native C impl of the problem.

In what world is this a useful comparison?

The fastest "real" c# solution is still faster than the java one though:

c# (real) - 3.1 seconds

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

I agree. That's really not usefull comparision. They should create categories for each benchmark, like:

  - very naive code (shortest, most readable & easy to write code)

  - idiomatic code

  - optimized code without other-language-libs wrappers and without SIMD, single threaded

  - optimized code without other-language-libs wrappers and without SIMD, multi-threaded

  - optimized code without other-language-libs wrappers and with SIMD and/or multi-threaded

  - optimized code with other-language-libs wrappers allowed and any other optimization technique

You are free to take the data, create whatever categories you want and publish.

How would that undo the unfair comparison done in the benchmark game?

1. Not “unfair”.

2. We wouldn’t look at the benchmarks game if we thought an alternative presentation was more useful.

I agree it's not a useful comparison. That's why I don't give much weight to statements such as "Java comes on top in every individual category of all the benchmarks".

> In what world is this a useful comparison?

In this world where we also compare to C++ programs.

In this world where — as you acknowledge — we also compare a C# regex-redux program that does not use a third party library.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

> Interesting, why does it need that?

2 out of 10 tasks regex-redux and pidigits accept third-party libraries, the other 8 out of 10 tasks do not.

> In what world is this a useful comparison?

If you're also separately benchmarking the same C library running on its own, then it's quite interesting to benchmark a .NET wrapper around the exact same library, as it allows you to estimate the overhead from the .NET runtime itself as separated from user code (ideally you'd try this with a bunch of different C libraries).

Of course, the program should be very very clearly labelled accordingly. Since it was just labeled as "csharpcore", then I am inclined to think the submitter was treating the benchmark as a competition.

Yes any benchmark will be invalid for some people, such is life. If you want to claim or know something specific you will have to so your own painstaking investigation or find someone who has done that work.

benchmarkgame does not attempt to compare idiomatic solutions for languages, it is closer to a “what is the best you can do” benchmark

> It is closer to a “what is the best you can do” benchmark

As I suspected. So of course this tells us very little about how fast idiomatic code is relative to other languages. "The best I can do" is to invoke hand optimised assembly language, but rarely is that the right choice.

A much more useful test would involve benchmarking some similar real world apps that solve the same problem.

Please show the objective rules that could be used to identify "idiomatic Java" and "idiomatic C#".

Please show the objective rules to direct how comparison should be done when one languages "idiomatic" is not the same as some other languages "idiomatic" — to avoid you can write Java in any language.

Generic boxing should be fixed “soon” if they ever release Valhalla.

Please show the objective rules that could be used to identify "typical idiomatic Java" and "typical idiomatic C#".

Please show the objective rules to direct how comparison should be done when one languages "typical idiomatic" is not the same as some other languages "typical idiomatic" — to avoid you can write Java in any language.

There's some low hanging fruit, like not permitting specialized collections in Java for a set of integers. Because of type erasure these are all heap allocated in Java but not in .NET.

I think there's value in benchmarks showing both the fastest you can go if you need to (specializing everything to eke out max performance), and benchmarks showing how fast you will typically go if optimizing for productivity.

> … how fast you will typically go…

Why would we think that would be similar for both you and `grumpyprole`.

You could probably constrain it sufficiently for some set of problems. Maybe something like: Solve problem X using the standard library associative map by elaborating the following pseudo code.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

> And which benchmarks games are those?

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Be aware that many implementations on benchmarksgame are much lower-level and using all kind of performance tricks than what you would normally write.

well the same for techemporer benchmarks. sorry but some of that stuff is as shady as the benchmarksgame. I really don't get it, why people don't create 100% benchmarks instead of specialized ones.

> … shady…

The benchmarks game puts the source code under bright light, a couple of clicks from the measurements.

That's why people have noticed differences and commented.

This: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

techempower is isolated to web framework testing as far as I know

While I'm not super familiar with the Java world, none of the frameworks that have a significant advantage sound familiar to me - I'm not sure how mature are they, whereas Asp.NET is the solution for writing servers under .NET.

Java's performance hasn't really mattered since Oracle took it over. There are things MUCH worse than poor performance, and being owned by Oracle is one of them.

Ownership by Microsoft isn't great either, unless you enjoy jumping through hoops to disable their telemetry[1].

[1] https://github.com/dotnet/sdk/issues/6145

The alternative being no one would have bought Sun and Java would have died as Java 6.

I understand Sun ex-employees have an axe to grid with Oracle, yet no one else cared for Sun assets.

Will this bullshit ever die?

OpenJDK has the same goddamn license as the linux kernel. It is (yes, the open source codebase) developed by Oracle 98+% alone, and other vendors are just forks of this code base (including oracle jdk, which contains only trivial changes AND a paid support option, for those that need it)

You can hate Oracle as much as you want but their Java division is a surprisingly adapt and capable team, doing very great job at stewarding the language.

IMO that link makes .NET look very good. Aspcore, the straight off the shelf, obvious choice, is the best performing .net server? It beats Jetty and Spring but loses to a long tail of less popular frameworks

You're looking at pretty old results, round 18 was in 2019. I also don't think that boutique web frameworks say much about the strength of the underlying language or runtime (e.g. look at just.js).

What in the Java world is in the same maturity tier as ASP.NET is open to opinion, but at least local Java devs seem to consider Spring or Micronaut as sane defaults, and of course modern ASP.NET runs circles around those.

https://www.techempower.com/benchmarks/#section=test&runid=9...

That seems a bit misleading of a comparison IMO and only one case (JSON serialisation) when I look at their data. You are also linking to a round from two years ago which is out of date. It's also showing a lot of frameworks that are not that mature and not well used in the Java camp vs ASP.NET that is widely used, full featured, has a lot of bells and whistles and a lot of plugins available for most technologies and standards. All of which could have negatively influenced performance, even the hooks to allow them to be injected in can do so even if not enabled. The fact that a full featured web framework makes it close to the top (sometimes the top) over several rounds over many of their categories of use cases I can't discount as pretty good.

i.e. Its hard to read benchmarks without context of each framework shown, the compromises they have taken, how usable it actually is for building software vs just a benchmark, what shortcuts are done in the benchmark, how idiomatic is the code, etc.

https://www.techempower.com/benchmarks/#section=data-r20&hw=...

My personal experience having worked on both platforms for several years is that Java is easier to get to an acceptable performance, but the .NET runtime when you have to put the effort in has a higher upper bound of performance. It just has more tools in the CLR to work with than the JVM (e.g. value types, proper generics, spans, and more) so you can express something with a little more mechanical sympathy. Java is left with some decisions from legacy IMO that by default hurt its performance (i.e. lots of default boxing has hurt me before especially with generics). With .NET Core and future versions I think .NET is also taking up Java's default perf area as well. YMMV but if I'm worried about performance being a risk in my project .NET gives me more tools to optimize it IMO should that risk eventuate.

> benchmarkgame benchmarks

Lol look at the code. N-body for example, the C# is horrific C-in-C# code with a million optimizations (just read the comments lol), the Java code is idiomatic and not optimized at all.

Well the fastest C# entry for n-body looks like a translation of the C/C++ versions. It is a meaningful result that the primitives of the language allow for it to hang in that company. The absence of a Java version using numerics is unfortunate, that'd be a nice addition.

A lot of the coding style seems optimized around copy-pasting the C code, e.g. trying to alias the Vector methods (Vector256.Create) to their instruction name (_mm256_set1_pd). That makes the code non-idiomatic, but it also doesn't really help performance, just makes the porting easier.

The F# example is on the same runtime and a better view of using the numerics directly. As a trade-off of performance, memory, and code complexity it is actually a pretty solid balance, which I wouldn't have expected.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

How is it meaningful?

I can inline C code in Ruby, does that mean Ruby is as fast as C now?

I'd much rather see a comparison of idiomatic code in different languages. When I choose a language to build something in I'm not thinking "How can I write C in this language?"...

Please show the objective rules that could be used to identify "idiomatic code in different languages".

Inline C/ASM is literally a different language. System.Runtime.Intrinsics is a library. Just because this one benchmark used the library in a C-like style doesn't mean that's required.

There are many simpler implementations in all languages. I actually like that there are multiple implementations, as this lets us estimate the benefit and complexity of adopting various optimizations. Limiting the benchmark to naive implementations would penalize languages with more broad capabilities for optimization.

I'd personally prefer a benchmark limited to memory-safe implementations, though.

Yes look at the code!

Look at this C# program —

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Look at this C# program —

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

And they're slower than Java... Albeit super close. The parent I responded to was referencing how C# is supposedly faster than Java... But idiomatic C# is about the same (slightly slower).

Isn't that writing Java in C#?

Why should C# programs be limited to what can be done in Java?

You can do the same thing in Java, but apparently no one wrote it yet because it's just a silly micro-benchmark. Either way I'd rather see idiomatic code. Or just write C/C++ directly.

> I'd rather see idiomatic code

As you know simpler C# programs are shown.

Well .NET are going in the same direction as Java - as the top comment here observes the speedups come mostly from profile guided optimization which has been Java's forte since Java 1.1 or so. And .NET PGO is off by default, still, so it's got a long way to go yet.

Just to add, Java now has simd in the form of the incubating Vector API. I found that it is one of the best high level low level SIMD APIs, with automatic fallback to for loops on ineligible hardware, as well as having an option for preferred lane width.

.NET has the ability to handle your memory layout explicitly with structs.

They expanded on that functionality with span and made sure that the common libraries is implemented using this.

If you do textbook OOP development for everything, you will end up with a lot of allocations and what not, which was the case, so they went through the entire base class library and rewrote all often used methods to be faster.

they even posted a bunch of posts with all their improvement tricks: https://devblogs.microsoft.com/dotnet/performance-improvemen...

>If you do textbook OOP development for everything, you will end up with a lot of allocations and what not, which was the case, so they went through the entire base class library and rewrote all often used methods to be faster.

They forgot to tell the others that there is life beyond OOP and GoF design patterns.

C# is filled to the brim with functional programming features. Much of the base class library is somewhat functional (although obviously not all or even most, given the age of the BCL and stability of the API).

> was .NET just alright in performance before all this

For some niche applications (i.e. financial exchanges), .NET 5 [was/is] arguably the fastest way to implement certain ring buffer abstractions because of its interesting blend of performance and safety. There is a variant of the LMAX Disruptor developed for .NET which leverages the value semantics of the C# struct to push things beyond what the Java implementations are capable of [0].

Certainly, with enough resources and manual memory management, you could best the C# implementations using a C/C++/ASM codebase, but this is a tenuous tradeoff with practical risks that must be accounted for.

  [0] https://medium.com/@ocoanet/improving-net-disruptor-performance-part-3-introducing-the-valuedisruptor-5b467730bbe

Mark and sweep garbage collection is optimal for some kinds of multi-threaded algorithms. If further minimised with judicious use of value types, it can be surprisingly difficult to outperform it even with carefully tuned C++ or Rust code.

It’s not the same, but there is this well-known framework benchmark [0], it always had the .net frameworks close to the top.

I’m guessing a lot of the speedups come from getting rid of legacy cruft. With .net core/.NET 5/6 they got rid of a lot of things compared to .NET Framework 4.8 and could play with optimizations that simply weren’t doable before. That’s just me guessing, though ;)

[0]: https://www.techempower.com/benchmarks/

.NET Core also introduced some new CLR features that are incompatible with the CLR used in .NET Framework. Span<T> for example.

This has happened before with .NET Framework 2, 3, 4 etc. but instead of making a .NET Framework 5 they rather made .NET Core cross-plattform and threw backwards-compatability-at-all-cost out of the window. While all .NET Framework applications (except the ones that do naughty stuff with reflection) that were compiled for .NET Framework 4.5 behave the same way on .NET 4.8, .NET (Core) got rid of this and lets developers bundle the CLR directly, giving them more leeway for incompatible changes.

It's that in part. Here are some of my additional observations and or guesses.

They invested a lot of time adding language features with compiler and runtime support to avoid e.g. heap allocations/copying, like Span<> and friends, (readonly) ref structs, in/ref/out parameters (ref and out parameters existed before but were used a lot less in the runtime), or ValueTasks to some degree. This in turn enabled a lot of potential for optimizations in the compiler (aside from essentially writing an entirely new bytecode compiler with Roslyn and entirely new JIT with RyuJIT, throwing out the crufty old compilers), in the general runtime, and in the specific runtimes/frameworks e.g. ASP.NET. Those optimizations have to be implemented first however, and more and more get implemented with each new version.

I have a project I maintain that sees an almost 50% speedup from net48 to net5, and another 10-15% speedup from net5 to net6 (based on the time it takes to run the extensive test suite). It isn't even that compute heavy. From profiling it appears that a lot of these speedups are due to internal copies of data being avoided, and a lot of additional fast-paths in the runtime (e.g. fast-paths for byte-arrays or character-arrays as opposed to taking the generic array slow paths).

Another thing of note is that they added a lot of `bool Try*(..., out result)`-style APIs meant to avoid exceptions and the associated handling, and switched a lot of internal code to use these functions. E.g. in the reference source of the net48 runtime I think there are still a lot of instances of

   try {
     var number = int.Parse(value);
   }
   catch {
     // slow path/error path
   }

instead of the new-idiomatic .netcore and later style of

   if (!int.TryParse(value, out var number)) {
     // slow path/error path
   }

try-catch was/is slow-ish, and throwing exceptions is too, aside from it preventing inlining by the JIT a lot of times.

And while #nullable (source annotations for what is nullable or not) and associated annotations such as MaybeNullWhen() had no direct influence on how the compiler could optimize, it probably helped people a lot writing correct code and as a side effect a lot more code became compile-time provable non-nullable which enabled further optimizations e.g. generating code that skips redundant null checks.

> the new-idiomatic .netcore and later style

`Int32.TryParse` has existed since .NET Framework 2.0, which was released on February 13, 2002: https://docs.microsoft.com/en-us/dotnet/api/system.int32.try...

Right, this one has, a lot of other public or runtime-internal Try* methods have not.

And even tho this particular one has existed for a long time, that doesn't mean it was used consistently in the runtime or in the popular first and third party frameworks.

I'd argue the Try*-style, while artifacts of it were present before already, only really became widely idiomatic with dotnetcore.

Surprising that in 2021 Java still doesn't have this in its standard library

Well, the Java approach is philosophically somewhat different. C#/.NET is closer to C++ where they are very willing to complicate the language and APIs to make the job of the runtime or compiler easier. Java just philosophically refuses to do that, more or less (perhaps you could argue that's changing a bit now with value types).

So in Java they just made exceptions really fast. There are lots of runtime optimizations around exceptions, for example, if you regularly parse strings that aren't numbers then the resulting exception will automatically stop having its stack trace filled out, which makes throwing drastically cheaper. The JVM can also inline the code and then convert try/catches to ordinary gotos.

Honestly I really like C#'s outvar and return success idiom. It's soooo ugly and yet slick at the same time. C does the same thing but I think the inline out var declarations make a huge difference to using them. Of course you miss out on error context an Exception or Result<T, Err> gives you but for many of the Try* functions it really doesn't matter.

Yeah, I became a fan of that too.

In a lot of cases Try* is the outright right approach, too. Like `IDictionary.TryGetValue(key, out var value)` is better than try { var value = IDictionary.GetValue(key); } catch (KeyNotFoundException) {}` and has no race like `if (IDictionary.ContainsKey(key)) IDictionary.GetValue(key);`.

Try* functions are still free to throw in actually exceptional cases, just not on generic not-so-exceptional errors.

If you really need context, there is nothing from stopping you from implementing Try* functions in your own APIs that either have another out param for the error information, return the error information instead of a bool or use a Result<T, Err> kind of type (or a tuple), either.

In the dictionary case, TryGetValue also saves you from having to hash and lookup twice unnecessarily (once for ContainsKey, and once for GetValue).

Thanks. This is an interesting comment…I had not realized coding styles in c# had changed in this way

There are two things:

(1) .NET Framework was slow and had some bad habbits (e.g. heap allocations, reflections, little optimizations, etc) ... especially the web stack. .NET Core/.NET fixes that, issue by issue. And since .NET is historically very close to the underlying platforms, we now see competitive outcomes (to e.g. Go, C++, etc).

(2) Performance = lower CPU/Memory Allocation = more throughput = lower Cloud costs. At scale, that makes a huge difference.

Software benchmarks are super subjective. Michael Larabel at Phoronix and Isaac Gouy of the benchmark game have done a lot in this area. But everyone says you need to take it with a gain if salt (which is often true).

There's also TPC-C benchmark suites where people benchmark their own software and claim results. Not really independent journalists there.

> Software benchmarks are super subjective.

No, they are not, but they are just a measurement tool, not a source of absolute truth. When I studied engineering at ETH we learned "Who measures measures rubbish!" ("Wer misst misst Mist!" in German). Every measurement has errors and being aware of these errors and coping with it is part of the engineering profession. The problem with programming language benchmarks is often that the goal is to win by all means; to compare as fairly and objectively as possible instead, there must be a set of suitable rules adhered to by all benchmark implementations. Such a set of rules is e.g. given for the Are-we-fast-yet suite (https://github.com/smarr/are-we-fast-yet).

It's subjective because it can't be used as a source of truth. Of course "I measured X and my results were Y using methodology Z" can be a statement of fact but X and Z are where the subjectivity lie.

For example, benchmark game allows for warmups and so does awfy. This favors jits because it allows them to warm up when they would otherwise be slower. This might give the mistaken impression that java is a great choice for command line tools due to the performance characteristics.

In contrast, most benchmarks I've seen don't use profiler guided optimizations for C or C++. Hence the subjectivity.

And the claim of only wanting idiomatic code in awfy. This is, of course, subjective as well.

Last I checked benchmark games didn’t care about warmups.

You could be correct. The point doesn't rely on what a specific benchmark collection in particular does but that there is an open discussion on what is appropriate in the context of what people using the results find important.

.NET used to be considered a little bit slow, especially when using the Reflection related features, but not nearly as slow as interpreted languages.

One SQL query going over the network will so dominate any micro-optimizations in the framework that it's a little silly for most of us to listen too closely when the ASP.NET team says they've sped up request processing another 40%. If JSON parsing request bodies and reading headers are significant, an API generally isn't doing very much.

I assume temp solutions and low hanging fruit was added in the move to .net core and the new compiler. Now that it's more stable, things can get tightened up.

Most of the performance gains are really in the middleware (for example Entity Framework) and getting rid of pre-.NET Core legacy cruft rather than the VM, AFAIK.

Not exactly. Here’s a really good blog post on the perf improvements: https://devblogs.microsoft.com/dotnet/performance-improvemen...

Reply on news.ycombinator.com