Ask HN: What's the fastest programming language with a large standard library? | Hacker News Reader

Ask HN: What's the fastest programming language with a large standard library?

An example of a language that does not fit the bill: rust

Common Lisp has quite a lot compared to most languages, and in most cases is just as performant. Of course C and Rust are always going to outperform in edge cases simply because of their proximity to hardware.

I don't remember the exact quote, but it goes something like "most [languages?] are a buggy implementation of half of Common Lisp"

If you go for fastest then C++ with its own standard library plus add a few third party ones will get most stuff done. Look here: https://en.cppreference.com/w/cpp/links/libs

Pick header only libraries if you are concerned about portability.

Now C++ is a pain in the ass to work with, and the world has moved on to better more convenient things. Do explore modern C# and Go for better developer productivity.

C#/.NET in my opinion. Extremely extensive and high quality standard library and it keeps getting faster every release.

Though it's kind of a difficult question to answer, what do you need the standard library to actually do?

Agree, no other standard library has comparable degree of code vectorization either.

What does that mean?

.NET's standard library is very heavily vectorized, vectorization is considered in all scenarios where it is applicable, the compiler will also apply it to copies of known length and string comparisons fully eliding and unrolling Memmove and SequenceEqual calls.

The gives languages that run on top of .NET massive performance advantage in a variety of scenarios versus any other language - C++ and Rust stdlibs are far more conservatively vectorized because neither language has stable SIMD vector API and even then out of modularity constraints a lot of routines have to either rely on autovectorization which is fragile or manually vectorized with intrisics for each individual platform.

A short non-exhaustive list of examples is

- Shared SIMD helper for Aho-Corasick, Rabin-Karp and other text search algorithms https://github.com/dotnet/runtime/blob/main/src/libraries/Sy...

- Bloom filter https://github.com/dotnet/runtime/blob/main/src/libraries/Sy...

- Base64 encoding and decoding https://github.com/dotnet/runtime/blob/main/src/libraries/Sy...

- Element search (memchr and the like) https://github.com/dotnet/runtime/blob/main/src/libraries/Sy...

- UTF-8 transcoding https://github.com/dotnet/runtime/blob/main/src/libraries/Sy...

The above are examples of 1% code that ends up used by 99% of other codebase in one way or another. Regex engine, JSON serialization and parsing, substringing and etc. all use these.

Rust has had a stable SIMD vector API[1] for a long time. But, it's architecture specific. The portable API[2] isn't stable yet, but you probably can't use the portable API for some of the more exotic uses of SIMD anyway. Indeed, that's true in .NET's case too[3]. So .NET has to rely on "manually vectorized with intrisics for each individual platform" as well.

Rust does all this SIMD too. It just isn't in the standard library. But the regex crate does it. Indeed, this is where .NET got its SIMD approach for multiple substring search from in the first place[4]. ;-)

You're right that Rust's standard library is conservatively vectorized though[5]. The main thing blocking this isn't the lack of SIMD availability. It's more about how the standard library is internally structured, and the fact that things like substring search are not actually defined in `std` directly, but rather, in `core`. There are plans to fix this[6].

With all that said, I wouldn't necessarily disagree with the broader point in this thread. .NET's regex engine performance is quite impressive[7]. (And I still have to merge the PR updating to .NET 8.) I'm really only replying here to correct more narrow points.

[1]: https://doc.rust-lang.org/std/arch/index.html

[2]: https://doc.rust-lang.org/std/simd/index.html

[3]: https://github.com/dotnet/runtime/blob/72fae0073b35a404f03c3...

[4]: https://github.com/dotnet/runtime/pull/88394#issuecomment-16...

[5]: https://github.com/BurntSushi/memchr#why-is-the-standard-lib...

[6]: https://github.com/rust-lang/rfcs/pull/3469

[7]: https://github.com/BurntSushi/rebar#summary

Getting clarifications from the OG himself! :)

Indeed, I meant to say "portable SIMD API" but it got lost somewhere in the stream of consciousness - sorry.

As for limitations of using such APIs in the standard library - in case of .NET platform-specific paths mainly exist for the following reasons:

- To achieve optimal codegen with more specific instructions

- To account for platform differences that are not expressible through crossplat vectors (vpternlog on x86, tbl/tbx on Aarch64)

- The code was written before crossplat API was available and has not been updated yet

Another aspect of such API is that all intrinsics still consume the same unified group of Vector128/256/512<T> so that writing scenario-specific helper is still relatively easy, which is what bits of standard library often opt into.

Right. I pointed it out because it isn't just about having portable SIMD that makes SIMD optimizations possible. Therefore, the lack of one in Rust doesn't have much explanatory power for why Rust's standard library doesn't contain SIMD. (It does have some.) It's good enough for things like memchr (well, kinda, NEON doesn't have `movemask`[1,2]), but not for things like Teddy that do multi-substring search. When you do want to write SIMD across platforms, it's not too hard to define your own bespoke portable API[3].

I'm basically just pointing out that a portable API is somewhat oversold, because it's not uncommon to need to abandon it, especially for string related ops that make creative use of ISA extensions. And additionally, that Rust unfortunately has other reasons for why std doesn't make as much use of SIMD as it probably should (the core/alloc/std split).

[1]: https://github.com/BurntSushi/memchr/blob/c6b885b870b6f1b9bf...

[2]: https://github.com/BurntSushi/memchr/blob/c6b885b870b6f1b9bf...

[3]: https://github.com/BurntSushi/aho-corasick/blob/f227162f7c56...

Heh, movemask keeps coming back. Rather than emulating it, it appears to be more efficient to separately handle IndexOfMatch, LastIndexOfMatch and GetMatchCount scenarios it is used for most of the time:

- https://github.com/dotnet/runtime/pull/94472/files#diff-5824... (it's closed for now but I'm hoping to get back to it at some point)

- https://github.com/jprochazk/tmi-rs/blob/ac3ce6aee8bbe038a98...

It can account for good 30% performance variance depending on the use case (on Apple's M-series cores).

That's what the `memchr` crate does. It uses `vshrn` just like in your links. And vpmaxq before even bothering with vshrn: https://github.com/BurntSushi/memchr/blob/c6b885b870b6f1b9bf...

Doesn't it have an ide problem on Linux?

Not with Rider anymore as far as I know. I'm still on Visual Studio but colleagues report Rider is as-good or better.

I use Rider on Linux, it's great.

Why is no one mentioning c++? Is that not the obvious answer here ?

Does it have a large standard library?

I always thought of Turbo Pascal/Delphi/Free Pascal as a fairly complete system. Static typing gets in the way at times, so you won't see TensorFlow in Pascal, but the TUI/GUI that came with it made it suitable for building almost anything, with no added pieces.

The compilation process itself is one of the fastest around. Libraries are "units" and separately compiled. You can have GUI executables that weigh in at < 500k if you turn off debugging.

Is it fast at runtime?

Ohhhhh yeah

Java

20 years ago yes.

Today it is a terrible monster that you use because of the lack of alternatives, not because it is fast or comfortable to use.

I disagree Java 21 is really nice. Lots of modern features, tons of rock solid, battle tested open source libraries. Amazing tooling.

> Amazing tooling.

Apparently you're comparing to java 20 years ago. Not to modern dev friendly ecosystems.

I've used vscode with typescript and python, but I found the amount of detail and coverage the inspections intellij has are better. The profilers that Java have are also the best I've seen. The profilers can show you crazy details on SQL queries being run, that get traced all the way through to the http endpoints (https://www.ej-technologies.com/resources/jprofiler/help/doc...).

Been a Java developer for many years but I've never tried JProfiler. Is it free, no big catches?

No the 3rd party profilers aren't free. But they have free trials. Java flight recorder is free and has a lot of the features of the paid ones (like the SQL tab)

https://blogs.oracle.com/weblogicserver/post/jdbc-in-java-mi...

Java is pretty fast though, according to benchmarks. Your point regarding comfortable to use ... yeah, fair enough to some degree.

Nodejs is extremely fast in networking (web, api, gateway) kinds of scenarios both dev-time (which is more important) and runtime wise

Node doesn't have a large standard library so it doesn't answer the question.

Without providing a defn of “fast” and within what domain, this is a meaningless post.

this. go has the best perf:dx:deployment ratio.

HolyC

Reply on news.ycombinator.com