Hacker News Reader

Comment by bluetomcat | Hacker News Reader

All of these "fast as C" claims about modern, high-level Python-like languages (be they statically typed and natively compiled) are missing the point. It is mostly the minimalistic and terse programming style that C encourages that makes C programs performant. You avoid allocations wherever possible, you write your own custom allocators and memory pools for frequently allocated objects, you avoid copying stuff as much as possible. You craft your own data structures suited for the problem at hand, rather than using the standard "one size fits all" ones. Compare that to the "new this, new that" style of programming that's prevalent today.

View on HN

There actually is one high-level Python-like language that really is almost "as fast as C".

http://nim-lang.org

I am using it for years already, and it is really performant, somewhere between C and Rust. I am still wondering why so few people use it.

Benchmark: https://github.com/kostya/benchmarks

Nim vs Rust: http://arthurtw.github.io/2015/01/12/quick-comparison-nim-vs...

Performance discussion: http://forum.nim-lang.org/t/2261

Embedded Nim: https://hookrace.net/blog/nim-binary-size/

Nim on LLVM: https://github.com/arnetheduck/nlvm

> I am still wondering why so few people use it.

While Nim is my favorite language, I can understand that it has a small userbase, for these reasons:

1. No major backer like Google for Go or Mozilla for Rust

2. No killer feature like "memory safety and performance without GC" for Rust, instead a mix of all the reasonable down-to-earth features I want in a programming language

3. Some unique decisions instead of what you're used to from other languages, for example partial case sensitivity

4. The fact that operator order is changed by the amount of white space between symbols

                2+2 * 5 = 20
                2 + 2 * 5 = 12

If you're consistent about how you space your infix operators, this will have no impact on your code.

If you put space around some operators and not around others, in a way that doesn't correspond to precedence, you're going to confuse anyone who reads your code, in any language.

Being confused by code is no excuse for code not doing what ya know... every other programming language has done for the past 50 years.

I get that trying new things, but somethings are pretty well agreed upon.

damn... I actually like that a lot!

I like it for distinguishing homonym operators, but not for the precedence stuff they seem to have there. I'd like something like this though:

  let a = 10;
  a / 5
  output> 2
  let b = pwd();
  b/temp
  output> Directory<"~/temp">
  b / 2
  error> b:Directory does not implement method "divide(:number)"
  a/temp
  error> a:int does not implement method "get(:string)"

I am a strong proponent of Nim but this is probably the worst idea I have ever encountered in language development. Honestly!

Partial case sensitivity and the special underscore case are features I can live with. Unfortunately this has actually become a stumbling block for a wider adoption of Nim.

All strange special features should be optional, not default.

What makes you think this is default? It most certainly is not and will be removed completely in the future.

Edit: here is a source: http://nim-lang.org/docs/manual.html#syntax-strong-spaces ("... if the experimental parser directive #?strongSpaces is used..."). The last time this was discussed I said that it would be removed completely, and I still believe it will be. It's simply not a priority for us right now.

> What makes you think this is default?

Some of those features are default in Nim, some (strongspaces) are not. I say that all such weird features should be optional in general so that newcomers don't get scared off.

Also case and underscore should work like in C per default since Nim interoperates with C seamlessly anyway. Case insensitivity and ignoring underscore are ok if optional.

wow... have never heard of that partial case sensitivity before.

I think this goes beyond syntactic sugar. Holding the hand of the developer too much?

Personally, as a Python programmer I like interfacing with C++ code like Qt via PyQt. If I see a camelCase method I know where it came from, but if I see a PEP-8 style name or method I know it's our own code, not from Qt.

I used both nim (back when it was still nimrod) and rust for a while, before eventually settling on rust. I tried to give nim a chance, and was told that "they will grow on you" ("they" being the things you mentioned that were "unique decisions instead of what you're used to from other languages"). They never did, and though I got used to avoiding the problems I initially had with them, the language just never "felt good" to me.

Shameless plug: is it considered totally uncool in 2016 for one to be developing a memory-unsafe, manual MM, non-OO, thread-denying language that preserves most of the C semantics?

https://github.com/bbu/quaint-lang

Yes. Memory unsafety doesn't work.

No, definitely not!

I'd be very interested in a language that is roughly as low level as C, but has some obvious warts "fixed" while still being able to run on bare metal or with a minimal runtime system. I also don't care about a standard lib as long as I can call open(), close(), read(), write(), socket(), etc.

Native threads is another requirement for me.

Things I'd like to see in a language:

- compile to native executable

- type inference

- module system without header files

- easy to call into native C code, and export functions so they can be called from C or any other language

- first class SIMD structures (this is missing from Rust!), so that you don't have to duplicate code for sin4f and sin8f (which would be line-by-line equal, except types)

- perhaps some kind of modern polymorphism (ie. not class based OOP)

- can target GPUs via LLVM or SPIR-V

- memory safety is optional, but nice to have. I'd be mostly interested in using this kind of language for GPU kernels and tight inner loops, where you wouldn't be allocating anyways

I have a bunch of design ideas and prototypes in my drawer waiting for a lot of free time and inspiration appearing.

I like my tools sharp, even if it means there's going to be blood occasionally.

My next big endeavour with Quaint will be to create a clean module and linking system (without header files or any textual inclusions). Each source file will be transformed to a corresponding unit which contains code, data and exported type definitions. The linker would then merge these units and produce a native executable that runs your program in the self-hosted VM which will be a part of that executable. Pure native compilation or LLVM integration is too much of a hassle for me at this point.

One of the virtues of the language would also be the direct correspondence between the HLL code and the emitted VM instructions, without any optimisation passes. This makes it much easier to reason about code performance and to write code which performs consistently and predictably (albeit a bit slower).

Nim fits everything you ask, except for "can target GPUs via LLVM or SPIR-V". Even that may eventually be fixed by having OpenCL C as a compilation target.

Also, I am not sure what you mean by "first class SIMD structures", but you can definitely have a single definition for sin4f and sin8f if they are line by line equal except types, by using union types.

Nim is definitely on my short list of languages to learn, however...

Targetting GPUs is a deal-breaker. I'm sure the Nim compiler would be pretty easy to retarget to GPUs via SPIR-V (the new binary IR for Vulkan/OpenCL shaders and kernels) or OpenCL/CUDA C. But I don't think that would work for Nim's runtime system or existing Nim libraries (including any standard libs it has).

Also Nim's pauseless low latency automatic memory management (I guess you can call it a "GC") is very interesting but it's not what I'm after.

> Also, I am not sure what you mean by "first class SIMD structures",

I mean this:

    def multiply_and_add(a : <n x f32>, b : <n x f32>, c : <n x f32>) : <n x f32> {
        return (a*b) + c;
        // TODO: figure out how to use "madd" from FMA4 or NEON instruction set
    }

The trivial piece of code above should be "generic" so that it can be called with any width of vector.

Now the example above is very trivial but more complex examples might have challenges for correct implementation of the type checker. In particular, doing vector shuffles (ie. equivalent __builtin_shufflevector in GCC/Clang vector extensions) would need to have a strange type. Shader languages typically use a syntax like `myvector.wxzy`, which might work.

This might perhaps be possible with an ungodly mess of C++ templates and explicit template specialization for each vector type (and hoping that the compiler is aggressive enough in inlining). But I'm not really a fan of template-heavy C++.

In fact, the kind of solution I've been thinking about would be semantically similar to what I'd do with C++ templates.

> but you can definitely have a single definition for sin4f and sin8f if they are line by line equal except types, by using union types.

I'm not familiar enough with Nim's union types to be sure, but my guess is that this would not compile to efficient low level code apart from the most trivial of circumstances. This is my (not very) educated guess based on other high level languages with some concept of union types.

Anyway, Nim is a very cool language that I will check out sometime in the near future. It just isn't what I'm looking for my very specific use case.

A union type in Nim can only be used in funciton arguments, and it does the obvious thing: when you actually call the function, it specializes to the type you are calling with. Think about templates in C++, where the type parameter can only assume one of two (or more) values. Hence it would generate exactly what you would write by hand, but the syntax is much less messy than C++ templates

You've almost described ISPC verbatim. If you don't already know about it you might want to check it out. http://ispc.github.io/index.html

It doesn't compile to a native executable, but since it produces .o files you should be able to just set your entry point and go from there.

> You've almost described ISPC verbatim.

Thanks, I've read about it before, but haven't spent too much time looking at it.

However, this "single program, multiple data" isn't exactly what I'm looking for (it would solve the sin4f vs. sin8f issue mentioned above, though). I need explicit, low level access to SIMD, coupled with genericity over vector widths. This means doing almost assembly-style SIMD code with explicit shuffles, blending, etc as well as access to intrinsics where needed.

I also need portability (ispc is from Intel, it probably doesn't support ARM NEON) and targetting GPUs.

I'm very well aware that my needs are very specific. I need to do math stuff for 3d graphics and physics applications.

All I need is for a lot of free time to appear from out of nowhere and I can write a prototype compiler for this myself :)

How would you use specific instructions yet have widths abstracted?

ISPC actually has some preliminary support for targeting Nvidia PTX btw. It compiles using LLVM.

See example above in this thread. In C + GCC vector extensions, I just use normal arithmetic operations (+, -, *, /).

However, when using specific intrinsics they are for a specific width. It might take some "library code" to take advantage of some instructions like dot products, etc.

Julia has all these things

Zig (https://github.com/andrewrk/zig), "a system programming language intended to replace C", has a lot of those features.

You might also be interested in Jai [0] which has many of those things but is not a 'real language' yet or possibly ever. Lots of interesting ideas though.

[0]: https://github.com/BSVino/JaiPrimer/blob/master/JaiPrimer.md

I love and use Nim (in production). I wrote about why I don't think it's gained wide adoption in a previous post here: https://news.ycombinator.com/item?id=11960814

The only thing I would add would be that compared to Ruby, Nim still takes you quite a while to put something together, so defaulting to Ruby isn't necessarily a great idea.

Just a small note for anyone reading the "Nim vs Rust" post; today, regex! is much slower than Regex::new.

All the benchmarks you posted suggest that Rust is faster than Nim, not the other way around. Did you have something else in mind?

A lot of them did show the other way, then came to Rust people's attention and improvements were submitted. As well as both languages' implementations changing over time.

The first benchmark ("Nim vs Rust") I looked at says

> Rust regex! runs faster than Regex

which is a very old claim - Regex should now be much faster than regex! ever was. Any pre-1.0 Rust benchmarks are probably wrong (to be fair, most benchmarks are probably wrong anyway).

> You avoid allocations wherever possible

If you don't have to write cutting-edge games or embedded software for tiny systems, why do you have to care about allocations at all? Today's systems and RAM's are so fast that garbage collections don't really matter in most cases. Consider SBCL (compiled Common Lisp) which is almost as performant as Java and C++.

http://benchmarksgame.alioth.debian.org/u64q/lisp.html

I used to develop software in C and C++ for many years, and a garbage collector was the thing I wanted the most. GC-free programming is unnecessarily tough in most cases, except you desperately need it for games and embedded systems.

You must worry about allocations anytime you write CPU-bound software which wants to maximise the output per clock cycle.

Granted, most software isn't like that, but it's certainly not only games and embedded software on tiny systems.

Because allocating something on the stack means adding a something to a pointer and puts the object somewhere that is likely to be in the cache during the function and allocating something on the heap usually takes a tree traversal and puts the object far away from the other stuff you might be using.

Also, using numbers from a benchmark game is not representative of the performance of real world applications. If you look at the code, you'll find that it's written in a style that avoids heap objects and GC wherever possible. Forcing heap allocation is what makes Java slightly slower than C in many cases.

> If you don't have to write cutting-edge games or embedded software for tiny systems, why do you have to care about allocations at all? Today's systems and RAM's are so fast

What? RAM is not fast at all, the latencies have almost not improved in 20 years (compared to the improvement of other subsystems like the CPU, of course).

RAII and smart pointers go a long way towards eliminating the need for GC in c++.

True, but usually one doesn't have control over the code others write to force them to use such tools.

Nitpick: everything you say is probably correct, but such performant C programming is also the very opposite of a "minimalistic and terse style".

Which one is more minimalistic, 'new Foo' or a collection of various custom-tuned allocation methods? Which one is more terse, 'myList.Where(foo).Select(bar).Aggregate(baz)' or an explicit for loop?

It is minimalistic in the sense that the language provides a narrow set of primitives and a skilled programmer combines these primitives in the most sensible way to solve the problem at hand. Higher level stuff in most other languages is much more generic.

Indeed, it may not be minimalistic in terms of the code size.

> All of these "fast as C" claims about modern, high-level Python-like languages (be they statically typed and natively compiled) are missing the point. It is mostly the minimalistic and terse programming style that C encourages that makes C programs performant. You avoid allocations wherever possible, you write your own custom allocators and memory pools for frequently allocated objects, you avoid copying stuff as much as possible. You craft your own data structures suited for the problem at hand, rather than using the standard "one size fits all" ones. Compare that to the "new this, new that" style of programming that's prevalent today.

Exactly! I cannot agree more.

I have a small test program I port to different languages to test the length of the code and the speed of the program. Of course it only represents a single use case.

* C is first, of course.

* twice as slow, come Pascal, D and... Crystal!

* x3 to x5, come Nim, Go, C++ (and Unicon).

* x6 to x9, come Tcl, Perl, BASIC (and Awk).

* x15 to x30, come Little, Falcon, Ruby and Python.

* x60 to x90, come Pike, C#, Bash.

* x600 to x1000, come Perl6 and Julia.

This list looks byzantine, I know :-) The trends I can get out of it:

* the last 2 are languages with JIT compilation, and that's horrid for short programs.

* the "old" interpreted (or whatever you name it nowadays) languages (Tcl, Perl) are not so bad compared to compiled languages, and much faster than "modern" one (Ruby, Python). (Again, this is only valid for my specific use.)

* compiled languages should all end up in the same ballpark, shouldn't they? Well, they don't. The more they offer nice data structures, the more you use them. The more they have some kind of functional style (I mean the tendency to create new variables all the time instead of modifying existing ones) the more you allocate and create and copy loads of data. In the end, being readable and idiomatic in those languages means being lazy and inefficient, but what's the point of using those languages if don't use what they offer? C forces you to use proper data structures and not re-use existing ones. It comes naturally. What is unnatural in C is to copy again and again the data, it is simpler to modify the existing one and work on the right parts of it, not to pass the whole chunks every time you need one single bit. In more evolved languages, compilation won't save you by doing some hypothetical magic tricks, it cannot remove the heavy continuous data copying and moving you instructed your program to do. And that is what made the difference in speed between C on one side, and D, C++, Go on the other side.

I'm curious where Rust fits in on your hierarchy with their emphasis on "zero cost abstractions". Of course that's more a lofty ideal than reality but it means that at least in some cases it does much better than C++. Is it a long program? Something that could be posted in a Github Gist maybe?

> I have a small test program I port to different languages

Please show one or more of those programs.

I bet C# is using Mono.

I don't think your list is quite right. Crystal is probably more on the tier of Go. Julia is also much faster than that, at least up there at TCL. Of course both vary a great deal depending on what you are using for.

Reply on news.ycombinator.com