Hacker News Reader

Comment by sidkshatriya | Hacker News Reader

I am ... confused by Nim a bit. Nim on the surface has a dream set of features: python like syntax, strong types, compilation to C, good garbage collection, good metaprogramming...

OTOH there seems to be a rapid accumulation of features. The language wants implement the "next shiny thing" (e.g. ownership ala rust) or at least that is how it appears to me from the outside.

For those people using Nim -- how do you cope with the ever increasing scope of the language? Isn't there a possibility that things aren't going to be well baked/abandoned?

I like the philosophy of evolving your language slowly and consolidating before adding newer and newer features.

Do you think the pace of change in Nim is OK?

View on HN

> Nim on the surface has a dream set of features

It's "too good to be true" is for real. The lack of friction when developing in Nim is what makes it so fun to code in (IMO). Metaprogramming in particular is just great in Nim and give you a lot of scope for doing things that would otherwise require new language features.

> how do you cope with the ever increasing scope of the language?

I've been using Nim pretty intensively for several years. From my perspective the language design has been very stable, even since before 1.0. I don't feel like the scope is expanding, and I don't think it really needs to either because the core language is built to be expanded with metaprogramming.

None of my code breaks when updating versions (and I do a lot of heavy metaprogramming). Nothing seems to be abandoned, only refined, and there's a big effort towards backwards compatability so breaking changes are exceedingly rare. The only one I only remember was some time ago when seq (the variable sized list type à la C++ vector) became not nil, so I had to replace `if list != nil` with `if list.len > 0`, so a good change IMO.

Most of the new things have been mechanical under the hood improvements and ownership is one of those. As I understand it, when using gc refs, you'll get free extra speed, better multithreadding, and better compile-time checks. No code needs to be changed, and when it's considered ready it will replace the previous GC, and you can add extra annotations for performance. It's refinement of the GC using ownership rules rather than a whole new set of rules for your program.

The stdlib is very stable (at least from my experience), and again great effort is spent on backwards compatability. Stdlib focus is on being small and "essential", so there's a high bar for stuff getting into it, and probably why it's rare for anything to change API there.

So personally the pace of change has been great - basically my code's just got free speed boosts when it's already fast enough :)

> Metaprogramming in particular is just great in Nim and give you a lot of scope for doing things that would otherwise require new language features.

Metaprogramming is v alluring to me, but I’ve been wondering, people that write code super heavy on the metaprogramming, doesn’t revisiting code and debugging become much more taxing?

I have written a lot of metaprogramming myself, like 5/10 of my repos are macro libraries. And I share your concern. Even the manual advices people to use the least "powerful" tool for the job. These are proc/func<template<macro.

Macros are prefered for creating dsl like https://github.com/treeform/fidget or https://github.com/pragmagic/karax

Otherwise the advice is to use them when they "pull their weight". The reason is they're a lot harder to write and even "design" them right (how they function, what code they write, etc) takes a lot of effort. And in most cases the simpler "tools" will fit your needs.

You have multiple ways to print the macros either as AST tree or even Nim code that can be copy-pasted to replace the macro:

- dumpTree

- lisprRepr

- treeRepr

- toStrLit

This means that for debugging you can generate your code. Replace the macro by regular code temporarily and understand what's wrong and then fix the macros.

I don't think I have a single library without macro but I try to keep them as reasonable as you can. But sometimes you need to give a Python feel (for example to provide a Numpy interface for a multidimensional array library) or you have to do code transformation (implementing async or multithreading as a library or autovectorizing high-performance computing code or implementing a GPU or Audio DSL) and there is no way around macros.

You generally want to use the simplest tool for the job first. Most of the time generics and templates really are as much as you need since Nim's generics are structurally typed and templates allow a lot of metaprogramming without getting into AST trees.

My experience debugging complex compile time transformations is pretty positive thanks to the tools available to manipulate and debug the syntax trees. For example you can `echo node.repr` to display the code it represents at any point, and going the other way you can pass code to `dumpAstGen` to display the code that will generate the AST tree and so on.

This also has the benefit of letting you quickly understand how the trees are structured by trying different bits of code, or for passing DSL syntax fragments to see the parsed tree structure for your macro to consume.

The main important thing is that stack traces at compile time get you to the place where the error is happening. How hard that is to debug depends on how complex your tree manipulation is, and how you are doing that manipulation.

For example if you're just got a load of `quote` statements that interpolate variables and you're chucking these together, the trace will usually lead to the start of the `quote` block, so if you've got a long `quote` block then you have to kind of find the error in that, but it's not too bad since the error messages are pretty precise and the language is strongly typed.

For direct AST manipulation, the error will point to the place where the error is triggered, so for example if you add a bunch of statements in a node, the error may be the place that node is added to it's parent to be 'connected' to the program as a whole. Or, if it's an attempt at a duplicate variable declaration it will point directly to the place where you're creating the `ident` that represents the variable in your macro, so that's pretty nice.

In short, yes it is more work, but it generally makes sense, is type safe, and we have tools that mean fixing stuff is often some `echo` statements to look at generated code.

Why does Nim have "nil"?

To represent a null pointer. Nim is a systems language (or at least can be used as one), and you need null/nil when working at that level.

Even Haskell has null pointers, if you know where to look:

https://hackage.haskell.org/package/base-4.14.0.0/docs/Forei...

Likewise unsafe Rust has null pointers, but I've never needed it or seen it used. (Same with Haskell).

You just mentioned using "nil" in a simple example which doesn't appear to be related to systems programming.

I didn't mention "nil" in a simple example. I wasn't the poster of that comment.

Oh, thought you were same poster. Well, that other person mentioned it in a context which doesn't appear to be related to systems programming.

Just checked Github for some of the top Nim projects.

Forum software: https://github.com/nim-lang/nimforum/search?q=nil&unscoped_q...

Web framework: https://github.com/planety/prologue/search?q=nil&unscoped_q=...

Twitter front-end: https://github.com/zedeus/nitter/search?q=nil&type=Code

These uses of "nil" don't appear to have anything to do with systems programming.

Fair point. I should mention that I'm not a Nim expert, and am not defending its design. It's true that 'nil' appears in Nim under both of its pointer types (GC traced "references" and untraced "pointers") -- which is unlike, say, Rust's safe references vs. unsafe (nullable) pointers. So 'nil' is not simply a systems level construct; otherwise you might expect Nim to have nillable pointers, but not nillable references.

You need nil/null when doing systems work -- and Nim is capable of that -- but I don't know the rationale for allowing 'nil' in higher level language features.

Thanks for the honest reply.

Nim interests me but this nil business seems questionable to me.

Just found this comment from the language creator,

https://github.com/nim-lang/Nim/issues/6638#issuecomment-487...

"We need the nil state to disarm pointers but that only means these can be nil and so would require a nil check before deref. Doesn't seem to be too hard nor too cumbersome."

Seems a bit dismissive to me

I agree that it's a sharp edge in the language. Or at least it seems to be -- I've written a little bit of Nim, and never encountered a case where I was actually bitten by 'nil'. The language does have a 'non nil' annotation that can be used when defining structs (and maybe in other parts of the language, I don't recall), so there must have been some thinking about the dangers of nil pointers.

I would still recommend playing with the language for a while. It's quite practical, and the performance is very good. I've used it to write some utilities where I might otherwise use Python, but either needed more speed or needed to use it in a restricted environment where the Python interpreter wasn't available. The experience was mostly positive, and I would use it again.

Thanks, I intend to give it a fair try

P.S., you can annotate procedure to help ensure you can’t raise certain conditions in code. Imho it’s nice for writing sections of “safe” code.

`non-nil` was deemed too breaking to introduce for 1.0 or even as a default because if you tagged your input as `T not nil` the compiler was not powerful enough to prove that T was indeed not nil except in the most obvious cases.

The new battleplan is to integrate the Z3 theorem prover into Nim as DrNim:

- https://nim-lang.org/docs/drnim.html

The first use-case will be proving array bounds at compile-time and elide them. Future use-cases include not nil tracking.

This sounds pretty cool, but, two thoughts

1) Echoing the other comments in this thread about uneasiness with feature creep

2) Wouldn't have been a breaking change if the language didn't have nil from the beginning

1: DrNim is basically additional annotations that are ignored, and you can choose to opt-in via a flag.

It's something you similar to MyPy for python and type annotations. The danger is if it becomes non-optional.

That said contrary too many languages, the core language doesn't need:

- threading

- async

which are huge parts to maintain, but thanks to metaprogramming you can implement them as libraries with very nice syntax as well. It also allows compiler devs to focus on what they know well, compilers and leave those 2 highly specialized areas to people who are experts (or become experts) in those.

2: There are 2 dimensions there: nil is used for C FFI, for higher-level primitives you can use Option. And C FFI was needed to bootstrap Nim with a useful set of functionality and has an escape hatch so that people could start building early on and not wait for the language. So both from a functionality and a time-to-market point of view it was necessary.

That's an old comment by the way. At that time the newruntime idea was tested (and now replaced by --gc:arc). One needs to be more careful when using others words, especially taken out of context.

what are you making in NIM? Finance stuff? Websites? Database tools? Science/Space?

Kinda curious what you've done "using Nim pretty intensively for several years"

I’m not the OP but have implemented a bioinformatics algorithm library with with Nim.

I had an implementation in Python and just ported it to Nim. I have been incredibly impressed with Nim to say the least. I managed to make the Nim version orders of magnitude faster with fewer lines of code while having more features. With the excellent Nimporter library, I can easily make a Python API and have it be pip installable.

I genuinely think Nim (or similar) may be the future of my field. Combined with its JS compile target and WASM support, it lets me target my three main “platforms“: web, Python API, and high-performance CLI.

This sounds very interesting! What are your thoughts on nim vs julia? In the scientific community, julia is popular for similar reasons as nim like easy syntax and compiled code. If you think nim will take over in bioinformatics, how is the support of numerical algorithms like BLAS and LAPACK?

https://forum.nim-lang.org/t/5242

There is https://github.com/mratsim/Arraymancer n-dimensional tensor (ndarray) library

Native plotting library: https://github.com/Vindaar/ggplotnim

as well as many other packages.

This is a good write up imo: http://lh3.github.io/2020/05/17/fast-high-level-programming-...

I have never written anything “production” grade with Julia, so do take my opinion with a grain of salt. However, I did once try to translate a simple algorithm library from Python into both Nim and Julia.

Julia was no faster than Python with my initial naive implementation. After consulting the forum and getting help, I did manage to get it running fast. My impression of Julia is that it can be incredibly fast but it takes work to get to that stage.

In contrast, with Nim, my code looked nearly identical to my Python code (i.e. quite simple) and ran fast from the start. This has been my general impression of Nim after doing a few projects in it: you write clean, readable code and get fast code out. There is no step 2.

With respect to numerical algorithms, Nim has good support for BLAS and LAPACK. I personally haven’t used them, so I can’t comment.

Nim’s metaprogramming allows for things like this [1], which is a macro library that translates between idiomatic Python and Nim. While I don’t use it myself (I find Nim’s idioms to make sense for Nim) it does make the transition a lot easier.

[1] https://github.com/Yardanico/nimpylib

Network protocol stuff, database tools, web clients & servers, games, graphics, AI, cryptography, and various random projects over the past 5-6 years. Some professionally, some for fun.

It's an incredibly adaptable tool. I mean, just take the example of writing type safe Nim code and compiling it to JS for a web client (already handy), then writing a high performance server in the same language, sharing the same types/code, but compiled to C. We can bring together front-end and backend! :)

Another generalist ability is with libraries you can get a high cooperation with Python - you can run Nim from Python and Python from Nim - so you can merge with that ecosystem too and use existing infrastructure as required. And of course an even better situation exists with C and C++.

Personally it's been a great way to diversify my ability to reach different domains with a unified tool.

> OTOH there seems to be a rapid accumulation of features...

I think you're stretching your narrative a bit. They're working on a new GC algorithm that (if you watch Rumpf's very interesting presentation[1] on it from today) is a significant performance improvement over their current GC.

What features are you referring to regarding the "ever increasing scope of the language"?

[1] https://www.youtube.com/watch?v=aUJcYTnPWCg

It is an impression (possibly an unfounded/unfair one) and the whole point of my question is for others to give me their opinions.

Nim is certainly a feature-full language. Look at the left sidebar on https://nim-lang.org/docs/manual.html and you will see a lot of things listed! I wonder how many of those features work well with others, how many work partially and how many are somewhat abandoned.

I'm looking to add Nim to my arsenal in the future. I'm positive on the language -- please don't misunderstand. I just wanted some opinions!

Everything that is new or was causing problems, moved under expiremental https://nim-lang.org/docs/manual_experimental.html These are the features you are describing, the ones needing more refinement.

stuff like concepts and code reordering are planned to eventually became stable. Nowadays everything is decided in the RFCs repo https://github.com/nim-lang/RFCs/issues before a PR hits the main repo.

> Look at the left sidebar...

Look at the Go language manual for comparison[1]. Go is simple (some would argue, simplistic) and there's a lot there too. Programming languages are complex beasts. "Number of things in the sidebar of the manual" is a poor way to judge a programming language.

> I like the philosophy of evolving your language slowly and consolidating before adding newer and newer features.

The Nim book[2] came out in 2017 and the language has maintained compatibility with it through their 1.0 release. Nim seems pretty stable and "well baked" to me.

One of the awesome things about Nim is that many features in the language can be implemented through macros. (AFAIK their async/await feature is implemented in Nim itself through macros.) So, I get the impression that Nim has a very solid core and builds things on top of that core. The fact that they can implement Rust-style move/borrow semantics without significantly changing the language is a really good sign to me of the language's power/design.

Personally, I wish Nim was what we were all using instead of Go. Nim has all the strong typing features (like generics) that Go is still working on, while also having the equivalent of goroutines and channels (though I haven't played with those in Nim myself yet), pleasant syntax, strong metaprogramming features, very strong operator overloading that helps make it suited for domains like data science, can generate a static binary, and on and on. I'd strongly recommend everyone take a look :)

I'm personally psyched that a repl (INim) is being worked on (and works now), because a good repl is something I really value in a language. There's a short presentation about it from today's conference as well[3]. The fact that the author keeps mentioning IPython gives me great hope for it.

Edit: in fact, upon rewatching, that INim video would be a great sales pitch for the language for you, as the author details his trip through a bunch of different well-known languages and why Nim meets the sweet spot of everything he's looking for.

[1] https://golang.org/ref/spec

[2] https://www.manning.com/books/nim-in-action

[3] https://youtu.be/Qa_9vut4TzQ

> > Look at the left sidebar...

> Look at the Go language manual for comparison[1]. Go is simple (some would argue, simplistic) and there's a lot there too. Programming languages are complex beasts. "Number of things in the sidebar of the manual" is a poor way to judge a programming language.

I don't think you're being fair to me here! Look at the _content_ of the sidebar of https://nim-lang.org/docs/manual.html and you will see the massive variety of language features listed there for the Nim language.

I am certainly not talking about the raw _length_ or _number_ of items in any list which is definitely a crude and silly measurement. Here your link to Golang's specification is missing my point.

> > OTOH there seems to be a rapid accumulation of features...

> I think you're stretching your narrative a bit. They're working on a new GC algorithm...

The thing is, according to the documentation [0], Nim already has six different garbage collectors.

[0] https://nim-lang.org/docs/gc.html

Do those represent a "rapid accumulation of features" that create an "ever-increasing scope of the language" as the OP claims, or are they just tuning options? Keep in mind the Nim team didn't create the Boehm or Go garbage collectors, and the new arc/orc are the same, with orc having a cycle detector. If anything, it's a testament to the design and stability of the language that the GC algorithm is swappable.

Edit: for comparison, Java has four different GC algorithms to choose from[1] with a whole lot of options for tuning[2]:

[1] https://www.baeldung.com/jvm-garbage-collectors

[2] https://www.oracle.com/java/technologies/javase/vmoptions-js...

Many languages have pluggable GCs. Modern Java VMs have at least four (serial, parallel, CMS, G1). As long as they are interchangeable, it's not really a language design issue.

also ZGC and Shenandoah

A lot of the things you mention are pretty self-contained (you have to enable specific compiler flags), or they're implemented using the macro system or templates, and so in theory at least it's not bloating the compiler.

Compared to something like Haskell (GHC), I think Nim is pretty lightweight and not too bloated.

Reply on news.ycombinator.com