Nim Language Draws from Best of Python, Rust, Go, and Lisp
infoworld.com
infoworld.com
On the other hand, I always considered Rust as the result of an ML programmer looking at C++ and thinking "this is bullshit, I can do it cleaner".
I might have a very ML view of the programming world, but that's ok, most other people have an ALGOL/C view of it. ;)
Its been around for a while. Felix has an ML like type system (stronger in fact), has pattern matching, has coroutines / fibers, and was designed to have a FFI less interop with C++ libraries. Very fast too.
* JSON String -> NIM String
* JSON Object -> NIM Hash
* and so on...
I've also had the same question when looking at https://crystal-lang.org.
PS: I do not claim Nim's type system to be bad as I do not have an exact counterexample of a "better" type system.
let s = parseJson("{'A': 'foo'}")
s["A"] # What type is this? We don't know until we parse the JSON.http://www.ats-lang.org/ [ml + c + everything dialled up to eleven. linear and dependent types. comes out of academia, but under active development and slowly beginning to gather a community.]
https://eigenstate.org/myrddin/ [not really familiar with this one but it looks interetsing. close to the metal, has pattern matching over algebraic datatypes but seems more towards the c than the ml end of the spectrum from the brief glance i took.]
http://mythryl.org/ [never took off, sadly; i really liked the looks of this one and its ambitious goals. strong sml influence, with better posix integration and some c-like surface features. developer is currently battling cancer: http://mythryl.org/pipermail/mythryl/2016-April/000942.html]
http://felix-lang.org/ [spawn of ml + c++ rather than ml + c. under enthusiastic if haphazard development.]
there have been a few others i've seen pop up over the years; i keep meaning to make a website or at least a github awesome-style list cataloguing and tracking them.
OCaml is far from being perfect, but it has lot's of features and a decent community (especially compared to all those hobby/research languages).
OCaml can do pretty much anything except things like image processing and video game programming. I mostly don't care about those things, and if I did, I would use Rust.
Now, from a research point of view, sure, those languages are interesting.
Maybe in the past, but not today. Today the perspective is from the very high level dynamically typed languages. Encoding things into types is out of the question.
I wish them however the best of luck. Compiling down to C means being able to leverage a lot of mature tools, and if they manage to simplify a lot of the language they may rival Go for projects when you need high performance but desire to retain high productivity regardless of how big and diverse your dev team is.
It is "as fun as" Python to start with, it has the knobs that made python popular in many niches. It has official JS, C, C++ and Objective-C backends (with perfectly compatible and natively fast FFI in each case). It compiles down to small, efficient executables like Go does.
It has some of the best metaprogramming facilities (in a non-lisp language), which makes it possible to introduce "source native" notation for e.g. XML and JSON.
It also has an LLVM backend, though it is not yet complete.
Nim is a big and complicated language, but so is Python. The size of a language itself is not a problem, as Python shows. Unlike C++, you don't have to be familiar with every small metaclass/descriptor/base-class-lookup issue to use the language. I believe Nim shares Python's "it's ok to ignore the details until you need them" feature.
Um, no.
Look at how long it took Python to displace Perl. And Perl was actively shooting itself in the brain for years or that probably wouldn't have happened.
Python is far more embedded in certain areas now than Perl ever was.
I'd very much like a more expressive and better designed than Go language, without the ceremony and straight-jacket of Rust.
That could be D or Nim or some schemes, for example, but I also want a community and tooling.
The problem is this mentality is one that produces unmaintainable software in the long run.
In the case of Go, the simplicity of the language can be frustrating when it comes to expressing one's ideas in "clever" ways; but the code you write may be easily grokked by a junior developer that started with Go last week. On the other hand, things like metaprogramming allow you to build a castle of abstractions which you yourself will have trouble navigating after six months without touching the codebase, and will be a pain of legacy code to any other engineer. Sure, you don't have to use all of it at first, or ever, but that's what I mean "in the long run", either you cave in or someone else in your team does.
It's hard to say Go is poorly designed, because it does what it was meant to do extraordinarily well, even if there are a few things that could be even further simplified and yes, things like generics (where the complexity tradeoff is not that high due to its ubiquitous nature) that could be added (and this discussion seems to be back on the table according to Russ Cox).
As for Rust, without all the ceremony you cannot summon the demons that grant you perfect memory safety without paying for GC as well as absence of data races. Nim simply cannot provide this despite all the complexity it comes with, and really, if you're okay with that much complexity you might as well just stick with C++ and the gargantuan momentum behind it. The committee is making strides towards making the language more palatable so the appeal for Nim diminishes with each new release.
What mentality? Wanting more expressive power than Go?
Then Haskell for one, or even OcamL and Ada, must produce some very unmaintainable code...
>In the case of Go, the simplicity of the language can be frustrating when it comes to expressing one's ideas in "clever" ways; but the code you write may be easily grokked by a junior developer that started with Go last week.
Only at the micro level (directly checking some lines of code). Grokking what 10 lines do easily doesn't mean you'll more easily grok the overall design, especially if lack of expression has you writing 1000 lines (and copy pasting e.g. for "generic" code) what could take 100 in a more expressive language (and still be perfectly readable).
In fact, the main, sure fire, correlation they found for number of bugs in studies of coding, is first and foremost lines of code (regardless of language).
>It's hard to say Go is poorly designed, because it does what it was meant to do extraordinarily well
It also violates the law of least surprise and orthogonality of concepts several times (e.g. special casing a generic "make", etc).
>As for Rust, without all the ceremony you cannot summon the demons that grant you perfect memory safety without paying for GC as well as absence of data races.
Sure, but as I don't care for GC pauses or perfect memory safety, I don't have much of a use case for it, hence wanting a middle ground between Go and Rust.
I personally don't need to care about either of these things to find Rust a nice programming language to use for general purpose stuff. Its type system and tooling are quite pleasant.
By far the biggest issue is dependency management/build tooling. Opam was a huge improvement in package management, so things are better than they used to be. But OCaml is missing something like Rust's Cargo, that makes creating and building projects simple and easy, and something that allows devs to avoid wasting time on build systems. OCaml also has issues with Windows support. OCaml works on Windows, but currently getting Opam to work on Windows requires Cygwin.
i'm personally enjoying ocaml a lot, but that's because i'm on linux; under windows i'd probably have gone with f# instead.
Nim compile and execution times are closer to C/C++ than most languages, maybe in the 10 percentile. Its expressiveness is, arguably, close to Python. It compiles to C, Objective-C, javascript, which many other languages do not. The C target runs even on arduino boards.
Presuming it does do anything better than those learning Nim is hard because the examples are stale and the documentation lacking. I don't mean just the official docs, which might be great, but where are all the answers on SO and community made books or tutorials.
Well, C++ is bloated and with tons of tricky parts, Rust has lots of ceremony and head-scratching to fit your programs into its "lifetimes" model, and Go is lacking expressive power and has several bad decisions baked in (possible forever).
That's a pretty low bar.
Nim is older than Go and Rust, and has what to show for it?
It sounds like a negative and leading question, but I really don't mean it that way. What are some interesting projects made in Nim? Even hobbyist project can be meaningful.
Metaprogramming, and it's not just about how powerful it is. Metaprogramming in Nim is easy, safe, and powerful.
You mention a lot of extremes: Go is simplest, Rust is the most safe, C++14 has the best execution performance. These extremes have trade-offs: Go's lack of generics mean that you end up writing a ton of unsafe boilerplate code, Rust's safety means that you have to spend extra time wrapping your head around lifetimes, C++'s best execution performance means that you have to deal with memory manually and in an unsafe manner.
These trade-offs are debatable, but my point is this: what if I want a language that is well balanced? Nim may not be as simple as Go, but it's definitely simpler than C++ and Rust. Nim may not be as safe as Rust, but only when you decide to manage memory manually, which is rarely necessary and when using Nim's GC it is just as safe as Rust. Nim compiles to C/C++ so its performance is identical in many cases.
The initial idea behind Nim was to have a small core language with a powerful metaprogramming system for expansion. Perhaps we have strayed from this vision a little bit, but it is still true. And we do aim to refocus towards it.
The closest competitor for Nim is probably D, not any of the languages you name.
Why does Nim get the attention that D does not? I think Nim only gets attention around here because its new and not because it brings anything directly useful. As long as it is still "new" there is still so much potential.
As for garbage collection, I think you are overestimating its value. There are great resource management schemes in both Rust and C++, and they can each manage much more than just memory. I am only qualified to speak in detail on C++, but if you simply avoid the "new" operator and automatically allocate everything (some say stack allocate), which is the default, then everything is cleaned up for you automatically.
Further down the garbage collection route, if your app doesn't need performance then why not stick with Ruby, Javascript or Python, they are all well supported and provide tons of flexibility and libraries. If you need performance, why settle for have measures why not go for C++, Rust or one of the blazing fast fully native languages without GC (even one of the really fast ones with GC like Haskell or Java)? I see the space between them filling up with a ton of languages that seem hard to distinguish. Nim looks like just another, piling into this gap that swallows otherwise decent languages like D, cyclone, darkbasic, pascal, and .Net language that isn't C# and so many other languages that I cannot count them.
It should get more attention as a Ruby/python alternative.
Saying JavaScript needs more attention is like saying the level of nitrogen in the Earth's atmosphere needs more attention paid to it.
Well, when you compare languages, you usually do it by feature set. Both Nim and D are statically typed, garbage collected, imperative languages with first-class metaprogramming features.
I'll also add that Nim has also about 9 years of development behind it. It's older than you think.
> Why does Nim get the attention that D does not? I think Nim only gets attention around here because its new and not because it brings anything directly useful. As long as it is still "new" there is still so much potential.
I see both Nim and D getting comparable attention. They both aren't mainstream languages, so coverage is mostly by enthusiasts and intermittent. People are interested because both are sufficiently different from mainstream languages, not because they suddenly plan to bet their careers on them.
> As for garbage collection, I think you are overestimating its value. There are great resource management schemes in both Rust and C++, and they can each manage much more than just memory. I am only qualified to speak in detail on C++, but if you simply avoid the "new" operator and automatically allocate everything (some say stack allocate), which is the default, then everything is cleaned up for you automatically.
This would be a long debate. Suffice it to say that for many people the benefit of avoiding GC is unclear. Both C++ and Rust manage shared ownership poorly (i.e. it being inefficient, leading to lots of syntactic noise, or both). Note also that I'm not saying that GC is always better, but that it's a differentiating feature.
As for resource management, RAII in my experience is primarily the consequence of poor support for higher order functions or equivalent functionality in a language. (The type of resource management that RAII facilitates is usually better done through higher order functions or a sufficiently expressive macro [1] system.)
> Further down the garbage collection route, if your app doesn't need performance then why not stick with Ruby, Javascript or Python, they are all well supported and provide tons of flexibility and libraries.
You seem to be have some serious misunderstandings about garbage collection. No, GC is not per se a performance drain, and adding the two orders of magnitude overhead that a typical bytecode interpreter for a dynamically typed language induces is not even in the same ballpark for even poorly implemented GC.
[1] And by macros I don't mean C/C++ preprocessor macros, but macros that are properly integrated in the language and its type system, such as in LISP, Nim, D, or Rust.
Both C++ and rust encourage exclusive ownership. And that's a good thing.
> RAII in my experience is primarily the consequence of poor support for higher order functions or equivalent functionality in a language.
HOF tie resources to lexical scope; RAII ties them to object lifetimes, recursively. In turn lifetimes may be bound to a lexical scope, but often is not the case.
It's not. It makes lots of important things unnecessarily hard and often leads to unnecessary copying [1]. It makes it hard to even do something like OCaml's List.filter and Array.filter properly. It gets in the way of doing functional data structures (ex: binary decision diagrams). Lots of common design patterns also require shared ownership.
> HOF tie resources to lexical scope; RAII ties them to object lifetimes, recursively. In turn lifetimes may be bound to a lexical scope, but often is not the case.
No, higher order functions don't per se tie resource management to lexical scope (though that's the easiest application). You can build an entire transactional model on top of higher order functions.
Second, tying resource management to object lifetime is dangerous, as object lifetime can exceed the intended life of a resource (ex: closures, storing debugging information on the heap).
Third, RAII is in practice little more powerful than lexical scoping. RAII works poorly for global variables (problems with initialization/destruction order) and thus is in practice limited to automatic and heap storage. Using heap storage leads to the aforementioned problems, where object lifetime can become unpredictable.
If you read the article referenced by that thread you'll see that a major issue with string copying is due to having a lot of bad interfaces taking raw C pointers so code at both side of the ibterface need to make copies exactly because the raw C pointer doesn't guarantee exclusive ownership. The remaining issues are due to a badly optimized string buider in Chrome and failure of pre-reserving vector memory which lead to many copies on resize. This last issue is fixed with move semantics in c++11.
>It makes it hard to even do something like OCaml's List.filter and Array.filter
Std::remove_if works genrically on any range. Boost (and the range TR) provides iterator views when you need lazy evaluation.
> You can build an entire transactional model on top of higher order functions.
I'm sure you can, in the end you can implement anything manually. With RAII propagation of lifetimes is done automatically by the compiler.
>Using heap storage leads to the aforementioned problems, where object lifetime can become unpredictable.
Only if you use shared ownership. Otherwise is completely predictable.
The point is that something like:
List.filter (fun s -> String.length s > 0) list
> Std::remove_if works genrically on any range. Boost (and the range TR) provides iterator views when you need lazy evaluation.
std::remove_if is destructive. Iterators are not the same thing as a functional filter operation.
> I'm sure you can, in the end you can implement anything manually.
The point here is that a transactional system is more powerful than RAII.
> Only if you use shared ownership. Otherwise is completely predictable.
If you don't use shared ownership, then you're basically limited to lexical scoping.
This is a very salient point. It bounced around in my brain a couple of hours before I came back to comment.
How often do you need to control memory layout and management so you get the absolute best performance? Compare that to how often you need to express filters.
For me, there's no doubt that expressing functional logic and having it be decently efficient is the most important need.
This little example of yours illustrates that a well-designed garbage-collected language has a HUGE advantage over RAII. I may be slow on the uptake, but this is the first time I've seen it that way.
* If the filtered result is used locally in a function (and then thrown away), a filtered view works just fine, is very cheap, efficient and is lazy (nice if you only consume a subset of it).
* Often only the filtered view is used, so you can destructively modify the original list, so no copies.
* If you need both the original list and the the filtered list, in a functional language you need to allocate new cons cells anyway. The cost of allocating, modifying and touching the new memory is going to dominate except for very long strings, so copying is not an issue.
Of course in C++ lists are frowned upon in the first place (as many algorithms can handle any data structure transparently), while they are kind of central in many functional languages.
There are cases of course where frictionless shared ownership is nice and GC shines. Filter is not one of them.
The problem is that you need a vastly more complex machinery to cover all the various ways to avoid copying/reference counting and then still don't have a general solution when you can't avoid multiple ownership (unless you count std::shared_ptr with its very high overhead).
As I said before, it's not that you can't do it, it's that there are costs associated with it.
> If you need both the original list and the the filtered list, in a functional language you need to allocate new cons cells anyway.
You can filter arrays also and allocating cons cells for a list is pretty cheap with a modern GC, as discussed before.
my point was that RAII trivially maps to transactions (destructors do rollback and commit is explicit). Transaction RAII object can be composed to make more complex transactions.
> If you don't use shared ownership, then you're basically limited to lexical scoping.
why would you say that? A common use case is having objects manually removed from collections triggering cleanup actions, like closing sockets, automatically de-registering from event notifications, sending shutdown events, rolling back transactions. That has nothing to do with lexical scoping.
RAII is limited in that it's tied to object lifetime, whereas more general transactional semantics can be linked to more general semantic conditions. Also, RAII does not have a good way to distinguish between commits and aborts.
> why would you say that? A common use case is having objects manually removed from collections triggering cleanup actions, like closing sockets, automatically de-registering from event notifications, sending shutdown events, rolling back transactions. That has nothing to do with lexical scoping.
If you use an explicit action to trigger destruction, then this doesn't have to be a deletion. In fact, having it tied to object destruction is unnecessarily limiting. The general rationale for having RAII is that it happens automagically; if explicit disposal is needed, then much of that rationale goes away.
In general, it seems to be me that you don't have actual experience with resource management outside of C++, so you're mostly speculating about what it's like and try to force your thinking about it into a C++-like model.
If you have issue managing lifetimes I can see why you might think explicit resource cleanup is better, but with so many scopes and even thread local scope, and move semantics ability to move an object into new scopes, there really is no limitation impose by tying resource cleanup to object lifetime. If you don't like that, then make you own classes to do it explicitly.
1. Modern functional programming languages generally come with compacting, generational garbage collectors that have bump allocators. This means in particular that the cost of heap allocations for temporaries is only marginally higher than that of alloca() and has good locality even for pointered structures (to the point where linked lists can outperform dynamically resized arrays such as std::vector, which is basically unheard of in C++). When heap and stack allocations are that competitive, that opens up a whole new set of techniques that aren't normally used in C++ and lifetime considerations become a lot more complex.
2. Functional programming languages use closures extensively, and closures can have effects on object lifetimes that are difficult to predict. The reason is that closures capture their environment – in particular local variables – and if they survive the stack frame that generated them, this can lead to objects living much longer than you think. It's a major reason why closures and RAII don't get along well (note that C++ didn't have closures until recently and in practice their use is much more constrained than in functional or multi-paradigm languages).
This does not mean that you do not want to have sane resource handling. But in general, you want resource usage to be a provable property of a program, so you will generally tie resource management to program state or program behavior rather than incidental language semantics.
Interesting. Would you mind naming a few such languages? I'm guessing Haskell. What about OCAML? Any others?
It's fairly common and needed for modern functional languages, as they can go through a lot of temporary objects when programming in a purely functional style.
First, allocation is only so cheap if it's temporary. If objects survive minor collections, then there's additional cost, as they get promoted to the major heap. The key idea that I'm getting at is that with temporary objects being cheap, you have more flexibility in creating temporary data structures and do not have to fit them in the constraint of a stack frame and (unlike with alloca()) do not have to worry about stack overflow and they can be returned from a function without copying (unlike stack frame contents).
Temporary data structures will still be small and generally fit in the L1 cache of any reasonably modern processor. And using pointer does not mean that everything is a pointer, and that you're necessarily sacrificing ILP.
I don't discount the power of being able to cheaply create (short lived) highly dynamic data structures. I do miss it in C++ and alloca never feels right.
yes, I'm a C++ programmer. I have experience with resource management in C# and python for example, which are a pale shadow of what is possible in C++.
I know nothing of resource management in functional languages, especially regarding transactions and I would love to read more about it if you have some pointers (ah!)
First, note also that it would not be particularly hard to add RAII on top of a garbage-collected language to coexist with GC for resource management; it's just not done in practice. And it's not because language designers are ignorant of it (Bjarne Stroustrop's Design and Evolution of C++ is part of the standard recommended reading list in the field).
Generally, you want resource usage to be a provable property of a program. Not that you'd actually write a formal proof, but you generally want to be able to explain at least informally why resource usage follows certain constraints (e.g. having certain upper bounds).
The basic insight that you need is that resource lifetime is just another semantic property that you can handle with basically the same techniques as other properties of programs; you do not need special language support for it (though, obviously, it helps if your language is a bit more expressive than a Turing machine :) ).
This means that you'll generally tie resource usage to program state and program behavior that you can reason about. The incidental semantics of object lifetime can be dangerous, especially in a functional language, as object lifetime can sometimes be unpredictable.
One of the major hiccups are closures. Closures capture their environment (including local variables) and if they survive the stack frame that created them (because they are returned or stored on the heap), then the lifetime of any captured object can be extended in a fairly unpredictable fashion. Obviously, that is not a good thing, as you have a hard time proving lifetime properties, but few functional programmers would limit themselves to a more trivialized use of closures just for the sake of RAII.
Instead, as I said, you tie resource management to program behavior or state. In the most simple case, that can be scoped resource management. But it can also be an LRU cache, a system based on transactions, or something else entirely. Here's a simple example of a library I sometimes use in OCaml:
class example = object
inherit Tx.resource
initializer print_endline "create"
method release = print_endline "close"
end
let _ = Tx.scoped (fun () -> new example)
[1] https://aherrmann.github.io/programming/2016/01/04/resource-...
I'm beginning to think that we're all going about things wrong :) . We like and rely on things like RAII, macros, etc. But these are nothing more than specific compiler features.
If we were to take control of generating our own code, we could have RAII, macros, and whatever else we dreamed up, easily. And the generated code could be in some readable, debuggable language, so much more visible and obvious than the results of a macro expansion.
For so long we've relied on an amazing black box called a "compiler", when perhaps we should take on the responsibility ("power") of implementing a compiler ourselves. (We could still, of course, generate some intermediate mainstream language, and then those amazing black boxes could take that as input and apply all their optimizations.
Seems to yield clearer code, less magic, more power, and more portability.
Rust is very different from C++ here. You wouldn't get those copies unless you directly called .clone(), so you'd know you were doing a deep copy.
This is especially noticeable and constraining when you come from a functional programming background and not C++ or when you're doing stuff that's more complicated than shuffling bytes (I've mentioned binary decision diagrams as an example).
The bigger point is: as it is extremely rare for me to write one of the few niche applications that are actively GC-hostile (such as web browsers, AAA video games, or OS kernels), I don't see the point of jumping through all the extra hoops that avoiding GC brings with it.
Hm, why would you not use multiple ownership then, instead?
That's exactly what I want. The problem is that (1) it comes with significant runtime overhead and/or syntactic noise in Rust/C++ (or alternatively, lack of memory safety); and (2), it becomes difficult to write code that works equally well for multiple and exclusive ownership (module APIs often become burdened with implicit or explicit ownership assumptions).
In other words, don't read this as "Rust sucks" (I actually rather admire Rust's design), read it as: for my purposes, the practical use cases of Rust are generally too niche to justify the software engineering tradeoffs.
At the end of the day, Rust can never be great for every single last programmer, and this is entirely okay.
After reconsidering, I think D while not mainstream has a much larger mindshare that nim. Even if it were the larger one I don't the need for another language in exactly this space where several other languages have failed demonstrating the difficulty here.
As for garbage collectors, I still hold the opinion that they leave all the other resources unmanaged. All but the best have large performance costs and that Rust and C++ solve these with RAII.
I apologize if you thought that I meant Ruby, Python or other newer scripting was a drop in replacement for a strong systems language. I see a typical lifecycle for software starting off as rapidly cobbled together pile of some scripting language. Perhaps someone makes a thing Ruby on rails and they assemble some product in a week or two. Then as they grow product owners often choose to rewrite (sometimes just partially) in something faster. Commonly Java, Go or C, so why settle for something slow here, when the problem is well define and it can be done with tools that produce incredible performance. People jump this gap because it is worth the time and effort to maximize the ability to scale a proven product. Why leave half or more of the performance on the table even when compared to Java?
I appreciate your input, but you seem more intent on attacking other languages than promoting Nim. I am still unsure why its good. Even if the others are bad, that doesn't make Nim good.
Closest competitors would be D, Felix, Crystal (for ruby folks) and pony (for Erlang/Scala folks).
Don't forget Swift. I think it outshines all of the above (and there is active discussion of adopting ownership concepts similar to Rust's).
Swift is already highly optimized. I found this article to be very interesting:
https://www.cocoawithlove.com/blog/2016/05/19/random-numbers...
TL;DR: 'Both the C and Swift are now the same. I don’t just mean “they take the same time”, I mean they are compiled to literally the same instructions.'
Initially the Swift Mersenne Twister code was faster, but he was able to apply an optimization to the C code that made it identical.
Really it is a Win/Win both do great things that might be able to fix some of the weakness of Python.
It's also very unlike Rust in that it's a very low ceremony language. Simple scripts are no more verbose than they would be in Python or Ruby.
After that debacle, Wikipedia decided Nim is finally good enough for them, and they even link now Nimrod_(programming_language) to https://en.wikipedia.org/wiki/Nim_(programming_language)