Nim for scientific computing: Back to the future
rnduja.github.io
rnduja.github.io
You mention Nim for scripting - can I pique your curiousity to add a Nim script example to the Num statistics project? http://github.com/numcommand/num
The basic infrastructure was in place after ~2 months. Since then, our team has been using it extensively, and I've been filling out the functionality & evolving the API (for example, testing different array iteration & access interfaces, to see what feels awkward vs convenient, balancing the trade-off between Nim-ness vs Numpy-ness vs Pythonic-ness, etc.).
You write your Nim procs, then annotate them with a special annotation (that's valid Nim, but inert most of the time). When you want to auto-generate Python wrappers around your Nim procs, there's a Python script you run that creates & compiles Python C-API code that exposes the Nim procs in Python as a shared library. This auto-generated code also translates Nim exceptions (including back-traces) to Python exceptions, calls the Nim GC before you return to Python runtime, etc. The auto-generated code also includes auto-generated Python docstrings, extracted from the Nim procs.
There's a Nim type that provides a nice Nim interface to Numpy arrays, so you can pass Numpy arrays into your Nim procs and access them natively. We've also recently added support for a few cute features, like the tuple return types & default parameter values. :)
The API is not yet solidified, so I'd call the project "alpha" or "beta", but it definitely works! The intention is to open-source it soon (once we're happy with the API).
Would you be interested in using something like this? What features would you need? Even better, are you a Python+Nim coder who could contribute?
Feel free to reply here or contact me via email (contact info in profile).
We're just waiting until we're happy enough with our Numpy array API that we'd be confident in calling it "mostly stable".
The (real) Numpy's C API is a perfect example of an API in which gradual accretion of new functions & functionality has given rise to a monstrous (both huge & horrible) API. The naming scheme & parameter patterns are inconsistent; sometimes the names are ambiguous (ie, unclear because they are overly general) or confusing (ie, the function does something other than what you'd expect from the name); and there are several very-similar functions, where it's not always clear which one you should use.
I do understand why the Numpy C API has evolved this way: Once you introduce a function in an API, you can't remove it / rename it / change its parameters, or you will break backwards compatibility with your earlier releases and cause headaches for many of your users. (And of course, there are constraints upon an API written in C, that do not affect an API exposed in Python or Nim.)
So we're really trying to dogfood our own API as much as possible before we inflict it upon the public. :)
Is there a good way to get in contact with you?
I would rather avoid the double ecosystem in the long run, but it would be a great thing to have until the Nim ecosystem grows.
In terms of ecosystems, I think that there is deployed Python code in the world that will never be re-written in Nim. I would prefer that Nim can be used to write drop-in extension modules for this Python code, rather than not at all.
I also think that there are situations where Python is a better fit for a problem than Nim. For example, handling JSON files in Nim (or any statically-typed language) will always be awkward (you have the choice between a double-dispatch Visitor Pattern or accessor methods that cast), but in Python, handling JSON is the easiest thing in the world. Likewise for any heterogeneous container.
By the way: Great article about Nim, as always. :)
* It's not entirely removed, just obfuscated so that if needed it can be still used as long as you "know what you're doing" and are willing to put up with it not being a general purpose REPL. Use
nim secret
to run it.
Nim feels a lot more like Python than one would think just reading the descriptions. I can't put my finger on why, exactly - maybe the type inference that gives an almost duck-type experience, maybe the incredibly helpful dogfooding community; I'm not sure.
Still, I think, the similarity in syntax can help. If there is one thing that python got right, in my opinion, is its straightforward syntax, and that helps saving brain power for something other than parsing
I'd rather work directly in a full-featured, internally-self-consistent language (and Nim is a very nice language) from the start. Hence -- for me, at least: Nim. :)
Aside from an entirely-subjective preference that caused me to fall in love with Nim at first sight, there are practical reasons for choosing Nim over Julia.
The overarching reason is that there is much more Python code in the world than Nim or Julia put together, and I think it will remain this way for a long time. (My startup previously worked almost completely in Python, for example, aside from some C or C++ extension modules and the usual Bash scripts for sysadmin.) Python also has some solid webserver frameworks, and is great for a variety of general-purpose tasks.
So, I wanted a solution that would enable us to incrementally rewrite code as necessary, without needing to rewrite everything from Python all at once. Also, I have no desire to rewrite our webserver code from Python. Flask & Django are just fine.
As I understand it (and please correct me if I'm wrong), Julia doesn't compile to C. So I can't compile snippets of Julia code and call them from my Python main-loop.
In contrast, not only is this possible in Nim, but Nim's macro system & well-developed AST spec make it very easy to traverse my Nim procs during compilation, to auto-generate per-proc wrapper code in C.
Cheers!
Cython is certainly great, but still Python objects and C structs differ, and you have to keep in mind the difference. Same for C functions and Python functions. And if you use the dynamic features of Python, you do not gain much in efficiency, because Cython does not have static dispatch, hence it has to rely on runtime checks.
In a way, Cython is its own language, and it is not a particularly powerful one. The help you get from types is limited, as many things will just have type object. At this point, I just prefer to use Nim, which is friendly enough, and has overloading, generics, macros and many other features.
I mean, if the money actually came from a different organization than Yahoo.
Its good they do have support from 3DICC though.
The way I see it is this. Python is not a particularly fast language, but has evolved a big ecosystem of scientific libraries. Now people are running into the language limitations, hence the development of various tools to compile parts of Python - either AOT or JIT - with PyPy, Numba, Cython, Numexpr and so on.
The problem is that all these tools (save from PyPy, which is just a different interpreter) introduce their ow sublanguage, slightly incompatible from one another. For instance, both Numba and Cython have to infer types, but their type systems are not equal.
Nim does not have the reach of Python ecosystem, but it is a solid language, that manages to combine well the speed of C and the simplicity and flexibility of Python.
I would rather start from a solid foundation and add libraries on top of it gradually, rather than start from a developed ecosystem and add a host of tools that try to fix the inherent limitations of the platform.
Hence, in the short term it is great that the projects created by Continuum exist, but eventually I would like to have something a little more consistent
However I didn't know yet that the Nim language is that powerful.
What? Nim's homepage has the phrase "C FFI is easy in Nim.." along with some example code.