Nuitka: a Python compiler
nuitka.net
nuitka.net
The author is alone, making steady progresses on a crazy complicated project, always communicating regularly, in an humble tone.
Then you download the stuff, and it works.
I'm still amazed on things like this happening. I though those events stopped existing after the 2000 bubble.
It seems like most people here are bashing on this in one way or another. People talking about Cython, people talking about how they don't even use Python anymore, etc. etc. This makes me super sad, because this is seriously one of the most technically amazing projects I have seen in several years. It is a labor of love by one single guy who frequently posts status updates where he's accomplished some new thing that was thought to be impossible.
It's really an inspiring story of somebody who isn't stopped by the haters, building something that he wants, that lots of people have told him is impossible or useless, but yet he still continues to make steady progress plugging away at it.
I really used to identify with the average commenter on this site, and feel that there was really a time when something like this would be truly understood and appreciated... But now, rather than some Python expert explaining how incredible this is, or some compiler wizard spotting this, being interested and offering to help we have lots of sniping from uninformed people who just seem to be annoyed that this won't magically speed up their Python now
Can we have a technical deep-dive on what this guy's accomplished? A few clever curious hackers tearing about the code spotting all the brilliance and sharing bits of it with us with commentary? No. Let's just all go on and on about how we should just rewrite everything in Rust. Again.
It goes a long way, and it's easier than contributing if you have no C++ background.
Ive never used either outside of a toy implementation, but Cython is based on Pyrex. They are different projects.
But I have a new project now that needs to be both highly concurrent as well as reasonably fast and I'm considering giving Numba a try since it seems like it's both faster and easier to work with.
Does anyone with experience with both have a comment as to the downsides of Numba vs Cython?
If you're writing fresh code, I think Cython is much better. Numba doesn't really give you a language to work in. With Cython you can basically write C or C++ --- you can declare structs, manage pointers, even have C++ classes with templates if you want. This hash table does similar work to Google's dense_hash_map, with similar performance: https://github.com/explosion/preshed/tree/master/preshed . I think it's hard to write something like this with Numba.
I'm not sure I agree with the statement that it's strictly easier to work with. One big problem with Numba is that it is very unpredictable performance wise. Sometimes it does an amazing job speeding up your function and sometimes it just doesn't, and I find it very difficult to guess a priori how Numba will behave. It's also not always obvious how to rewrite code that Numba doesn't like to code that Numba likes.
Cython on the other hand is very easy to reason about. The way it compiles your code to C is obvious and consistent and assuming you have basic familiarity with C it's very easy to reason about how a certain piece of code will perform after being compiled to Cython.
Basically when Numba is at its best behavior it is much easier to work with than Cython, but when it isn't it's much harder to deal with. Cython gives consistent and reliable performance every time.
There's also a Python package for integration with setuptools.
But ran into compatibility problems with most libraries we used (one application were a native QT-based and one were for web based on Falcon). Not sure how it looks today but if it's anything like before I'd recommend developing with it in mind from scratch if you want to use it.
[0] https://talkpython.fm/episodes/show/127/shipping-software-to...
Hasn't changed much, but Docker has made many of the distribution problems more-or-less irrelevant (but only if you use Docker).
Sure, it might be a better language than JavaScript, but you would have to rip out and replace the entire standard library, and Python without its libraries wouldn't be any fun.
So we are stuck unless the browser vendors decide to ship a Python VM next to their JS VM. And they won't do that.
Can it actually? I've certainly seen benchmark's where a JITed program is comparable/marginally faster than a C program, but I've never seen one where it outright trounces C.
Further, a jit can compile to specific hardware. An aot must anticipate what hardware is possible. If you aot compile to several possible hardware configs, then you'll bloat the binary.
Not since the development of "profile-guided optimisation", no.
That's certainly what HotSpot's promoters want us to believe, but I'm not convinced there's a lot of extra value in having detailed data on the current run, compared to 'static' profile-guided optimisation. I'm not sure if there's been any serious study of this question.
As to the specific hardware: this might matter for things like SIMD, yes, but often won't make any appreciable difference. Some Linux distros have the user compile almost everything locally, but aren't all that much faster for it.
If two small programs are about the same speed in C vs JITlang, then the theory says the JITted one should take over as the program gets bigger.
The folks at Anaconda can probably help you. Use the conda package manager.
I especially encourage to use virtualenv too keep your application completely disjoint from the rest of your system. It really helps when in the future you decide to upgrade the OS.
What? As long as you compile it in the virtualenv, it is. And now with wheels that's basically the case for any modern library. Yes, you will share the C libraries in stdlib, but if you really want you can isolate those too.
Being able to package as an MSI or exe is a fundamental requirement for adoption of Windows client side applications.
The nice thing with using setuptools is that you can create different distributable package types depending on the need.
My gripe is not with that sort of deployment, but rather the web-related stuff. Lack of boilerplate is a big part of what makes Python attractive for web development, if they start ramming EARs down my throat I'll be very sad.
Not that much to complain, the community is doing an awesome job. Especially given the very delicate balance we need to maintain since we have a _very_ heterogeneous user base.
Imagine if cPython would have been integrated in Web Browsers.