Also is this actually faster than just python? I tried it out with some bad expensive looping using lists with python3.9 and it was 100ms slower (1.3sec vs 1.4sec).
Just in general why should I be using Nuitka?
> In the future Nuitka will be able to use type inferencing based on whole program analysis. It will apply that information in order to perform as many calculations as possible in C, using C native types, without accessing libpython.
I believe the difference with PyPy is that PyPy tries to do this using just-in-time compilation and nuitka uses ahead-of-time compilation.
I did try this. With the "stock" python3 (3.8.something) I couldn't get nuitka to compile my code (or the example from the docs). With python3.9 I could start the compilation, but I didn't have libpython-dev shared object and finally with python3.9-dev I was able to make the binary. However after transfering it to another Ubuntu 20.04 machine I couldn't run it since it was linked against libpython3.9-dev.
Just as side note. The same python code (i.e. the .py file) would run on both machines out of the box.
The only dep would be libc, and if you compile against an old version, it should be forward compatible.
Mind sharing that code? I'm sure someone would like to be able to look into this.
python -m nuitka --clang --follow-imports main.py
repo: https://github.com/MRCIEU/gwas2vcf
If someone can make the program run faster by whatever means, it will make a bunch of people quite happy.
import time
start = time.time()
list = []
for i in range(10000000):
list.append(i)
sum = 0
for item in list:
sum = sum + i
print(time.time() - start)
I expected the binary to essentially be
1. start = time.time()
2. pre-populated list since it should be constant e.g. list = [1,2,3,...] or completely remove it since it is not used anywhere
3. pre-filled sum since it too is constant or same as above, removed since it is not used anywhere
4. getting second time.time() and printting the difference
Have you tried xrange?
And I am fairly sure xrange() is not in Python3
Come to think of it, you don't particularly know what sum = sum + i is doing either. There could be an __radd__ implementation in whatever your custom range spits out that indexes the WWW, renders a Mandelbrot or whatever.
And yes, of course overloading range and __radd__ that way would also be considered stupid, but if the loops were optimized away, it wouldn't be a drop in replacement for Python.
Since the `sum` is never accessed by anyone it can be optimized away and now since the only accesser of `list` is gone it too can be optimized away.
Thus both loops most certainly CAN and SHOULD be removed from the final compiled code. Only reason Python can't do it is because it is not compiled, so it has to actually run through the code at runtime and see what happens.
This is basic static analysis and any other compiler for any other language would do this. This is literally why we have keywords like `volatile` in compiled languages.
I don't understand your first argument at all. Why wouldn't the compiler know what the standard `range()` function does? The second one only makes sense since we aren't type hinting, however that too could be reversed by analyzing the code.
If we can't trust the functions in standard library then what is the benefit of compiling the code?
I'm sure it knows what the standard range() function does. It just doesn't know that that's the standard range function you use when the code is being run. Any Python program is also a module. Any module can be imported. Any imported module's global namespace is unknown at the time you write the program.
Imagine I have your code, above, in a module called nextlevelwizard, and I write my own little progam as this:
import builtins
def myrange(n):
cur = 0
while cur < n:
print(cur)
yield cur
cur += 1
builtins.range = myrange
import nextlevelwizard
This is a silly example, of course, but the dynamic nature of Python means that it really is very hard to skip steps you assume are irrelevant.
Even in the example you gave it could replace the loop with a bunch of print calls since the result (yields) do nothing.
Compilers are hard.