Hacker News Reader

Comment by linkingday | Hacker News Reader

Can't wait for the language to be as widely used as Python or R. Been playing around with it at home since the 1.0 release and it really is a joy to use.

View on HN

I think Python has a large advantage in this space since it is not as specialized. For many projects that I work on the actual core number crunching code is small (or part of a library I don't have to worry about). The rest of the code could be scraping data, or a web application. These are things that there are numerous packages available for in Python but seem to fall outside the focus of Julia.

I would highly recommend reading this recent post in the Julia forums [1]. It's from a "relatively new" Julia user explaining why Julia is actually a very good general purpose language, e.g. for string manipulation of scraped data. You do have a good point about packages, though: The problem is that the user base is smaller than Python's so some packages have yet to be created. And since the userbase is currently largely numerical programmers, non-numerical packages are more likely to be missing.

[1] https://discourse.julialang.org/t/not-only-for-technical-com...

Julia would be ideal for those task as well, but alas the libraries aren't there. Julia can easily call Python functions with PyCall, although I still wouldn't want to extensively use a complicated Python library in this way.

Seems like it would be better to call Julia from python.

Python doesn't have the typing support or even availability of many types. Calling Julia from Python in some cases would require something like cython or require even using the C api ahem.

They're way ahead of you: https://github.com/JuliaPy/pyjulia

I wish that Python feels a bit of Julia's heat, as that might be the only way for more PyPy love.

Python without GIL to provide real shared-memory threading would address 90% of my issues with the language.

In my experience, CPython is 100 to 1000 times slower than C (or C++, Rust, Fortran, etc...). CPython would be better off fixing the speed (perhaps by embracing PyPy) and staying single threaded than removing the GIL. At least if you're choosing threads for performance gains.

Think of it this way: A single threaded C++ or Rust program can do more work per second than a hypothetical GIL-free CPython with 64 cores...

> CPython is 100 to 1000 times slower than C

That's very context dependant. With annotations, typical calculations compile to C code roughly equivalent to native. The more of the actual runtime you use, the more you call into cpython which can't be sped up this way. Cython compiled code will be somewhere between C and cpython in terms of speed, but putting a single number to it will be always misleading.

>> CPython is 100 to 1000 times slower than C

> That's very context dependant. With annotations, typical calculations compile to C code roughly equivalent to native. The more of the actual runtime you use, the more you call into cpython which can't be sped up this way. Cython compiled code will be somewhere between C and cpython in terms of speed, but putting a single number to it will be always misleading.

OP said "CPython", not "Cython". CPython is pretty reliably 100X to 1000X slower than C.

Oops, you're right. One day I'll learn to read :-)

No worries. I suspect many of your downvoters were confused as I was that you appeared to be arguing that CPython could compile your Python code to efficient C! Took me a few passes to spot the misunderstanding.

I'm using compiled libraries for all my heavy lifting—the Python code is there to glue those libraries together in convenient manner that allows for exploratory programming. GIL-free CPython would let me distribute computations across multiple cores without having to deal with the overhead of multiprocessing (serializing objects across process boundaries, etc.).

If your compiled code is thread safe, you can releae the GIL in your library and achieve parallelism with current CPython. If your compiled libraries are not thread safe, you can't get parallelism anyway.

Maybe, if your heavy lifting functions are very short lived, releasing and acquiring the GIL is holding you back...

Btw, I'm not defending the GIL. Python has enough visibility and significance to deserve a good implementation. It's just that most people don't realize the slowness of CPython itself is a bigger impediment to performance than the GIL. Python should be in the same speed league as JavaScript, but there are too many things hindering progress.

Pypy is basically abandoned project. Main version is Python2 and It has poor support for python 3. The latest version They are supporting is python3.5. Beside that even with Pypy they wouldn't be even close to nodejs. Julia is on par with Fortran or C. Reason behind Julia amazing performance is its type system. e.g multiple dispatch and value types. Python lakes both. Python also has parallelism problem with GIL. Mypy team currently works on compiler based on static type annotation: https://github.com/mypyc/mypyc I think they could gain much more performance than Pypy.

This is not correct. Pypy is actively developed.

Numerical code is always written in numpy which can drop the GIL.

I recall that the main issue with PyPy is that it doesn't have as good interop with C and a lot of CPython codes just call C code and hence is difficult to widely adopt given the amount of reliance on CPython.

Reply on news.ycombinator.com