Comment by ummonk | Hacker News Reader

Comment by ummonk | Hacker News Reader

I love Julia, but what I don't get in this particular instance is to what extent it actually allows you to write code that you can't with Python. Sure, cassette allows you to do more powerful automatic differentiation, and various GPU arrays packages make writing code that runs on GPUs very natural, but it doesn't seem like that big an improvement over using a framework like TF / Pytorch / Numba in Python. Sure, those libraries aren't written fully in Python, but neither are Julia gpu libraries - you obviously have to convert to CUDA or similar code at some point to run on the GPU.

Julia brings great improvements in ability to write simple idiomatic serial code that runs at near C speeds on the CPU (whereas idiomatic Python code is at least 10x slower if you use an optimal compiler, and 1000x slower if you use the standard python interpreter). But for highly parallel code dependent on element-wise or broadcasting array operations, I just don't see the issue with Python.

""" [...] it is possible to perform kernel-like operations without actually writing your own GPU kernels: a = CUDA.zeros(1024) b = CUDA.ones(1024) a.^2 .+ sin.(b) """ [https://juliagpu.gitlab.io/CUDA.jl/usage/overview/#The-CuArr...]

As far as I know, that example code creates an ad-hoc kernel that performs the computation in a single pass.

I honestly don't know if that is possible in other frameworks.

Yeah, that's definitely available in Python frameworks. (I was wrong to mention Numba - I think CuPy would be a better example: https://docs.cupy.dev/en/stable/overview.html)

That's right, I found it: https://docs.cupy.dev/en/stable/tutorial/kernel.html#kernel-...

Calling out to array libraries in an interpreter (Python+Numpy, MATLAB, whatever) leaves a lot of optimizations on the table. The abstraction is not zero-cost.

For example, the simple array expression v+cw (where v,w are vectors and c is a scalar) can be calculated using fused multiply-add instructions. But this is not possible when called in an interpreter, which has to make separate calls for each array operation. For the same reason, this same calculation may saturate the memory bus. This is not an issue with Julia or other compiled languages. Not sure how Numba and/or PyPy optimizes it.

>neither are Julia gpu libraries

Wrong. Julia can call out to Cuda kernels, but it also does codegen for custom ops and types. Technically with enough engineering time, someone could rewrite all those C cuda kernels in julia, and julia would just generate low level machine GPU code on the fly.

Reply on news.ycombinator.com