High-Performance GPU Computing in the Julia Programming Language (2017)
devblogs.nvidia.com
devblogs.nvidia.com
using PyCall
np = pyimport("numpy")
res = np.fft.fft(rand(ComplexF64, 10))
No casting back and forth. This is a toy example, julia ofc has fftw bindings.
Interop with C++, MATLAB, Mathematica etc is similarly simple.
In practice, things either don't exist, or are poorly implemented:
Plotting simple things take 30 seconds.
And that's if you don't count the time it takes to `] add Plots`, especially on Windows!
And the REPL is broken.
And the editor is slow and annoying (Juno or vscode).
And documentation ranges from poor (no examples, buggy between platforms, broken links due to version updates) to non-existent. For example, lots of tutorials will often link to broken links to official documentation, links that one time were thought to be working but now aren't.
And so on...
Documentation is always a problem (especially for smaller packages), but I don't feel I had more issue with Julia than other languages (with a few exceptions, some languages managed to have an exceptional culture in terms of great documentation). Most of the issues were due to the fact that Julia just got to 1.0 a year ago, and the breaking changes made so most documentation became outdated, but this will only become less of a problem since the language became stable.
Julia has one of the best REPL of any language I used, and thankfully I didn't meet with any problems with it. Might be a good idea to create an issue on the github.
[1] https://discourse.julialang.org/t/compiler-work-priorities/1...
1. It's about compiler latency. Compiling happens when you call the function. That indeed can and should be improved.
But there are other things that contribute to the experience being shitty.
2. Is about adding a package, when adding a package it downloads all the dependencies (which includes cairo, and WinRPM on windows which all have problems of their own).
4. Is about the poor atom experience - I'm not a fan of electron apps myself - and about the slowness of the LSP on vscode, which just does not provide a good experience on vs code and things are often broken, especially on the latest versions.
If you think all these things are "compiler latency" then perhaps you're part of the problem.
Visual Studio Code is definitely fast enough in my machine (especially with a Revise.jl workflow), which is why I also assumed it was stuff like running part of the code or using the Language Server Protocol, which will hit the same compilation lag. I agree that Atom is slow, and it's one of the reason I don't use it.
Though I'm clearly biased since my experience is entirely in Linux, it's possible that the Windows experience is just worse.
What happens in between figuring out how to make basic tooling work and actually getting something done is frustration.
Building is so clearly not a one-time thing if you use the language to explore solutions, and play with things.
Sure, the usual answer is "well its an initial cost, its faster after that" but not all my code would otherwise take days to run. As long as "using CSV" takes 10 seconds, I'm out.
Currently I'm mainly using Jupyter Notebook and that is by far the best experience I've had(it's like Revise but much, much faster). But to me it seems Jupyter Notebook wasn't designed with code outside of a single isolated file in mind, which makes it cumbersome in some cases.
I like the language, but I hope the situation improves soon. Editing code in the browser is not that much fun.
You should really back up that claim, because in my experience it's absolutely great
Also your documentation issue is also strange. Yes certain things don’t exist but I would say the Julia docs is quite well made. In particular if you use the REPL documentation I find it much better than Python. Tends to be quite nice examples, color coding etc.
This is true, but that's a _very_ low bar to pass. Julia is a Lisp, and deserve to be compared to other Lisps rather than to lesser languages. Every Common Lisp or Scheme I have used has a vastly superior REPL experience than Julia. Even Clojure is better.
Don't get me wrong: I love Julia, and I hope it will eventually replace Python as the main language for scientific computing, data science and machine learning. But the REPL experience, at this point, leaves a lot to be desired. I'm sure it will improve in the future.
If so (which isn't), then we can start suggesting new features, perhaps better text editing capabilities, or introspection, better access to documentation.
It's broken because it has poor and puzzling exceptions, because output lags on Windows from gtk bugs (known for years, never fixed, huge GitHub discussion),and because the shell mode is often broken.
In fact, I have run tens of thousands of lines of essentially equivalent CUDA and OpenCL code (automatically generated) on the same hardware, and performance was in all cases very similar[0]. If anything, CUDA was actually slower than average (but in the cases I investigated, this was down to arbitrary differences like the CUDA compiler not unrolling some loops as aggressively and such).
[0]: https://futhark-lang.org/blog/2019-02-08-futhark-0.9.1-relea...
CUDA doesn't require rewriting code with ${OSS} framework of the year, every year. They need to earn that lock-in with future compatibility guarantees which none of OSS projects has.
> CUDA doesn't require rewriting code with ${OSS} framework of the year, every year.
How so? Change the GPU from Nvidia, and you are forced to rewrite code. That's the whole point of lock-in, it's a tax on developers. CUDA doens't guarantee you anything, if you don't stick with their GPUs.
Vulkan on the the other hand has conformance requirements.
It's not. It uses LLVM, which can easily target AMD GPUs. (Whether the Julia folks have invested in making this work, I dunno, but it's not Extremely Hard.)
Understandably nvidia gives you the wrong impression.
The focus on CUDA comes from the fact that most HPC systems for scientific computing are using Nvidia GPUs. That is finally slowly changing.
Only when they started getting a beating of PTX bytecode and multi-language deployment on CUDA did they woke up and came up with SPIR (later SPIR-V) and SYCL, which still isn't widely deployed.
Still the Julia team did a great job in making all those diverse features feel part of one connected philosophy instead of an ad hoc pile of functionality, even if it does take a little while to fully internalize it.
How exactly CUDA is "democratizing" anything, if it's tied to Nvidia? Vulkan backend would make more sense for that purpose.
While it would be better in a democratic sense for GPUs to be accessed using a fully free API, having an easily usable proprietary API is still more democratic than a difficult-to-use API (especially when, as here, the easy-to-use layer is actually fully free, and can perhaps be retargeted to fully free lower layers later).
[0]: https://gpuopen.com/compute-product/hip-convert-cuda-to-port...
I'd say, Nvidia are being hypocritical here, with this whole "democratizing" claim. They are direct beneficiaries of the lock-in they are advancing with it.
OpenCL on the other hand is C FTW and now kind of supports C++ if one has luck with the drivers.
From that point of view is democratizing GPGPU programming to anyone that doesn't want to deal with either C or C++.
I also don't see you complain that so far the only mature SYCL SDK is available from Codeplay, thus making it a single vendor "standard". At least until Intel (One API) and others actually come one with their SYCL extensions, because naturally nothing that Khronos does can be without extensions and its multiple execution paths.