Nvidia Warp: A Python framework for high performance GPU simulation and graphics
github.com
github.com
I recently saw a 2-3 orders of magnitude speed-up of some physics code when I got a mid-range nVidia card and replaced a few NumPy and SciPy calls with CuPy.
>> sympy.utilities.lambdify.lambdify() https://github.com/sympy/sympy/blob/a76b02fcd3a8b7f79b3a88df... :
>> """Convert a SymPy expression into a function that allows for fast numeric evaluation""" [e.g. the CPython math module, mpmath, NumPy, SciPy, CuPy, JAX, TensorFlow, SymPy, numexpr,]
sympy#20516: "re-implementation of torch-lambdify" https://github.com/sympy/sympy/pull/20516
In case you haven't tried it yet, Pythran is an interesting one to play with: https://pythran.readthedocs.io
Also, not compiling to C but to native code still would be Mojo: https://www.modular.com/max/mojo
Simply copying a chunk of data between two libraries through Python is already painful. There are so-called "buffer API" in Python, but it's very rare that Python users can actually take advantage of this feature. If anything in Python as much as looks at the data, that's not going to work etc.
Similarly, concurrency. A lot of native libraries for Python are written with the expectation that nothing in Python really runs concurrently. And then you are presented with two bad options: try running in different threads (so that you don't have to copy data), but things will probably break because of races, or run in different processes, and spend most of the time copying data between them. Your interface to stuff like MPI is, again, only at the native level, or you will copy so much that the benefits of distributed computation might not outweigh the downsides of copying.
When I was doing performances stuff: Intel was our main platform and memory consistency was stronger there. We would try to write platform-agnostic multi threading code (in our case, typically spin-locks) but without testing properly on the other platforms, we would make mistakes and end up with race conditions, accessing unsynchronized data, etc.
I think Python will be the same deal. With Python being GIL'd through most of its life cycle bits and pieces won't work properly, multi-threaded until we fix them.
Only a restricted subset of Python is efficiently compilable.
C is much cleaner and portable. Easy to use in Python directly.
Otherwise, I’m not going to bother.
https://github.com/NVIDIA/warp?tab=License-1-ov-file#readme
Looks more "source available" to me.
> Open-source software (OSS) is computer software that is released under a license in which the copyright holder grants users the rights to use, study, change, and distribute the software and its source code to anyone and for any purpose.
"2.7 You may not use the Software for the purpose of developing competing products or technologies or assist a third party in such activities."
vs
"California’s public policy provides that every contract that restrains anyone from engaging in a lawful profession, trade, or business of any kind is, to that extent, void, except under limited statutory exceptions."
https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml...
(Owner/Partner who sold business, may voluntarily agree to a noncompete, (which is now federally https://www.ftc.gov/legal-library/browse/rules/noncompete-ru... banned) is the only exception I found).
I'm not a lawyer. Any lawyers around? Could the 2nd provision invalidate the 1st, or not?
Question: why would NVIDIA, makers of general intelligence, which seems to compete with everyone, publish code for software nobody can use without breaking NVIDIA rules? Wouldn't it be better for everyone if they just kept that code private?
As for why the “to the extent” phrasing exists, consider an example: an employment contract consists of two clauses, A: that prevents the employee from disclosing confidential customer data to third parties, and B: a non-compete clause (which does come under the same provision mentioned by grandparent). If the employer ever sues an employee for violation of A, they shouldn’t be allowed to argue that they aren’t subject to it because of clause B.
Most of the magic is behind closed source components, and it’s posted with a fairly restrictive license.
The main optimizations (scheduler, vectorizer, etc.) are hidden behind these shared libraries. If open-sourced, they might reveal hints about proprietary algorithms and provide clues to various hardware components, which could potentially be exploited.
i like the value they bring, but the trend is against the existing paradigm of how python ecosystem used to be.
It’s quite frustrating that there’s seemingly no long-lived framework that allows me to write simple numba-like kernels and try them out in NVIDIA GPUs and Apple GPUs. Even with taichi, the Metal backend was definitely B-tier or lower: Not offering 64 bit ints, and randomly crashing/not compiling stuff.
Here’s hoping that we’ll solve the GPU programming space in the next couple years, but after ~15 years or so of waiting, I’m no longer holding my breath.
yes, the heterogeneous compute frameworks are largely broken, except for OneAPI, which does work, but only on CUDA. SPIR-V, works best on CUDA. OpenCL: works best on CUDA.
Even once you get past the topline "does it even attempt to support that", you'll find that AMD's runtimes are broken too. Their OpenCL runtime is buggy and has a bunch of paper features which don't work, and a bunch of AMD-specific behavior and bugs that aren't spec-compliant. So basically you have to have an AMD-specific codepath anyway to handle the bugs. Same for SPIR-V: the biggest thing they have working against them is that AMD's Vulkan Compute support is incomplete and buggy too.
https://devtalk.blender.org/t/was-gpu-support-just-outright-...
https://render.otoy.com/forum/viewtopic.php?f=7&t=75411 ("As of right now, the Vulkan drivers on AMD and Intel are not mature enough to compile (much less ship) Octane for Vulkan")
If you are going to all that effort anyway, why are you (a) targeting AMD at all, and (b) why don't you just use CUDA in the first place? So everyone writes more CUDA and nothing gets done. Cue some new whippersnapper who thinks they're gonna cure all AMD's software problems in a month, they bash into the brick wall, write blog post, becomes angry forums commenter, rinse and repeat.
And now you have another abandoned cross-platform project that basically only ever supported NVIDIA anyway.
Intel, bless their heart, is actually trying and their stuff largely does just work, supposedly, although I'm trying to get their linux runtime up and running on a Serpent Canyon NUC with A770m and am having a hell of a time. But supposedly it does work especially on windows (and I may just have to knuckle under and use windows, or put a pcie card in a server pc). But they just don't have the marketshare to make it stick.
AMD is stuck in this perpetual cycle of expecting anyone else but themselves to write the software, and then not even providing enough infrastructure to get people to the starting line, and then surprised-pikachu nothing works, and surprise-pikachu they never get any adoption. Why has nvidia done this!?!? /s
The other big exception is Metal, which both works and has an actual userbase. The reason they have Metal support for cycles and octane is because they contribute the code, that's really what needs to happen (and I think what Intel is doing - there's just a lot of work to come from zero). But of course Metal is apple-only, so really ideally you would have a layer that goes over the top...
> Intel, bless their heart, is actually trying and their stuff largely does just work, supposedly, .... But they just don't have the marketshare to make it stick.
Isn't OneAPI basically SYCL? And there are different SYCL runtimes that run on Intel, Cuda, and Rocm? So what's lost if you use oneAPI instead of Cuda, and run it on Nvidia GPUs? In an ideal world that should work about the same as Cuda, but can also be run on other hardware in the future, but since the world rarely is ideal I would welcome any insight into this. Is writing oneAPI code mainly for use on Nvidia a bad idea? Or how about AdaptiveCpp (previously hipSYCL/Open SYCL)?
I've been meaning to try some GPGPU programming again (I did some OpenCL 10+ years ago), but the landscape is pretty confusing and I agree that it's very tempting to just pick Cuda and get started with something that works, but if at all possible I would prefer something that is open and not locked to one hardware vendor.
IMO if they had focused from the beginning on ML similar to Mojo, they would be in a better place.
It feels like the ball is entirely in Apple's court. Well-designed and Open Source GPGPU libraries exist, even ones that Apple has supported in the past. Nvidia supports many of them, either through CUDA or as a native driver.
Architectural elements of _all_ graphics cards.
> The efficiency of executing threads in groups, which is known as warps in NVIDIA and wavefronts in AMD, is crucial for maximizing core utilization.
[0] https://www.xda-developers.com/how-does-a-graphics-card-actu...
>The threads are executed in groupings called warps. The amount of threads making up a warp is architecture dependent. On AMD GPUs the warp size is commonly 64 threads, except in RDNA architectures which can utilize a warp size of 32 or 64 respectively. The warp size of supported AMD GPUs is listed in the Accelerator and GPU hardware specifications. NVIDIA GPUs have a warp size of 32.
https://learn.microsoft.com/en-us/windows/win32/direct3darti...
Its basically the software implementation of DirectX
A recent simulation code when pushed with gcc openmp-simd matched performance on a 13900K vs jax.jit on a rtx 4090. This case worked because the overall computation can be structured into pieces that fit in L1/L2 cache, but I had to spend a ton of time writing the C code, whereas jax.jit was too easy.
So I'd still like to see something like this but which really works for CPU as well.
I've thought about extending JAX with custom primitives and a custom lowering process to support constructs which work on CPU (but don't work on GPU) -- but if I did that, and wanted a nice programmable substrate -- I'd need to define my own version of abstract tracing (because necessarily, permissive CPU constructs might imply array type permissiveness like dynamic shapes, etc).
You start heading towards something that looks like Julia -- the problem (for my work) with Julia is that it doesn't support composable transformations like JAX does.
Julia + JAX might be offered as a solution -- but it's quite unsatisfying to me.
What's this community's take on Triton? https://openai.com/index/triton/
Are there better alternatives?
What does this mean? I've mainly heard the term "spatial computing" in the context of the Vision Pro release. It doesn't seem like this was intended for AR/VR
> Overall the biggest distinction as of now is that Taichi operates at a slightly higher level. E.g. implict loop parallelization, high level spatial data structures, direct interops with torch, etc.
> We are trying to implement support for lower level programming styles to accommodate such things as native intrinsics, but we do think of those as more advanced optimization techniques, and at the same time we strive for easier entry and usage for beginners or people not so used to CUDA's programming model
It's noteworthy that Taichi also supports AMD, MPI, and Kokkos.
behind the sytax, there is plenty of heavy lifting for writing sophisticated code, when need be. that surely helps with the network effect.
NERFs backpropagating pixels colors into the volume, but also semantic information from the image label, embedded from an LLM reading a multimedia document.
Or something like this. Anyway, wanna buy an NVIDIA GPU ;)?
> import warp as wp
Can we please not copy this convention over from numpy? In the example script, you use 17 characters to write this just to save 18 characters later on in the script. Just import the warp commands you use, or if you really want "import warp", but don't rename imported libraries, please.
Pandas and Numpy are popular, sure. As is Tensorflow (often shortened to tf). But where do you draw the line, then? should the openai library be imported as oa? should flask be imported as fk? should requests be imported as rq?
It seems to happen mostly to libraries that are commonly used by one specific audience: scientists who are forced to use a programming language, and who think that 1-letter variables are good variable names, and who prefer using notebooks over scripts with functions.
Don't get me wrong, I'm glad that Python gets so much attention from the scientific community, but I feel that small little annoyances like this creep in because of it, too.
import os as o
import sys as sAt the end of the day, this is a minuscule issue either way. I won't import stuff your way, but if you hand me code like that, I won't change it.
import warp as np
Now you can re-use your old code as-is!Anyway, I also don't really care that much about character count -- but it's the only reason I could think of for why someone would want to rename "warp" to "wp". If anything, abbreviating libraries like this will add more cognitive load because you have to remember what all the abbreviations mean and how to import them.
OpenCL seems like it's just obsolete
The fact that OpenCL 3.0 is basically OpenCL 1.0 rebranded, as acknwoledgement of OpenCL 2.0 adoption failure, doesn't help either.
Very tactful from nvidia. I have a lovely AMD gpu and this library is worthless for it.