cburdick13

The sample is really designed to show the simplicity of the syntax, and the performance is just a side effect. Where you'll see a bigger performance difference with numPy/cuPy is when kernel fusion happens where MatX is typically able to fuse many things into a single kernel at compile-time and cuPy launches many kernels. If you have a specific type of expression you'd like to compare please let us know.

Hi, if you don't mind opening an issue asking for this we can run these and put in the readme.

Hi, I looked into this and it seems that it is indeed using >=

Depends: cuda-libraries-11-8 (>= 11.8.0), cuda-drivers (>= 520.61.05)

Hi, yes, the original development was started for radar users who did not know CUDA but needed to write in c++. Many of our examples and code are radar related for that reason.

Hi, I addressed the comparisons in other comments, but in general this is for c++ users and not Python. It's more of a comparison to numPy/cuPy, and we do have a table showing the comparison in the docs.

We don't support automatic differentiation (yet).

We have benchmarks in the benchmarks directory, but these are for things like convolution, matrix multiples, etc. It's not for running a traditional benchmark set like resnet.

Like most benchmarks it really depends on what you want to do, and since it's a general library everyone might care about different things.

Hi, those libraries don't have a GPU counterpart, and for matrix multiplication we only support GPU right now.

I have to admit I'm only tangentially familiar with gnuradio, but matx should be able to integrate with any C++17 codebase. We have several examples of integration with our streaming sensor pipeline called holoscan.

see this radar pipeline example for one:

https://github.com/nvidia-holoscan/holohub/tree/main/applica...

That's a fair point. I'll look into it.

Hi, what specifically are you looking to benchmark on the K80? Users are free to contribute and we've had many external PRs.

Contribution guide is here: https://github.com/NVIDIA/MatX/blob/main/CONTRIBUTING.md

If you use the run files rather than the package manager files it's always installed in completely separate folders inside /use/local/cuda.

What you're describing with an "all" install can somewhat be accomplished with containers right now and none of the dependency problems.

I agree we should update it given that it's also a very old comparison (2 years now). The cuPy comparison should be fair since it was the same GPU.

I addressed it more here: https://news.ycombinator.com/item?id=37760120

Hi, you can choose not to install the driver with a CUDA install, and to download the driver separately.

The main difference is Jax is for python primarily, while MatX is c++. This might seem like a poor answer, but in many domains (quasi-real time, signal processing, etc) the language is important to give certainty guarantees on performance.

MatX has been used in several projects with hundreds of microsecond deadlines, which is not usually something you'd choose Python for out of the box.

We are instead targeting users who already have python or high-level code that they need to port to c++ for whatever reason, and want to do it in the easiest way possible. With c++17 we're able to provide a simple syntax without compromising on performance compared to most native code.

The main difference is the GPU part. This is a large difference because the same lazy evaluated template type can be run on the CPU or GPU through what we call an executor. On the CPU it's likely very similar to how xtensor is already using expression trees, but on the GPU it's quite a bit different because if the libraries backing it and optimized kernels.

The syntax of MatX also allows us to do more kernel fusion in the future to improve performance without any changes.

It was mentioned elsewhere, but we support down to cuda 11.4, which supports down to the Maxwell architecture (nearly 10 years old now).

Good point, and agreed the landing page is a bit sensational. I mentioned it elsewhere but between MatX and cuPy we see a 3-4x performance difference on average. The gap tends to widen with more complex workflows where compile-time kernel fusion gives more improvements compared to something like a single GEMM.

We typically support whatever the underlying library supports. For int8 the support would come from cuBLASLt currently. I don't believe that or Cutlass supports mixed precision inputs, but I can check.

We've tried our best to match python as well as we can, or falling back to matlab-style if Python doesn't have it. Many of our unit tests are verified against python, so the conversion is typically very easy. The one thing that python has that makes this much easier is keyword arguments. We've tried to use overloads to mimic this as best as we could.

That being said, in the example on the home page there are notable differences:

1) we have the run() method. The reason is that the expression before the run is lazily evaluated for performance and does not execute anything. Having the run() method allows you to run the same line of code on either a CPU or GPU by changing the argument to run()

2) in MatX memory allocation is explicit. Python does it as-needed, but this causes a performance penalty with allocations and deallocations that are not under your control. Specifically in the FFT example, numPy will allocate an ndarray prior to calling it, but on the same line. In MatX the allocation is (typically) done before the operation so you can control the performance of the hot path of code.

If you have any specific suggestions, we would love to hear it

Hi, what matrix sizes and types are you working with and how many batches? In general it sounds be similar to eigen, but with GPU support. We have several svd methods for different scenarios, so if you give us the info above we can let you know the performance.

That's right, we've tested down to pascal, but this should work on Kepler too since CUDA and the underlying libraries support it.

Hi, besides an (subjectively) easier syntax, the performance should be higher compared to libtorch. Every operator expression (think of it as an arithmetic expression) is evaluated at compile-time and is often fused into a single GPU kernel. This also removes the need for JITing. If there's a specific workflow you're curious about comparing libtorch vs MatX please let us know and we can try it out.

Hi, having feature parity with cuPy is a daunting task, especially for a C++ library. At this point we feel we have a good foundation for all kinds of basic and advanced tensor manipulations, and have a growing number of library-based functions on top of that. The low-hanging fruit was wrapping the CUDA math libraries, so that has the most progress.

Since MatX was originally intended for streaming/real-time processing the focus has been on making C++ for CUDA easier to use for those kinds of applications.

There are also a lot of things in cuPy and sciPy that don't make a lot of sense to do on the GPU, like offline tasks such as filter design in signal processing.

Also, since C++ users typically are writing in that language for maximum performance, we've put a large focus on making sure we are as close to writing optimized CUDA as possible. In general, most workloads we've tested are about 3-4x faster than the cuPy counterpart due to better fusion and language overhead.

We have discussed supporting cusparse as well, but cusparse typically requires a different input type like a CSR matrix. This is not something you'd typically want to detect/convert, so we're still discussing ways of getting this integrated cleanly.

We do have several versions of SVD, including one that calls cuSolver.

Feedback is always appreciated!

Hi, MatX currently has partial support for CPUs too. Please see this comment:

https://news.ycombinator.com/item?id=37758635

Currently we support both CUDA and CPU to some extent. CPU is done through standard C++ (and soon stdpar). Obviously standard C++ is problematic since it doesn't include everything we support (FFTs, matrix multiplies, etc). One option is to use open-source libraries that do these, but then it ends up being a lot of dependencies that are hard to manage. We have plans to improve CPU support soon, so stay tuned.

Hi all, I'm one of the maintainers of MatX. I didn't expect it to hit HN this soon, but happy to answer any questions.