Introduction to GPU Programming in Chapel | Hacker News Reader

Introduction to GPU Programming in Chapel

chapel-lang.org

105 pointspjmlp24 comments

Happy to see this here. I was following Chapel pretty closely back around 2010-2012, but hadn’t paid much attention to its recent evolution.

They mention vendor neutrality but I don’t see a mention of which OSes this is supported on, or if they have any form of support for Metal on macOS (e.g. through one of the cross-compilation routes like through SPIRV).

Thanks, @subharmonicon!

While Chapel can run on many different systems, the main goal is making HPC programming much easier. Therefore, we are currently focusing on hardware that you can find in HPC systems (NVIDIA, AMD and Intel). Metal doesn't fall into that category, unfortunately. So far, the name came up infrequently in our discussions IIRC (especially targetting SPIRV), but we haven't heard from any [potential] user who may be interested in it. I would encourage you or anybody else interested in it to create an issue asking for the feature: https://github.com/chapel-lang/chapel/issues/new. Seeing public interest in that direction can change our prioritization.

One thing that I wanted to add that's not in the blogpost is the "cpu-as-device" mode. With that mode, you can use any machine, even one without a GPU, to write applications using Chapel's GPU features. That mode is for those who want to do initial development/debugging on their personal laptops before putting their application on an HPC system. In other words, while you can't use Metal directly, you can still write GPU-enabled applications in your Mac using Chapel, if the end goal is to run it on an HPC system. More details on cpu-as-device: https://chapel-lang.org/docs/main/technotes/gpu.html#cpu-as-...

Seems like it currently supports only CUDA and ROCm. Are there plans to support any other GPU vendors, such as Intel?

The example also seems a little bit high on 'magic'. Chapel is told to execute on GPU, yet it somehow decides "this loop is order-dependent so it should run on the CPU anyway"? It's not a bad approach necessarily, but I don't think you can get any such implicit serialization when directly programming CUDA C++ or similar. (I think Futhark, another language for high-level parallel computing, also doesn't suffer from this kind of thing, thanks to being purely functional.)

Re Intel support: That's definitely in our plans. However, there are also many other areas that we are actively working on to add more features, fix bugs and improve performance. When prioritizing we typically make decisions based on what our current and potential users might need in the language. Frankly, we are not seeing a big push for Intel GPU support so far. So, currently it is not near the top of our priorities. If you (or other readers) have any input on that matter where lack of Intel support might be a blocker for testing Chapel and/or its GPU support out, definitely let us know.

Re implicit serialization: To clarify; the serialization based on order-dependence is not implicit. The users should use `for` loop if their loop is order-dependent and `foreach` (and `forall`) if their loop is order-independent. In other words, the Chapel compiler doesn't make decisions about order-dependence. In particular for GPU execution: a `for` loop will never turn into a GPU kernel.

There are however some cases where a `foreach` does not turn into a kernel. You may be referring to those cases, but that's not related to order-dependence. Some Chapel features cannot execute on GPU. If your `foreach` loop's body uses any of those features then it will not be launched as a kernel even though `foreach` signals order-independence. Now, a subset of such features that makes an order-independent loop GPU-ineligible are there because we haven't gotten a chance to properly address them, yet. Another subset of such features will remain thwarters for a longer time and maybe forever. For example, your `foreach` loop could be calling an extern host function.

What about Apple Silicon / Metal support?

Thanks for bringing this up. I posted an answer to the same question here: https://news.ycombinator.com/item?id=39009566

But seeing that we already have two questions about it makes me think whether this should be something we should think more about when we are prioritizing work.

This so going to sound extreme, but it is true: I personally and professionally won’t use it for anything until it has Metal support.

The simple fact is that me and my team do our development largely on MacBook Pros and Mac Studios. We have some GPU rigs for running production code on Lunux or Windows with NVIDIA GPUs, but all of our developer tooling is on macOS. Anything that can’t run natively in recent Apple hardware gets second-class support.

That doesn't seem extreme to me, as I generally feel similarly. If you (or other readers) are genuinely interested in using Chapel with Metal, please open an issue on our GitHub repository capturing your request, as that would be valuable to us.

Just to make sure it didn’t get lost, note that it is possible to develop GPU code in Chapel on a MacBook using the cpu-as-device mode Engin mentions above, and then deploy it on NVIDIA GPUs on production systems by recompiling. This is how I develop/debug GPU computations in Chapel.

> If your `foreach` loop's body uses any of those features then it will not be launched as a kernel even though `foreach` signals order-independence.

Is this signalled/warned about so that you don’t accidentally use one of these features and kill your performance? Or a way to indicate that you specifically intend it to be run on GPU?

It's not signaled / warned about by default. However, if you want to make sure your `foreach` loop runs on the GPU, you can use the `@assertOnGpu` attribute. For instance, the following program would not compile:

  @assertOnGpu
  foreach i in 1..10 { /* Do something that can't be done on a GPU. */ }

The compiler will print a message explaining why the loop was not eligible for GPU execution.

I'll also add to @danilafe's reply that we have a GpuDiagnostics module which would count kernel launches or report them as they occur in a section of code. Something like that can be used to debug parts of your code where you do or don't expect kernel launches to occur. See https://chapel-lang.org/docs/main/modules/standard/GpuDiagno...

Thanks for joining us! I enjoyed studying Chapel a while back. I have a few questions about it.

Most AI in the press is done on expensive NVIDIA’s. Many papers have techniques with lower cost or higher effectiveness. Their algorithms are described in high-level form with little or limited implementation. Many in OSS and non-DL are using smaller models that can run on diverse hardware, if one has expertise to program it. It would be helpful to have a language that maps high-level techniques in papers to diverse hardware for use in training or review.

Can Chapel currently implement the concepts in papers on NN’s and LLM’s to run on multicore, clusters, and GPU’s? If so, can it implement hybrids where specific functions are GPU optimized but the overall design is split across machines? If not, what is missing for using Chapel for rapid prototyping of AI concepts?

These are great questions, and ones we’re very curious about as well. I don’t believe that our current Chapel team has much experience programming NNs and LLMs, having focused on other areas. That said, I’m also not aware of any intrinsic barriers to implementing such algorithms in a portable way within Chapel, potentially calling out to vendor-optimized implementations when available and appropriate.

If you, or others, would be interested in exploring this topic, we’d be very interested in either partnering with you or supporting your efforts.

(Also see Engin's response about programming tensor cores for some thematically related thoughts: https://news.ycombinator.com/item?id=39020703 )

@ColonelPhantom: Thanks very much for your questions. The following are answers I'm relaying from Engin Kayraklioglu, who heads up the Chapel GPU effort:

Re Intel support: That's definitely in our plans. However, there are also many other areas where we are actively working on to add more features, fix bugs, and improve performance. When prioritizing, we typically make decisions based on what our current and potential users might need in the language. Frankly, we are not seeing a big push for Intel GPU support so far. So, currently it is not near the top of our priorities. If you (or other readers) have any input on that matter where lack of Intel support might be a blocker for testing Chapel and/or its GPU support out, definitely let us know.

Re implicit serialization: To clarify; the serialization based on order-dependence is not implicit. The users should use a `for` loop if their loop is order-dependent and `foreach` (and `forall`) if their loop is order-independent. In other words, the Chapel compiler doesn't make decisions about order-dependence. In particular, for GPU execution a `for` loop will never turn into a GPU kernel.

There are, however, some cases where a `foreach` does not turn into a kernel. You may be referring to those cases, but that's not related to order-dependence. Some Chapel features cannot execute on a GPU. If your `foreach` loop's body uses any of those features then it will not be launched as a kernel even though `foreach` signals order-independence. Now, a subset of such features that makes an order-independent loop GPU-ineligible are there because we haven't gotten a chance to properly address them, yet. Another subset of such features will remain thwarters for a longer time and maybe forever. For example, your `foreach` loop could be calling an external host function.

Sorry for what now appears to be a double-post. Engin had just registered for HN, hadn't seen his reply going through, so asked me to relay it.

Re-reading this Q+A this morning, I also wanted to clarify one thing, which is that when a 'foreach' or 'forall' does end up being executed on the CPU, that doesn't mean it has been serialized. 'foreach' loops on the CPU are candidates for vectorization while 'forall' loops typically result in multicore task-parallelism with each task also being a candidate for vectorization.

I always liked Chapel. It is one of the very few HPC-first languages (the only other one I know of is Regent). And probably the most convenient way to port or write numeric code in heterogeneous computing environments (the only other choices are C++ or Fortran).

Do you have any thoughts on adding tensor core support? Tensor cores are most of what modern GPUs are used for but are devilishly hard to program well (they totally break most of CUDA's abstractions).

This is a good question, and I completely agree that programming them can feel... strange.

We approach this as two separate problems:

1. Can Chapel utilize tensor cores under the hood? We have LinearAlgebra and BLAS (and LAPACK) modules in Chapel. They have not been integrated with the GPU support so far. But we want them to be able to use libraries like cuBLAS for GPU-based array under the hood for GPU-allocated arrays. That model can enable Chapel to exploit tensor cores efficiently and seamlessly. Of course, you can extrapolate this to ML/AI APIs and potential Chapel modules correspond to them for example.

2. Can Chapel enable general-purpose tensor programming? This is definitely more challenging, where the main challenge is whether we can make tensor programming portable in such a way that the same code can be used on a non-GPU (and non-vector) processor. Probably a relatively shorter-term solution is to provide a low-level interface that's not much different than CUDA's Warp Matrix Functions or ROCm's rocWMMA interfaces and live with that for a while even though it is not portable. One of the thoughts that I can ignore is whether general-purpose tensor programming is _that_ common vs using tensor cores under the hood through some library as I described in (1).

To me this seems like poor design just like locales for languages are. As the computing devices a program runs on is dynamic it shouldn't me modeled as global state. Five tasks are run and should be scheduled on N cpus and M gpus.

Chapel was designed for the high performance computing community where programmers often want full control over mapping their computations to their hardware resources without needing to rely on techniques like virtualization or runtime load balancing, which can obscure key details. That said, higher-level abstractions can be (and have been) written in Chapel to insulate many computations from these system-level details, such as distributed arrays and iterators. Users of these higher-level features need not worry about the details of the underlying locales. We refer to this as Chapel's support for multiresolution programming.

That said, other communities may obviously prefer different approaches due to differing needs and constraints.

What you primarily want in HPC is control over where your data is stored. That is is subtly different from where your computations are performed. E.g an HPC computation may use N heterogeneous devices and require fine-grained control over how data is communicated between those devices. The examples with "locales" are too blunt to handle such scenarios.

We agree that the placement of data is important for HPC programmers to control. Locales are the means of controlling such placement in Chapel, whether directly (as in this article’s simple examples) of via abstractions like distributed arrays (whose implementations rely on locales).

Once the data is created, computations can be executed with affinity to a specific variable in a data-driven manner using patterns like `on myVar do foo(myVar, anotherVar)`. Alternatively, an abstraction can abstract such details away from a user's concern and control the affinity within its implementation, as the parallel iterator implementing `forall elem in MyDistributedArray` does.

According to the article, locales control where the code is running, not where the data is stored. Maybe that is implied in some cases such that if you create data in one locale that is also where it is stored, but it tells you nothing about how data created in one locale and accessed in another locale is handled (or even if that's allowed). As you mention other Chapel features that I don't know about they may fill in the gaps. My only point of contention is that the locale feature is poorly thought out and not a good way to address HPC needs.

Locales do control where the data is stored. For example:

  var HostArr: [1..10] int;  // allocated on the host memory
  
  on here.gpus[0] {
    // now we are on a GPU sublocale...
    var DevArr:[1..10] int;  // allocated on the device memory
    ...
  }

In the near term, we are planning to publish our 2nd GPU blog post where we will discuss how to move data between device and host.

Reply on news.ycombinator.com