Chapel 1.32 | Hacker News Reader

Chapel 1.32

chapel-lang.org

123 pointspjmlp29 comments

Wow, I didn't realize Chapel was still under development, I haven't heard about it since 2011. Its contemporary rival was Fortress from Sun Microsystems (https://en.wikipedia.org/wiki/Fortress_(programming_language...), which does appear to be defunct, both of which are languages intended for highly-parallel computation on supercomputers.

And works on less than super computers.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Related. Others?

Chapel: Programming Language for Parallel Computing - https://news.ycombinator.com/item?id=36158373 - June 2023 (2 comments)

AoC 2022 in the Chapel language blog - https://news.ycombinator.com/item?id=34274056 - Jan 2023 (1 comment)

A Look at Chapel, D, and Julia Using Kernel Matrix Calculations - https://news.ycombinator.com/item?id=23403748 - June 2020 (2 comments)

The Chapel Parallel Programming Language - https://news.ycombinator.com/item?id=22708041 - March 2020 (60 comments)

Show HN: Parallac.js – A JavaScript clone of Chapel for distributed computing - https://news.ycombinator.com/item?id=13072797 - Nov 2016 (4 comments)

Chapel: a parallel programming language designed for productivity at scale - https://news.ycombinator.com/item?id=11747709 - May 2016 (19 comments)

New language built from the ground up for productive parallel programming - https://news.ycombinator.com/item?id=11257792 - March 2016 (2 comments)

Learn Chapel in Y minutes - https://news.ycombinator.com/item?id=10029449 - Aug 2015 (13 comments)

Why we need Chapel for large-scale parallel computing - https://news.ycombinator.com/item?id=7972971 - July 2014 (1 comment)

The Chapel Parallel Programming Language - https://news.ycombinator.com/item?id=7951706 - June 2014 (10 comments)

The Chapel Parallel Programming Language - https://news.ycombinator.com/item?id=5047197 - Jan 2013 (2 comments)

The only viable replacement for Fortran we currently have. The rest are not getting it.

What are C/C++/Zig/etc. missing that Chapel has?

Working in heterogeneous computing environments (and having first-class facilities for that), while still being easy enough to immediately start adapting numerical code from any reference implementation.

What are these first class facilities? Just curious

Roughly, the sets of computational problems that people used (use?) MPI for. Things like numerical solvers for sparse matrices that are so big that you need to split them across your entire cluster. These still require a lot of node-to-node communication, and on top of it, the pattern is dependent on each problem (so easy solutions like map-reduce are effectively out). See eg https://www.open-mpi.org/, and https://courses.csail.mit.edu/18.337/2005/book/Lecture_08-Do... for the prototypical use case.

My answer would be that Chapel supports a partitioned global namespace such that a variable within the lexical scope of a given statement can be referenced whether it is local to that CPU's memory, stored on a remote compute node, or stored within a GPU's memory (say). The compiler and runtime implement the communication on the programmer's behalf and take steps to optimize away unnecessary communication. Other key features include first-class support for creating parallel tasks in high-level ways, including parallel loops.

Julia? not only it's 1-index, it's also very imperative

Julia is great, but not HPC-friendly.

This is interesting. I've had good experiences, but admittedly haven't done problems with huge amounts of communication. Is there some fundamental issue? Lack of supporting libraries?

I wonder if the JIT model prevents predictable benchmarking/optimization, making long-term robustness difficult? It might also make using MPI difficult. But this is mostly speculation on my part.

Seems to be down, so here's a cached version

https://web.archive.org/web/20231009141106/https://chapel-la...

I really like Legion as an alternative for HPC applications, among many others (https://legion.stanford.edu/)

Legion has been used to develop distributed, drop-in replacements for libraries like NumPy and SciPy Sparse -- https://github.com/nv-legate/cunumeric/, https://github.com/nv-legate/legate.sparse

Legion is great! I like that its developers really get HPC. But it is C++, with all of its baggage. (Update: There is also Regent language on top of C++ API, looks interesting).

Hey, I'm the creator of Regent. Feel free to ask me questions.

Cool! I wondered how production code would look like, then I found this: https://github.com/stanfordhpccenter/soleil-x/blob/master/sr..., and was thoroughly impressed.

I wonder if it would be possible to easily set it up on our university's HPC cluster (which is not a supercomputer, but still has SLURM and everything there).

please email us at hpccenter@stanford.edu, and ask to be put in touch with someone at PSAAP. I would be happy to help!

How would you characterize/contrast Regent vs. Chapel? What do you see are the main drawbacks/benefits of each?

Chapel's killer feature, in my opinion, is being able to take something that looks like a simple loop and turn it into distributed code (via domain maps). To the extent that you can write your program in terms of that feature, you can get very clean programs. But once you step outside of that feature, you've basically back to full-on parallel programming (i.e., explicit PGAS programming with nearly every parallel programming construct under the sun). Chapel offers a nice syntax, but it's semantically not so different from writing SHMEM or MPI or one of these other explicitly parallel programming models.

Regent does also support loop auto-parallelization, though it's not the focus of the language and not generally how people write idiomatic Regent programs. Regent fundamentally is a task-based programming model. "Task" is a fancy word for a function that can run in parallel. The key is that (a) tasks execute with sequential semantics, and (b) inside of a task, you can do basically whatever you want. The compiler doesn't need to analyze the code aside from verifying that you're passing data around correctly. This means the set of programs you can write in the "nice" subset of the language is much, much larger. The vast majority of Regent programmers never encounter any explicitly parallel programming constructs, there is no way to make code that deadlocks or races, etc. On the other hand, organizing programs in terms of tasks does still take effort and a degree of cognitive shift. You still have to divide the program into parts that can be parallelized, even if you're not responsible for parallelizing them.

Thanks, this is helpful. It seems like (based on your reply) there are people successfully using Regent for scientific computing (I'm assuming); do you think the language is a viable choice for industry, or are there particular milestones you're looking reach?

Yes, we're focus mostly on scientific computing. One of our major apps is S3D, which we ported to Regent from Legion C++ [1]. This has been the first time the domain scientists have been able to modify the code themselves; previously the C++ was too much for them. There are other examples of Regent apps elsewhere in this thread.

If by "industry" you mean in areas related to HPC, then Regent is likely to be applicable to what you want. The further you get away from HPC, the less likely that would be. You probably wouldn't use Regent to write a web server, though it's not impossible....

Right now my biggest item is making sure we support all the DOE supercomputers, which means adding support for Intel GPUs. (Regent currently supports NVIDIA and AMD.)

[1]: https://theory.stanford.edu/~aiken/LegionRetreat22/slides/ch...

Thanks, yes, I was thinking along the lines of HPC type applications in industry.

I should also note that there is Pygion if you want to use Python. Not a lot of great reference material right now, but there's the paper:

https://legion.stanford.edu/pdfs/pygion2019.pdf

And code samples:

https://github.com/StanfordLegion/legion/tree/stable/binding...

This is great.

We definitely don't have an excess of supply in the "HLLs for vendor neutral GPU programming" dept, especially if you have a healthy reaction to C++. Chapel and Futhark, what else?

If you know Python, you can also use Taichi:

https://www.taichi-lang.org/

I’d love to see all these communities come together to share ideas with each other even more.

Taichi solves a different problem. It's (mainly) a tensor compiler that targets kernel generation on different architectures, while Chapel is for orchestrating and communicating between computations over distributed memory spaces (and heterogeneous devices).

Reply on news.ycombinator.com