Comment by bee_rider | Hacker News Reader

Comment by bee_rider | Hacker News Reader

> In particular, our implementation does not utilize OpenMP or any other high-level multithreading APIs, and thus gives the programmer precise low-level control over the details of the parallelization, which permits more robust optimizations.

I totally disagree with this line of thinking. The person who knows most about the hardware on which the program will be run is the person running it, not the programmer. The OpenMP API somewhat complicated in an attempt to allow the programmer to, at a high level, express ideas about data locality to the runtime.

Unless we're imagining a universe in which the programmer is around to tweak their source code for every hardware platform, the idea of "giving the programmer more control" is a dead end. The programmer must be given expressiveness.

Threading libraries are complicated because hardware is complicated. I mean first generation threadrippers are a little old, but they still exist: do we really want to have everybody re-write the code to handle "I have multiple packages on this node, NUMA between the dies in a given package, NUMA between the packages in the node!"

Unless we're imagining a universe in which the programmer is around to tweak their source code for every hardware platform

It doesn't have to be the whole universe just a region in which a particular kind of specialized scientific computing is done. I have not idea what this specific thing is good for but looking at it in through the lens of general purpose computing truisms probably misses more than it illuminates.

There's some truth there, but I think it is actually flipped.

Scientific computing codes generally stick around a while (or at least they might, or at least we hope they will), they are run on complicated platforms by motivated people. There's incentive to do run-time tuning, and there's value to making them general.

On the other hand, you might have things like a videogame console, where the program is tuned for a particular platform. There's plenty of room there for programmer control I think (not sure though, that isn't my field -- in the Unity/Unreal era maybe they try and make things general? But I bet the programmer can exert great control if they want).

Scientific computing codes generally stick around a while

I don't think this is generally true and wasn't at all true in my limited exposure to scientific computing. There's lots of one-off, ad-hoc code that isn't building the next BLAS or whatever. The whole trope of 'scientists are terrible programmers' doesn't come from some universally prevalent incentive to write general, long-living code.

Scientific computing is not scientists doing computing. Its weather simulation, CFD, and other engineering FEA software. Lots of it has been around for decades and may even be written in Fortran.

Come to think of it, this may be targeted at scientists after all.

It says it is for "high-performance scientific computing" though. Could actually be intended for scientists who happen to be writing programs and mislabeled though. But do they typically care about manual hardware tuning, the sort that this library claims to enable?

It's common among scientists who write low-level code. Also, the author of the library we are talking about is a physicist, and the library probably exists for a reason.

Totally agree. Plus, OpenMP is a standard that’s grown to take into account new hardware. It works in C, C++ and Fortran, which is a huge bonus. It’s stuck around for a long time for a reason - before it and MPI were standardised, every set of processor hardware has it’s own parallelism libraries and the programmer had to learn a new set.

> The person who knows most about the hardware on which the program will be run is the person running it, not the programmer.

Then again, in scientific computing, often these are the same person.

AFAIK in scientific computing, the research group is targeting their university's particular cluster and the cluster's admins/operators are there to maintain and protect the shared facility, not to get the research result.

I think it depends on the group.

For example, I have more-or-less free rein a small cluster which belongs to my professor (my group is small enough to just manually coordinate to not annoy each other), then there's a university cluster if I feel like messing around with bsub and waiting in a queue. I'm also under the impression that it isn't super difficult to get a handful of xsede credits when starting a new project.

I prefer to debug on my local machine or the group cluster, so I don't have to risk putting pointless buggy runs into the shared queues (which might be costly, depending on policy, but even if they aren't costly I don't want to hog resources for no reason).

If I write the code that tunes performance for a particular machine, then I have codepaths that I wrote and can't test locally very easily. Better to use the existing OpenMP and MPI knobs to tune. Sure, you can end up with an evil thread interleaving that only shown up when you have 100 cores, but this should hopefully be rare.

That's my perspective at least. I do numerical stuff, so I'm shielded by some of the real nitty-gritty "perfectly synchronized threads" stuff by the fact that my costly operations are usually decent sized linear algebra calls, so it is usually best to give the lowest NUMA level(s) to BLAS, LAPACK, or maybe some sparse solver (which should be reasonably scalable and pretty well tested).

Reply on news.ycombinator.com