Comment by andrewmcwatters | Hacker News Reader

Comment by andrewmcwatters | Hacker News Reader

andrewmcwatters

I'm too unknowledgeable to know, but I consistently get the feeling when I interact with a new programming language that they all fail to encompass everything that a modern system is capable of, or that they provide mechanisms with such high overheads that they're no better, or even worse, than programming languages of yesteryear.

One example of this for me is that you don't need special drivers for making your program run on a CPU; that would sound quite strange to most developers I think.

But many high-end systems have full blown GPUs on them that we basically never use unless they're specifically interfaced with for video purposes, scientific computing, etc.

Even then, you still have to use specific interfaces to access that hardware. It's not intrinsic to the language.

But say I wanted to run some generic routines across CUDA cores/Stream Processors, I have to explicitly move into the realm of GPGPU programming to do that, and then you have to write "compute kernel" code. It's not just... programming. It's fully terminology'd, it's own thing.

There's no standard library for any programming languages that automatically sends your routines to the GPU.

I don't know how all of this should be approached. I've been exposed to all of these systems, but they're all clearly isolated.

All I know is that it doesn't feel right. I fully engage development efforts feeling like I'm not using the full compute power I have available to me, and a part of that I think is that today we just don't think like developers did 50 years ago.

We have so much compute resource available to us, I think in some very fundamental ways--industry wide--we no longer understand how to use these systems. I know I don't.

I'm consistently asking myself why certain operations I write take milliseconds for some reason instead of microseconds. That's tremendously concerning to me.

C++/AMP (https://docs.microsoft.com/en-us/cpp/parallel/amp/cpp-amp-ov...) is something along those lines. I dunno if it strictly satisfies your "intrinsic to the language" requirement - you still have to explicitly request GPU processing by using the appropriate parallel_* algorithms and restrict(amp) - but it's intrinsic in a sense that compiler takes a C++ lambda and compiles it to run on the GPU.

But I don't think that you could reasonably make something that just automatically sends any random routine to the GPU, or at least not efficiently. The processing model is just inherently different.

While i agree that it is more difficult to distribute computational load between cpu and gpu than it aught to be, Gpu is not necessary faster for all workloads than cpu, it is very specific. Most of computations can probably be executex faster on cpu.

agreed, but i take the GP's argument to be that the languages aren't sufficiently expressive to allow us to communicate generically about workloads in a way that allows a system to correctly determine whether they would be faster on a CPU or not.

Correct. Another good example here is that to my knowledge, developers don't do things like write directly to L1, L2, L3... they tune so that those caches are used effectively.

What are your compiler, assembler, glibc, and the kernel, except specialized drivers for your cpu?

They are certainly more polished than the tools around GPUs or other io co-processors, but I don’t see the difference.

To be honest, we don’t even manage parallel computations on the CPU well. In most languages there is basic SIMD at most with some autovectorization. But if you don’t write your code with threads explicitly, you will only ever use one core.

Most (I assume you meant implementations) don't have even basic SIMD support like that. Even big and popular ones, such as JS, JVM, Python, Lua, .NET, Go... A couple of those do have simd intrinsics.

The JVM will have very good vector instructions coming with Project Panama.

SIMD instuctions were available to JVM already by Java 1.0 (Java 1.0 and VIS instructions in UltraSPARC both came in 1995). So if this Panama stuff comes next year, they got around to it in 27 years. I predict we'll have autovectorization in another 27 years in 2048.

Autovectorization is done by the JIT compiler since forever. But even state of the art autovectorization is very finicky. What’s coming in Panama is an abstract API over SIMD instructions that will reliably compile to the corresponding instruction, while providing safe fallback in the form of serial for loop on platforms not providing the means. It is not behind in times, it will be ahead of most languages.

Interesting!

This seems a good article on the developments in JVM autovectorization: https://inside.java/2021/01/27/extending-c2-autovectorizatio...

Isn't that something that Mono had for ages?

You say we don't need drivers for the CPU, but there is still software that had trouble running on the M1. And there is plenty I can't get to run on a raspberry pi.

Quite arguably, we do - such a driver would be a "compiler". Most compilers do some really wild things to reshape your code to take (some) advantage of new processor features.

There's a huge amount of "brain offloading" that's deferred to the compiler guys, and in a ton of ways, that's the point of having a driver as well; they adapt generic commands like "write a file to disk" to work for you on any kind of disk; compilers adapt your C (or whatever) code to work on e.g. any of the current intel chips of the last N years (within which there are quite a few breaking incompatibilities; I've had code compiled for a really particular i7 CPU actually not work on a different one, because it was taking advantage of new instructions only available on that particular chip - when I reconfigured my compiler to hit a wider compatibility target, the issue went away.

So yeah - there's a huge amount of knowledge "hidden away" there.

Yeah. I don't know why I only hinted at that argument. This was exactly what I meant.

Julia with ArrayFire seems to be straightforward, efficient and easy to use.

Reply on news.ycombinator.com