Why programming languages matter [video]

Then my apologies to you and everyone else who replied to me: I really should have just said "I feel we scatter our attention and energy too much and we don't collectively evolve our craft as much as I believe we are capable of".

It's not about them being a new concept. Do we have Erlang actors or Golang goroutines there? Are they fully safe like Erlang's actors or Rust's futures would be (provided you don't use escape hatches)?

If not, they remain toys.

Same, by the way. I am ashamed that I didn't do it more often over the course of a 22-year long career. I really should change that.

I just launched 2 experiments to production this week without going through the launch approval process. I'm such a rebel.

> How does one "move the art forward" without understanding the landscape first?

Practice data structures, make complex algorithms described in books (most of which are free, and the rest are maximum $35), code a small program for an embedded controller for once to see how the sausage is made first-hand, participate in different kinds of open-source. There are plenty of ways outside of "the world needs one more LISP interpreter".

> Just because something "is known" to humanity doesn't mean it's known by every individual presently.

Sadly. I wish I belonged to another species where this assertion was not true. Sigh.

> Viewed this way, how many people in the world, presently, have sufficient knowledge to "move the art forward"? How many have the means? I'd put it at a few hundred. Maybe you're one of them?

Me? Absolutely not, I am 43 and at this point I am severely burned out. I had so many ideas and ambitions but working for the man for 22 years has crushed me. Maybe if I get a bag of money in the upper 7 digits I'll be able to reignite my tinkering spirit but we don't live in fairly land and this is not happening so nope. You're looking at one more person who was crushed by the free market forces.

That being said, you have people working on e.g. the Rust compiler or OCaml in general. Likely a bunch of geniuses and I love it that they are actually funded and are keeping up the good fight. Gives me some hope.

> maybe you could assist them by sharing your experience?

Yes, that's the best idea really: education. But as we all know (1) younger people never listen to advice, and (2) I probably never found the right audience (though thinking of it I am starting to see some lectures popping up lately in my city again, maybe I should try to go). I've been told I am excellent mentor (as recently as 3 months ago) but my health and time constraints prevent me from doing it more.

But how do you educate people to be unhappy with the status quo? I seriously don't know. The human brain is kind of like this: "Hey I am not hungry, I have where to sleep and most of my basic needs are covered, I guess the world is perfect and I don't need to change anything at all anywhere". Observed it thousands of times in my life and I am bitter about it to this day, likely to my grave.

> But maybe we need to play more efficiently?

Yep, I would not be against some central "authority" website / app that distributes "play" tasks to whoever is willing to do them (obviously with plenty of repetition and redundancy, you can't have 10 people agree to tackle on 10 different tasks and then they never show up again). And this does not remove choice, you can still present people with like 50 choices and they surely will identify with at least one of them.

All this free energy, wasted all the time. [sighs deeply] I wish we had more structure and direction is what I am saying all along.

repl based development, or until people see how cool structural editing really is

Oh I do like that for sure, but as you said it's a solved problem in very few languages.

But in general, very few such unquestionably good practices are adopted.

You can put a program with 50 lines and 10 bugs in production. :)

To me "production" also means "runs reliably over long periods of time, does not fall over under load, and has no unexpected panics". Well, and also "makes full use of the available computing resources and prevents lag as much as possible".

"Toy" is basically "I am such a huge fan of LISP, I am convinced that the world absolutely needs one more interpreter!".

Just because something's old doesn't mean it's not relevant, yes might not've been the best language to suggest - the more modern BQN, Uiua and Singeli exist too - but it's still a fairly niche paradigm. Ideas tend to come in cycles too - look at the 1980s ideas of the transputer or connection machine.

I wanted to point towards a programming paradigm that's approach enables you to take advantage of the parallel execution possible within chips today due to the notation being both precise in intent yet vague in execution. Take summing an array (`+/vector`) or selecting values given a boolean mask (`mask/values`) - both these very simple expressions are expressible directly in SIMD instructions, as there's no for loop index enforcing an order.

The parent comment was talking about aligning programming languages with the hardware. I am not commenting about the viablity of those languages, but rather that if your goal is to write the most performant code by understanding the strengths and weaknesses of the hardware than using fp concepts is not the way to do it.

"FP" camps tend to come in two flavors: "I'm a mathematician writing a computer program, and all problems will be made to look like math problems even if it means the program becomes an inscrutable mess of types and dense syntax" and "functional-ish idioms are included". The latter is useful, sometimes, for cloud computing and parallel computation; the former tends to have too many problems (slow build, slow execution, poor jargon laden syntax, etc.) to be very useful outside of academia.

I am (perhaps obviously) biased, here, but I tend to just roll my eyes whenever any of my colleagues suggests we should use functional programming to solve a problem. There are actually very few real-world use cases where it's objectively better.

Research around functional languages involves their execution models. Even ignoring execution models immutability is a staple of functional programming which is not good for performance.

> "Is the CPU really that serial, when it literally reschedules your instructions based on the graph of their interconnections?"

Yes. Yes, it is because all of that rescheduling and reordering is completely hidden at great effort and expense to make it seem like the instruction stream is executing in exactly in the order specified. If it weren't, lines of code would essentially execute in an indeterminate order and no programs would function.

Functional languages have a core that is well suited for parallelism, but all mainstream functional languages have various features that are unsuitable for parallel execution - particularly if you want to target limited machines like GPUs.

The Futhark programming model is standard functional combinators: map, reduce, scan, etc. You can do that in any Functional language, and it is mostly trivial to rewrite a Futhark program in Haskell or SML or OCaml. Futhark removes a lot of things (like recursive data structures) to make efficient parallel execution easier (and even possible), and adds various minor conveniences. But you don't really need to add something for functional programming to be suitable for parallelism; you just have to avoid certain common things (like linked lists or laziness).

There are many cases where you can replace a call to `.iter()` in your Rust code to a call to `.par_iter()` from `rayon`. Those cases are trivial, and it's great.

It's just about compiling a neural net down to run as efficiently as possible. Either on GPU, CPU, or your own accelerator. Neural nets are very computational intensive, while being pretty uniform internally and as a class. Everyone making silicon and a fair few other companies as well has an AI compiler team right now. At the moment the hot product would just be LLM tokens for as few cents each as possible.

I believe it's either related to the new AI-specialized chips, or maybe to the factoring or neural network graphs.

Any specialized domain tend to have its own domain specific language, so obviously it would be true for AI, too.

Ai problems boil down to compiling linear algebra problems onto very complicated chips.

In standard processing, the code is so branchy that we often resort to heuristics in order to get 'good enough' perf.

The FLOPS difference between a cpu and gpu is huge. It makes things that are intractable on cpus possible. Without gpus there is no deep learning.

That being said, writing code for gpus by relying on cpu compilers will result in terrible perf. In order to take advantage of the hardware you have to take into account minute details of the architecture that most cpu compilers ignore.

Cache oblivious algorithms are algorithms that know that there is a cache but don't rely on particular cache sizes. It's the way a lot of cpu code is written because it means not having to deal with particulars.

On gpus, particulars matter. For example, to compile a matrix multiply on an Nvidia GPU, you cant just use vectorized multiplies and adds. No. In order to achieve max performance you need to utilize the warp level matrix multiply instruction which requires that you split an arbitrarily sized matrix into the perfect native tensor sizes and then orchestrate the memory loads (which are asynchronous on gpus, and transparently synchronous on cpus) correctly. If you don't you waste millions of dollars (literally).

So whereas on a cpu you might just modify your matrix multiply loops to get contiguous memory access and add some vectorization in and cross your fingers, on a gou your compiler needs to take the trivial three nested loop algorithm, look up the cache size particulars and instruction capabilities for the particular generation of the chip, and then rewrite the loop nesting to make it optimal. All while making sure you don't introduce further memory hazards (bank conflicts), etc. So your simple three nestled loop algorithm gets turned into a nine nested loop monstrosity.

The stakes are much higher here and the optimizations much different. Whereas on a cpu, we kind of give up with the branching complexity, and just do our best since we never truly know the state of the program, on gpus, the algorithms being executed are extremely amenable to static analysis so we do that, and optimize the shit out of them.