Mastering Dyalog APL
mastering.dyalog.com
mastering.dyalog.com
Anyway, I had a fun time a while ago translating APL programs to NumPy. At some point you get what APL is all about, and you can move on with life without too many regrets. Turns out most of the time it's more like a puzzle to get an (often inefficient) terse implementation by torturing some linear algebra operators.
If you're after a language that's OSS, has terse notation, and rewires your brain by helping you think more clearly instead of puzzle-solving, TLA+ is the answer.
Edit: if you're curious to see at a glance what APL is all about:
APL code:
(2=+⌿0=∘.|⍨⍳N)/⍳N <- this computes primes up to N and is presented as the 'Hello world' of APL.
Equivalent NUMPY code:
```
R = np.arange(1, N + 1) # ⍳N
divides = (R[None, :] % R[:, None]) == 0 # 0=∘.|⍨⍳N
divisor_counts = divides.sum(axis=0) # +⌿
result = R[divisor_counts == 2] # (2=...)/⍳N
```
As you can see, the famous prime generator is not even the Eratostenes' sieve, but a simple N^2 divisor counting computation.
Honestly this is how computers/software/programming feel in general these days and it’s ruined it all for me.
It's sort of sad, but really I think it is a weight off my shoulders.
solutions in APL can be very efficient if they are written in a machine sympathetic way
or in cases where the interpreter can map them onto one
for the curious:
https://aplwiki.com/wiki/Performance
https://www.youtube.com/watch?v=-6no6N3i9Tg (The Interpretive Advantage)
https://ummaycoc.github.io/wc.apl/ (Beating C with Dyalog APL: wc)
You focus on the 'often inefficient' parenthetical, yet, to me, your response highlights the puzzle nature of the thinking APL encourages. If anything, it shifts the question from 'how do I express this tersely' to a still narrower 'how do I express this tersely in a way the interpreter can also optimize'.
I'm not sure APL has more or less of it compared to other languages
for example in Python, even though the language has a concept of "There should be one-- and preferably only one --obvious way" (PEP 20) it is quite multi paradigm, which I think is a strength of Python
oop, functional, imperative, …
and you get tons of libraries to choose from
e.g. numpy, pandas, polars, pytorch, keras, jax, … etc
but you still also have to figure out the algorithm and data structures you want to use (like in any language)
and you also kinda want to know (if you care about performance) how pytorch differs from numpy and how that differs from using a list with boxed values
Not saying this is not the case with APL
it definitely helps if you are familiar with the APL implementation you're using if you care about performance
I just don't think it's a disadvantage of APL over other languages
What I was trying to point at was more specific: the way I experience APL thinking tends toward 'expression search' and 'notation compression', which feels, to me at least, somewhat at odds with clarity about the underlying problem. More often than not, I seemed to produce an APL-shaped model of the problem rather than a problem-shaped model expressed in a language.
When I first learned about APL, I was looking for new ways to think about computation. What I found was a language that rewarded deciphering APL programs and generating clever new ones. That is interesting and beautiful, but it was not quite the kind of brain-rewiring I was looking for. My original comment was targeting people in a position similar to mine and trying to set expectations about what they would learn best from APL. APL may change how you think about array expressions and how far they can go, but TLA+ is much closer to what I'd recommend if what you want is clearer thinking about programs, systems and state.
In vector function space, no one can hear your eigen-scream.
The reason for this is because APL is quite popular in fintech, and of course that industry has no qualms about things not being free.
Unfortunately, this seems to be a common experience. A lot of smart people only engage with APL via toy puzzles, like you did, and bounce off because that gives no insight about how to use the language in real life. IME, to really start getting APL you need to write and rewrite a full application 20 times.
It helps to read code from the masters, too [0, 1, 2, 3, 4]. These all approach architecture in different ways: pedagogical FP style, OOP heavy, data-oriented design, event-driven state-machine, or a mix of the above.
[1]:https://github.com/Co-dfns/MicroUI-APL
[2]:https://github.com/Dyalog/ewc
[3]:https://github.com/Co-dfns/Co-dfns
[4]:https://github.com/Dyalog/Jarvis/blob/master/Source/Jarvis.d...
> As you can see, the famous prime generator is not even the Eratostenes' sieve, but a simple N^2 divisor counting computation.
Well, that's because you wrote a divisor function, not a seive. Arguably, the ease of typing an outer product (i.e ∘.|⍨⍳N) can tempt us into writing quadratic algorithms unnecessarily, but this is just an experience issue, IMO.
If we want a seive, we can just write one directly:
p⊣{ω~n×1+⍳⌊N÷p⍪←n←ω↑⍨1⌊≢ω}⍣≡1↓1+⍳N⊣p←⍬
There's also a faster seive by Roger Hui [0] in the dfns workspace as well as a family of prime number functions [1] for things more than just prime generation.
I think part of this is because that is how most (possibly all) sources teach APL and array languages, solving puzzles and manipulating arrays. If you learn to write programs in an Algol derived language, you can write programs in most common languages without having to learn how to write programs, you just need to learn the language. Modern array languages sort of allow us to use them like the Algol derived languages, but this does not seem to work out so well and often does not work to the strengths of array languages.
The algorithm iterates over numbers ⍺ from 2 to N, removing the multiples that are greater than ⍺ and no greater than N from p. If the removal with ~ has to inspect all of p, then all the primes are there so we have asymptotically at least N/log N entries by the prime number theorem and we get N^2/log N time (when ⍺ is over N/2, no multiples are in range so this can be skipped, but that just cuts a constant factor 2 from the time). Conceivably p could be marked sorted, so the entries to remove could be found with a binary search. This is a bit harder to analyze, but I think each prime under √N will cause the list to change, and incur N/log N data movement. So you get at least (N/log N)^(3/2) cost, still quite a bit worse than linear.
Edit: changed the algorithm while I was writing... the new one is better, it keeps one list p of primes and one list ω of not-yet-marked-out numbers. However, primes are removed from ω one at a time, so that each of the N/log N primes has to be moved for each one before it, giving at least (N/log N)^2 cost (I mean, maybe an interpreter could binary search and also recognize when ~ only drops the first entry and do that by slicing? But the (N/log N)^(3/2) from above definitely holds). Mutating a bit array in place is pretty important to classical sieve performance.
i←¯1⋄{⍵~n×n↓⍳1+⌊N÷n←i⊃⍵⊣i+←1}⍣≡1↓1+⍳N
It's a tad tricky to carefully analyze the asymptotics of my above prime generator, since the search space of Without (~) shrinks on each iteration. I think Merten's theorem gives an estimate of e^-γ/log(p_i), which we need to sum for all primes up to N. Taking prime density 1/log(n) and integrating N/(log x)^2 over our range is O(N^2/log N), I think.
> Mutating a bit array in place is pretty important to classical sieve performance.
Challenge accepted:
p⊣{x[⍵+n×⍳⌊N÷n←p⍪←x[⍵]]←0 ⋄ 1+⍣{(x⍪1)[⍵+1]≠0}⊢⍵}⍣{⍵=N-2}0⊣x p←(1↓1+⍳N)⍬
I found MATLAB/Octave was good
Matrix conjugate transpose:
H = A';(author)
I’d really like to properly get into APL though. My plan is to solve a bunch of problems on Kattis [3].
I'm really enjoying this way of learning a new language in the age of LLMs - starting with easy problems on an online code judge website and work with an LLM to come up with/explain simple solutions. It gives me dopamine hits, lots of reps, allows me to start coding right away, and is a nice way to slowly ramp up difficulty and get practice with different features of the language.
[1] https://github.com/ebanner/dyalog-mode
https://www.dyalog.com/uploads/documents/MasteringDyalogAPL....
There are certainly valid arguments that you hive certain things up when moving to an array language, but loops are not one of those.
That said, you won't use loops as much, but that's not because loops are not available.
That said, learning APL isn't about learning the symbols any more than learning mathematics is not about learning the meaning of the various symbols it uses. To continue with that parallel, it also isn't about memorizing formulas. It is about using the tools to solve problems and, over time, changing the way you solve problems...now in 3D.
I learned APL in the early 80's and used it professionally for about ten years. The way I think of solving problems is fundamentally different in many ways because of this experience.