J can look like APL or English
wjmn.github.io
wjmn.github.io
I think part of the reason is that J's syntax for function composition is too heavy. In an improved visualization of J like the one shown here, I would find very interesting to try something like, for example, composition by simple juxtaposition and trains with under(or over)lining, maybe even playing with colors to indicate if verbs form a train or other composition. I understand that is a quite different problem from just substituting @: with o̲.
Nevertheless, very nice work. It is interesting to see the English and APL versions next to each other. The video is also very good.
With regards to how different of a problem that would be, J's syntax is context-sensitive. If you were to accurately identify trains and forks, you could not use regex, and would probably end up implementing a subset of the J parser.
For an example of a very incomplete and largely suboptimal parser that uses the above: https://github.com/Rscho314/J-compiler/blob/master/ast.ijs
The main idea is to change visual representation instead of using delimiters. Different fonts or text sizes sizes or cursive/inverse/underline are also valid ways of displaying different "colors".
BC To ask a slightly broader question, what is the connection between computer language and thought? To what degree does our choice of how we express software change the way we think about the problem?
AW I think it does a lot. That was the point of Ken Iverson's Turing Award paper, "Notation as a Tool of Thought." I did pure mathematics in school, but later I was a teaching assistant for a graduate course in computer algorithms. I could see that the professor was getting killed by the notation. He was trying to express the idea of different kinds of matrix inner products, saying if you have a directed graph and you're looking at connections, then you write this triple nested loop in Fortran or Algol. It took him an hour to express it. What he really wanted to show was that for a connected graph it was an or-dot-and. If it's a graph of pipe capacities, then maybe it's a plus-dot-min. If he'd had APL or K as a notation, he could have covered that in a few seconds or maybe a minute, but because of the notation he couldn't do it.
Obfuscation is not the goal of this style of programming, even though it looks like it's been deliberately golfed. Being able to express complex operations in terse but well-defined ways makes the intention clear. If you know the meaning of the primitives in +/ . * then there's little to debate that the combination is the matrix product. Whereas, if you have this operation expressible only with multiple explicit loops and intermediate variables, there are greater opportunities for error and a greater reading overhead.
I don't think J/APL are suitable for every sort of programming, and the sort of domains where J/APL excel are array-focused. J/APL's notation are a poor fit for writing a frontend single-page application (SPA), for example, and I would never want to maintain a SPA that merged all their functions into one line. I would not write complex integrals in English, and equally I would not write a medical document in mathematical notation.
But the goal of these languages is to improve the readibility of array manipulation by using composable and well-defined operations, not to obscure it. There is a large upfront cost to learning the notation, no doubt about it, and this is a necessity before you can use the language at all. But the intention is not for it to be harder to read - it's for it to be easier.
a, array programming concepts
b, apl like syntax
I think the combination of these two makes J and co interesting. I am just not sure if the second is required. Not sure how much is lost if we use the following:
kmc =: right (step power infinity) initialise
initialise =: (left deal tally at right) from right
step =: (left groupby allocate rank 1 2) then (mean every)
allocate =: euclidean then (equal minimum) then indices then head
euclidean =: (minus then square then sum) vectors
We lose manipulability.
I can perfectly read an expression like this:
(a add b) multiply (a add neg b)
(square a) minus (square b)
APL languages have a considerable number of symbols to memorize (with the exception of k, which has very few primitives, J in particular has many of them), but the core is always the same and relatively small (see the APL Dictionary, by Iverson), and many symbols are related so they are easy to remember. Nevertheless, it takes a while. Once you learn the symbols, you need to learn common idioms and constructions, which takes even more (J, with a prevalent tacit style, makes this quite difficult) but once you start getting it and can manipulate the symbolic expressions in your head, the English version does not matter any more.
There are other reasons, like keeping a large amount of the program on screen (see the videos of arcfide), but mental expression manipulation is what convinced me.
With J and APL, one pragmatic benefit I do see of the symbols over the words is the ability to see and use patterns if one is familiar with the language. This is harder to do with the words, which spread concepts thinner. I think Iverson expresses these ideas very nicely in Notation as a Tool of Thought.
The APL/J/K family strongly assume that readers of the language are familiar with their parsing and semantics, so idiomatic usage of each language makes no concessions for what is felt to be unnecessary. It's a different mindset for sure. When the users of the language share an understanding of what the symbols mean though, they no longer think it's obfuscated.
J takes such strong inspiration from English (to the point that it calls functions "verbs", operators "adverbs" and "conjunctions", and arrays "nouns"), that I once thought J would be better written entirely as English words instead. After having made such attempts, I actually found it harder to read and maintain the English versions. I couldn't attach strict behaviour and meaning to the words like I could with the special symbols. Perhaps I didn't entertain this idea long enough, but the words felt considerably less malleable.
That is interesting that you have this experience with maintaining the two versions. I guess I should try it out as well.
I'm very fond of a number of videos by the late John Scholes narrating APL solutions step by step, including the well-known demonstration of Conway's Game of Life in APL (0). His videos on depth-first search (1) and solving sudoku (2) are also brilliant.
(0) https://www.youtube.com/watch?v=a9xAKttWgP4 (1) https://www.youtube.com/watch?v=DsZdfnlh_d0 (2) https://www.youtube.com/watch?v=DmT80OseAGs
https://github.com/danlm/QNial7
It is a fairly standard APL which inherently uses english-like notation instead of symbols.
``` kmc =: right (step power infinity) initialise initialise =: (left deal tally at right) from right step =: (left groupby allocate rank 1 2) then (mean every) allocate =: euclidean then (equal minimum) then indices then head euclidean =: (minus then square then sum) vectors ```
That's quite readable and compact. I'm fascinated by J now.
What many people lack is a sense of history. Why does something like J even exist? Do you know? I do. It was a horrible mistake on the part of Iverson in response to the limitations of the hardware issues we had to deal with at the time (think 80's).
How does a man who authored a paper exalting the advantages and power of notation as a tool for thought [0], turn around and create something like J?
Simple answer: Business, for the most part, and, if I am going to be benevolent, a naive attempt to bring something to the masses (while destroying APL).
Here's what you had to do at the time if you wanted to run APL on IBM PC's (or clones):
- Buy a specific motherboard (if it was not an original IBM PC)
- Buy a specific graphics card (which really wasn't a graphics card at all, just text)
- Purchase or program a character EPROM that replaced rarely used characters with APL characters
- Swap out the stock character ROM with the custom EPROM
- In some cases install a toggle switch outside the computer to be able to switch between APL and standard ROM's
- Purchase either an IBM Selectric or pinwheel printer
- Purchase an expensive character ball for your IBM printer or an equally expensive APL character for your printer
- And then you had to buy your APL interpreter from a company like STSC for hundreds of 1980's dollars
- APL was outside the purview of "civilians"
- Normal office environments were never going to consider something like this
- There was a cost and technical complexity to wanting to use APL
- Installing a modified APL character EPROM broke applications that relied on the characters
in the character table that were replaced (mostly characters you could use for basic drawing on a terminal)
- APL was very much limited to those who absolutely-positively had to have it
- That meant academia and very niche applications (financials)
- Widespread adoption was pretty much impossible
And it was a failure. Not the least of the reasons for this being that, to the average person, J code looks like ASCII vomit. APL has distinct symbols, just like mathematics. One might not be able to understand the code but the end product not only looks very different from ASCII vomit, it is. Because notation has power.
The correct path would have been to understand technology would catch-up to the requirements of APL and focus on extending and improving the language. A simple example of this would be adding an object-oriented programming layer to the language. Compilation and distribution through executable files and (civilian) installation packages, which are valuable for commercial closed-source security and deployment, would also have been a valuable add-ons. I was working with Assembler, FORTH, C, and C++ at the same time I worked with APL. I could deliver executable user-installable programs in any of those languages --and sell them!-- while my use of APL was limited to a range between academic and internal tools. It would have been brilliant to have been able to deliver finished user applications indistinguishable (by the user) from those written in C or C++. That was impossible. They should have worked on that, and more.
Frankly, I have been wishing, for decades now, for someone to stand on the shoulders of APL and develop it's successor. A rich SYMBOLIC language capable of allowing rich and effortless expression of ideas at an entirely different level of thought and abstraction than for-loops and indenting lines by four spaces.
Anyhow, it's a soapbox. I know. Getting off it. I need to go do some useful work.
[0] http://www.eecg.toronto.edu/~jzhu/csc326/readings/iverson.pd...
From reading, because the glyphs were too hard to deal with in a pre-Unicode world. Too hard to type, to display and to print. J was intended to be ASCII based, column-first and play more with natural language grammar features. But I have wondered, was Ken Iverson happy with J? Did he like it? Regret it? Is there any record of anyone asking him or him discussing or writing about this?
Frankly, I have been wishing, for decades now, for someone to stand on the shoulders of APL and develop its successor. A rich SYMBOLIC language capable of allowing rich and effortless expression of ideas at an entirely different level of thought and abstraction than for-loops and indenting lines by four spaces.
What properties might this have?
A brief touch of APL has left me chiding mainstream languages for not having vector/collection oriented transforms, and for not having a core of carefully thought out compose-able symbols for common operations. But it has also shown me a view of people having to work around APL's limitations. The whole environment feels a little like watching Prof Kernighan demonstrating a Unix pipeline to spellcheck a document from a text file of known words in the 1970s, and being impressed that it happened, but comparing it to a probabalistic context-aware spelling and grammar checker today and thinking "you couldn't build that with a shell pipeline" - but with my dim view, I'm not able to see much room for a notation which is significantly better across (m)any domains without just being "build your own DSL". Even APL implementations don't agree on the exact behaviour of the core APL functions. I believe NARS2000 and Dyalog handle ⊆, ⊃, ↓, ↑ differently. How much room is there for a notation to be bigger and more capable but still agreeable to enough people?
What does "effortless expression of ideas at an entirely different level of thought" mean in concrete terms? What level? What ideas? I'm enticed, but unseeing.
Compilation and distribution through executable files
Dyalog APL can make executable files. The main catch is that you need a license to run them, as well as a license to develop them. There seems to be no way to buy a Dyalog APL license then distribute free (unpriced) executables. They have an object system, and .Net integration, and have been filling in some of the symbol combinations over the years - as a beginner I can't really comment on that except iota underbar is now a builtin function.
Another thing (restriction) was to use available hardware. So some symbols were created in a way which was possible to achieve with teletypes of the days.
It's possible one should start with careful (re-)reading of "A programming language".
And maybe we should think once again about the power of the whiteboard and how we could express that into an executable language.
Hard to say. What does a person who devoted decades at IBM and around the world promoting and speaking about the idea of a symbolic language as a tool for thought think about abandoning that idea entirely? I actually heard him give this presentation in person way back when at an international ACM APL conference (I attended to a few of them).
> What does "effortless expression of ideas at an entirely different level of thought" mean in concrete terms? What level? What ideas? I'm enticed, but unseeing.
I understand. It's hard to communicate something like this. I sometimes equate it to a bird trying to explain to a human being what it feels like to fly. One can get to an approximation, but at some point the only way to see it is to become a bird.
The best explanation I have found requires the reader to have formal musical training. Far more people have this than the opportunity to use APL on a daily basis for a couple of years. Rather than repeat it here, I just offered the explanation to Volt. Here it is:
https://news.ycombinator.com/item?id=20770420
> Dyalog APL can make executable files. The main catch is that you need a license to run them, as well as a license to develop them.
Right. Because APL ended-up being used in the financial industry we have a situation where you have various versions of it offered as paid licenses rather than FOSS. This, right there, pretty much killed the language. There are interesting experiments out there, NARS2000 perhaps being the most accessible. However, this isn't going to bring the language to forefront in any imaginable way.
Put simply:
If you can't develop useful software solutions with APL better and faster than with other languages, well, it is useless.
For example, I can't build a website with APL. I can't build and deploy ML applications. I can't program an embedded system. I can't build a CAD program or a game. Etc.
What is it good for then?
Today? Not much. As much as I love what the language represents it is my opinion that the only reason it still exists is because there might be codebases in corners of financial institutions that rely on it. With time this too will evaporate. At that stage APL becomes a language of academic interest, and that's about it.
> Even APL implementations don't agree on the exact behaviour of the core APL functions.
Well, yeah, that's today. I'll call it "30 years after the start of the end". Back in the 80's, for the most part, you had the IBM APL/APL2 flavor and, at a popular level, STSC's PC implementation. There were others, but, for the most part, the APL core followed Iverson's original APL. IBM added such things as nested arrays and a few other nice improvements. In many ways IBM dropped the ball. I can't blame STSC, they did not have the staying power IBM has. IBM could have evolved, open sourced and popularized APL. They did not.
They are called "idioms" precisely because these expressions can be like words once you internalize them (something that happens naturally to a professional user of the language). It's interesting to explore the library and maybe try a few of the idioms on something like NARS2000.
Finn APL Idiom Library:
1. Why does the article in particular upset you? You talk about how J looks like ASCII vomit, and the article is trying to mitigate that. Perhaps it doesn't go far enough, which I would agree with.
2. What does an APL successor look like? Beyond object-orientation (which I don't see as contributing much).
No, the article does not upset me. In fact, none of this upsets me at all. If it did I would stop my work in other areas and devote all of my efforts towards bringing "NextAPL" to the world for all to enjoy.
APL is, for the most part, and sadly, a dead language, has been for quite some time. Yes, yes, there are corner cases of usage but that's about it. It's more of an academic oddity these days than anything else. Again, sad to say this because I learned it when I was about 19 and used it for about ten years with great success.
Precisely because APL is a dead language virtually everyone who writes about APL doesn't really know APL and "the APL way".
The best I can explain what this means is to compare it to musical notation. This, of course, requires that the reader know and understand musical notation beyond the basics. In other words, if the reader has a few years of traditional musical training they'll get this immediately.
Simple concept: Once you grok musical notation, not only do you read it without effort, you see patterns in it, your brain can "see" the music, you can talk about it, you can write it and you can communicate your thoughts with this notation.
Now imagine someone comes along who doesn't want to put in the effort to learn and use this notation. What comes out of that are things like tablature and other stuff. Let's call that ASCII vomit just to stay with the theme.
Useful?
Sure?
The same?
Not even close. Not by a long-shot. In fact, it is primitive, inelegant and most-definitely not conducive to working "at the speed of thought", something Iverson clearly indicated APL enabled. Something anyone who has used APL for a non-trivial amount of time --defined as typing and reading it like a trained musician does musical notation-- will attest to.
Programming in APL, in this sense, can feel very much like playing the piano using sheet music with traditional musical notation.
Another way I always thought about programming in APL was to envision data structures floating in space in front of your while having powerful symbolic tools to manipulate them. It's almost like those scenes with Tony Stark using an advanced holographic CAD system to design his hardware. No mechanics, all thought.
In terms of an APL successor, well, this is hard to fully define until you dive into a project that would aim to product such a thing. I wrote about this on HN a few years ago. Rather than copy the whole thing, here's a link: