J for C Programmers | Hacker News Reader

J for C Programmers

jsoftware.com

102 pointsmcguire63 comments

Can someone who knows all those one letter languages well succinctly describe the differences between J, K, Q and APL?

What I think I've got so far (this is all hearsay, and none should be taken as fact):

APL: The original. Non-ASCII. APLers hate the successors, because "notation is tool for thought" and ASCII is a poor representation. Main implementations GNU APL (slowish), Dyalog (fast and expensive, free version available, good documentation)

K: ASCII, used in finance (among other sibling languages, dependent on the concrete company and concrete group within a company), comes with kdb+ (columnar data store, integrated with K), very fast (main interpreter loop in processor cache), one real implementation (very very expensive, limited free version available), several cvariations on the language, some of them open source, none comparable

Q: syntax sugar upon K, but otherwise exactly the same

J: another evolutionary strang of APL, impeccable pedigree (Iverson), free software, experiments more with language features, is more complete when it comes to the "linguistic" style (adverbs etc.), database available (Jd, but this database is commercial), slower than K, more extensions like for GUI programming

On K - there are now two "real" implementations, one by Kx Systems (i.e. the version under the hood of kdb+) and one by Shakti (new company from Arthur Whitney, original author of k). The Kx one is k4, Shakti is k9. Having said that, k9 is still quite a "work in progress", don't think it's considered production ready yet.

May also be worth nothing that k4 is essentially "undocumented" and Kx encourage to use q instead of writing any code in k directly. Most functionality can be pieced together using docs from other versions of k (I think k2 manual is floating around online) & the docs from q.

I thought Shakti started at k7, and k5 and k6 have also a Kx implementation. BTW, https://ngn.bitbucket.io/k.html is a great overview of K implementations and which version they are implementing.

edit: k5, k6, k8 were apparently never released (https://aplwiki.com/wiki/K)

Yep, for a while Shakti was k7, but then Arthur started over & it became k9. Not sure what happened with k8

k5 was also known as "kOS" and was to be a version of k as a full OS that could run directly on hardware - I believe that got abandoned due to the pain of writing drivers

http://archive.vector.org.uk/art10501320 - article about kOS before it was abandoned

How does q relate to K? I dabbled a little but the distinction was never clear to me.

Q is K syntactic sugar to make the system easier and more "traditional".

q is k with all of the monadic verbs being replaced by English words: `+` is `flip`, `=` is `group`, `-` is `neg`, and so on. It also has q-sql, a SQL like sublanguage for conveniently working with tables. k has the verbs that implement it, but they're not really as convenient.

The main difference is the introduction of qsql (SQL-ish statements can be treated as expressions) and some utility functions that are defined in K. QSQL is mostly for business analyst folks, and is just syntax sugar over the K-native `?` and `!` functions (which is more powerful than qsql).

Some additions:

The major difference separating j from k is that the former is optimized for multidimensional arrays (_i.e_ math), while the latter is optimized for vector processing (_i.e_ financial data) and uses lists of vectors when working with higher dimensions.

J is also fast, but not for the same things as k. It's interpreter is also designed to fit in the cache.

Dyalog APL has been experimenting with bytecode compilation, although I don't know whether it's implemented in the current version. It also has a gpgpu extension with restrictions on the language (Co-dfns). Unlike j, APL does not implement operations such as matrix multiplication by calling to BLAS, so if you're doing matrices on the CPU, use J. As an aside, j also shines for graphical output with its opengl bindings and its 2d/3d Matrix viewer.

BTW Jd is indeed commercial, but is free to use in non-commercial applications.

Personally, I think that k is much more popular on hn because they're aggressively promoting it. J is better if you're not doing financial work.

> j's interpreter is also designed to fit in the cache

  $ du -sh /usr/lib/j9/bin/libj.so
  3.2M    /usr/lib/j9/bin/libj.so

Might be a tight fit...

Kdb k is only 771kb, and shakti k is 120kb.

J is a general-purpose language. It is oriented toward math and data handling far more than K.

Speed relative to K, for example, is largely dependent upon what operators are used, with what size and shape of data. Certain operations in J are highly optimized, comparable to C (or exceeding) what you'd find in matrix math libraries.

J came along particularly to fix the ASCII problem, but it took some design decisions for things that were bugging Ken Iverson from years of APL work, that var[4] style indexing is really a function application, that all functions should have configurable “rank”, that tacit programming with trains of functions should be a focus, among other things.

Many of those choices have been added to Dyalog APL since then, SQUAD indexing function, trains, rank adjustment operator.

Dyalog APL has traditional APL functions and keywords which looks a lot more procedural than the golfed symbols mostly shown - for loops, if statements, etc. which GNU APL has not, IIRC; and has an object system :Class and a bridge to the .Net framework and C# interop both ways, GUI support possible through that (Windows Forms I think).

> APLers hate the successors, because "notation is tool for thought" and ASCII is a poor representation.

I'm perplexed by this one. I'll concede that the APL symbols are prettier but why does it magically stop being a notation just because you use combinations of ASCII symbols. Especially when many APL symbols are composed of units themselves.

⍲ vs *: One is 'notation' because it has the two units stacked on top of each other and the other is not because they are stacked horizontally?

I've wondered that myself, because the zoo of symbols doesn't speak to me, like iota, but maybe the mathematical use has something to do with the APL use.

So I simply assume there is something deeper going on than "it looks ugly". Maybe someone here is in that camp and can tell us. At least the argument was made on HN in some earlier APL or J thread.

Are you really saying that non-ASCII is notation and ASCII is not? When I read your first comment, I understood it to mean that better notation is better than worse notation.

I'm not saying that. Some APL fans say that ASCIIfication of APL is a bastardization, because notation matters. (search through earlier HN threads)

Notation is obviously a shorthand for "sensible notation" or even "better notation" there, because obviously just about anything can be notation. Including laying out pebbles on a beach.

But further than that, I don't know.

The usual argument is that "one should APL like one reads music", meaning that with experience you immediately recognize specific patterns which help you understand what is going on. As for why ASCII can't do that, only the critics know their true reason, which I suspect amounts to "it's different from what I'm used to". I've done both, and I found J notation no more obscure. Critics should also note that (Dyalog) APL departed from APL2 to copy many J patterns, so it appears difficult to argue that J was such a mistake to begin with.

Some people hold prejudice against parentheses and prefix notation, for others it's ASCII...

ASCII is a much smaller space of possible operators, so an ASCII-based language is always going to end up significantly more verbose than one that allows non-ASCII symbols. I don't use APL but I definitely appreciate being able to write ≤ or ⊆ or ∀ if that's what I mean, rather than having to expand those out in ASCII.

≤ "expands" in J to <: Definitely not more verbose.

J uses . and : as modifiers to a base operator to create a pretty big set of primitives. See https://code.jsoftware.com/wiki/NuVoc

> ≤ "expands" in J to <: Definitely not more verbose.

I mean, in a fixed-width font (as people tend to use for programming) it's literally twice as long. That adds up.

Actually, J is often less verbose than the equivalent APL, hence its use for code golfing.

This is less true since APL has begun integrating J semantics and operators, but still..

I should have used the full sentence. Why does it stop being "notation as a tool of thought" See comments like this one https://news.ycombinator.com/item?id=20732924

There is something I call "the whiteboard test". I work in research, and I spend a good part of my time working with formulas in whiteboards and discussing about them with my colleagues. I am constantly confronted with the irregularities and limitations of standard mathematical notation and see APL as a huge improvement, I would really like to use it more, and I have even taught some basics to colleagues so we can use +/ for sum and ×/ for prod, or ⌈ and ⌊ for max and min, for example.

Iverson solved some remaining inconsistencies in J, and it includes some elegant concepts as tacit programming (+/ % # for the arithmetic average is beautiful) that were not in the original APL, but I do not see myself writing J in the whiteboard or a notebook.

But ASCII may not be the main problem. Curiously, I find that K performs much better in my whiteboard test than J, perhaps because the set of symbols is much smaller. And if you think it may be a matter of getting used to it, I have worked with APL far less than with J and still find the APL symbols more appealing.

There are many things I like about J, and I do not have a very rational explanation to give you, but I agree with those that say that APL is a better tool of thought.

At least one person [1] has played with using APL symbols in J, at least at the display level. I gave it a try, but was surprised to find that I preferred the ASCII version. Perhaps if J had its own custom symbol set (extended from APL?) it might have been better -- not all of the APL symbols have exactly the same meaning in J, and I remember finding some of the differences to be a bit jarring. For example, conjunctions like '&>' and '&.>' are heavily used and deserve their own symbols -- just putting a little-circle before a '>' feels like a hack. (No disrespect intended to the author, though, this was a great experiment.)

[1] https://wjmn.github.io/posts/j-can-look-like-apl/

If a symbol is made of multiple side-by-side ASCII characters, are there any problems telling where one symbol ends and the next starts?

In ordinary text, we separate words with spaces, and add punctuation at the end of sentences to further make the structure apparent. (Well, now we do...a lot of pre-modern writing would justwritethewordssmushedtogetherlikethis [1]). In most of the sample programs in APL successors I've seen there usually didn't seem to be space between symbols.

[1] https://www.bbc.com/culture/article/20150902-the-mysterious-...

Yes, at least for me, trying to learn it. For example, / in many examples is an "adverb", modifying the previous operator, although xx/ is frequently introduced as a single thing. (Are all built-in verbs one or two characters?)

'xx/' is a single thing -- in the sense that 'xx' is a thing, and '/' is a thing that modifies other things. By applying '/' to 'xx', you get a new, standalone thing.

Re: one-or-two characters, there are a few exceptions, like the folds ( F.. F.: etc.) which are new to J9. Also some older ones like {:: (Map) and &.: (Under) where `x u&.:v y` means `v^:_1 (v x) u v y`. Some of these aren't verbs but conjunctions, adverbs, etc. For the most part they stick to one or two. There are parsing rules that dictate what the second and third characters can be (that's why you see so many colons and periods), and these rules are actually discussed in the JforC book.

Reading J can be tough, as a fellow learner I feel your pain. :) The one that usually gets me (in my own code, even!) is

    5 myverb \ n

...which runs the verb over 5-element slices of 'n' at a time, and joins up the results. The '5' is too far away from the '\', and it can be hard to see the connection right away. I understand why it's so (it's really running the modified verb 'myverb\' with arguments 5 and n) but I still find it hard to read.

When writing my own code, I often leave extra spaces between the multi-character words, and anywhere else that helps legibility. J might be a terse language, but there's no point being crazy about it (unless you're golfing).

No idea about APL languages but it sounds similar to asking whether the code is uniquely decodable:

https://en.m.wikipedia.org/wiki/Variable-length_code#Uniquel...

> J: ... slower than K

This is debatable. Given the clause in Kdb+ on demand license [0], it is healthy to give all the published performance figures the benefit of doubt.

[0] "1.3 Kdb+ On Demand Software Performance. End User shall not distribute or otherwise make available to any third party any report regarding the performance of the Kdb+ On Demand Software, Kdb+ On Demand Software benchmarks or any information from such a report unless End User receives the express, prior written consent of Kx to disseminate such report or information." -- https://ondemand.kx.com

https://i.imgur.com/sJJWBtq.png

i'm a C programmer, not for any great love of the syntax, but because i cross-compile for embedded hardware with limited resources and there aren't a lot of options.

there are also other use cases for C. since it is close to the metal, it is a good fit for security-sensitive, heavily-audited codebases, because there are fewer layers of abstraction to audit and a smaller attack surface. manual memory management makes it a good fit for performance-sensitive applications, real-time encoding, signal processing, etc., because a garbage collector isn't competing for CPU time.

there's a lot of overlap between all these use-cases, though, and most of it comes down to the fact that C:

* is compiled,

* has manual memory management,

* is a relatively small language with a relatively small standard library,

* (and of course, we can't ignore the network effect) is ubiquitous.

so, when i see a document to introduce me to a new language, and it claims to be written for C developers, the first thing i am going to look for is some information about the language so that i can judge whether it is even remotely suitable for the contexts in which i would normally write in C.

i read "foreword", "introduction", "culture shock", and "running a j program", looking to answer some questions like: is it compiled or interpreted? if interpreted, how complex is the interpreter? if compiled, does the compiler use LLVM or some other intermediate AST? is it garbage collected? can the garbage collector be turned off? how does performance compare?

the only passage that tried to answer "why should i bother?" is a couple of sentences in the foreword:

> To begin with, for the productivity. J programs are usually a fifth to a tenth as long as corresponding C programs, and along with that economy of expression comes coding speed. Next, for the programming environment: J is an interpreted language, so your programs will never crash, you can modify code while it's running, you don't have to deal with makefiles and linking, and you can test your code simply by entering it at the keyboard and seeing what it does.

all of that is true of python too, but that doesn't mean python can do what i do with C.

I think you are misunderstanding the intention of the book.

When you see a book titled "Japanese for English speakers", what it means is that it builds on your knowledge of English to teach you Japanese, not that its goal is that you replace your usage of English with Japanese. Certainly, the book will not include a justification of how you can use Japanese to speak in London and New York as you can do with English, and probably it won't even bother explaining you why learning Japanese may be a good idea. And, yes, you probably could learn Chinese instead, but that is not the point.

> it is a good fit for security-sensitive, heavily-audited codebases, because there are fewer layers of abstraction to audit and a smaller attack surface.

C is a language that has a reason for existing. However C exposes programs to categories of errors that are not even possible to represent in other languages. In my opinion security code should be as error proof as possible, and C is definitely not a good language to accomplish that goal in. (I used to be a professional C programmer)

security code should be as _provably_ error-free as possible. C's strength in this regard is not due to any syntactical strengths, but due to how few layers exist between the code and the hardware.

it's not the only option, maybe there are even some better options these days from a purely technical standpoint, but C is used in heavily-audited security-sensitive applications despite its syntactical/design shortcomings because it keeps you close to the metal.

> C's strength in this regard is not due to any syntactical strengths, but due to how few layers exist between the code and the hardware.

Any language compiling into native code has this advantage.

You perhaps meant that in C the programmer actually codes closely to how it's going to look like in native code. That could be an advantage; but for that C brings disadvantages, which come from one actually having to code like this - there is no other way.

The question what's more important - language constructs which are closer to actual hardware or language abstractions which prevent certain bugs from appearing - isn't that clear.

The whole book is pretty good. After reading and applying the earned knowledge one wishes the book would be longer, with more material covered, but there are other books on J, essays and phrases on jsoftware.com, articles and forums - especially forums.

This book is quite suitable as a first book on the subject. It's a good reference, too, for basic mechanics; for style in organizing longer thoughts one would go besides the book.

There easily could be a few "aha" moments while reading. One could be in the discussion on verb ranks - then the words at the beginning, about the general commanding the army as a whole, becomes clearer. Another is a classification of loops - an unusual viewpoint for traditional C approach.

I read it putting all of myself into it (even caught a couple of typos/bugs). I also read Iverson's.

These languages (APL, J, K) promote a programming paradigm that sits somewhere in between the continuous spectrum between imperative and functional paradigms.

There is lot of good to absorb from J, and not little of it has fortunately permeated into the famous Python's numpy library. Many people have been inadvertently using APL/J deepest concepts thanks to it.

I do think that, even if they are beautiful, they are still generations away from a community Zeitgeist that would render as widely spread as, for example, Haskell.

In the section on "Culture Shock": https://www.jsoftware.com/help/jforc/culture_shock.htm

  The triple quotes are ugly.
    Quit bellyaching...

I love it. So many arguments in PLT are really just about not ever wanting to leave ones comfort zone. Sometimes you just have to suck it up and do a thing.

I would like to read such a culture shock section for every language I don't know yet!

I like that it doesn't evangelize and it doesn't disparage C, it just explains the logic behind the differences.

I prefered the sequel K for D programmers.

Funny though anything that competes with D better be damn good. The more I study and write small projects with D the more I love about D!

Same!

The more I think about what we do in Software the more I value readability. I think one of the reasons I love D is that it clearly treats readability as a design requirement.

I just wish there were more jobs out there using Dlang...

What does D offer over OCaml or any other ML-family language? I never really got what its selling point was at the time - "compile-time reflection" sounds like the worst of both worlds (but I guess it means something akin to typeclass derivation?), and the idea that it could kinda-sorta work without GC was never really explained clearly enough.

D shouldn't really be compared to ML. D is really just a reimagined C++, free of a lot of C++'s absurdity and incoherence

But I'm not looking for "a reimagined C++". I'm looking for "a programming language". What is the question for which "a reimagined C++" is the answer? (Maybe a language for use in environments where GC is not acceptable, but D fails to make a convincing case that it is such a language).

D lets you do away with the GC and is working on a borrow checker type system like in Rust.

> D lets you do away with the GC

But you can't use the standard library if so, right? Or rather you can use some parts of the standard library but not others, and some third-party libraries but not others? It was never clearly explained what does and doesn't require GC.

> and is working on a borrow checker type system like in Rust.

But it only started work on that after Rust became established. So why wouldn't I just use Rust, which has a borrow checker I can use today?

It wears off.

I used d for quite a while, and I still think it's much better than c++. However, it's ultimately just more of the same: an incoherent, tangled mess. Slightly more coherent, slightly less tangled, sure. It's possible to iterate on garbage, but you still wouldn't want to serve it at a restaurant. Worse, it has politics. Prominent d users regularly talk about forking the language/compiler; not because of any specific language pain points, but because of pathological problems with the way the language is developed.

The main compiled/native language to watch imo is cone[1], though most of its ideas are not yet implemented. Zig is also decent, and closer to production-ready.

1. http://cone.jondgoodwin.com/

Neat, have you looked at V yet?

https://vlang.io/

V is a joke. I don't understand why anyone takes it seriously.

It's current capabilities are probably why. I don't see it as ready yet, but its interesting how much it does in such a short span of time.

Damn it, I thought this wasn't a joke!

I did enjoy fumbling around in ike [1], a javascript-based K playground :-)

[1] http://johnearnest.github.io/ok/ike/ike.html

I still feel bamboozled by the mockbuster "J++ for C++ Programmers".

J++ was actually a real thing. I wouldn’t be surprised if that book had been written.

If these books have APL adjustable rank ⍤ used on the `for` function, we can make many more combinations:

    "J for⍤0 0⊢ C Programmers" ≢ J for C programmers.

    "J for⍤1 0⊢ C Programmers" ≢ J++ for C programmers.

    "J for⍤0 1⊢ C Programmers" ≢ J for C++ programmers.

    "J for⍤1 2⊢ C Programmers" ≢ J++ for C# programmers.

    "J for⍤2 2⊢ C Programmers" ≢ J# for C# programmers.

etc. Then with `commute` ⍨ we can swap the argument order as well:

    "J for⍨⍤1 0⊢ C Programmers" ≢ C++ for J programmers.

etc.

(Assuming C has a single implicit plus, C++ is a 1D array of plusses, C# has a 2D matrix of plusses)

APL-likes have been a fascination of mine for some years, but sadly at a distance. For those of you working in these languages, what do you use them /for/? What industries and/or kinds of applications do you write? Why are these languages used for them vs others?

I'm looking for some inspiration. (I think) I'd love to use one "for real"/"in anger" for something but are they too niche to use as CV/Resume enhancement?

I'm afraid this isn't a particularly good answer. Anyway I use Matlab, which I think shares some ideas with the APL-like languages. (Coders who already know they hate Matlab can skip to the paragraph labelled "Downside" below.)

What's good about Matlab? Maybe the best comparison I've seen in a normal language would be LINQ in C#. If you've been writing (or reading) code that makes heavy use of LINQ, imagine re-implementing all that stuff using explicit loops with dummy indices and all the rest of that line noise. Yuck. Hard to write and worst of all hard to read.

Matlab takes that further than LINQ does ... in certain directions. Basically stuff with arrays.

I don't know whether the APL-likes take the same idea even further than Matlab, or just use short keywords.

Downside: when your problem doesn't break down into array patterns that Matlab is good at, then Matlab stops being fun. At all. For stuff like that, where you really do need to supervise every line of C-ish code personally, something like Ruby or C# is way more fun. (Except if you're doing say EE or optical physics. Then Matlab's libraries are still a huge time saver.)

I took some time and used APL to write a little query DSL for answering questions about a legacy C++ codebase. I had a collection of little tools that linked against clang and dumped out XML files containing various cuts of information of the codebase. I then borrowed some techniques form Aaron Hsu to do structural queries that joined these tables together in ways that gave me useful information about the codebase.

It was alright. I don't think I was able to invest enough in APL to make it really stick but have the impression that if I did it could be very useful.

Fwiw, Aaron Hsu is really the guy to follow for all things APL. He's also super nice and helpful and a model for anyone who wishes to represent a technology.

I use J for general signal processing/machine learning development. It allows for clear expression in terms of arrays. It's also decent infrastructure for talking to C libraries, and the database is quite powerful (it's fast and you can apply verbs directly to column groups). Downside for anything like this is lack of libraries.

There's utility in knowing it in the same sense as knowing Lisp; makes you better at other languages. There's also social capital in that people who use and work on the APL family are a sort of "mafia" working on very interesting and lucrative problems (generally in quant finance).

I've been teaching myself J and dabbled in a few other APL-family languages before that (Klong and Dyalog's APL mostly). I don't have any specific goal in mind and have been doing it for fun more than anything -- I recently tried writing a minimal ray tracer[0] for example.

I've been surprised at how much I liked writing it. J can be derided for being a "puzzle language"[1], which isn't entirely unfair, but I've found the paradigm of array programming carries over into more widely used languages and libraries ("Iverson ghost"[2]).

I wouldn't bother learning to put it on a resume, I would try it out for fun.

[0]: https://idle.nprescott.com/2020/ray-tracing-in-j.html

[1]: https://prog21.dadgum.com/219.html

[2]: https://analyzethedatanotthedrivel.org/2018/03/31/numpy-anot...

> For those of you working in these languages, what do you use them /for/?

Sometimes, when the logic of a program is clear, J could be like a calculator for quickly getting a result. Actually, with J you're pretty soon at a point where you're refining the specification, not actually solving the problem (because solving is easy). There is a saying that an hour-long task APLer solves in 5 minutes and then spends the remaining 55 minutes refining the solution - and the problem - so they'd be clearer.

At some point J feels easier than other tools and it does feel like a "tool for thought" - a shorter way between what you think and what the program does. There can be even shorter ways, I don't know.

K is heavily used in finance (especially trading) since it's the language of KDB+ (a super fast time-series DB).

One thing to be careful of; J has evolved quite a bit between j805 and j902[1]. While JforC is usually fairly simple examples, I don't think anyone has done the work to bring it up to date (in fact it's probably j5 vintage). Might also be true of the dictionary and some of the phrases. Nuvoc is the bleeding edge language reference. [2] Great book though; it's how I learned, more or less, along with the "labs."

[1] https://code.jsoftware.com/wiki/System/ReleaseNotes

[2] https://code.jsoftware.com/wiki/NuVoc

Reply on news.ycombinator.com