Klong: a Simple Array Language
t3x.org
t3x.org
If you'll forgive the style, I took a swing at implementing k-means clustering in Klong a few months ago [1]. While I did find a few slow-downs with a modestly sized data set (Fisher's iris data), I think Klong is built on a Forth-like VM using the core of a scheme implementation (all of which the author wrote!). It wasn't slow enough to really bother me, as I was more interested in learning about array languages and algorithms.
I'm a fan of Klong -- it's just fun.
[0]: http://www.t3x.org/klong/book.html
[1]: https://idle.nprescott.com/2019/k-means-clustering-in-klong....
In Q, abstraction is wholly unnecessary. Libraries are unnecessary. I can fit the entire language in my head and all code in the language is built out of the fundamental building blocks of it. There’s no layer after layer of abstraction, just a small base of operators and functions.
One might be tempted to compare it to other small languages, like Scheme. But for Scheme it’s intended that you build higher and higher levels of abstractions. With Q it’s both small and compact, and one rarely if ever feels the need to actually build a library of utility functions or anything really: it renders unnecessary the false prophet of code reuse.
I got interested in Q because some of my friends at other financial institutions were using it for HFT and research on historical tick data. I found it surprising at the time I that their software stacks could be so simple, essentially just a single small executable that entirely fit in the CPU cache that managed both the runtime and their entire database. At the time I thought this was impossible, but now I realize how possible and eminently reasonable this was. Instead of Kafka and Redis and SQL Dbs and Kubernetes and god knows what else, you just have Q. No DevOps bullshit, no containers, no complex shit to manage.
In the past 3 weeks I’ve had to rethink my philosophy of what software needs to be and have concluded that most of us are doing things very very wrong: we’ve created this unnecessary world of complexity that is tantamount to masturbation.
Now let’s talk about the language itself. It may look complex, but it’s actually quite simple. The only real data types are lists, dictionaries, symbols, and numbers. And surprisingly, that’s enough. When programming in it I have no desire for classes or objects or trees or whatever, in fact, just using lists and dictionaries can usually solve the problem in a better way.
Also, while Q is interpreted, it’s so goddamn fast. I’ve done some benchmarks against C and Q is about as fast, sometimes even faster. The optimizations the interpreter does are just insane. Not even joking, a single line of Q can accomplish somewhere between 100 to 1000 lines of what the equivalent C code would look like.
Another plus is that editor tooling is unnecessary, I have no need for autocomplete or inline docs because I can literally remember the entire thing. And since Q is so compact, I can view an entire program on a single screen, no scrolling, no jumping between files.
I’ve also noticed that I can remember the verbatim definition of my code. I can almost perfectly remember my exact code for each Advent of Code solution.
Anyways, good luck on Klong, I’m going to check it out. I don’t like J and wish there was a good open source K/Q like language. I don’t actually like K because the overloads on function arity creates a lot of complexity (Q does away with this), so Klong might be right up my alley!
For anyone who hasn’t tried an array based language, I highly recommend it. Q has expanded my mind more than I thought possible at this point in my career. My last really mind expanded experience in programming was when I picked up Scheme and Haskell in high school. And now after using Q, those languages aren’t that great anyways!
Another question - are there debuggers for these languages? Can you set breakpoints?
How do you interact with the www? Can you get some soap/json data and write a simple web service (even just a json endpoint)?
Or do you mostly see it useful to export data in some tsv/csv format, filter it through k, and write it back out?
Without comoments, it might be a little tough. I've gotten into the habit of commenting each line.
>Another question - are there debuggers for these languages? Can you set breakpoints?
There's an integrated debugger that supports breakpoints. I really haven't used it at all since the REPL works quite well, but I could imagine it being useful in production. Especially since you can debug and fix your program while it's running.
>How do you interact with the www? Can you get some soap/json data and write a simple web service (even just a json endpoint)?
There's included functions for parsing and generating html and json. I haven't used them though.
>Or do you mostly see it useful to export data in some tsv/csv format, filter it through k, and write it back out?
I would imagine that live Q programs work by leveraging the real-time streaming functionality and just operating directly on the stream or in-memory database. For offline analysis the ability to load and store data in binary is probably useful. You can also read and write the entire working image in order to transfer state across machines or time.
If you don't understand something in k or APL after writing it, you probably wrote it wrong to begin with.
A cool example of how much APL lends itself to this is co-dfns, which the author rewrote multiple times:
https://news.ycombinator.com/item?id=13799067
In particular, the desire to make the code as "disposable" as possible has lead to me using strictly combinatory/points-free style programming in the core compiler. It makes it very easy to change things because it's easy to see the whole picture and easier for me to delete any given piece of code. Many "fixes" in my code are single character APL fixes, or maybe two character APL fixes.
This whole discussion got started because of the amount of change that has happened with this code base (hundreds of thousands of lines). You can read more about that in the other historical threads referenced by dang. I've rewritten the compiler at least five times.
What if you need to scale beyond what one machine can handle? Or if you need redundancy and failover? Transactional guarantees? Or need to talk to external systems?
Its easy to build a single-executable no-DB no-container no-devops application in most languages if you don’t care about the finer details of building a high scale resilient system that can survive a machine going down or has to talk to third party external systems.
You can literally run the database for a multimillion dollar company on one such server.
I don't know what transactional guarantees Q has though.
Also, your comment on "noDB" is unfounded. k and q are exclusively distributed with kdb+, which is probably the fastest SQL database on the planet.
Though if you want an open source k, ngn/k was just released under the AGPL a few days ago.
I'd recommend starting with J, though, because there are a lot more resources for it, there's a mobile app for it (that's really nice), and it's the last thing Iverson worked on.
K/Q does scan/reduce from left to right, which is easier for most people to reason about and use, even though it makes "-/1 2 3 4 5" different from "1-2-3-4-5" and thus less "pure"/"consistent" in some sense.
J's trains and forks (and APL's simpler products) are much more confusing for people than K/Q's juxtaposition and projection.
Can't remember or find Iverson's comments now. He might have made more, or I might be misremembering/misattributing. But I agree with these and think K is a better start than J (though not sure between APL and K).
This is the arthurese micro APL implementation, which inspired the J implementation - although IIRC the design for J was already well on its way at the time (and indeed, Iverson did not consider J a new language, but rather an improved APL dialect, so you could say the design goes back to 1956...)
It likely also formed a draft for A/A+ which Arthur did at Morgan Stanley (open sourced at aplusdev.org, but unmaintained and rotting theese days) - which themselves led to the creation of K.
Also the reference and blog posts are useful: https://code.kx.com/v2/
It’s not as easy to get information as other languages, but the mailing list is super helpful and all the questions I’ve asked have been replied to by paid Kx (the people that make Q) employees:
{&/x!:\2+!_x^1%2}
This is function that checks if the value given as first parameter x is a prime by running through the numbers 2 to square root of x and applying the function x modulus y, then it looks at the minimum of the resulting list and see if it is 1 or 0. If it is one then x is a prime.
Written in a language with c-style syntax it would be something like:
x -> min((2 .. sqrt(x)).map(y -> x % y)) == 1
Why is Klong simple?
Is it easier to implement, parse and interpretate?
Or is the syntax somehow simpler once you get used to it?
The former maybe; but it doesn't seem to be a part of the language's pitch.
The latter is surely in the eyes of the beholder. I find it counter-intuitive to have so many single character operators & ! : _ with uncommon meaning and precedence rules.
The only reason you find the c-style version simpler is because that's what you are used to. But while you have been able to interpret the Klong expression with no experience at all, ask yourself if someone familiar with array languages (and only array languages) would be able to understand the c-style version in a few minutes without having to read an extensive tutorial.
I do not agree simplicity is in the eyes of the beholder. If a language can be totally specified by a few paragraphs and a table, it is simpler than a language that needs a long specification. However, this does not mean the simpler language is simpler to use. That, indeed, is a subjective matter.
Well, yes. But we're taught infix notation early (x+y), and function application quite early too (y=f(x) - and in combiation (also note short-hand eliding multiplication): f(x) = 2x + 1).
I still find array languages to be too dense for me - but I do understand that they are simple from a certain point of view.
f(x)=2x+1
In Q is
f:{1+2*x}
You could call it with: f 5 or f[5]. It’s really not any different.
Otoh, I'm still considering working through advent of code - such domains should be well suited, I guess.
For advent of code, it really shines though. Read in some big chunk of data, do a bunch of matrix and list operations to get the answer.
To be clear, a little language pride is always warranted :-)
Other than that, I find it a bit funny that the factorial function is singled out as an example of why naming (short) functions are redundant - seeing as how it's one of the first "complex" functions/operands thaught/named in mathematics, and heavily used to shorten notation for for example combinatorics (that is "n!").
But I think this just reflects on the (for me) somewhat alien culture of array languages - its not wrong, but it is different.
,/:~
Klong is simple, because its syntax is very regular. In K, many symbols are overloaded, so their meaning can only be deduced when knowing the types of their operands. In Klong, the syntax is unambiguous. See http://t3x.org/klong/ambiguity.html
Regarding precedence rules: there are none. Everything evaluates from the right to the left (as in K or APL).
And .. well, if you want C or Java, use C or Java.
Quickly, which is higher precedence, "&" or ">" ? You may have memorized them, but many people get it wrong often enough. APL has a very simple and consistent rule: There is no precedence in an expression except paranthese, and you go right to left.
And it may seem to not help with onboarding if you skim, but if you take the time it clicks quickly for most people, especially if you consider that "right to left" is actually "left of right", e.g. (note, a|b is the maximum of a and b, not the bitwise or)
1 + 2 * 3 + 4 | 5
(1 + ...) applied to ((2 * ...) applied to ((3 + ...) applied to (4 | 5)))
def f(x): return 1+x
def g(x): return 2*x
def h(x): return 3+x
f(g(h(max(4,5)))
f g h 4|5
It is different - but it is simple and consistent without any arbitrary precedence and associativity rules.
Also, it's not a new idea; APL predates every other programming language still in use except, perhaps, Fortran, Lisp and COBOL -- of which only Fortran uses operator precedence and associativity in the modern C/Algol sense. Lisp is also consistent (though with a different route than APL took). And I'm afraid to describe COBOL in fear that cthulu will rise.
I have been playing (quite a lot; to the extend my wife got me a t-shirt with some k on it for my birthday :) with Shakti(k7) for a few months now and been working through the Dyalog tutorials.
Keep up (that part) of the good work!
Edit: bought the book and started reading
Also the various libraries and functionalities that plug into Numpy is countless and extremely powerful.
There's also https://futhark-lang.org/ which might be an improvement in writing array processing kernels.
What does that mean?
I do find some J choices questionable - e.g. '}' and '{' are used as individual operators, rather than a parenthetical pair - which confuses editors without a special J mode, and humans who switch between languages often - but if you use it enough, it's easy to get used to; Also, K keeps () {} and [] as parenthetical pairs, like they are in C/Java/Python etc
Note: This text used to be interactive, but is no longer, thanks to the General Data Protection Regulation (GDPR), which makes it virtually impossible for small, non-commercal sites to "process user data" (like your input) without contacting a lawyer first.
For what it's worth, this URL is the first result on search engines.
Mind you, the attempt to be technology-independent rather than clearly mandate solutions has been a huge problem of the tracking law.
Theoretically, you could enter some personal in a free-text field and it would be stored in the log files. I am not saying that people actually would do this, but they could. Now they cannot.
But, in general, if you want to be 100% safe; just have a static site that collects nothing and logs nothing. No need to nag about it; it makes the internet a better place anyway imho.
In my experience, if you are not frivolously collecting user data and make sure that the personal data you do collect is something you can explain in the line of your business (even if that data is very secret data, but that usually has extra laws/rules on top). And, finally, that you make sure that data is properly stored (encrypted on properly updated servers), you have no issues.
In this case, IF the owner would get an email from the governing body of the country (I think Germany in his case), he would very easily be able to explain the reason for the input and the body can see that he is not asking nor frivolously collecting user identifiable data. If people choose to enter it, for some reason, en masse, in his site, then they would ask him to throw it away when it comes (so in his case, don't log it, but why was it being logged anyway?) in and that's it. No fines, no nothing.
What is uncalled for, IMO, is the name calling in the posting further up the thread.
So I may have such a strong opinion on the GDPR because I live in Germany.
Those C&Ds are pretty insane: I heard about them before, what are they for? In NL we had, for a little bit, lawyers who searched out illegal photograph usage and tried those letters. But according to Dutch law, I can remove the picture (we had a large image site) and that's it. So they made $0 of us, not did the lawyers. Is it something similar?
Eg many EU countries also have laws against hate speech, but I saw nobody take down their hobby hacks because of those.
If you can, you should. There are too many totally bogus laws!