YAML Parser for Dyalog APL | Hacker News Reader

YAML Parser for Dyalog APL

github.com

12 pointstosh9 comments

As a tool for thought, APL is wonderful. I mean that in a true and non-snarky way. Wish I'd invested in the details. Snippet:

c←∊LF,⍪⊆⍵ ⋄ k←0⍴⍨≢c k c⌿⍨←⊂n≤(⊂¯1++⍀m)⌷(m←c=LF)⌿n←+⍀(≠⍀c∊'"''')<(c='#')∧¯1⊖c∊LF⍪WS k[⍸(⊢∨¯1⊖⊢)≠⍀(k=¯1)<c∊'"''']←Q k[⍸(k∊¯1 Q)<(c∊'-?:')∧1⊖c∊LF,' ']←S k[⍸sm∧im←m<(n-i⌷m⌿n←⍳≢n)=n-(i←⊂¯1++⍀m)⌷(m←c=LF)⌿n←+⍀(k=S)∨sm←c=' ']←¯2 k[⍸im<k=S]←0 d←n-(⊂¯1++⍀m)⌷(m←c=LF)⌿n←+⍀0⍪¯1↓k∊¯2 S k[i←⍸(k∊S Q)<(c=':')∧1⊖c∊LF⍪WS]←S d k c m⌿⍨←⊂1+m←(⍳≢k)∊(i-1)(⊣I⍸)⍨⍸¯1⊖(k=S)∨c=LF ⋄ k c(⊣@(⍸m←m∧1⊖m))⍨←S'?' d+←(c=':')<n>(⊂¯1++⍀m)⌷0⍪¯1↓(1⊖m←c=LF)⌿n←+⍀¯1⊖m d k c⌿⍨←⊂k≠¯1 ⋄ p←⍳≢i←⍸(c=LF)⍱k<0 _←((¯1⊖⍪⍀){p[⍵]←(⊂a⍸⍵)⌷a←⍺I⍋⍺}¨⊢){⊂1↓⍵-≢x}⌸d[i]⍪⍨x←(⊢I⍋)∪d[i] k[i]×←S=k[i] I I@{k[i][⍵]=S}⍣≡⍨p ⊣ k←k[i][y I x[y←⍸≠x]⍳x←¯1@{p=⍳≢p}p]@i⊢k ld←d ⋄ d←(⊂+⍀m)⌷0⍪d⌿⍨m←(¯1⊖m)∨m←k=S d k c⌿⍨←⊂(m∧k=¯2)⍱(d>ld)∧~m←(i⌷m⌿n)<(i←⊂¯1++⍀m)⌷(1⊖m←c=LF)⌿n←+⍀k=S d⌿⍨←m←(1↑⍨≢)⍣(~∨⌿m)⊢m←k=S ⋄ k←S M Q I'-:'⍳m⌿c ⋄ k[⍸k≡,Q]←0 ⋄ c⊂⍨←m c[i]←1↓¨c[i←⍸('-?:'∊⍨⊃¨c)∧k∊S M Q] c⌿¨⍨←(∨⍀∧∨⍀U⊖)∘(~⊢∊LF⍪WS⍨)¨c ⋄ d k c⌿⍨←⊂(k=0)⍲0=≢¨c c←(1↓¯1↓⊢)¨@(⍸'"'''∊⍨∊1↑¨c)⊢c q←(k=M)⍀x⍳⍨qq←∪(⊂'')⍪x←c⌿⍨k=Q ⋄ d k q c⌿⍨←⊂k≠Q d k q c⍪⍨←(d+←S M∊⍨⊃k)↑¨d k 0(⊂'') k⊖⍨←1 ⋄ k[⍸d≥1⊖d]←K

Is it possible to learn how to fluently read APL or do people create programs for sport, similar to Brainfuck?

As someone unfamiliar with it, this looks like the product of some sort of charset/unicorn decoding error rather than a cohesive program.

Based on e.g. [this classic](https://youtu.be/a9xAKttWgP4?si=oMQQVbIziq60Cd1F), it seems people are fluent and productive in APL in a way I doubt anyone can become in Brainfuck.

But I would also like to know from APL aficionados whether it is a “write-only” power tool akin to a “Perl for math”, or if it’s also easy to read other people’s APL code once you get used to the most common symbols in APL.

I don't put a lot of stock in the "write-only" accusation. I think it's mostly used by those who don't know APL because, first, it's clever, and second, they can't read the code. If I remember I implemented something in J 10 years ago, I will definitely dig out the code because that's the fastest way by far for me to re-learn how it works.

This project specifically looks to be done in a flat array style similar to Co-dfns[0]. It's not a very common way to use APL. However, I've maintained an array-based compiler [1] for several years, and don't find that reading is a particular difficulty. Debugging is significantly easier than a scalar compiler, because the computation works on arrays drawn from the entire source code, and it's easy to inspect these and figure out what doesn't match expectations. I wrote most of [2] using a more traditional compiler architecture and it's easier to write and extend but feels about the same for reading and small tweaks. See also my review [3] of the denser compiler and Co-dfns.

As for being read by others, short snippets are definitely fine. Taking some from the last week or so in the APL Farm, {⍵÷⍨+/|-/¯9 ¯11+.○?2⍵2⍴0} and {(⍸⍣¯1+\⎕IO,⍺)⊂[⎕IO]⍵} seemed to be easily understood. Forum links at [4]; the APL Orchard is viewable without signup and tends to have a lot of code discussion. There are APL codebases with many programmers, but they tend to be very verbose with long names. Something like the YAML parser here with no comments and single-letter names would be hard to get into. I can recognize, say, that c⌿¨⍨←(∨⍀∧∨⍀U⊖)∘(~⊢∊LF⍪WS⍨)¨c trims leading and trailing whitespace from each string in a few seconds, but in other places there are a lot of magic numbers so I get the "what" but not the "why". Eh, as I look over it things are starting to make sense, could probably get through this in an hour or so. But a lot of APLers don't have experience with the patterns used here.

[0] https://github.com/Co-dfns/Co-dfns

[1] https://github.com/mlochbaum/BQN/blob/master/src/c.bqn

[2] https://github.com/mlochbaum/Singeli/blob/master/singeli.bqn

[3] https://mlochbaum.github.io/BQN/implementation/codfns.html

[4] https://aplwiki.com/wiki/Chat_rooms_and_forums

Yeah, about fifteen minutes for lines 15 to 21. Which includes getting used to this n-(⊂¯1++⍀m)⌷(m←c=LF)⌿n←+⍀ pattern for sum-within-lines that appears everywhere. Pretty sure the k=¯1 checks do nothing (planning ahead for a future feature?) and I think it breaks if there's a quote character in a comment. But, like... line 19 sets k to ¯2 where there are whitespace characters that come before any non-whitespace character on the line other than - ? or : followed by whitespace? I don't know YAML well and I have no idea how this fits in. So it's hard to remember what ¯2 means as I read later code.

You absolutely rock. Thanks for taking a look at my project. It's currently on the back-burner right now, but I do have future plans. The ¯2 is specifically something I'm hoping to get rid of. It has to do with some specifics of handling leading sigils, but it's slightly broken and the ugliness of ¯2 signals to me that it needs to be fixed.

Hoping to hear your thoughts in the future once the project is more in shape!

In the BQN compiler I'd write (⊂¯1++⍀m)⌷(m←c=LF)⌿ as the equivalent of (⊂⌈⍀m×⍳≢m←c=LF)⌷ using the function IT (indices-times but it also includes the ⌈⍀). Although I'm not sure it's inherently faster. Most of the cost is in selection, and there's an algorithm taking a shuffle of a sliding window from the right argument that would work better with the denser indices. And those indices have a smaller type which may be useful. The annoying thing is that this whole set-and-hold operation could be implemented in SIMD directly and none of those complications would apply. But it's pretty hard to expose in a nice way without just providing a builtin.

I'm the author. This is definitely a serious project of mine. I presented about it at the Dyalog '23 User Meeting last October:

    https://tv.dyalog.com/Dyalog23/?v=bb6sIAJjMeM

In a sibling comment, mlochbaum impressively just reads my code without too much effort. However, it is first and foremost written for a domain expert, with all affordances for outsiders discarded in favor of parsimony and direct expression of thought. Even if you know all the APL symbols, you'd need some domain knowledge about parsing and YAML specifically to really understand the why.

Learning to read the APL symbols is super easy. Just give yourself a couple weeks of 30 minutes a day or so. That's definitely enough to get you started solving challenge problems.

However, "fluently reading APL" is like talking about fluently reading math. The focus is on the concepts and communicating those concepts. Fluency is tied to familiarity with the underlying ideas of the problem domain.

Thank you, yes.

Reply on news.ycombinator.com