Štar: an iteration construct for Common Lisp | Hacker News Reader

Štar: an iteration construct for Common Lisp

tfeb.org

100 pointsTomte35 comments

A couple of years ago I finally started to understand the Waters series facility [1], which was offered as an alternative to Loop. It's a bit finicky, and the error messages are often mystifying, but I have come to quite like it for some more complicated looping situations. The thing that finally made it accessible to me was a blog post by Joe Marshall, Series tips and tricks [2]

Ironically enough, once my use of it advanced enough, I found myself writing producing forms which are just a restricted form of Loop invocation - Series turns out to compile down to Loop. It's just that I (and others, apparently) find Series expressions more pleasant to deal with than Loop expressions.

[1] https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node347.html#...

[2] http://funcall.blogspot.com/2022/07/series-tips-and-tricks.h...

I shared the urge to avoid loop until I read Pascal Costanza's Highly Opinionated Guide to Lisp[1]

>> Seemingly, the intended way to use the LOOP facility is to just "guess" a way to express an iteration and see if it works. If it doesn't you can either look up the specifics ...

Since then, I do just guess at the syntax and it strangely does what I want most of the time.

It seems that a library like this has a lot to prove because a) it doesn't provide a new capability, b) it adds a project dependency, and c) creates yet another way to do a standard task. If you really don't like the loop macro, you probably don't need much persuading, but I would have liked to have seen more discussion on the these trade-offs.

[1] http://www.p-cos.net/lisp/guide-v1.html

Huh, this is such an unusual design goal in programming languages "just try something and it'll probably work"

Only css is done this way... but not even intentionally

That's not the design goal, it's more a "haha but kind a true" thing.

https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node235.html

The grammar for what's accepted by Loop is well specified, but the results also read very clearly. Once you learn a few of them, you start to understand how the rest would be written and can guess, but the design is not that you would guess.

Haskell works like this: if you can manage to make it compile, it'll work the first time. Might leak gigs of space too, but it'll work.

> Huh, this is such an unusual design goal in programming languages "just try something and it'll probably work"

Perl was famously that way. It's great for the initial write but not so good for maintenance.

Very very handy for people just learning (what's often referred to as "baby perl" in an affectionate rather than derogatory way).

Learning to use perl -properly- is a whole other barrel of monkeys though.

(it does have the advantage that it's very easy to spot code written by a confused perl programmer, whereas I've experienced a number of unfortunate events where python code initially looked fine but turn out to be logically utterly confused; this is not a complaint about python, though it does darkly amuse me)

Funny. I'm pretty sure I failed a technical interview many years ago due to the whiteboard exercise in which I used LOOP pseudocode to solve the problem. They weren't Lispers and assumed the code wouldn't work, but I was sure it did (I couldn't explain why though, at the time, as they made me doubt my solution).

Either way I think they were looking for a recursive solution. Oh well, they still invited me to lunch, though.

Loop works well, is widely used, also available in Emacs Lisp and comes with zero dependencies. For these reasons, I don't see this gaining much traction.

Anecdote: In almost 20 years of CL usage, I've never needed an extensible iteration construct. I also find 'loop' a lot more readable than 'iterate' [1], another fad that has come and gone.

[1] https://iterate.common-lisp.dev

TIL that there is a whole package for “Common Lisp Emulation”[1] in Emacs that adds functions and control structures from CL “to make Emacs Lisp programming significantly more convenient”. Never developed a package for Emacs myself, but I feel like this can be of great help if I ever wanted to.

I know that there are projects like Lem[2] that try to build some kind of editor (Emacs-like, I guess) on top of CL, which made me wonder if we would have a CL-based Emacs today if it would have been standardized by the time and if this would have been a better choice than creating a custom dialect.

[1]: https://www.gnu.org/software/emacs/manual/html_node/cl/ (C-h R "cl" in Emacs)

[2]: https://lem-project.github.io

I'd say that lem is an emacs. Emacs is a genus, not a species. Before GNU Emacs (& XEmacs) there were others. And I'd classify lem as another.

> I'd say that lem is an emacs. Emacs is a genus, not a species.

The question is what makes this species.

For example see https://www.finseth.com/craft/#c10.1

It's the command set (key bindings, extended commands, ...), buffers with modes and minor modes, the user interface for buffers, marks, the "cursor", extensibility, ...

For example in a typical Emacs the "cursor" / current mark is always visible, even in GUI versions. This is very different from other editors, where the current cursor can be non visible.

Basically there are a lot of similarities between the early Emacs editors (EMACS, EINE/ZWEI/ZMACS, Multics Emacs, ... and others).

Does LEM try to be similar to an EMACS editor or is it only emulating one on demand?

GNU Emacs use might also think of Emacs Lisp as an extension language, a GUI version, menubars with menus, self-documentation, ...

Theological questions. The reality is that one "knows an emacs when one sees it." And lem behaves like GNU Emacs in some fundamental respects. Default keybindings, buffer management, macros, etc. Even newer things like lsp support + modes, etc.

It is not clear to me why the marketing for lem is "a Lisp IDE" when it clearly is "an emacs written in Common Lisp"

In what sense is iterate a fad that was gone? I see 1161 systems in Quicklisp (out of 5499 total) that transitively depend on iterate; so about a fifth of the open-source ecosystem.

Transitive dependencies are a poor indicator of actual utilization. How many direct dependencies are there?

For what it's worth, I quit using it since it broke if you ever used the CL function `count` inside an iterate construct. Reportedly it was fixed, but it's still failing to work correctly using the Quicklisp version (just installed CL on and setup QL since it's a new laptop, so it shouldn't be pulling an old version). It was fun for a while, but having to remember "Oh yeah, don't use count" every time I reached for it for something natural was annoying and not worth bothering with.

It has 201 direct dependencies, making it the 9th most directly-depended-on system in Quicklisp, one spot ahead of UIOP.

Methodology (yes, I could've stayed in Lisp, but I started by awking systems.txt...):

    (with-open-file (*standard-output* #p"deps.txt" :direction :output)
      (iter
        (for system in (ql:provided-systems t))
        (for name = (ql::name system))
        (format t "~a ~a~%" (length (ql:who-depends-on name)) name)))

    $ sort -h deps.txt | tail
    200 uiop
    201 iterate
    206 split-sequence
    274 closer-mop
    344 cl-glfw-opengl-core
    350 bordeaux-threads
    387 fiveam
    391 cffi
    408 cl-ppcre
    1006 alexandria

I find iterate much better on the other hand. Especially the part where I don't have to use :do to switch back to CL syntax and I can use the usual nested if else without having to remember the :end/:then stuff.

Nice! Looks like most modern list-comprehension syntaxes.

> iteration and value accumulation are orthogonal problems which should be solved by orthogonal constructs

This is also covered to an extent by "Why Functional Programming Matters" in the discussion of laziness: https://www.cs.kent.ac.uk/people/staff/dat/miranda/whyfp90.p...

For a direct comparison of combining this syntax with "accumulation" into a lazy sequence, see Clojure's `for` macro:

https://clojuredocs.org/clojure.core/for

A few remarks:

* Syntactic sugar for nested loops is welcome!

* I would have preferred a comparison with https://github.com/Shinmera/for/, as it's nearer in concept than loop/iterate.

* Eschewing collection is a mistake in my opinion. Yes, you can do your own with (let ((ret)) (for ... (push it ret)) (nreverse ret)), but it'll be verbose.

As other said, it's a very Scheme-y vision of looping, which has simplicity/purity/explicitness advantages but lacks in pragmatism. Personally, I firmly stay in the iterate team, even with its issues.

I don't lisp, but `in-iterators` is badly named unless there's precedent elsewhere in the lisp community. I initially expected it to implement (Cartesian) `product` rather than what's usually called `chain` (though admittedly the former is less useful in a language where `for*` is directly possible). Note that `product` really needs optimizations depending on whether its inner iterators can be trusted to be repeatable or not.

I also find it a bit fragile for iterator implementations to use 2 functions, rather than a single function returning `value | unique-end-of-iterator-sentinel` (there are at least 3 obvious ways to lay this out such that there are no values an iterator cannot produce).

It looks to be heavily influenced by Racket's for loops and sequences; what this calls `in-iterators` is `in-sequences` in Racket.

Learn iteration here :] https://lispcookbook.github.io/cl-cookbook/iteration.html loop, transducers and friends.

Looks similar to the Racket for macros. I like it.

``` It also has no inherent knowledge about how to iterate over any particular structure: it doesn’t know how to iterate over lists, or ranges of numbers. Rather it knows that iterating has to answer two questions:

    is there more?
    what is the next thing?

In addition it knows how to ask another question:

    is there any information I can use to make asking the first two questions faster?

```

This approach sounds so good. Focus on what is needed to solve the main task. Don't do less, but please don't do more. And... Do Not Assume.

Yes, and that's why Java and C# use pretty much this exact approach (without the acceleration part, to be fair) for their Iterator/IEnumerator that power their for-each loops.

> Rather it knows that iterating has to answer two questions:

> is there more?

> what is the next thing?

> This approach sounds so good.

Does it? I think separating the checking for presence of a next value (hasNext) from retrieving that value (next) isn’t the best idea, and most performant implementations will secretly already do the latter when asked the former, but then have to add an additional check in the “what is the next thing?” method in case there isn’t any.

That’s doing extra work in cases where the program only is interested in whether there are more items, but that rarely happens, doesn’t it?

Yes, good compilers can often combine the two through inlining and the like, but why make the compiler do extra work if you also could have “what is the next thing, if any?” as the sole method? (That’s more or less what IEnumerator.MoveNext does in C# (https://learn.microsoft.com/en-us/dotnet/api/system.collecti...))

> Similarly, Štar doesn’t contain a mass of syntax letting you select only certain values, or allowing you to terminate iteration early: you don’t write

> (loop for x in l while (numberp x) do ...)

> Instead you write

> (for ((x (in-list l)))

> (unless (numberp x) (final))

> ...)

While loop has arcane grammar and can be hard to write correctly, it will be read many more times and so I think is preferable to the latter.

For a Lisp user the s-expression variant would be more useful, because it is easier to write and read. It also helps that it is easier to indent / pretty print, because the usual pretty printer algorithms can be used. For LOOP one would need an indent / pretty printer with either some heuristics and/or knowledge of LOOP grammar.

Personally I use LOOP because it is built in. But generally I prefer something like ITERATE, which is similar powerful but fits better into the host language (-> Common Lisp).

I, too, am lisperati :-)

"I am sick to death of knee-jerk anti-LOOPism and I am beginning to irrationally regard it as a plot to disable me as a programmer by excommunicating my useful tools." [1]

[1] http://ccs.mit.edu/papers/CCSWP150.html

I tried to clean the dirt on top of the Š with my finger..

Me too. Don't name your thing with diacritics most people don't know how to make on a keyboard.

You have no idea how much comments like that make me want to name my pet programming language something like 鹪鹩 and then post it to "Show HN", just to see how many people know from the top of their heads which language has more speakers, English or Chinese (it's English, by 300 million people; but 1.1 billion of Chinese speakers can make those hanzi on their keyboards just fine).

Imagine if we had to write `整数 size = 0;`.

Which compiler was used for the benchmarks?

Reply on news.ycombinator.com