Zuo: A Tiny Racket for Scripting | Hacker News Reader

Zuo: A Tiny Racket for Scripting

github.com

106 pointshomarp31 comments

There is more information about the reason for this here^1 Below is an excerpt

> As in Non-recursive Make Considered Harmful: Build Systems at Scale, we can build a better make. This commit mostly imitates Shake as described there, but in a Racket style and called "Zuo" (based on the Chinese word for "make"). The Zuo implementation started at https://github.com/mflatt/zuo, but here it's in the intended long-term home, racket/src/zuo.

> The goal is for Racket to be be just as easy for end-users to build as now after converting scripts from make and /bin/sh to zuo. So, make should work in Racket's top-level directory, and configure plus make plus make install should work in a source directory or distribution, all with no new pre-installed packages required. The new makefiles will be small, however, just ensuring that the zuo executable is built and then handing off to Zuo scripts.

1. https://github.com/racket/racket/pull/4179

I've quickly played with it, and from what I've seen it's almost perfect for scripting, at least due to the following reasons:

* it can be linked statically, thus it should run on any Linux with a simple copy; (not by default, use `LDFLAGS=-static`;)

* it can embed in the binary the core library (and other libraries of choice), thus there is no need for other files to be installed; (this is not by default, but by using the `image.zuo` directly one can get this;)

* it has minimal startup latency, almost 2x lower than Python2.7, and many times less than Python3.10; (this is without the extra libraries embedded, with the extra libraries embedded it's still faster than Python2.7, but not by that a large of a margin;)

I haven't yet checked the available API, but it does seem to be quite enough for simple system related scripting.

For those interested in another Scheme implementation, that compiles to native code, with a fair amount of community libraries, I recommend Chicken Scheme.

I'm super interested - I'm always on the lookout for a replacement for Ruby as my primary scripting lang. LISPs seem super cool for scripting because of the exploratory nature of the REPL - writing a script, at least for me, is a process where I have to run it dozens of times, tweaking it every time, and then finally running it regularly without monitoring, and the REPL would save so much time in the first part of this process.

But while I'm interested, I can't find any docs for what's included. For scripting, I think I would at least need the following in the standard library, no package management required:

1. A good API for filesystem ops, such as reading and writing files. Ex: reading the lines of a file into a list, writing a list to a file as lines etc.

2. Making network requests easily, with persistent connections if you need it. (see Net::HTTP in Ruby, which has become extremely usable for all but the most advanced use cases)

3. Parsing JSON at the very least, with YAML and CSV being good to haves.

4. An easy way to run shell commands for stuff that's just better handled there. (like `` in Ruby)

In addition, if the stdout natively supports colouring text that would be super sweet, but that's not a hard requirement.

I can see 1) to some extent in the examples, but without a docs site I'm not sure about the rest. Does anybody here know for sure? Otherwise, is there some other lisp that would meet these requirements?

Does it have to be a Lisp? I was searching for a language to replace Python for quick scripting, and eventually settled for Raku. If you use RakudoStar distribution, you'll get all the functionality you list (and if you use bare rakudo, you can install it with zef).

Why: Python can be verbose, and when I want to write a one-off script to rename some thumbnails in a directory somewhere, that verbosity doesn't help. The strong typing, while good for correctness, also doesn't help much in this use case.

What I tried: V, Nim, Clojure, Scala 2, Scala 3, GNU Smalltalk, Rust, Zig, Clojure, Racket, Emacs Lisp, Raku, Ruby, OCaml, and LiveScript.

Why Raku[1]: expressive power is insane. My test case of writing a script to merge requirement versions coming from two differently formatted files needed ~30 lines of code on average, while Raku needed just 10. The next best was LiveScript with 23 lines, then Elisp with 25 (counting imports; I have all the libraries already imported in an Emacs session, so I could go down to 19 lines, but thought that would be cheating).

The language being "write-only" is not important to my use case, but I also don't feel like Raku is write-only. Yes, reading something like this:

    my @toml = "script.toml".IO.lines.grep(none "" | /^'#'/).map:{S/'='.*$//.trim};

can take some getting used to, but it's not really much worse than BASH.

[1] https://docs.raku.org/language/faq#Why_should_I_learn_Raku?_...?

Thanks for the rec. It doesn't have to be a lisp, but if I'm switching from Ruby I wanted to get something much more major in return, and the REPL features fit the bill. The reason is that when scripting, I often need to debug and explore the problem while the script is running, and most languages don't make that easy. Ruby's debugger (and formerly, pry) is pretty useful, but a full fat REPL would make the experience far more joyful.

As for that line, maybe it's my Ruby heritage, but I didn't find it particularly unreadable! Chaining messages is pretty tight and very useful for one liners.

I don't think you'll find a Lisp that would be anywhere near things like IPython (which I'd consider an example of "far REPL", with context-sensitive completions, inline documentation, rich history with searching and filtering, magic commands (eg. %timeit), ability to call shell commands, ability to mix code and shell (eg. for x in range(10): !echo $x - for lack of better example), ability to load and reload modules, ability to place breakpoints & integrated debugger, etc.). Racket with xrepl enabled is nice and offers some of the features, but far from all of them. You might want to take a look at rash[1], but the project is still experimental and some parts of it are not finalized. Common Lisp implementations coupled with SLIME are nice, and with some utility libraries you can go a long way, but then you're basically tied to Emacs. I'm not sure about Clojure - I didn't investigate that much due to startup time making it not very well suited for my use case (that's also a reason why I ruled out Elixir, even though it's also a very nice language). Other Lisps all have REPLs, but they tend to be really bare-bones compared to more advanced implementations. I used Chicken Scheme for scripting, making use of its ability to compile to native executables, and it's quite nice with a lot of eggs (packages) available, but the REPL is also pretty standard, with just the default readline functionality.

> maybe it's my Ruby heritage

Definitely :-) My Python-only colleagues screamed in horror after seeing that line...

[1] https://rash-lang.org/ - I have a similar project running on GNU Smalltalk, but didn't get to work on it in half a year :(

Thanks for the incredibly detailed response! I should probably consider ironpython too, though I’m not particularly fond of Python myself. But for now, as others in this thread have pointed out, I’ll try babashka :)

Give babashka a try. It’s a (very batteries-included) clojure dialect that has the specific goal of replacing shell scripts like that: https://babashka.org/

bb is great. There's a Node based version now too that can leverage npm libs: https://speakerdeck.com/borkdude/nbb-at-london-clojurians-ma...

Yeah, this looks about perfect. Thanks!

If you're interested in a lisp for scripting, you should give Racket a try. This is just a Racket DSL (created I believe as a make-like system for the actual Racket project).

Regular racket has all of the things, and if easy interaction with the shell is what you desire, you might try RASH, the Racket Shell DSL.

Another example of using a Scheme for scripting is scsh, the Scheme Shell.

The last update was in 2006, but it's (in)famous for the acknowledgements section written by Olin Shivers.

  Who should I thank? My so-called ``colleagues,'' who laugh at me behind my back, all the while becoming famous on *my* work? My worthless graduate students, whose computer skills appear to be limited to downloading bitmaps off of netnews? My parents, who are still waiting for me to quit ``fooling around with computers,'' go to med school, and become a radiologist? My department chairman, a manager who gives one new insight into and sympathy for disgruntled postal workers?

  My God, no one could blame me—no one!—if I went off the edge and just lost it completely one day. I couldn't get through the day as it is without the Prozac and Jack Daniels I keep on the shelf, behind my Tops-20 JSYS manuals. I start getting the shakes real bad around 10am, right before my advisor meetings. A 10 oz. Jack 'n Zac helps me get through the meetings without one of my students winding up with his severed head in a bowling-ball bag. They look at me funny; they think I twitch a lot. I'm not twitching. I'm controlling my impulse to snag my 9mm Sig-Sauer out from my day-pack and make a few strong points about the quality of undergraduate education in Amerika.

  If I thought anyone cared, if I thought anyone would even be reading this, I'd probably make an effort to keep up appearances until the last possible moment. But no one does, and no one will. So I can pretty much say exactly what I think.

  Oh, yes, the *acknowledgements*. I think not. I did it. I did it all, by myself.

https://scsh.net/docu/html/man.html

I really hope that's a very odd joke. This makes me extremely sad for this person.

This is one of those before-columbine jokes.

Olin likes writing, obviously, and I suspect he started with "I did it. I did it all, by myself" and found a way to get there.

I came here to see this. I used to hack scsh scripts in grad school, back when I was entirely too obsessed with functional languages. It's weird shell, but fun to use if you like that kind of thing.

I still use it with scheme48. Mostly for the process notation (which you can get in chicken these days).

Lisp are in a great place where you can extend syntax to the problem domain. This means that you can bolt on a system like this and it feels just like home.

Racket has a lot of fancy features that might have runtime support. Glancing through the reference: Contracts, threads, futures, custodians, impersonators, wills & executors, ephemerons, etc.

How feasible is it to run an arbitrary Racket application on top of this small core? (Even if all these features are implemented badly for ease-of-implementation)

Standard Scheme seems to allow a small base, but Racket has a lot of things built-in and I'm not sure how much can be done as a library vs. needing runtime support.

There actually is a small base, called racket/base. It excludes a lot of built-in features - you can see a lot of "this module is provided by racket but not racket/base" in the docs - and people tend to use it for modules that don't need the full functionality, because it tends to make the code faster.

Are there any actual separate implementations of it? Could r7rs Scheme be used to write a library that allows running any racket/base program, for example?

I'm sorry, I don't know that much about it. I don't think there are any separate implementation of it. I'd guess that as long you have call/cc you could do it, but Racket has its own specific macro system (syntax-parse, but syntax-rules and syntax-case are supported), and I don't know how much of the racket/base is built on syntax-parse; or, on the other hand, how syntax-parse is itself implemented and if it would be possible to port easily to other Schemes. The good news are that once you have racket/base, most other things in stdlib should work.

What exactly do you mean by this question?

R7RS could totally be used to write a library that allows running any racket/base program. It's called an interpreter :D.

The nice thing about Racket is the #lang system allows you to work with many different languages in the same project. Racket has an R5RS [1], R6RS [2], and R7RS-small [3] implementation as a #lang.

[1] https://docs.racket-lang.org/r5rs/index.html

[2] https://docs.racket-lang.org/r6rs/index.html

[3] https://github.com/lexi-lambda/racket-r7rs/blob/master/READM...

I'm looking for one alternate implementation of a subset of Racket so that I can justify using the language for things.

All of your links are someone implementing Scheme in Racket, not Racket in Scheme. So that would make Scheme "safe", but not Racket.

It would be possible to write an interpreter in any language, but I can't tell if that would take weeks or years before it could run most racket/base programs.

I’m still not sure what you mean by “safe.” Are you worried about Racket becoming unmaintained?

Racket itself is written on top of Chez Scheme, so does that satisfy your safety condition?

> I’m still not sure what you mean by “safe.”

Probably means "portable". If I'm reading it correctly, they want to be able to run Racket programs using a portability layer instead of writing a full interpreter. That would be something like using WINE vs. VirtualBox running Windows inside.

> Are you worried about Racket becoming unmaintained?

This is quite understandable concern, and even if it doesn't happen, having an "exit strategy" prepared is not a bad thing.

> Racket itself is written on top of Chez Scheme, so does that satisfy your safety condition?

Right, I forgot about it! After the rewrite/porting to CS Racket should have become easier to port to further platforms. I think much of Racket's reader and macro-expander was rewritten in pure Scheme.

@MichaelBurge - take a look at the blog[1] and follow links from there[2]. Racket was ported from a custom JIT-capable implementation to Chez Scheme. It took 4-5 years, but it was finished last year. The performance is now at the same level it was before, but a lot of C code is now written in Scheme.

[1] https://blog.racket-lang.org/2020/02/racket-on-chez-status.h...

[2] migration done post: https://blog.racket-lang.org/2021/01/racket-status.html

I believe all of #lang racket is written in #lang racket/base.

Zuo is only superficially like racket. You likely can't do any of the fancy things in it.

Possible contenders.

https://janet-lang.org

https://fennel-lang.org

And https://practical-scheme.net/gauche/index.html

I think part of the issue with using another language can be highlighted here

> The problem with the current build system could be characterized as too large a gap between C and make and the next runnable element, which is either Chez Scheme or Racket BC; neither of those are simple to build. Zuo, a tiny Racket, is an intermediate step to bridge that gap.

maybe babashka too? lightweight clojure with scripting features.

https://github.com/babashka/babashka

A few years ago I've written my own (still experimental) in Rust:

https://github.com/volution/vonuvoli-scheme

The main usecase for this was exactly scripting and systems-programming in Scheme.

Reply on news.ycombinator.com