Guile Steel: a proposal for a systems Lisp | Hacker News Reader

Guile Steel: a proposal for a systems Lisp

dustycloud.org

104 pointspabs377 comments

Uhoh. Author of the post here. You know, when a post leaves its target audience, it can really make all the difference, huh? Last week I had a post on HN's frontpage and it was aimed at a general audience, it was an introductory topic (an intro to Scheme, without assumptions of involvement in that community). This time it's a post of mine I wrote off the cuff talking to a specific audience about a specific topic (specifically, me musing out loud about a project that would be interesting to the Guile community, to the Guile community). And that's fairly reflective in the replies here.

- Common Lisp is the only real lisp and anyone who's not using it reinvented it badly? Check.

- Questioning what is "systems programming" and yes I actually already did that in the post itself, it's a large portion of the post's text? Check.

- "It really ought to be in Rust because Rust is awesome?" Check.

The point of this post is, to the Guile community, "This would be an interesting thing for us to explore, here's what I've been loosely thinking about exploring it... what do you think? Anyone else excited to talk about this with me?" (It turns out, within that community: yes.) It's not hating on Common Lisp, it's not saying that existing lisps aren't already "systems languages" (I agreed on that in the post), and no I'm not aiming for this to be Rust because we have Rust and you can use Rust and that's cool!

And the other point of the post is: this is a hilarious name, "Guile Steel"... "Guile" + "Guy L. Steele" + "close to the metal". And also, PreScheme is a pretty cool thing that's been underexplored outside of Scheme48 and hey maybe we should think about whether that's a cool direction we should modernize on!

So anyway, if you feel annoyed because this isn't targeting Your Favorite Thing, that's cool... it isn't! You're super welcome to read the post, just know who the audience is, following the above. :)

> And the other point of the post is: this is a hilarious name, "Guile Steel"... "Guile" + "Guy L. Steele" + "close to the metal". [...]

Exactly what the name reminded me of. It makes the name even more fitting.

Interesting stuff! I hadn’t run across Carp before, so thanks for that too.

I thought “Guile Steel” was obviously partly a play on the fact that steel doesn’t rust as easily. :-)

Yes that was part of the joke also :)

You can't steal guy's name that's improper ;)

Someone should send an email telling the guy to steel himself!

Also fun fact: Guile was already a pun on Guy L. Steele, I just finished the pun :)

Oh I never considered Guile to be a pun (weirdly..). Thanks for patching the gap :)

It's not stealing if you first introduce it properly :-)

https://www.youtube.com/watch?v=_ahvzDzKdB0

I think it's odd that I was thinking the same thing about a month ago. What it would take to create a "systems" level Lisp.

I think this is great, and will be watching this space.

p.s. Name is perfect.

There is actually already a Scheme interpreter implemented in Rust (https://github.com/mattwparas/steel) which has the same name for the exact reasons you stated.

May I suggest looking into Lokko Scheme[1]? It's a R6RS Scheme capable of booting in bare metal, even has a small OS available that you can boot up in QEMU, it even runs Doom! It's a great project, check it out.

[1]: https://scheme.fail/

I've never done more than a little dabbling in scheme or guile, but I'd love to see this happen.

I’m gonna sound like a broken record, but … if you’re looking for a “systems programming scheme”, consider Gerbil Scheme (https://cons.io/)

  This is would be a great project to provide a mature Lisp that developers could use to target WebAssembly.  Guile Scheme is an amazing Lisp to utilize for this purpose. Here are some other Lisp-to-Wasm type projects from my brief search.

-The Wisp project to compile Lisp-to-Wasm: https://github.com/mbrock/wisp

-Chaitin's Lisp- This is a WebAssembly port of Gregory Chaitin's minimal Lisp interpreter: http://weitz.de/chaitin/

-HN post: Common Lisp running natively over WebAssembly: https://news.ycombinator.com/item?id=31590819

Great idea!

Wait a second, wasn't wisp something else as well? searches for a few seconds Ah here it is: https://hg.sr.ht/~arnebab/wisp (a fewer parens lisp notation)

Yeah, naming conflict. Haven't tried the web-oriented-lisp but Arne's Wisp is awesome.

and then https://github.com/wisp-lang/wisp/blob/master/doc/language-e...

and I'm sure I've seen a time travelling debuggable js-transpiled thing long ago

ps: oh and this https://github.com/paulitex/Wisp

pps: ok i'm digressing https://lips.js.org/

I feel like something like this could be incubated INSIDE Rust.

Look at this tiny proof-of-concept lisp in a rust macro: https://crates.io/crates/macro_lisp

- Full LSP support (because rust macros have full LSP support)

- Compiles down to bare-metal code

- It's just Rust at the end of the day, so all the incredible tooling is already there.

If I were to start making a new lisp today, this is what I'd do!

It could be! Btw, macro_lisp is hilarious, nice job.

My post was specifically aiming at talking to the Guile community as in terms of something useful and befitting of that community's use. A "Lisp Flavored Rust" would be interesting if someone wanted to do it though.

The idea that you can have an alternate ISA that's designed for high level languages like lisp with fast pointer tagging and garbage collection is a myth that isn't really supported by the history. A huge reason for the demise of Lisp machines (other than their outrageous price) is that while they were very cool, they were also incredibly slow compared to commercial Unix systems written in C. This is actually addressed in the preface to the Unix Haters Handbook[1] (ctrl-f "Michael Travers" to get to the relevant part), which is written by someone who had to move from working on a Lisp machine to a VAX workstation. It's a great (and humorous) read and kind of illustrates the point that as cool as Lisp machines were, they had terrible performance compared to other computer architectures at the time. Worse is better, and all that.

Another great example of this is Jazelle[2], which was an ARM ISA extension that added extensions to support running Java bytecode natively on the processor. It's a really cool idea but the reality was that it was too complicated and didn't work well. High performance JDKs like Azul or OpenJDK that are written the traditional way for regular ARM/x86 perform better, and they make CPU design much simpler. The reason that there's a weird symbiotic relationship between CPU design today and C-like languages is that's the model that actually is the best tradeoff between performance, safety, and cost. I know this may sound strange to people, but the market doesn't lie.

There's obviously an argument to be made that there was a chicken-and-egg problem here, where Lisp machines and Jazelle had low adoption rates and that led to low investment in the technologies from companies (compared to say x86). Maybe in an alternate reality where the industry had gone all in on this kind of technology the technology would have actually been fast and cheap. But that's highly speculative and the actual historical examples we have don't support this idea.

[1] http://web.mit.edu/~simsong/www/ugh.pdf [2] https://en.wikipedia.org/wiki/Jazelle

i think the authors point is that you don't need to wait for alternate ISA in order to use lisp as a systems language

I have a feeling (not based on sys prog exp) that by not going with Common Lisp you will be reinventing the wheel

Perhaps that's intended.

sure. when i wrote that i thought it was meant for general public. i also think that its great that the guile community is looking at this. my point wasnt so much as "use my favourite tool" as it was that i just think that a lot of good work has been done in cl and cl is most performant of all lisps. cl already targets assembly and c and has ml-level static typing. the author might find thinlisp [0] interesting

  [0] https://github.com/ska80/thinlisp

Slightly off-topic, but what would be a good book to learn guile (scheme, lisp) for someone with solid programming background (C, ruby, php, golang but no lisp at all)? I always wanted to give it a try but not sure where to start.

Structure & Interpretation of Computer Programs!

https://xuanji.appspot.com/isicp/

I think that is the interactive-inside-the-browser version? Of course the original is the MIT Press version at https://mitpress.mit.edu/sicp/

The Little Schemer and the Seasoned Schemer

The Little Schemer is a playful introduction and SICP is more impactful but also more challenging book. Even if you already know how to program, it will force you to re-wire how you think about programming.

SICP is great at teaching concepts and testing your knowledge, but it lacks an important part that is not quite obvious at first: Interactive programming AKA how to actually use a Lisp.

I recommend you watch some videos of people using an IDE/Emacs or some text editor with sufficient support like VSCode to write Lisp code interactively. How they build a program while it is running. But also doing mechanical things like navigation and structural editing.

Step 1 is the basic mechanics of that, leveraging an integrated REPL and language constructs. Step 2 is to make this productive and fluent, by using workflows, tricks and tools that let you introspect what is there, manage program state, move things around effectively etc. Step 3 is then to go beyond and not just write the program in a direct fashion, but experiment, generate data, visualizations, interactive debugging etc.

> I recommend you watch some videos of people using an IDE/Emacs or some text editor with sufficient support like VSCode to write Lisp code interactively. How they build a program while it is running. But also doing mechanical things like navigation and structural editing.

Could you please provide a link to some example videos you recommend? Greatly appreciated!

Author of the post recently wrote A Scheme Primer.

https://news.ycombinator.com/item?id=32010490

I second this, emphatically! Between all the books and talks I've been through, this stands out for how much it packs in and how approachability it does so. I can't really give a perspective from a beginners view, but for a programmer with existing experience in other languages, think this is a great piece to read.

Aw thank you both <3

Paradigms of Artificial Intelligence Programming by Peter Norvig

https://github.com/norvig/paip-lisp

To learn Lisp, rather than Scheme, Practical Common Lisp is awesome. It's available online for free at https://gigamonkeys.com/book/, but I recommend purchasing it if you can.

David S. Touretzky, Common Lisp: A Gentle Introduction to Symbolic Computation.

It's the most beginner-friendly Common Lisp textbook but it might be worth it even for a seasoned programmer of other languages just for how nicely it explains lisp-specific concepts and constructs.

The first edition is available for free: https://www.cs.cmu.edu/~dst/LispBook/

Sounds like bitC: https://web.archive.org/web/20160529063027/https://www.bitc-...

It was basically a lisp syntax for C, which enabled things like lisp macros. It died when the author went to work for Microsoft.

Or Schemetran: https://gitlab.com/codetk/schemetran A lisp syntax for writing Fortran.

I think bitC was significantly more than that but I never used it in anger.

Years ago I came across another system, c-amplify, which was more literally a lisp syntax for C, intriguing but at the time wasn't able to get it to work. https://voodoo-slide.blogspot.com/2010/01/amplifying-c.html

A whole lot of people, including commenters on the link above, don't really get the difference between something like that and a lisp-to-C compiler. I'm not sure where the cognitive gap lies.

> It was basically a lisp syntax for C

It was more like a proto-Rust. The Lisp-like syntax was a placeholder while the developers (plural) tried to solve language problems on a semantic level. Memory safety was their white whale. They never came up with a solution for that. The original author left because he had given up.

> "something like C"

SBCL? Or ECL (compiles to C), CLASP (C++, LLVM)?

The older ThinLisp? (compiles a subset of CL to C) https://github.com/ska80/thinlisp

The very new NPT, a CL implementation in C? https://github.com/nptcl

> Static typing for ahead-of-time more-correct programs

the Coalton library. A ML/Haskell as a CL library. https://github.com/coalton-lang/coalton/

> Static typing for faster or leaner programs

SBCL

> WebAssembly

Common Lisp running natively over WebAssembly: https://news.ycombinator.com/item?id=31590819 (very much POC)

I know, I know, but he didn't mention any of these in the blog post, nor any CL project. Justice is done.

> he didn't mention any of these

"she didn't mention any of these"... my name is Christine, it's right at the top of the post...

At any rate, most of the approaches you're taking resemble Chicken Scheme's "compile to C" approach, which is all good and well, but doesn't accomplish the (questionable, and the post questions it) quasi-definition of "systems language" I was exploring in the post... it's something that compiles to it, but doesn't have the underlying characteristics of something written directly in it. More in the post I wrote in reply https://news.ycombinator.com/item?id=32058142

> "she didn't mention any of these"... my name is Christine, it's right at the top of the post...

Apologies. This old blog theme rings in me the old reaction of "Christopher from Mediagoblin!".

Apology accepted, thanks :)

I guess that's yet another reason to re-theme my website....

The article mentioned the value of static typing for "ahead-of-time more-correct programs" and "faster or leaner programs", but overlooks an even greater contribution: being able to quickly understand unfamiliar or ancient long-forgotten code without having to run experiments on it or call someone out of retirement.

It's the difference between having a plain cadaver, completely drained of its life fluids, and having one where every empty tube, regardless of size, has been labeled with something like "blood, to heart" or "lymph". If you aren't intimately familiar with human anatomy, the latter is an immense time saver.

Most software that has been continuously developed over a long period of time is either C or C++, even though Lisp predates both by over a decade. There's a reason for that.

> Most software that has been continuously developed over a long period of time is either C or C++, even though Lisp predates both by over a decade. There's a reason for that

also a reason for memory leaks :)

I run hundreds of C/C++ programs on a daily basis. Many of these have been running for months on end, yet my memory chart remains a flat line.

Can the Mossad hack me? Sure. Would Lisp stop them? Probably not. Ron Minnich gave a talk a few years back titled, "If You Trust Your Computer, You're Crazy!" How many of Ron's points do Lisp & Rust fully address? By my count, zero.

they might make the job harder

The reason is called free beer UNIX V6, with C compiler on the box, and the subsequent POSIX specification.

Additionally, people keep forgetting that C++ was born on the same building as them, thus rode on their wave as companion in crime.

Just like there is a reason most Web software uses JavaScript on the browser.

I've seen more VB (1991, proprietary) programs than Lisp programs. I've seen more Delphi (1995, proprietary) programs than Lisp programs. Using a jumble of symbols language without any text hint as to what they are, or even when they run, is a crippling tax on scale--be it LoC, timeframe, or contributors.

It's the same reason why people are flocking to TypeScript.

And I like Lisp--as a not-systems language.

And yet LispWorks and Allegro are still in business, go figure.

You can specify types in Common Lisp just fine if you so desire: https://lispcookbook.github.io/cl-cookbook/type.html

Yes, I know this. Most people on HN know this. When most of the base libs, built-ins, & legacy code don't declare types, static typing loses a lot of its value.

In something like Go/Pascal/Java/Rust/C, if I introduce an impedance mismatch during a re-factor, the compiler catches it in seconds or less, and can tell me every file & line to adjust. In classic Lisp, this becomes a runtime error, and it can be days/weeks/months/years (if ever!) before the bugs are shaken out. In typed Lisp, it's a split between the two.

Splitting the difference for the sake of a legacy codebase (like PHP->Hack), or because you've had a language thrust upon you (Javascript->Typescript) is a reasonable choice. To double-down on a predominantly type-declarations-optional language in 2022 is nuts. Lisp is good, but it's not that good.

FWIW, Common Lisp is already a "systems Lisp".

It really saddens me how much Scheme implementations have proven that Greenspun's Tenth Rule — any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp — applies to more than just C and Fortran.

Rather than incompatibly reimplementing all the great things which make Lisp work so well for so many problems, why not just use Lisp? Is it ignorance? Is it hubris? Maybe it's genius, and I just don't get it.

I have a theory about it, as someone who used Lisp since 1987 and Scheme more thereafter: there's no concise version of what https://docs.python.org is and that makes ALL the difference.

I know what exists, have the books, etc, but nothing is as well organized to be as easily searchable and without digression as what the Python world has set up for itself.

I tried to get into SBCL a few months ago and I found all of the documentation I encountered to be really good (really good) but there was just too much to learn. I'm not complaining, I just didn't have the time personally to climb the learning curve (despite all the available lore from previous climbers.)

When you start with a small system and reinvent things, you know them from the inside out. Often that's easier and more fun than reading up on an existing large system. The downside, of course, is that we wind up with incompatibilities and myriad idiosyncratic ways of doing things.

This is called “The Paradox of the Active User” and it underlies an enormous amount of why our world sucks (not that it all sucks, but this underlies much of where it does). In short: I’m too busy to learn something that will make me so much happier and more productive forever. (Aka. “I’ll just sit in the dark.” :-)

Perhaps perversely, your comment is almost, but not quite, enough to push me to make a second run at it. I mean, I goddamned know CL is better. :) Cheers!

I've pondered what strange reasons might lurk in the depths of me psyche that could be inhibiting from using Lisp.

Hint: Don't let the parens turn you off. I promise that once you get over the hump, you will learn to love them, and won't ever want to use a language that doesn't have them!

I don't think that's the right mindset to have. One should just learn to appreciate diversity. One shouldn't approach any language with the view that one is about to discover a language that rules them all. Just learn with a view to discover the strengths and weaknesses of whatever language you are dealing with at that moment, as every language must have those.

http://www.lispworks.com/documentation/HyperSpec/Front/Conte...

True enough, but the spec is a specification, not a guide. It gives a lot of the 'what,' but not a whole lot of the 'why.'

That's part of what made Practical Common Lisp so revolutionary.

the spec kinda sucks for usability, but thank goodness it exists; it's exactly what i meant docs.python.org does way better

As a lisp novice, I found this to be a principal challenge. Most of the docs I find will have either function signatures or examples, but the examples rarely cover the bases. I think it's pretty common for dynamically typed languages to have very hand-wavy documentation, but lisp is big on the arcane and powerful. I'd often find answers to my questions in the entirely wrong places, e.g. pearls of wisdom tucked in an example that I stumbled across looking for something entirely different.

One of the inventors of Scheme literally wrote the book on Lisp: https://en.wikipedia.org/wiki/Common_Lisp_the_Language

I'm aware! Although Common Lisp as specified does differ somewhat from the language he describes in his book.

But you kinda implicitly assume my point: Scheme and Lisp are different things. That doesn't make Scheme bad or Lisp good, although I will assert that Scheme is underspecified and Lisp is pretty decent.

Carp Lisp is one lisp that has Rust like memory management without garbage collection.

A good reason not to use it then.

Ooh, ahh, can’t we have for Rust what Clojure did to Java, and call it Lust?

I've had similar thoughts and am developing a language for them: github.com/civboot/fngi

It targets it's own VM, which I plan to eventually be able to compile to native code. So it can both execute code at compile time (literally execute functions just declared) as well as compile to native code later.

It's not scheme (or even lisp) -- it draws more from FORTH but with a much more readable syntax. However, the goal is to be a language that fits this kind of bill -- it can bootstrap other languages (and whole operating systems) with ease.

> Before we get into this kind of stream-of-consciousness outline, I'd like to note that very topically to this, over at the Spritely Institute (where I'm CTO, did I mention on here yet that I'm the CTO of a nonprofit to improve networked communication on the internet on this blog? because I don't think I did)

At this point it seemed like the article was purposefully written in a stream of consciousness style instead of just ending up that way. There’s not much here.

> [PreScheme] doesn't do automatic garbage collection itself, and I think Rust has shown that this area could be improved for a more modern PreScheme system.

I strongly doubt this (because Rust's ability to manage memory safely and efficiently comes from its saucer-science static type system), so i strongly encourage the Guile/Scheme/LISP community to attempt to do it!

That would be a good idea, if rust actually would implement memory safety, type safety and concurrency safety. the first goal is stated in the docs, but violated by its implementation, the other two are not even fulfilled by their specs.

what's missing is concurrency safety for lisp. and enforcing compile-time type safety in the proposed lisp kernel. not just the usual run-time type safeties.

lisp can ignore the unsafe POSIX architecture (esp. blocking IO), so there's a chance.

PreScheme was statically typed! It used Hindley-Milner!

Yes, why not.

Reply on news.ycombinator.com