Comment by varoun | Hacker News Reader

Comment by varoun | Hacker News Reader

I spent the last year writing substantial Common Lisp, including a FoundationDB client and distributed systems primitives layered on top of that, an observability and operability library for CL systems, and a large scale log, metrics and tracing platform that leverages these - in about 200K lines of code and tests. My take on this, apart from one of familiarity and personal preference/taste in choosing a programming language, in the age of LLMs are - 1. From a “functional requirements” perspective, any language will do. You can build substantial systems in something low level like C or higher level like Python or Common Lisp for example. 2. From a security perspective, if you are using LLM assisted code generation, whether you use C or Rust or Common Lisp does not really matter I think - current models are capable, and future models perhaps more so, in producing secure code.

My understanding is that LLM are deflationary, resulting in code being priced as a commodity (cost plus). If this bears out, then lower token costs and higher performance (esp in my case, where good performance results in lower infra costs to host the service) will start to matter more and more. Common Lisp code bases are famously dense (in the good sense), resulting in perhaps lower token costs when an LLM needs to review and make a change (smaller code bases also help humans / small teams). Common Lisp can achieve performance that’s close to that of an equivalent implementation in C or C++.

Taken together, they make for one case for using CL and other similar languages, although one that’s grounded largely in economics.

I couldn’t help but notice the parent commenter’s post history. Account from literally before this was called HN, one comment 6 years later, then 13 years later this comment.

The pro lisp/s expression comments check out.

hopefully in 20 years we'll get another

But will an LLM write dense Common Lisp? A lot of that density comes from well-designed domain-specific macros. An LLM will have no incentive to write these on its own, though it could if a human prods it sufficiently.

Yes, human direction will still be required for the foreseeable future (whatever that means in this fast-paced field) to recognize where introducing a DSL or a macro makes sense.

The only issue have faced with generating Lisp/scheme code, especially on small models, is that it always misses some closing parenthesis. However, because the small core language, you can use structured output in the LLMs that support it and it works quite well.

I found that explicit instructions to keep nesting depth below a limit helps limit the damage. That limit was 5 in my case, which can be inconveniently tight in some situations.

In the Clojure world, there is a small utility that is pretty popular called clj_paren_repair (part of clojure-mcp-light: https://github.com/bhauman/clojure-mcp-light/blob/main/src/c...) that uses parinfer’s indentation algorithm to help repair out of whack Lisp parens (and brackets and braces for Clojure). It’s easy to expose this as a skill or tool. I had Pi create me a tool in 60 seconds. I have to say that before I discovered it, I had no time for parinfer (paredit all the way, baby), but this really does the trick as often the model gets the indentation correct (from being trained on so much Python??).

They don't write dense Python. I'll tell you that. The individual functions might look good, but there's just so much repeated or nearly repeated code.

Yea, this seems to be a consistent issue. The models don’t know when to think strategically and often make local edits that keep growing individual functions with more and more conditional logic.

> an observability and operability library for CL systems, and a large scale log, metrics and tracing platform that leverages these - in about 200K lines of code and tests.

This sounds really really interesting. I don’t suppose you can share more about this?

I’m ok to release the first three (the observability/operability layer, the FoundationDB client, and the distributed systems primitives library) on Github, under the MIT license. I would like to do this properly, the code currently does not have any developer or end user docs, along with a few other nice to haves, so it will probably take a week or so. My Github username is the same as my HN username, and I’ll try and post here again when the release happens. Happy to chat more then.

I’m not expecting much from the comment side of things! But I’ve played with FoundationDB over the last couple of years, and see it as such incredible technology for the enterprise.

The trouble is that as a “foundation”, it’s not immediately useful unless you are Java and can thus consume the record layer.

I also have a huge soft spot for Lisp, so your work here dovetails with two of my favourite things. I hope you get the chance to publish!

Do you really think current LLMs can implement a large application in C as securely as they can in e.g. Go? I will freely admit that I have not tried to do so but I have a hard time imagining it. Perhaps my biases are outdated...

Danluu had an article circulating on HN a few days ago that apparently shows that yes, they can. He implemented Zstd and most models did it almost perfectly in most languages. Pandoc was a challenge they still cold not one shot though, which seems correct to me as the LLM would need proper guidance still for such a large enterprise.

Zstd is decidedly not a "large C application" though, more like a smallish library at best. The C reference implementation is also open source and thus likely part of the training set for coding LLMs. Any model re-implementing Zstd would probably be "remembering" how it was done originally, instead of implementing it from scratch.

By your argument it should have been able to just remember Pandoc existing implementation then? But it didn’t, you think they can only “remember” small code based well enough? I doubt that. Especially since after using LLMs quite a bit I am confident I could write my own specifications for something and LLMs would be able to do it properly, despite definitively not having seen it before.

Why not? The vulnerabilities you read about are due to oversights, not inherently lacking capabilities of the language. For every program exhibiting any given of such, you'll find thousands which don't make the same mistake at the same place. After all, most reported vulnerabilities are just a short patch away from being fixed.

Now human programmers might find it more difficult to get certain things right than others, but to a LLM only quantity of examples matters, no?

Out of curiosity, what is the largest C program you have written pre-LLM, and did you ever try to run it through valgrind*? The "oversights" in C tend to be extremely subtle and dangerous, and of a type that would be impossible to make in Go (or Rust or Common Lisp or whatever.) I do not have confidence in LLMs not falling victim to these subtleties at least once in a large application. I could be totally wrong! Perhaps they can all be found with a sufficient adversarial loops or something. I just have a really hard time imagining no problems of this sort occuring.

*(Memory leaks can probably be found by just having the LLM run valgrind itself and chase them down, but this gives you a good feel for the difficulties of writing safe C. Again, never tried any of this with LLMs myself, and I haven't written a nontrivial C program in years.)

It's more opportunities for a screw-up to lead to a vulnerability in ways the tooling won't catch by default, in a system where P(screw-up) > 0.

> Do you really think current LLMs can implement a large application in C as securely as they can in e.g. Go?

That's not my experience: I've seen an LLM generate a C++ use-after-free (1.5 month ago).

Reply on news.ycombinator.com