A Personal Lisp Crisis (2012) | Hacker News Reader

A Personal Lisp Crisis (2012)

blog.jacius.info

87 pointstbirdz38 comments

I enjoyed skimming over this again -- I happen to be the guy who wrote the first comment on it.

I'm pleased to say that I've stuck with CL, and I'm glad I did. I've managed to use it at work, and have been using it to build something I can't picture doing (easily?) in any other language. I've managed to pique the interest of a few coworkers, too.

Much of what the article says is indeed true, but I just don't feel they're as problematic as they're made out to be. There are some ugly corners of the language, but they're, in my opinion, vastly outweighed by a fearsome arsenal of tools at your fingertips. I'd contend that it's warts are "warts in the small" -- some weird naming, some inconsistencies in usage. Other languages -- even pretty good ones -- have problems in things as basic as their scoping rules. I believe that by comparison, CL is actually pretty reasonably designed.

> I've managed to use it at work, and have been using it to build something I can't picture doing (easily?) in any other language.

Could you expand on that? I'd love to know what you built and why you think it would be some hard work to do it in any other language.

Sure, s/built/are building/g.

Without going into too much detail, I basically am subsuming another language (statically-typed, Algol-ish) wholesale into Common Lisp. It replaces that language's syntax with an s-expression syntax, but also provides a way to embed its native syntax. So you can use CL macros to build syntactic abstractions, and use them as low-ceremony wrapping around native code. It also makes many of the primitives of that language into first-class objects in Lisp, so a REPL for the embedded language is emerging (or expressed another way, you can evaluate the embedded language in the CL REPL). I don't think Clojure would do as good a job (gut feeling; I'm not a Clojurist). Racket would probably be a decent choice, but not sure if I'd get good performance (again, not a Schemer).

One of the more arrogant things I've heard Lispers claim of their language is "a superset of all other languages". While that's not entirely true, this work has made me feel that there's more than a kernel of truth in that hyperbole. Warts and all! :)

That's pretty amazing. What sort of tools and features did you use to do that? Genuinely interested as CL is a language I always keep in the back of my mind for the "next project" and this is giving me a big push to use it.

Programmable syntax (reader macros), programmable printer are 2 big ones. Also regular macros are key, manipulating code as structures not text, having both lexical and dynamic variables, closures and all the other nuts and bolts stuff. Type and compiler-optimization declarations help keeping things fast (well, not fast enough, but I'm trying!)

Thank you :)

I think you were correct about Clojure. I read recently reader macros are not an available feature.

Yes, mostly. reader macros are not extensible by the user. The compiler has and uses reader macros.

Rich Hickey has said he hasn't seen an implementation of reader macros that is compose-able across libraries. i.e. two different OSS libs using a two different incompatible macros.

If I understand well, you wrote/are writing a macro that parses your algol-ish language, so you can do, if your language is java, something like:

    (let ((my-java-class  
           (java public class MyClass {
                   public MyClass() {}
                   public myMethod() {System.out.println("hello");}
                  })))
          (myMethod (my-java-class)))

And it shows on the REPL:

     hello

That's quite impressive, and I can see how it opens new horizons.

Bit late replying to this, so I hope you see it, but as I noted elsewhere, the key here is something called a "reader macro". This is a mechanism by which you can control the behaviour of the Lisp reader (i.e., the READ function), and occurs before regular macroexpansion. I use [] to denote s-expressions in the embedded-language domain, and {} to introduce native syntax, which passes its contents through to an actual parser - which returns s-expressions. I can print the resulting forms in native syntax, embedded-s-expressions, or plain Lisp. It's a neat demonstration of the superficiality of syntax.

Perhaps another interesting observation is that what I'm building today is not actually what I originally set out to do, which was more limited in scope. Perhaps another weakness of CL is its tendency to seduce you into feature-creepism. :)

Author's point wasn't that Common LISP couldn't do such a thing. It's whether Common LISP's issues and troubles are justified in process vs examples like Racket cited. How hard would it have been to do your project in Racket which seems to be able to do whatever Common LISP can do and with more consistency in language?

Haven't spent enough time with Racket, but my understanding (correct me if I'm wrong) is that you won't squeeze as much performance out of it, and performance is quite important. I'm also quite partial to Lisp's condition system -- I don't believe Schemes can support something that comprehensive (a trade-off for having nice things like call/cc, I think?)

Chez scheme was recently released as free software. Not sure how it compares to SBCL, but it blows all the other scheme implementations out of the water: https://www.nexoid.at/tmp/scheme-benchmark-r7rs.html

https://github.com/cisco/ChezScheme

Interesting! I'm wide open to more Scheme in my life - so thanks for the pointers.

It's not just an impressive Scheme that I plan to try but it's history from 8-bit to modern implementations was a fun read. Many clever tradeoffs made. See here:

https://news.ycombinator.com/item?id=10210931

Hmm. That's a good point. It didn't benchmark as well as Common LISP last I looked. Another time, Racket developers updated the compiler and improved a benchmark on a spot due to complaints on a forum. I rarely see that haha. It would still have to be tested to ensure performance is within what you need.

"I'm also quite partial to Lisp's condition system"

You're not the only one. Many like that. It's been a while since I used Common LISP. Here's Racket's condition system or a brief intro to it below. Is there anything specific that it seems weaker on?

https://docs.racket-lang.org/r6rs/r6rs-lib-std/r6rs-lib-Z-H-...

While I don't doubt the author's claims about the CL community being unwelcoming, my admittedly anecdotical experience with it was very different:

I wrote a small utility library this year (cl-strings) as my first ever open source project, showed it on #lisp and r/lisp asking for feedback, and received mostly encouraging comments and very constructive criticism.

I've also made some PR's to jscl as a complete newbie to the project, and felt very welcomed.

Hopefully this meme about the CL community is taken with a grain of salt, it certainly is not true universally.

Despite the interesting read, I had the same experience too except as a complete newbie.

I tried redefining the plus operator to see how flexible the language really was (after, you know, all the talk of its power). I'd led myself into a rabbit hole with a macro. The guys in #lisp fairly and in humour told me that using eval in a macro indicates my macro is wrong. Second, I'm trying to redefine a core operator so I should read up on packages. Despite that, they guided me to get my macro working anyway to conclude my sandbox play with success.

I remember being very grateful for their help and feeling like it was a community I wouldn't mind being a part of.

I imagine with a dumb rookie mistake like that, I should have had my self esteem shredded to dust. Instead, I carried on dabbling and reading. I'm no CL programmer but I know more than I did that day so I wasn't put off in the slightest.

EDIT: I just remembered they advised I could make my day job more interesting by writing my own way to inline CL code in PHP as a good project to up the ante.

CL is designed so that code written for earlier Lisp dialects can run with little or no modification, which means there's a lot of bloat and historical baggage in the core `common-lisp` package. But even if that weren't the case, you'd still need conventions like you do in any other language. Clojure is a young language with a BDFL. It already has historical baggage that prominent members of the community will tell you to avoid.

The fact that so many people write their own cleaned-up wrapper libraries like CLUEL isn't a sign that CL especially needs them; it just means it's easy to do. If a language doesn't have macros and doesn't let you shadow built-in operators, you have to play the hand you're dealt.

It's trivial to deal with cryptic function names and inconsistent parameter ordering in a language's core library. I can read code with `car` and `cdr`; I write code with `first` and `rest`. What makes a poorly designed language is something you can't paper over, like broken lexical scoping. There are definitely some things worth complaining about in CL as a language, but I didn't see any in this post.

The author is right. This was a great read. The LISP family has some of the highest potential of any language. Amazing things were done with it, from software to hardware, in the 70's and 80's. Then, after politics and Common LISP, it's been pretty stagnant while most of the best stuff happens outside it in about every attribute. Often, someone ports such works right back to Common LISP to show it could do it all along. It just didn't due to a combo of language and community issues it will never fix.

Leads to two conclusions. The first is for the author to open-source, if it hasn't been, these analyses and work in the article so people can start fixing up a version of Common LISP. The second is to ignore Common LISP entirely in favor of a newer project like Racket Scheme that builds on better, more-consistent foundations with good community and batteries included. It's become most of what I expected of the Common LISP's while Common LISP withered.

The third result is some brave academics need to implement a Genera clone using Racket or Chicken. The rapid iteration, live updates, and whole-system debugging should be preserved. Also feature a deploy-to-executable model that can run in more suitable environments (eg desktops, containers). That would be badass.

Common Lisp certainly has plenty of warts, and the author is not the first person to attempt a rationalization.

The problem with any such thing is getting people to use it. Experienced CLers have already learned how to spell 'makunbound' [0] and that the argument order of 'gethash' and 'assoc' is inconsistent with that of 'aref', and that '+', while generic in a sense, is not a CLOS generic function (you can't add methods to it), etc. etc. etc. So it's going to be a tough sell to these people (myself included).

Your real audience is newcomers to the language. But to appeal to them you need excellent documentation including substantial tutorial material, at least as good as what already exists for CL itself. That's a massive effort on top of the effort of just designing the rationalization.

And then you have a third problem. While implementing your rationalization on top of CL using the package system -- so as to be able to use existing CL libraries -- is the obvious approach, the fact remains, all those existing libraries are written in standard CL rather than in your dialect. So once a newcomer learns your dialect, if they want to read or modify any of those libraries, they will still have to get some familiarity with existing CL.

I'm not saying it's impossible, but boy it seems like a tough mountain to climb.

[0] Randomly omitting letters was a bit of a thing in the old days; consider 'creat' and 'umount', from a different tradition altogether.

Anyone know what he/she ended up writing their lisp game project in? I've been learning Clojure for the past couple of months and now I'm thinking about writing a game in Lisp. I've considered Clojurescript obviously, but it seems a bad choice since everything is immutable by default. Chicken Scheme seems interesting since it compiles to C and it's multi-paradigm so I can program functionally or imperatively.

Any thoughts on a Lisp choice for game development?

does it matter much? I'd suggest working in the language you know best. You can always improve performance of slow code. Struggling to learn a language and write a game handicaps yourself, because you've got to learn a whole language to express yourself. Computers are super fast now, you can write a fine platformer in any language. If you're going 3d, the heavy lifting is handled on the video card.

Slow code does matter. Especially in 3D. Although the video card is doing most of the work, your slower code is still the one pushing vertices and textures into the GPU and telling it where to draw them. If your code is slow enough, you won't be able to do that at a good refresh rate (remember, 60fps == 16.6ms per frame!)

For some inspiration see here:

https://github.com/lispgames/lispgames.github.io/wiki

Thank you!

He wrote at least one game in Chicken Scheme (see later blog posts talking about a Lisp game jam, and work on a SDL binding for it)

The latest Lisp Game Jam page is pretty packed with Lisp game programming resources: https://itch.io/jam/spring-2016-lisp-game-jam

I don't think anyone really thinks CL is all that special or that it is clearly "the best" LISP dialect -- it's just that part of the problem with LISP is that there are so many dialects and CL is a reasonable enough dialect and is kind of a standard, so why not use it?

"kind of" a standard? In some reasonable [1] sense it was the first language to be a standard.

[1] - yes, there are other reasonable metrics

The author forgot to mention the most used Lisp dialect: Emacs Lisp.

There's an awful lost of AutoLISP out there still too.

AutoLISP is barely worth the "Lisp" name, except for the syntax.

> There is a half-jocular saying among atheists, that the surest way for a Christian to lose their faith is to read the Bible, cover to cover. The idea is that if you really scrutinize it, and don’t gloss over the strange, antiquated, or inconsistent parts, it stops seeming so profound and special.

Reading the Bible had the opposite effect on me. It made me into a Christian. It's so Holy and so truthful. It's exact. I've never read anything like it.

About a third of the text is wasted whining about the "negative" community, which actually makes me want to look at CL again.

Which perhaps is evidence for their claim of personality-based self-selection?

The opening paragraph about becoming a skeptic after researching the bible seems to be a bad analogy. One could list contradicting examples of skeptics who became believers by looking further into the subject (C.S. Lewis, perhaps Lew Wallace who later wrote Ben-Hur, and Lee Strobel in our own day).

"The Christian ideal has not been tried and found wanting; it has been found difficult and left untried." - G.K. Chesterton

I disagree. I used to cite them back when I was a Christian, esp Stobel. You find looking further into them that they didn't really become Christians by investigating evidence, esp alternatives in religion. It was always a faith-based thing. Strobel, one I pulled myself outside the echo chamber, was especially one-sided and fraudulent as shown below. Author even encourages you to whip out Case for Christ and follow along with him.

http://www.bidstrup.com/apologetics.htm

You have the logical arguments that most ex-Christians come up with on their own. Also, recall Genesis says we have ability of "gods" to know right from wrong. So, we know the shit is wrong. Especially all the rape and slavery linked to on the right. ;)

http://www.evilbible.com/evil-bible-home-page/god-is-impossi...

Then there was the contradictions we kept noticing. Some of these are justifiable but others (esp integer section) clearly show unreliability.

https://www.youtube.com/watch?v=RB3g6mXLEKk

So, it's not as if we don't read the Bible. The number of ex-Christians that spent years dedicated to it, reading it daily, studying its history, doing apologetics, and so on counters that easily. It's that the book itself & evidence behind it doesn't add up to reason. So, once faith leaves, reason takes over and we leave it for rational things. That simple.

Note: I'm not going to continue a Bible debate from here. Believe what you want to. It's not the point. I'm merely countering this myth that anyone that rejects it didn't read it, try it, commit with their heart, whatever. That's straight-up lies perpetuated by a subset of Christians. It's actually a combo of logical and emotional reasons plus issues with Bible itself. That leads to opposition to it.

I think it's not how much you scrutinize the bible, but whether you look beyond the surface and ask whether on the whole it does in fact represent a cohesive view of reality. Sure, you will find what you go looking for -- for both good or ill. All find what they truly seek.

But beyond motivations, there is the deeper question of whether it is really true.

"If Christianity is untrue, then no honest man will want to believe it, however helpful it might be; if it is true, every honest many will want to believe it, even if it gives him no help at all." - C.S. Lewis

Reply on news.ycombinator.com