Lisping at JPL Revisited | Hacker News Reader

Lisping at JPL Revisited

blog.rongarret.info

219 pointslisper100 comments

I find those exaggerated:

> I don't really program in Common Lisp any more. I program in a little custom language that I have incrementally built on top of Common Lisp over the years

so yeah, that's CL. ergolib is a CL library, like any other, with a cool macro to learn, if you want.

> A CL programmer looking at my code would probably go, "WAT?"

it's OK :] There are still defun and opening parens.

> a fragmented ecosystem of little sub-languages

but what sub-languages are we talking about? I only see a library with helper functions and macros. That's Common Lisp, not a derivative. You can pick any other library of the ecosystem, it's compatible, like Serapeum, another common library with many useful stuff baked-in, compatible with all CL implementations. I don't call that a fragmented ecosystem. Only some authors have their fav' utility and that's ok.

Last words re implementations, for new readers: SBCL is thriving, LispWorks and ACL deliver, ABCL and ECL are active, CLASP (CL on LLVM) is financed and active, GNU CL got a new release, someone in Japan started a new implementation in C, someone else on Savannah another one in C again… And CCL is not thriving unfortunately :( There is room for contributors (there's some backing, contact the CL Foundation).

In Lisp, the boundary between a little macro library and new language is fuzzy. But my code looks pretty radically different from textbook CL code, much more different that, say, Scheme code looks from textbook CL code, and no one would dispute that Scheme is a different language.

Maybe I should have said that I don't code in idiomatic CL any more.

> SBCL is thriving...

Indeed. The CL ecosystem in general is doing very well. It is only CCL that is fading away, and hopefully even that is only a temporary setback.

> but what sub-languages are we talking about? I only see a library with helper functions and macros. That's Common Lisp, not a derivative.

If your language is a DSL factory, the line between your language and DSLs naturally blurs. If https://github.com/y2q-actionman/with-c-syntax exists, does it mean that C is a DSL of Common Lisp, given a good enough standard library? If https://github.com/calyau/maxima exists, does it mean that Maxima is just Common Lisp with more maths? If https://github.com/Shen-Language/shen-cl and https://github.com/coalton-lang/coalton/ exist, does it mean that Shen and Coalton are just a fancy way of writing Common Lisp in an immutable way? If https://github.com/froggey/Iota exists and we can play sdlquake on Mezzano, does it mean that LLVM-IR is a dialect of Common Lisp?

The above series of questions is not meant to be fully credible - it's meant to be food for thought.

> Different people like different things, and that's OK. Drink whatever is the best fit for your palate, and code in whatever is the best fit for your brain.

Doesn't this ignore the need to work together and build on each other's work? To me, that's not about companies being able to treat programmers as interchangeable, but about us collectively building higher, i.e. building towers of Babel rather than suffering from the curse of that story.

Take one: We need people moving together just as much as we need scouts who can push ahead and find the best paths for the rest of us.

Take two: Working in the best fit for your brain is not incompatible with working with others -- enough companies are Java shops, why can't there be CL shops? (There are assuredly Clojure shops at the very least.)

Take three: Not everything has to be about work. Programming is a skill, not an occupation; there are lots of ways to apply programming as a skill that don't require you to build large edifices in teams.

And how nice when those scouts lay groundwork that is useful for others, rather than creating something only they can use?

Yes, you may think better in lisp than c++, but that doesn't help your team when you created something and moved right along. Better to speak the same language.

I was referring to the discipline as a whole, not scouting at the granularity of a single team. Also refer to take three: working in an industrial team is not a prerequisite for an application of programming to be valid.

But to your very specific point: I have in the past developed prototypes in Rust whilst on a Java-only team. It's actually a good thing: these prototypes were meant to be thrown away, so working in a different language (and one that helped me model things effectively) meant we were forced to throw it away then write it right in Java once we knew what we were doing.

(We did the same thing for another feature, but it was someone else and they preferred Python. Again, we were able to prototype more rapidly and then implement a solid solution in Java once we knew how things should shake out.)

Another thing that is good to encourage is portable data. It's good for its own reasons, and it makes that sort of exploratory environment much more flexible.

I've reimplemented a few side projects more than once, using common data back-ends because I can. I find it helps for say, messing with a new programming language, because I can apply it to a task I'm already familiar with.

Better for the others, but is it better for the scouts? Too often individuals are treated as drones obliged to work for a collective, which feels entitled to the work of the drones. From your perspective, they're not a good team mate. From their perspective, they're not part of your team.

I think this example is being carried to extremes unproductively.

All being equal, it's better to have a team writing code that is useful to the system or project or whatever ASAP.

If an individual insists on prototyping in a language that nobody else can use, then they should either carry that prototype into the team's ecosystem, or start prototyping in that ecosystem and learn to be productive that way.

If option A, let's hope that "write twice" is fast enough that they are productive. If option B, they may take a productivity hit but will probably catch up.

The third option of "I'll do lisp you do <teams ecosystem>" is not very pro-social. Come integration and debugging and maintenance time, that Lisp-er is not in a position to help. That's an undue burden in the rest of the team.

I think you're excessively hung up on the example of everything happening within a single team inside a single company or some such thing, when the author of the submission wrote about making a choice for themselves. You don't start coding in Lisp out of nowhere when working in a Java shop on a CRUD app. But if you're charting your own course, it's great to be able to choose whatever suits you best.

I can't count the number of times someone published their open source project and inevitably people in the comments started questioning their language choices, because they wouldn't be able to use it in their "team", and how that choice is ultimately harmful for "the community". Now, you're not doing that exact thing here. I'm just thinking about how all that focus on "the team" and "the community" often goes too far.

Personally, I'm not going to do anything in Common Lisp, but I'm always happy seeing other people doing their projects in that language (and many others).

> I think you're excessively hung up on the example of everything happening within a single team inside a single company or some such thing, when the author of the submission wrote about making a choice for themselves.

That's fair.

> I can't count the number of times someone published their open source project and inevitably people in the comments started questioning their language choices, because they wouldn't be able to use it in their "team", and how that choice is ultimately harmful for "the community".

Yep - not paying means not entitled to anything. I agree there.

So we are doing now what the others do? It defies any reason.

If you are part of a team, then yes, do what others do. And no, it doesn't defy reason. Code is written once, but it's maintained for a long time.

Or it's thrown away. But if it's thrown away because nobody else on the team knows that language, that's pretty wasteful. And if you make someone on the team learn a new language just to maintain that one program that's also wasteful. The person who insisted on using their language is optimizing for their own programming, but actually slowing down the team as a whole. We have words for that kind of behavior; they aren't compliments.

Now, in any given instance, it could be that the program is much easier to write in one language, and more difficult (or even impossible) to write in others. But even in that case, if you're in an organization, you need to get buy-in from the team or management or both, rather than just doing your own thing because you can make it work. Otherwise, your program may work, but it's an orphan. You need the organization to be willing to maintain your special program, or it dies.

Sometimes hard to predict whether something that works well for you would, or would not, work well for others, and whether others could, or could not use it.

Take four: what, specifically, about ergolib should prevent anyone from building on top of something written with it?

[Edit: A programmer should be able to Reverse Polish Notate, nested-paren a tree, reduce some arrays, make references in nested scopes, evaluate without mutation, send a message, spin up a fail-fast supervision tree, produce finite answers from infinite graphs, synchronously react, rendezvous with logic variables, combine the above. "Specialization is for insects."]

Maintainability. If you're the only person who can edit the library, you've just created a bus factor that would be unacceptable to most project managers.

I mean, that specific library isn't really all that hard to work with. It's not doing anything completely crazy, just somewhat unusual. But in general I think it's probably better to keep the crazy custom environment to personal projects- at least for awful utility macro libraries like mine.

Narrowing down on a single library is one thing. Maybe case by case it's ok. In general if someone showed up and said "I will prototype everything in lisp and you can take it from there" I'd say they are a bit egotistical and not being much of a help to the org. Prototypes are 1% of the work.

And I say this as someone whose full time job was to do research prototypes of robotics libraries for a while.

As a complement to my other reply to you, I'll note that I developed a simulation system in Java (a very common language), and strove to keep it small, avoid complexity, and generally make it approachable -- but for a long time I remained the only one who understood how the system worked, because you simply had to understand the business domain in order to understand why things worked the way they did.

This is my personal opinion, but I think that domain expertise is a much stronger component of bus factor than language choice is. You can write unmaintainable code in any language. But good code often still has prerequisites to understanding. Meanwhile, if you have the domain expertise, you can get quite far in understanding semi-bad code just by pattern matching on manifestations of your domain.

not only project managers. i love lisp but i dont use it for hobby projects partially because i don’t want to maintain libraries in a small ecosystem where they are often abandoned

The fact that you adapted the famous Heinlein quote for programming just goes to show me a problem with that quote. Recall that the original quote mentioned programming itself as one of the many things that a human should know how to do. But your adaptation illustrates that programming alone is deep enough to merit its own specializations. So just as, IMO, specialization is not just for insects but also for humans living in a functioning, populous civilization, specialization is also fine and expected for programmers, particularly since our numbers have grown over the decades. And to bring this back to my original point, yes, this means that we shouldn't all be writing lowest-common-denominator code in mainstream languages. But I still think it's a good idea to avoid needless fragmentation.

Fair enough; I think I'm describing the horizontal bar of the "T" and you're pointing out the vertical, and we differ on the amount of fragmentation ergolib represents. (Despite not using CL on the daily, upon perusing ergolib's source I found it a relatively straightforward mod-con library, so I don't think it necessarily fragments)

(But I was pretty serious about that list: I would hope a programmer just out of school would have heard of all of them, and expect anyone experienced to have done most of them, if not necessarily in the languages that make each one easy. Similarly, from Heinlein's original list, I currently lack only practical experience in planning invasions [though I have theoretic hex-board experience], butchering hogs [though I have a friend who has offered to teach me], and dying gallantly [though I have "I am just going outside and may be some time" as an example to follow should the occasion arise])

I'm waiting for WebAssembly components to be standardised then I don't care anymore. The idea is to create "libraries" written in any language that can be used by any other language, with WASM being the low level ABI.

When that happens, I can go all in with Lisp because I care about my personal productivity (I have always been a lone wolf programmer) yet still being able to use code written by anyone in their comfy language.

As I get older, I find I just want to mold and bend the computer to my every will, and Lisp is the tool to do so. That the resulting code is hard to understand for a junior engineer is not my concern.

But one thing is for sure: I want to be able to create any abstraction, and I am done writing glue code, writing yet another PostgreSQL library for yet another language. How many date/time libraries have been written since the invention of UNIX? Imagine the amount of wasted time, money and effort. Then multiply by the number of languages that have been invented because the syntax of the previous one is too inflexible.

I stand by the Lindy effect: the longer something has been around, the longer it will survive.

Looking forward to Wasm Components to use a language from 1960 to its full potential. Then the goal is to have a WASM-native Lisp that can dynamically load any other WASM library, and I don't have to write yet another date/time library ever again nor care about junior engineers getting confused by crazy macros in my code.

(I have been suffering from the Lisp curse for 6 months and counting. Please send help.)

I sympathize, but if you're working for a company then somebody is either going to need to be able to read and understand your Lisp code, or the WASM it generates, in order to maintain it after you move on to some other company.

sph probably has a clause in there to own the code. A traveling salesman among a graph of corporate nodes. Their own tools, always honed; their own solutions, always at hand.

But owning the code doesnt mean removing the runtime after he "blessed" the company with his proprietary "help" and some poor underpaid sob now has to "only maintain" the work of the great master.

He'll code it in C or Java or Javascript and that'll be it thank you very much, so that we can maintain it forever.

Not sure what this subthread is worried about. I never said I was going antisocial and forcing Lisp upon my fellow coworkers.

I just said that, where I have complete control (i.e. on personal projects, or companies I bootstrap) I will choose Lisp because I wouldn't have to care for junior employees.

This forum is so preoccupied by only considering something that could be understood by a team of thousand average-skilled engineers, and only the easiest to understand languages matter, which is why all Lisp discussions devolves into mentioning how hard it is to adopt in large corporate teams.

The power of Lisp is inversely proportional to the headcount, and sometimes you're the only one working on a project, so you want max velocity and flexibility. Not everything has to be optimised for a FAANG development model.

Ah I misunderstood. You mentioned junior engineer and I interpreted the context as being in a company. But for personal projects, or any other ones with like-minded contributors, more power to you! I try to do all mine in Haskell fwiw, for similar reasons :).

That's a good point, the maintenance burden of vendor libs. It may not necessarily be closed source, but for other reasons--hard math, unfamiliar language, or domain depth--might as well be a proprietary blob.

However, re-reading sph's comment above and seeing their response below, this truth table may help fill in context for me (thinking aloud and all that):

  A: Running Lisp stuff for personal
     and/or own startup
  B: Embedding the libs as a B2B solution
  C: Maintenance overhead

  A  B  C   A & B & C
  T  T  T   SkyMarshal and xwolfi's comment
  T  T  F*  I agree there would be a cost
  T  F* T   sph using for own stuff or biz
  T  F* F   Same as above
  F* T  T   N/A
  F* T  F   N/A
  F* F  T   N/A
  F* F  F   N/A

The asterisk (*) above meaning we can abort there since it wouldn't make sense to proceed to the other values.

So then B is where things went one way or the other. And admittedly, I must have assumed B.

Nevertheless, I am cheering sph on :)

> How many date/time libraries have been written since the invention of UNIX?

Indeed. This service (especially the time zones) should be part of the standard operating system API.

What I mean is that I should be able to use TensorFlow, or a Ruby library or Qt6 from any other platform and language, without someone having to write bindings to it or porting it wholesale. We've been wasting thousands of man years just rewriting the same code everywhere, with the quality of implementation being dependent on how popular the language is. So you get the best libraries on Java, and smaller ecosystems never reach critical mass because there's no good libraries. This is unsustainable.

I really am hyped for WASM components.

Wasm is just java for people who weren't around for Java.

Unfortunately, making a new isa is the easy part. The hard part is defining the abi.

.net came closest. Wasm is focusing on the wrong problem.

The standard operating system API should provide abstractions for all general real-world objects the machine is expected to represent or interact with: dates, times, durations, recurrences, abstract time concepts like "the second Monday of every month except when it falls on a holiday", people, places, events, to-do lists...

These are things that businesses have to deal with every day. The fact that we don't have those things, standard, as a part of a programmer's toolkit is a great failure of language and OS designers.

https://web.archive.org/web/20180223094826/http://www.xent.c...

As a language designer and implementer, I have spent a lot of hours writing time&date code, and disagree that I'm to blame for them not being standard.

I've also released them with a Boost license, so anyone can use them for any purpose.

The JVM lets you program in whatever you want to and then use code written in whatever comfy language (Java, C/C++ via native interop, Kotlin, etc) or a Lisp dialect (Clojure). I frequently use Java and Clojure together in the same project to great satisfaction, but I know what you mean anyways. We need WASM to get around the web question. I just hope it can fulfill the promise you have set upon it.

We already had it 10 years ago with PNaCL but then politics happened.

I've been JVM languages sharing libraries, but is the WASM ABI going to be rich enough to support that?

We (sadly) rarely see languages that target binary (whether x86 or ARM or just LLVM) sharing libraries. Partly it's because the .so and .dll interfaces only support the C ABI, and the absence of a rich well-defined OOP abstraction.

But, yea, I'm not totally sure if a low level WASM ABI would lead to a rich ecosystem of multi-language libraries, unless the WASM ABI is at a high enough level that it defines higher-order functions, classes, etc.

WASM has a low level ABI, but that's the thing WASM components are trying to rectify. Components is a proposed spec that builds on top of WASM.

Imagine a .so library with a native concept of high level objects that can be understood and passed between libraries written in different languages.

I have been doing that in JVM and CLR ecosystems for about 20 years now, somehow it is a novelty in WebAssembly, latest trend is even to re-invent application servers.

I haven't been keeping up so maybe someone who has can comment but doesn't WebAssembly have a similar problematic relationship to tail calls and deep recursion as the JVM does?

When collaborating by the mechanisms you describe is not a priority, it is reasonably sound engineering practice to ignore those mechanisms.

Though there are tradeoffs (and assuming 10x programmers exist) a ten-ex programmer can replace a team of ten one-ex programmers and eliminate mechanisms facilitating intra-team efficiency from the process.

Anyway, to me the Tower of Babel is a metaphor of collective action...building construction requires teamwork. Icarus is better for lone wolves.

exactly, in the author's vision the individual matters more than the group ; I couldn't disagree more with this.

> In every other language I have to think about whether to write op(arg1, arg2 ... argn) or arg1.op(arg2, ... argn) or "arg1 op arg2"

In D these are interchangeable: op(arg1, arg2 ... argn) or arg1.op(arg2, ... argn) which is called Universal Function Call Syntax. It works very nicely. It enables a Left to Right reading of stacked function calls:

a(b(c(d(1)))) becomes 1.d.c.b.a() which is a lot more readable.

If d takes two args, is there an inverted syntax as in “(1, 2).d.c.b.a” or must you do d(1, 2).c.b.a or something else?

`1.d(2).c.b.a` and `d(1,2).c.b.a` both work

I wasn't aware of this:

  Clozure Associates is no longer a going concern, and the volunteer effort to port CCL to the M1 appears to have stalled.

Sad news if true (and I have no reason to doubt it). I had kind of assumed that an M1 version of CCL would appear as a matter of course.

AFAIK the task is looking for volunteers (who have the necessary low-level knowledge) and/or funding. The mailing list at openmcl-devel@clozure.com has some threads related to it - see https://lists.clozure.com/pipermail/openmcl-devel/

> I have no reason to doubt it

Point your browser at clozure.com if you want confirmation. (There's a reason I didn't include that link in the post.)

As a Lisp neophyte, I've been watching the Clozure CL situation with some interest -- I really want to give it a try, especially the macOS/Cocoa bindings, but my only Mac at the moment is an M1. A couple of specific questions I've been wondering about:

1) Does the port require a rewrite of the compiler? It seems like it was only ever released for 32-bit ARM, not even generic ARM64? How much of that work can carry over to an Apple Silicon port?

2) Will the announced rewrite of Core Foundation in Swift and/or the transition to SwiftUI put the CCL Cocoa bindings at risk of obsolescence anyway?

My current job doesn't demand much coding out of me, so I feel lucky that I'm able to focus my personal learning efforts on interactive development environments like Lisp and Smalltalk, and grateful that solid free implementations are available in any event. If somebody does start up Apple Silicon CCL crowd-funding, I'd be in for at least $100.

> It is super-simple to customize Lisp to suit your personal tastes, and so everyone does, and so you end up with a fragmented ecosystem of little sub-languages, not all of which (to put it mildly) are particularly well designed.

This happens with any language that has macros, and why I have strongly resisted adding macros to the D language.

I can offer a counterexample: the Elixir community, despite having a (very good) macro facility at our fingertips, is largely free of macro-mania as you describe it.

Culturally, most folks use macros to import a few things into the caller namespace, or to wrap functions somehow (e.g., add timing metrics to every function in a module), and virtually all the functionality should reside in regular functions in regular modules, rather than having the macro inject a bunch of code that doesn't exist in a testable form.

I'm not sure why we have it so good. Dreams can come true I guess. But I bet a community of low-level systems programmers might not be so well behaved :D

> I haven't seen your code, so take this next comment as you see fit:

Every person who advocates macros argues that they are not susceptible to the siren of creating their own incomprehensible and undocumented language. Every one who's code I had to work with, I'd point to it and say "what about that!"

Even I was lured in by the Lorelei.

To be fair same applies when the languages have template metaprogramming with enough knobs, and nowadays it seems the answer to many inference issues in D is "write a template".

Plus at least macros are much easier to debug than current infrastructure for D's string based mixins.

For me this is the year of Lisp. Julia and R for warm up, and then Common-Lisp.

R7-Large Schemes do not seem to me ready (apart from Racket which is not R7-Large). Look the beautiful Handbook of Julia. R is full of documentation.

Common Lisp is steadily improving.

Schemes need something similar.

Ou of curiosity , what is the main problem you see with Gauche Scheme?

I have started spending most of my time programming again after a long hiatus where I was primarily focused on business. One of the things I am picking up in the process is learning Common Lisp. I am tracking my progress here[1]

Not for any particular reason, but because I think Lisp Macro/Metaprogramming is a fundamentally different thing that can only help expand my mind.

1: https://github.com/keeb/learning-lisp

Or contaminate it with rot. Meta programming is an enormous risk in software, at least if the software has to last more than 20 years and maintained by more than 10 people.

Every exemple I've seen, be it annotation aspects, script side loaders, preprocessing macros, "configuration" parsers etc has been a complete mistake vs writing what you want clearly, generate nothing or destroy the generator to let people touch the concrete thing rather than navigate a maze of indirection.

I hope you ll end up expanding you mind with the conclusion that we should never program stuff that output other programs if what we want to program is the output.

Ah, a natural encourager, I see.

I am familiar with code generators (in general) and macros from other programming languages like C. And "no"/"lo" code solutions which behind the scenes are code generators.

I have the same bias as you at the outset of this endeavor. But I also like to learn. Maybe there will be an aspect that will grow on me.

Author here. AMA.

What are you using Lisp for lately?

I'm not doing a lot of coding nowadays. But for the past five years or so I have had a consulting gig helping to maintain and extend a tool for VLSI chip design written in Common Lisp. Unfortunately, I can't say anything more about that at the moment.

Before that I wrote an e-commerce system that I used to run my company, Spark Innovations. It used Stripe for billing, Easypost for shipping, and Hunchentoot as the web server. I also wrote a spam filter that is still in production on my personal web server.

What does the Common Lisp consulting market look like?

If one puts themselves out there (build reputation on the Internet) as a CL programmer, are they likely to land consulting gigs?

Well, Clozure Associates was in that line of work and they are out of business. So that's a data point.

There are few job announces, but positions are found thanks to one's reputation and online presence. It definitely happens, that's a data point.

Thanks for writing this followup, Ron.

This paragraph really resonated with me:

> All this is a reflection of the so-called Lisp curse, the fundamental problem with Lisp -- its great strength is simultaneously its great weakness. It is super-simple to customize Lisp to suit your personal tastes, and so everyone does, and so you end up with a fragmented ecosystem of little sub-languages, not all of which (to put it mildly) are particularly well designed.

I have a career-long love and admiration for Lisp, but I've never used it for "real" things because of this precise reason. I wanted my code to be understandable and modifiable by others who are already familiar with the programming language I'm using. With Lisp, it's almost guaranteed that any sufficiently large program will be written in its own dialect no one but the author knows.

nah, disagree, how do you come to the conclusion it is "almost guaranteed" without having tried it for "real" things? (no mean for ad-hominem, I'd like to read details)

It's also a great language where you can adapt anything, any library, to your needs, bypass its shortcomings, etc.

What is the ecosystem of fragmented sub-languages? I don't know any in CL. (ergolib isn't a crazy sub-language either, it's just a library with helper functions and macros). We have solid libraries like Serapeum, supporting all existing implementations. Not sub-languages.

I agree in the weakness that it's too easy to fix a library and not contribute the fix back. Individuals and companies are guilty here. I wish I saw more companies contributing to the open-source libs they use. But other persons send great contributions. It's a small world, so we easily see the good and the ugly.

I believe the OP's ergolib provides an example. From https://github.com/rongarret/ergolib/blob/master/core/bindin..., the examples show code like:

   ;;; (bb
   ;;;  x 1
   ;;;  :db (y z) (foo)
   ;;;  :mv (a b c) (bar)
   ;;;  (do-something)
   ;;;  :with open-file f "foo"
   ;;;  (do-something-else))

And the code this replaces. This seems to me hard to understand without first digging into ergolib and grokking what bb means. It's just one simple example.

I forget now where it was, but there's a famous Lisp quote which goes something like (paraphrased from poor memory): in Lisp you don't just solve problem X, you first grow a new programming language designed for solving X-like problems and then simply solve X with that.

This has its charms, for sure. But it makes the code harder to understand for folks not familiar with your "language designed for solving X-like problems"

It's a little helper, so as in any language we have to check the doc and an example. Just like how to use a rest api, or a high level function that does a lot of stuff under the hood. Once you know it's a binding macro it's even easy to guess. I don't find it difficult, it is similar to many similar macros out there.

so yeah, I don't find the example extraordinary enough to speak about sub-languages. Plus, as another comment said very well, the most important is domain knowledge to understand the code. And the quote is right but we can also use Lisp like a dumb language and get shit done!

You seem to imply in TFA that you believe Clozure Common Lisp specifically is the best Lisp distribution. (Please correct me if I’m misinterpreting your stance here.)

What about CCL makes you describe it as a “technological marvel”? What advantages have you gotten from CCL that can’t be obtained from other Lisp distributions?

I’m asking as someone curious about Lisps and who always thought that the various Lisp distributions were roughly equivalent — so I’m sincerely curious, not doubting you.

> What about CCL makes you describe it as a “technological marvel”?

The fact that it started out its life as 1) a full CL implementation 2) with an IDE that is still usable today (you can run the original CCL on an emulator and it is till competitive) which 3) included an interface builder that was super easy to used, in fact easier than anything that exists today AFAICT, and 4) ran on a machine with 1MB of RAM and an 800k floppy disk.

The other thing that CCL had/has was a blazing fast compiler. It was/is fast enough that it was/is practical for me in many cases to recompile my code from scratch every time I run it.

When reading your section on Python and ending statements with "pass", and after not finding other references to it in your blog, I have to ask about Ruby. It seems like it would check many boxes (highly extensible, no significant whitespace, begin-end blocks, quite lispy...). If you've played with Ruby, what drawbacks have you found?

What are the major advantages you found in Python and in what cases do they exceed those in CL (aside of e.g. the type system you wrote about in 2008)?

I've never written any Ruby code but I did look into it once. What turned me off was its extreme approach to object orientedness. If I want to add two numbers I want to be able to write (+ x y) or x+y. I don't want to have to write x.add(y). That's weird syntax in service of an ideology that I don't subscribe to.

There were a couple of other weird things that turned me off but I don't offhand remember what they were.

This doesn't mean I think Ruby is a bad language, just that it had some in-your-face features that didn't resonate with me.

? Are you sure you're not confusing Ruby with something else?

x+y works just fine in Ruby.

Yeah, I was misremembering the details. But here's an example of the sort of thing that I don't like.

If I want to convert a number to a string, I do:

    x.to_s

which would lead me to think that to take the square root of a number I would do:

    x.sqrt

but I don't. It's:

    Math.sqrt(x)

So for every operation I have to remember whether it is invoked by x.op or op(x). In Lisp I don't have this cognitive load. It is always (op x).

That is not unique to Ruby. What is unique to Ruby is that the x.op style is applicable to literals. So you get weird things like 3.14159265.to_s which just looks bizarre to me.

Although the early versions weren't so, after the replacement of the base object model (see new style classes), in Python everything is an object as well, and many seem to miss that.

    print(type(1))
    print(dir(1))

produces,

    <class 'int'>
    ['__abs__', '__add__', '__and__', '__bool__', '__ceil__', '__class__', '__delattr__', '__dir__', '__divmod__', '__doc__', '__eq__', '__float__', '__floor__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__getnewargs__', '__gt__', '__hash__', '__index__', '__init__', '__init_subclass__', '__int__', '__invert__', '__le__', '__lshift__', '__lt__', '__mod__', '__mul__', '__ne__', '__neg__', '__new__', '__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__', '__ror__', '__round__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__', '__sizeof__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__trunc__', '__xor__', 'as_integer_ratio', 'bit_length', 'conjugate', 'denominator', 'from_bytes', 'imag', 'numerator', 'real', 'to_bytes']

That's true, but actually using a literal as a object is forbidden by the syntax.

    Python 3.8.9 (default, Mar 30 2022, 13:51:16) 
    [Clang 13.1.6 (clang-1316.0.21.2.3)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> 1.__abs__
      File "<stdin>", line 1
        1.__abs__
          ^
    SyntaxError: invalid syntax
    >>>

Ruby allows this:

    irb(main):001:0> 1.2.to_s
    => "1.2"

I think that's weird.

Only because those are special methods only allowed to be directly called by the runtime, doesn't change that the language model is actuall OOP based.

You could have used one of those that is allowed instead, example:

    >>> (3.2).is_integer()
    False

The parentheses are needed due to parsing precendence.

So here is your Ruby like example in Python.

Fair enough. I guess I should probably take another look at Ruby.

Thanks for answering my earlier question. Here's hoping you catch another one!

In a lot of my clojure scripts, I forget the shape of the data, and so to help future me, I've use tricks like pre/post constraints, defstructs, schemas, etc. In my experience, TypeScript has the best typing ergonomics (block-local, partial interfaces!). But since I prefer lisp language and interaction ergonomics, I'm always on the lookout for new developments, like https://github.com/coalton-lang/coalton, which has pushed me to start playing with sbcl.

Maybe CCL has already solved this, but I'm not versed in CL, so what methods do you find most effective for passing down knowledge of constraints and data shape to your future selves?

That's a really good question. No, CCL has not solved this. CCL is just a CL implementation with a really fast compiler and a nice IDE, but it doesn't really extend the language. Ergolib, which has nothing to do with CCL (it works on most CL implementations) extends the language, but it doesn't really deal with the "shape" of data. I haven't actually heard that term before, but I presume it's referring to data type constraints. If that assumption is correct, then the answer to your question is that this is a very hard problem. The solution that is currently in ergolib is not even half-baked. I had a vision of evolving ergolib into a fully-fledged new programming language which would be called Ergo and include a state-of-the-art type inferencer, but I have yet to get around to implementing it. It turns out that dealing with the vaguaries of external data sources is MUCH harder than dealing with inconsistencies within my own code. I hardly ever get type errors in my code, despite the fact that I have minimal support for typing, and even what little there is I hardly ever use. The closest I get is to be fairly anal about creating classes for everything and avoiding punning cons cells like the plague. Punning cons cells is a vert common practice in idiomatic Lisp code, but I think it's a Really Bad Idea (tm). So, for example, I really hate association lists. Ergolib has support for abstract associative maps, which allow you to use an alist as the underlying implementation if you really want to, but the default is a hash table, and I find I hardly ever need to change that.

The hard stuff for me is dealing with external interfaces, which almost always comes down to parsing. This is one reason I love Lisp -- once you get your data into the form of an S-expression, a lot of the heavy lifting has been done and everything just becomes much easier. So any time I have to deal with json, or xml, or csv, or whatever, the first thing I do is parse it to turn into a sexpr, and that almost always turns out to be the hard part of whatever I'm doing.

Did that answer your question?

Yea, for the most part.

The problems you describe with external data are supposed to be clojure's forte, but clojure also makes cheap footguns because maps fill most of the use cases for alists / hash tables, and implicit nil punning is everywhere. Once data comes through the wire and is turned into a map, if there isn't an enforceable structure in code, I have to run the program to check the data's shape. Hardly new in dynamic languages, and there are many solutions (e.g. spec, malli, schema, just to name 3 for clojure), but I haven't seen one matching the ergonomics that TypeScript 3+ offers today.

Thanks again! Always enjoy insight from experienced lispers.

> even inspiring some people to plagiarize it

That's not what plagiarizing means. Plagiarizing means to pass off as one's own. The website only copied the essay, without trying to pass it off as their own. They also linked to the original.

What they did is maybe a pirate copy?

Bb looks suspiciously like an identity monad comprehension :)

I was thinking similarly, reminds me of `do` notation in Haskell. Which is great, Haskell and Lisp are my two favorite languages :)

Is a do notation same as chain of responsibility GoF pattern?

Not exactly (although maybe you could implement that pattern with the right choice of monad).

    do
      x <- [1, 2, 3]
      y <- [4, 5, 6]
      return (x + y)
    -- [5, 6, 7,  6, 7, 8,  7, 8, 9]

Here we have a series of assignments followed by an expression computing a result. You can't tell directly, but this is using do-notation in the List monad -- an assignment `x <- [...]` picks a value from the list, then continues. In fact, this code is run once for every possible choice from the given lists, and all of the results are aggregated into a list.

The earlier commenter referred to the Identity monad, which would look like:

    do
      x <- 3
      y <- 5
      return (x + y)
    -- 8

No special lists this time -- what you put on the right is what gets stored in the variable on the left. This is what you're used to in any imperative language.

I suspect `bb` is a little more special, because Ron mentioned it has similarities to LOOP. I'd guess it supports recursive definitions somehow, but it's not clear what the difference actually is.

Curious to know how to implement Do pattern in an OOP language consistently.

Good question. "Do notation" is sometimes referred to as "the programmable semicolon", because how you go from one statement to the next depends on which monad you're in. (In the List example, the assignment `x <- [1, 2, 3]` forked execution three ways before proceeding to the next assignment.)

In Java, the semicolon is pretty much fixed; you can't change the relationship between statements in any meaningful way. So you have to play tricks. You can implement the List example in a crafty but leaky way:

    do(() -> {
      var x = choose(1, 2, 3);
      var y = choose(4, 5, 6);
      return x + y;
    });

When `choose` is called for the first time, it remembers which choice it returns to you. When you `return` from the block, your result is stored, and then the function is re-executed -- except this time, `choose` will now return a different value (the next one). You can implement this in a rather small handful of lines, but it's very tight and clever code, and not at all obvious how it works at first glance.

There's another way to do this, with chained `flatMap` calls, like:

    list(1, 2, 3).flatMap(x ->
      list(4, 5, 6).flatMap(y ->
        list(x + y)));

...but that's so different from "normal" use of a language like Java that readers and writers alike are likely to balk to some extent.

'do' in Haskell is like PROGN right?

I guess in a OOP langague like C++ it would be something like:

foo1(); foo2(); return foo3();

progn has a fixed meaning in how to sequence sub expressions

do, afaik, just flips the written expression and use actual bind functions depending on the typeclass at point

Do isn't a pattern, its a notation for monad operations. Any pattern implementable in it can be reproduced in any language where there are objects which support monad operations by using those operations directly. Some languages (e.g., Scala and its comprehensions, which use monad operations rather than imperative operations over collections the way Python does) also have a do-notation-like syntax sugar, as well.

Monads and 'do' and stuff are just a form of abstract control structure. They're hiding information about the actual nature of a computation, making it look like a bit of imperative code, but in the background they could be doing anything. To be honest, despite the mystique, this isn't so different to virtual calls in an OO language, or just having objects hide their internals. Can you create an algorithm accepting the same steps but performing some magic in the background, around and in between every step? You have decorators, iterators, visitors, all that sort of stuff, so the answer is probably yes. Monads happen to be a very simple way to do this, and so you end up with entire ecosystems and fundamental ways of thinking built on that pattern.

i wonder what's the difference between CoF and a sequence of conditionals (except maybe being reified at the application layer and potentially adjusted without changing source, a bit like a macro)

Is the source code for any of these really interesting JPL lisp programs available in a repo somewhere? e.g. I'd love to look through something like Eggemeyer's Plan-IT system referenced in the article.

Been almost 15 years since I was at the Lab but I don’t recall Lisp being at all popular. And I started there as a Perl & PHP developer before transitioning into Java.

The original article was written about 20 years ago and he says that by 1999 lisp was already falling out of favor (if it had ever been in favor). So, 15 years ago (2008?) would've been almost a decade after that.

I started at JPL in 2002 and worked with dozens of groups. Including many folks who had been at JPL since the 70s. No one ever talked about Lisp. I remember helping a former team member who had transitioned to ground systems port an old Mars Rovers Motif/Xt app to Qt (2 or 3?) on Linux. Blew his mind how much performant Qt was on the same hardware and with simpler code. He only ever mentioned being forced to use Ada for a few years and then being forced to port the code to C/C++ a budget cycle later. Probably around 2006. He had been at the lab since the 80s.

But to be fair to the TFA’s author, prior to about 2007 when we got our first CIO, we could literally use almost any technology as long as it wasn’t illegal and wouldn’t get NASA, JPL, or CalTech in trouble. I ran databases & web sites with public IPs, ran myPHPAdmin, phpNuke, QTSS hosts and bunch of other crazy stuff with public IPs.

Can you say how you made AdWords' billing system impossible to hack by employees that had root access?

Reply on news.ycombinator.com