How the Curse of Lisp impacts your business (even if you don't use Lisp)
medium.com
medium.com
And a blogger stumbled upon this piece, and decided there was a widespread problem with people rubbing oranges on their heads, and, further, with the general problem of the author's entire nationality of people, and that this urgently needed to be exposed to the public, in a patient explanatory blog post.
Seeing this blog post, others, realizing that there's been a misunderstanding, but knowing more about that nation (and/or fruits, and/or hair care), than about PR damage control, proceeded to respond, with a flood of various corrections, all over the map.
For example, "I prefer fresh-squeezed lemon, so long as you're careful to avoid traumatic brain injury." And "Those people are not that bad; the many stories of kidnappings are often exaggerations of muggings." In all seriousness.
And anyone observing is thinking "I'm even more confused than before, but those people sure are defensive about whatever the heck ungodly things they're doing with fruits. It might be a nice place to visit, for a day trip, but I wouldn't want to live with those crazy fanatics."
Again I would never (and didn’t) draw an issue with an entire nationality of people squeezing oranges in their hair, particularly being an orange-squeezer myself. If anything, it would be better if almost everyone would be squeezing citrus in their hair every day, as we would more readily be able to point to the side effects associated with doing so without it seeming like an affront to one group of people.
True, this analogy is confusing, but it’s also great that you created it.
One of the key skills of a a senior dev is to be able to judge the tradeoff between rolling your own and borrowing someone else's. In your life as a dev you'll be making this choice all the time. The pros and cons are generic: time spent, potential bugs, security, free upgrades, specific vs general solutions, external reliance, and so on. But you will exercise this judgement constantly.
Also not being a LISP guy, is it really so common to roll your own everything? Would you write your own JSON parser, is it that easy in LISP?
As someone who has dabbled in lisp: IME, everything is easier - enough so that I more-often _consider_ inventing something that solves my subset of the problem.
I suspect this contributes to the poor library availability situation (I understand that's improved markedly in recent years, but still).
The latest video I've found about it, and Kris' interviews are usually great: https://www.youtube.com/watch?v=fytGL8vzGeQ
No, that's more of a Haskell thing to do (you start with your own parser combinator, of course).
Common lispers write their own testing framework - I guess that's Peter Seibel's fault ;)
https://sabracrolleton.github.io/testing-framework
https://gigamonkeys.com/book/practical-building-a-unit-test-...
I only dally with Lisp(s) but probably not, I'd reach for a library in quicklisp.
However, in Janet (https://janet-lang.org/) there's a PEG parser and I wrote a JSON parser for fun in 134 lines that passes most of the test suite by Nicolas Seriot.
So I'd say it's reasonably easy for a better-skilled programmer than I am.
Well, there are a lot of them. Here's a link to a guide someone wrote to compare a bunch of them: https://sabracrolleton.github.io/json-review.html
That said, Lisp developers learn to fight their instinct and not redefine the language all the time, just as Haskell developers learn to fight their instinct and not make complex types all the time, and C++ developers learn to fight their instinct and not use STL all the time. But those take some experience.
Ironically this blog post isn’t even targeted at developers but more management types, but it wouldn’t be life without being spectacularly wrong on HN and raked over the coals on it. In short though I agree it’s not like you can just use off the shelf for everything or that that is even a good idea to try as that has its own set of problems.
But that said, obviously Clojure with deps.edn exists. Looking at Quicklisp's package manager, it looks like the packages are updated on at least a year's interval? That seems quite slow if you are looking to socialize your changes, use someone else's recent bug fix, or to defragment a project.
You could have asked in the community for opinions.
> This is because Lisp makes it very easy to write your own libraries from scratch;
That's not generally true. Some libraries are maintained for decades, since it would be difficult to reproduce them.
The reasons for writing new libraries, instead of an existing one are more like:
* disagreement with coding styles
* very different underlying platform
* need for a different license
* lack of maintainers for code written decades ago
> people who use Lisp tend to do that instead of reusing third-party dependencies.
I don't think that's the case. There are widely used libraries for a bunch of things and having a stable and standardized language makes it actually easier to write platform independent code and enables code sharing.
There are reasons why people write their own libraries, often because existing libraries were written for different platforms (especially the operating system).
In the free software area of Common Lisp, a system like SBCL is popular because of the network effect in a small niche of Lisp programs. Libraries tend to be better supported in SBCL -> more use for SBCL.
True, and I was focused on making a non-Lisp point so I didn't and apologize for that. I do find the feedback valuable although I wonder how to reconcile what's been posted here with what I've heard from other Lisp coders.
Trying not to beat a dead horse, but I recognize there may be many reasons people come up with for writing a new library instead of reusing one. My point is that it has a business impact to do it that way, because unless that library is open-sourced, the next time it happens it will be an overall loss for the industry due to solving the same problem again and again in slightly different ways.
One of the reason to use Lisp was to leapfrog the competition. See for example: https://paulgraham.com/avg.html Applications were written in novel areas by small teams. The commercial Lisp vendors had their development environments bought and be used by these teams, but they could not post success stories, because they were not allowed to mention the customer and its application. Here often substantial and complex software had been written, at the core of some new business venture. The reason this software code could not be used by, then was not that it easy to copy, but that the developers did not want to give their competitive advantage away.
For Lisp libraries -> if any library domain gets complex, there will be only a few ones which actually get used, because it was non-trivial to write. People might try to write simpler or new ones, but the better ones are hard to replicate and Lisp developers will tend to use those. For example the Lisp cookbook recommends one Regular Expression library: https://lispcookbook.github.io/cl-cookbook/regexp.html That's what most developer tend to use - it is widely ported, written in good style, well tested, ...
A strange thing in the Common Lisp world is that new developers stumble of old libraries. They expect libraries with lots of bug fixes and a long maintenance log. But the library hasn't much touched in since some years.
They detect then, that this library is "done". It has reached a state where not much needs to be done. This is also an effect of Common Lisp, where the language is mostly stable for several decades and code can be used for a long time without the need for a new attempt.
It feels like Lisp and Open Source are old enough this should not be a relevant concern anymore. Many newer Lisps exist that have been OSS their whole lives, including the community infrastructure. What remains are the inherent properties of the language instead of any legal artifacts.
That is a great set of responses to the Lisp Curse article. Again I will note that my points are not about Lisp as I'm just taking that article as granted (which it turns out it's not, but still). The point about the pareto principle is also language-agnostic, and an interesting point. It's sort of a proof that the 80% problem is going to persist to some extent even if you do use many dependencies. (Note not to the fullest extent — the 80% problem can at least be mitigated by everyone not reimplementing everything every time.)
I think it used to be closer to monthly. It has slowed down a lot recently. I've seen talk on IRC about ideas on how to improve the bus factor for that situation.
Also, there are some alternatives to Quicklisp that exist, such as ocicl, which seems to be gaining in mindshare. I haven't used it personally (yet), but so far I've only seen positive feedback from people that have tried it.
Qlot can and does install from github, gitlab, ultralisp, etc.
Ocicl, if we are not talking about your Mother's Common Lisp ;)
...most of which are cargo-culting carbon-copies of "gurus" and "authorities" (and "Uncle Bob"). No critical thought in sight, but a lot of mental gymnastics to justify conclusions that don't follow from whatever was written above.
The massive influx of poorly prepared junior programmers in the last decade doesn't help. Somehow, beginner programmers tend to overgeneralize from too narrow a perspective, leading to fallacious but firmly held beliefs. This is nothing new and happens to most beginners; the problem is that we're dealing with a massive wave of beginners that seems to be still gaining momentum instead of subsiding. One of the effects is that the experienced, mature programmers' voices are drowned in the noise, save for a few "gurus" who are often better writers than programmers.
The whole field shunning its own history doesn't help either. The culture of moving fast created a misperception that things older than half a year have no value. Similarly, agile and co. made it so we live in perpetual "now," rarely considering the long-term consequences of our work. This leads to resume-driven development and contributes to enshitification of everything, yet nobody seems to realize there even is a problem, much less have any intention of fixing it.
Honestly, I'm struggling to find my place in the field after two decades of coding.
It's been more than a decade. Take a time machine back to 2014, and we're complaining about religious zealotry among followers of the GoF book, debating whether SOLID means anything useful at all, and enjoying spicy rants like "Rails is a Ghetto." I think the problem is not so much that junior programmers are generally ill-prepared to do professional work -- that much is to be expected -- but that the industry continues to be unable or unwilling to evaluate competence.
With the advent of Quicklisp, many repos on public services like GitHub, etc., it is now easy to find and use good libraries for Common Lisp.
Of course there is a network effect: a language like Python is much more popular so there are many more libraries available. That said Common Lisp, Racket, and other Lisps all have good ecosystems and are fun and productive to use.
The early .com companies on the Internet were all deeply involved in Lisp/AI companies: Symbolics, BBN, Thinking Machines, MCC, Xerox, SRI, ...). Plus a bunch of universities (MIT had hundreds of networked machines running some Lisp, including Lisp Machines).
Many of the R&D groups were on the Internet. Larger amounts of code were for example shared via tape. For example the Symbolics User Group already shared code. The CMU CL wrote their Lisp implementation as public domain and shared all their implementation code: Lisp, editor, UI, ...
Lots of old code from that time is all collected in the CMU AI Archive, which was established early 90s: https://www.cs.cmu.edu/Groups/AI/0.html
At that time a lot of new stuff was developed, like CLX, the X11 interface for Common Lisp. Every Lisper on X11 was using it. TI developed CLUE and CLIO on top of CLX. LispWorks developed their gui toolkit on top of CLUE and CLX.
Sure, there were lone developers but Lisp in the 80s was widely used in universities and R&D. All were networked. MCC had 200 Lisp Machines in their network, IIRC. MIT more. SRI similar. Xerox. BBN even developed their own Lisp OS + machine. Symbolics had support for DECNET, TCP/IP, IBM Mainframes, ...
The Common Lisp standard, developed in the 1980s was probably the first developed with heavy e-mail support.
There was a lot of stuff shared.
Is there any actual evidence for this?
> When you have an internally-maintained piece of code, it does not benefit from the incoming bugfixes that an external dependency might enjoy.
But, even this one is not always good. Your internal code might not have the bugs that would affect your program, might be less efficient due to doing things that are not relevant to your program, and you might be able to fix the bug more easily in your own program than you would with an external dependency, especially if they do not fix it in time. There is also vulnerabilities, and other stuff. Furthermore, the external dependencies might make the program larger than it should be. There are many good reasons to avoid external dependencies.
Yes, you can point to xz-utils as an example of the ineffectiveness of groups at catching attacks, but it’s also famous because it’s exceptionally bad and probably not normal.
Another alternative is that if the dependency is only one or two files, you can just make your own copy of them and use those instead, and maintain them by yourself, reviewing any changes to the official version and applying them to your own if you consider them necessary; this might also allow you to fix bugs that haven't been corrected and to omit features that you do not need.
Another reason you might wish to avoid external dependencies is if they cannot be packaged together with the software; if they come from a different source, then there is the possibility that the external source may be no longer available (which might or might not be malicious; again, both kinds can and do happen). (If they are not very large and only one or two files, then you can just include it with your software.)
If your software uses external dependencies, you might also specify exact versions by both version numbers and cryptographic hashes. The use of cryptographic hashes will mitigate some types of attacks as well as to potentially allow finding other copies if the source that you got it from is no longer available.
Another issue, depending on the kind of dependencies, may be interfaces with other software (possibly including the operating system), in which case there may potentially be multiple possible implementations, and the system administrator may wish to install their own.
In the package manager for operating system design that I had considered, this is also an issue that I had considered. A package can have multiple identifiers of any kinds. Additionally, there are other features of the operating system which can help many things. Hopefully, these things will be able avoid many kind of problems.
Is it really hard to replace engineers when you have a lot of dependencies? A lot of dependencies means a lot of things you have to learn. Let's consider the case where every line of code invokes a different dependency. That's clearly insane and not the ideal case, even if they are all well-known dependencies. Everyone on the team will be constantly chasing down documentation for the dependencies they don't know.
The optimal point is somewhere in the middle. I just am pushing for more dependencies because I see it being too far in the "no dependencies" direction.
What? Lisp makes implementing libraries no easier or harder than most other languages. Even the original author afair criticised his own article about the lisp curse some time ago.
Because we are few in number, it's hard to defend against, and harmful.
It is not easy to write your own libraries in Lisp. You're not going to write your own cryptography, web serving, database access, regular expressions and whatnot, and if so, not to the level of quality of available libraries.
People developing applications in Lisp use libraries.
My point is something that applies to every (other) language, and it's a point about fragmentation. If you use packages in your favorite language, that's great, and not who I'm talking to.
Who cares what is the intent or not if all that remains is the effect at the end of the day.
Honestly, having a curse named after you makes you cool in my book, at least when you adopt the "no publicity is bad publicity" outlook. Especially when the core critique is that the curse is that the language is too powerful.
> [Your article] (Lisps) is essentially immune to the network effect.
You're making a strong inverse argument of "nobody got fired for using Java" against Lisp specifically.
I'm talking about actual economic harm here, though. Decision makers in organizations being influenced by made up articles that trash-talk Lisp people, resulting in reduced opportunities in that area.