Lisp has had decades to break out of its niche if it delivered a really advantageous solution, but somehow that never happened.
I think a huge part of it is that it is not immediately obvious that one needs what Lisp offers, and by the time the system has grown to the extent that the need is obvious, it has also grown to the extent that one no longer sees the fores for the trees. One doesn’t think ‘oh man, I need garbage collection’; one thinks, ‘oh man, I need to manage malloc and free better!’. One doesn’t think, ‘oh man, dynamic scope would really fit this problem well’; one thinks ‘oh man, I need dependency injection.’ Peter Norvig famously noted that 16 of the original 23 design patterns were invisible or simpler in dynamic languages such as Lisp†. Heck, there was a time when one couldn’t rely on recursion, or even conditionals! But the programmer who has managed to get stuff done without recursion, or without conditionals, or without macros doesn’t really see the point. He’s even worried: those things may add too much expressivity to the language. Why, folks could write unmaintainable code with them!
Of course, folks write unmaintainable code without them, too …
Anyway, I think a huge issue is one of education and experience. Ours is a massively growing field. The vast majority of folks are juniors, and don’t know any better; a portion of their education was miseducation. The seniors often have one year of experience, twenty times (rather than twenty years of experience). Objective standards are rare to nonexistent. Norms and standards are absent.
But yeah, when I’m working on a large project in a language other than Lisp, I often think, ‘man, this would be so much easier in Lisp!’ or even ‘man, this would be practical in Lisp!’ (because anything is possible in a Turing-complete language …).
One no longer sees the forest for the fire.
Also, don’t forget that Lisp machines were once the most coveted development machines on the market. But Symbolics had to develop not only the language and IDE, but also the OS, the hardware, the microcode, and everything else all at once. It’s pretty telling that they soon began running Unix (on a separate processor) and then their next product was an add–in card for an Apple Macintosh II containing a Lisp processor ASIC. By then the C++ hype train was gathering steam and the AI winter had begun. Symbolics didn’t survive, and their direct competitor LMI had even less chance. So it’s not that Lisp offers no advantages, it’s just that market conditions killed off the companies that were offering it. Note that these market conditions were created by advertising and shifting public perception.
I thus return to my thesis, which is that the market success of a language has little, if anything, to do with the advantages of the language. Instead marketing and advertising rule the day.
This was forty years ago. Doing LISP advocacy like this just makes people sound like they're that Japanese guy who refused to surrender until the 1970s. The world has moved on; there have been other opportunities; and LISP has not won them either.
I would say that language popularity is highly correlated with the actual usefulness of the language. But "actual usefulness" covers far more than the "intrinsic qualities" of the language. It also covers the scope and quality of the standard libraries, the third-party libraries, the available tools like compilers, IDEs, and debuggers (which may be third-party), available documentation and training, and people available to hire who know the language. Of those items I listed, the only parts that could be considered "intrinsic" are the standard libraries and the tooling that comes with the language by default.
And with that the edit-and-continue paradigm loses much of its value. If you have to commit changes to a shared source file anyhow then you'll be not much worse off with debugging a core dump.
But don’t forget that in Common Lisp, you can redefine classes at run time as well as functions. All existing instances of the class will be updated to the new definition, and you can provide the code that decides how the old fields are translated into the new ones if the default behavior is insufficient. Good luck doing that in C or C++.
My favorite story involved a race condition that was discovered in the code running on a satellite, after it had been launched. The software on the satellite was mostly written in Common Lisp (there was a C component as well), so they opened a connection to the satellite, started the REPL, debugged the problem, and uploaded replacement code (which obviously added a lock or something) to the satellite all through that same REPL. While the satellite was a hundred million miles away from Earth, and while it kept performing it’s other duties. You can’t do that on a system which merely dumps core any time something unexpected happens.
https://flownet.com/gat/jpl-lisp.html
> During that time we were able to debug and fix a race condition that had not shown up during ground testing. (Debugging a program running on a $100M piece of hardware that is 100 million miles away is an interesting experience. Having a read-eval-print loop running on the spacecraft proved invaluable in finding and fixing the problem.
https://www.jetbrains.com/help/idea/altering-the-program-s-e...
But you can't easily collaborate with that style of development.
The REPL (or scratch buffer) is typically used for testing/observing. Not the actual source code development. Although it is possible to never write your source code to a file if you're just playing around with a toy experiment.
Says who?
I'm not a huge fan of lisp, but I do like the language. Unfortunately, it's way too hard on 90% of the developers. They need some more structure so they can think about one thing at a time, which is why C-like syntax won in the end.
All the best developers I know are into Lisp or Haskell (or both). They can crank out ridiculous code which then goes unused because maintenance would be too much of a burden. Sometimes I write some really complex one-liners (which are like 5-10 lines long) to do some tasks using all the possible hacks to avoid having to type an extra character. I might be able to do that, but most developers wouldn't be able to see how the data get transformed and keep all of that in their mind. Whatever I wrote is unmaintainable by most people.
The reality is that the majority of people can't grasp their mind around complex concepts. Which is ok, most developers write a few API endpoints and some UI components, they don't need much to create value.
We can get some good concepts from the functional world and transfer them to C-like syntax languages though. We can even have some of the programmability of Lisp (but not all of it) via macros.
Technically, you could run e.g. a Python program under pdb, break on certain exceptions, and fix things inside a living system. It's just not a customary way to do that.
there is no full-featured web framework (although you can write web apps of course),
no satisfactory GUI lib (now Gtk4, Qt5 (hard to install), IUP, nice-looking Tk themes, more low-level bindings to graphics libraries etc)
the package manager came late,
good open-source compilers came late,
less choice in editors (now many https://lispcookbook.github.io/cl-cookbook/editor-support.ht...),
and, well, lots of FUD and a language not for everyone.
smalltalk can do the same btw. i had been working on a small website where a specific request from the browser would fail. instead of sending a failure message the request would just hang. in the mean time on the server in my pharo smalltalk window an error would pop up. when i fixed the error, the download of the request resumed in the browser as if nothing had happened other than a delay.
We like to make sure everything running in prod is verifiably built from source in-repo. So that's the thing, while it's a really snazzy feature for sure, the value over the rest of the world is on the questionable side. At least for our use case, but I think it's true for most use cases.
edit: Also really curious about your smalltalk and pharo experience. Sounds fascinating!
in smalltalk on the other hand you get a nice IDE with all the comforts of a GUI. you have your windows where you browse your code neatly structured in classes and methods. there is a window where you run your app and manage your tests which light up red or green if they fail or pass, another which logs error or other print messages, and if an error happens while an app is running a new window pops up, showing you a trace of what was running and a text field with the code that failed, like in a debugger, and right there you can edit the code and resume.
the code is written to your class, and when you go back to your code browser the change is reflected there, and you can commit it to a version control system. pharo btw has pretty good integration with git, and already a few years ago it almost acted like a git gui. it's probably even better now. the primary downside is that the text editor in pharo is simple, like a browser text area, and not a sophisticated editor like emacs or vim.
> [Smalltalk] the code is written to your class, and when you go back to your code browser the change is reflected there
I feel your pain. "writing the changes back to the source code definition" seemed like a no-brainer desirable feature of a Lisp REPL, yet I could not find a way to do that out of the box using Slime. I'm sure one could program it, however! Bet someone has...
Here's what I use: edit code, save file, tell slime to eval current defun. I haven't yet suffered indiscipline to hook `slime-eval-defun` to call `save-buffer`. Would that work for you?
Someone's going to argue with me. Fair enough. Provide your explanation.
CL has had no real “unknown unknowns” for a very long time. While folks who newly discover it feel they found the gold idol in the jungle cave, the cave is, in truth, well explored, mapped, and documented but the idol is left behind.
All excuses to not use CL have long been, or have had the opportunity to be, addressed. Today, it’s fast enough, small enough, empowered through utilities and libraries enough, has different build and deployment scenarios to work with a vast array of applications. And yet here we are...still.
ABCL runs on the JVM, which runs everywhere on everything. Clojure, first class system on top of the JVM, but no real adoption. Some, to be sure, likely (I have no data) more than CL itself. But it’s still an blip on the radar.
Meanwhile, a bunch of hackers threw together a language sharing many aspects of the core feature set made popular in Lisp and Scheme runtime environments, made it look like an Algol step child with curly braces and everything, and since then an entire ecosystem of software has been written (and rewritten) into this system and it’s runtime is the focus of some of the largest companies in the history of civilization.
Raise your hand if you think that if the creators of JavaScript went with an S-expression syntax instead of a C/Java derivative, we’d be running a VBA clone in our browsers (but nowhere else)?
Because at this juncture, THE thing that distinguishes CL and other Lisps from where we are today, is the syntax. Every other charm these systems enjoyed have been cherry picked away.
Advocates say the syntax is not an issue. It’s a feature m, not a bug. But the “wisdom of the crowds” has spoken, and they stay away.
1. Computing is extremely blinkered. More than in other professions, people in computing are either unaware of what has been done before. If they learn a little bit about what has come before, they look for reasons to be dismissive of it, so they can turn their attention away. They know a few things (languages, platforms, tools) and just live in that world.
2. At any given time, a small handful of things are popular. This changes over time. The amount of stuff we have produced in computing vastly outnumbers what is popular. It's like a game of musical chairs in a packed sports dome, where there are seven or eight chairs on the floor.
3. The field is still growing; there are probably more people who joined in the last 10-15 years, than those who joined the field before that. Almost every newcomer plunges into whatever is popular at the time, and will never look at anything else unless it is new and popular, which will happen at most some 3-4 times in their career before they are out.
Those are generalities. Then there are Lisp specific historic items.
Lisp specifically had a bit of a heyday in the 1970s and into the 80s. People developing Lisp systems were very ambitious and their work eventually demanded hardware that only big companies and well-funded institutions could afford. They did fantastic things, but Lisp did not scale down to the emerging single-chip microcomputer with a small memory (or not in that fantastic form). Typically, Lisp would have liked a few megabytes of RAM compared to tens or hundreds of kilobytes.
The microcomputer was something new and popular, bringing with it new people who had nothing but microcomputer experience. Most of them didn't know anything about Lisp other than reading about it in books or some magazines like Byte and Creative Computing, which is something that only a curious minority would engage in.
Eventually, consumer microcomputers became powerful enough to run Lisp well, but by that time, the people who remembered Lisp were vastly outnumbered by new people.
(Speaking of memory sizes, GNU's implementation of Lisp, even, GNU Emacs, was absolutely derided for its memory use, well into the 1990's. For instance, one joke interprets its name as an acronym for Eight Megabytes And Constantly Swapping (EMACS). Eight! Not Eight Hundred or Eighty. Eight megabytes is ridiculous today; the resident size of a Bash process can easily be that.)
Another problem with Lisp is academia, which has played a role in actively destroying interest in Lisp.
After the downturn in Lisp popularity, schools continued to teach Lisp dialects, but often badly, leaving students with a bad taste. They used scaled down dialects not suitable for software development work, like certain Scheme implementations, and gave students assignments that focused on doing things with recursion and lists, and other nonsense that is far removed from making a text editor or game or whatever.
This practice is still continuing. If you follow the [Lisp] tag on StackOverflow, you will notice that from time to time, students post Lisp homework questions. E.g. "we are required to write a recursive function that removes matching items from a list". The comments will say, there is a built-in function remove, why don't you use that. The student will reply, oh, is that right? But, in any case, we are only allowed to these five functions: cons, car, atom, ... and we can't use loops, only recursion.
Never in a programming course that used C, or Modula-2 or whatever have I had homework forbidding me to use any language statement type, operator, or library function! This is purely a Lisp teaching problem, and it leaves students with wrong ideas and impressions. They might misremember things and spread misinformation like "Lisp has only linked lists and nothing more, and everything must be done with recursion; it is useless".
(do-it (do-it first))
I get control right where it's broken, so I can fix the do-it defun. Great, I like that. But by fixing it, this means I changed the result of (do-it first).
So the point at which the machine (?) is is a point that's unreachable anymore by the current code.
I hope my example is clear enough.
I really don't understand how that works when fixing what would allow you to continue would change the state at which you're given control to fix things?
~ cat do-it.scm
(define (do-it x)
(if (> x 0)
x
'error))
(define (do-it-fixed x)
(if (and (number? x) (> x 0))
x
'error))
~ gsi do-it.scm -
> (do-it (do-it 0))
*** ERROR IN do-it, "do-it.scm"@2.7 -- (Argument 2) REAL expected
(> 'error 0)
1> ,b
0 do-it "do-it.scm"@2:7 (> x 0)
1 (interaction) (stdin)@1:1 (do-it (do-it 0))
2 ##main
1> ,e
x = 'error
1> (set! do-it do-it-fixed)
1> ,(c x)
error
> (do-it (do-it 0))
error
i think the approach here is to accept that you fixed the bug for the second call, but you will still have to go back and retest the first call.
You're right.
> i think the approach here is to accept that you fixed the bug for the second call, but you will still have to go back and retest the first call.
Gotcha. It looks like a very useful feature. I may actually just try it to try to understand how it works: especially since TFA says the CL integration with Emacs is good (I happen to be an Emacs user).
This just makes me wanna bust open a smalltalk image...
Gambit v4.9.5
> (let ((x 10) (y (- 1 1))) (\* (/ x y) 2))
\*\* ERROR IN (stdin)@1.30 -- Divide by zero
(/ 10 0)
1> ,e
x = 10
y = 0
1> (set! y 2)
1> ,(c y)
4
>
Unless you are restricting this to only development in which case there are a lot more languages other than common lisp that support hot-reloading/re-definition.
Like what? The only notable one I know of is log4shell. And no one advocates not to use java because of rce. Nor javascript, nor python, nor erlang. Compare with c...