Mark Watson's Lisp books | Hacker News Reader

Mark Watson's Lisp books

github.com

235 pointsKRAKRISMOTT49 comments

Having worked with Mark for a couple of years (he was my manager), I got to hear many stories.

One of the more interesting was when he got special permission from his employer at the time to purchase / access a lisp machine.

Wealth of knowledge, good programmer, definitely worth reading his books (if interested in a topic he covers)

Nice, what was the year? Approximately.

More importantly, he’s a GOFAI engineer who understands modern ML. Folks like him are rare (especially those of his generation who are still actively coding).

Had to look that up: "Good old fashioned artificial intelligence"

I always read it as "gang of four ai" first.

GOFAI is basically "artificial intelligence" algorithms that don't involve neural networks or statistical learning. Deep Blue and Prolog (knowledge bases aka "expert systems") are your typical examples. They usually involve sophisticated tree construction and hand coded search algorithms. Modern ML also has tree stuff and symbolic algorithms but they are all heavily augmented by deep learning and probabilistic methods like Monte Carlo.

When people say Lisp was used for "AI research" and that it suffered from the "AI winter", GOFAI is what they are referring to. Neural networks exist back then but they were very small scale, good enough for some signal processing applications and as research curiosities but most of the powerful emergent capabilities were never truly realized until post millenium.

I’d love to read a write up of someone buying a Lisp machine and then coding in it. It’s a fascinating historical what could have been, if they had survived to the present day.

Lisp is cool. It has that hacker feeling when you code in it. No matter the dialect :) Downloaded the whole repo, will have a look.

Lisp is the way

Lisp has that cool hacker feeling because it's how software development should be done: a relatively minimal core of concepts exploding into endless possibilities, the code you write itself expressed in terms of a runtime data structure which can be traversed and manipulated, the immediacy of a REPL and directly changing a live system.

It never took off because developers are modern Sisyphi, who when freed of their burdens immediately yearn for the feel and heft of the cold stone boulder in their hands. That's how we got Java: enough of Lisp to be tolerable and familiar to C++ programmers.

If lisps are that good why there's so little killer software written in it?

With rare exceptions like emacs and clojure's datomic I feel like such a lauded language should have more to show.

And yet, there's comparatively much more killer applications written in php alone than all lisps combined.

I am proficient in several lisps by the way (Racket, CL, Emacs Lisp, Scheme) and honestly every time I need to start a "serious" project I just feel like any other language depending on the use case is a better fit.

Popularity does not mean good. Also I assume most killer apps use right tool for the job - For example in a corporate setting with multiple developers something like Java would be better.

I speculate the following - if there are few developers who love lisp, then all can try to mob program a lisp code base -- multiple cursors, same time (code review and development all happen parallel). (Not sure if such an editor exists) This would be a good way to maintain a lisp code base I think (speculation). (Compared to Java where you can work mostly alone and make a mess and it will still work somewhat OK)

Visual Studio Code can do that, Emacs it'd be harder. But not many Lisp programmers are interested in working that way as I understand it.

Mark is right here on HN:

https://news.ycombinator.com/user?id=mark_l_watson

I have learnt a great deal about modern state of Lisp from comments written by Mark and others like pjmlp on HN. By reading their comments I've learnt that Lisp is relevant in non-Lisp languages too. So many programming languages borrow concepts that were explored in Lisp languages. And not just Lisp. ML too. Their comments helped me learn this kind of background.

If I could get a directory of active Lisp devs who comment on HN that would be great! Maybe even create a feed out of their comments. Is there any feed of any kind on HN? Even a basic RSS feed could be useful. I could pull the whole feed and filter them out for their comments.

You may enjoy this old book.

http://www.paulgraham.com/onlisptext.html

Title isn't accurate, this seems to be a link to all of Mark Watson's older books: including books on Lisp, Haskell, Java, and personal stories.

Looks like this PR from a couple months ago was him moving a bunch of older books from his personal website into this repo: https://github.com/mark-watson/free-older-books-and-software...

+1 for "Brady_story".

what score change +/- should there be for bragging about having "... 55 US patents"?

And also a fairly active HN user as well!

https://news.ycombinator.com/threads?id=mark_l_watson

"Bragging" seems unnecessarily pejorative. Reading the information about his patents didn't sway me one way or the other the first time I visited his site, but I can see it being relevant data for some of his audience. The (superior) content of his books did sway me, as did Mark's willingness to make many of them available for free. I've since had occasion to interact personally with Mark, and I've unfailingly found him to be gracious and personable; "braggart" does not apply, in my experience.

It's just a basic resume credentials, the same as job experience. There is another HN thread talking having personal projects online. Whatever increases your changes to put the leg in the door, do it.

I wish github just linked pdfs to the google viewer, so that it could be read without having to download the file to disk!

for example: https://docs.google.com/viewerng/viewer?url=https%3A%2F%2Fra...

I guess its possible to make a bookmarklet for it...

I clicked the first one, it's readable in the browser for me?

https://github.com/mark-watson/free-older-books-and-software...

Can't you change this in your browser config?

I remember changing this back and forth a number of times during my life, depending on what was more convenient for me given the nature of the jobs I was predominantly doing during those days.

Nowadays I would prefer PDFs (notably including those served as data URIs) to be automatically cached to a dedicated directory and opened in an external PDF reader but I had no time to configure this so far.

The historical hype around Lisp is from back when the alternative was C.

https://manifold.markets/Alana/convince-me-will-i-learn-lisp...

Haha, no. Modern PLs have reached parity with Lisps but comparing C and Lisp is apples and oranges. C was fast no doubt but much lower level than LISP and served a different purpose.

And then you proceed to link someone’s comment as a reference?

Nobody is trying to convince you to use Lisp, it’s just free books you could learn from if interested. The hostility is hard to understand.

Their point was that the "superpowers" and mythology of Lisp come from a time (the 80s) in which C was the primary alternative, popular language. Standard ML came later and the further MLs in OCaml and F# came much, much later. Erlang was proprietary. Prolog was around but misunderstood and remains so. Smalltalk was there and super powerful (maybe moreso?), but as far as I understand, kept to certain machines.

But now we have Elixir, Erlang, F#, OCaml, Scala, Racket, Chez Scheme, Gerbil Scheme, Chicken Scheme, Ruby, Python, Clojure, Pharo, etc. all competing with the standardized Lisp in Common Lisp.

I love Lisp, or rather Scheme and its ilk, but the point is that its mythology comes from the 80s and early 90s in which it did sort of set the standard for modern languages. It's really a fantastic family of languages, but there doesn't seem to be one that really stands out as pragmatic, industrial alternatives to the other languages it has heavily inspired. It seems Clojure and Racket come close, and Common Lisp is just too "hardcore" for pragmatic use, in my opinion.

>Common Lisp is just too "hardcore" for pragmatic use, in my opinion.

In what ways? Asking as Lisp newbie.

It's mainly that the core editor is Emacs or proprietary ones that have indeterminate futures, there are different implementations that are incompatible for things not defined in the standard, such as for GUIs, not a well defined and vibrant ecosystem or community, the package system is strange, and other things of this nature. As a language, I think it has a bit of cruft from being a mismash of everyone's favorite features from all the Lisps around the time that the standard was created. I find that it is hard to read, and this is coming from someone who likes Schemes and doesn't care about parentheses. Furthermore, you will struggle finding a job. Although admittedly, the jobs you do find may be cool. But they may also be giant balls of mud, which is my main impression and true from what I have read.

I think for personal or dedicated use cases within a larger system it is probably awesome. For example, I believe the CLR (.NET) garbage collector was generated via Common Lisp code. I personally use other languages for my personal projects, namely F# and Elixir, and when I want to reach for a Lisp/Scheme, I choose Racket. I find Scheme a more focused language. I don't think I'll ever have the time to actually understand things like CLOS and the metaobject protocol in Common Lisp. The condition system seems pretty cool. I have practically every Lisp book there is, and certainly all the major ones. I just can't get into it, and when I try, it just feels like it's stuck in the past as an ecosystem, and I revert to Scheme or another language. The notebooks in F# and Elixir are gamechangers, in my opinion, for learning, going through books, and even production uses. The various Common Lisp books are awesome, but I use F#, Elixir, and Racket when working through the bits I do write code for.

Today other editors have good to very good support, such as Vim or Atom/Pulsar: https://lispcookbook.github.io/cl-cookbook/editor-support.ht... the VSCode plugin is here and evolving, we can use Jupyter notebooks, and even a very cool editor written in CL, working out of the box for CL and other languages with its built-in LSP client, Lem.

> It's mainly that the core editor is Emacs

What’s the problem with that? Emacs is, on balance, the best editor and ecosystem. Nothing else comes close, and time spent on anything else is ultimately wasted.

> I don't think I'll ever have the time to actually understand things like CLOS and the metaobject protocol in Common Lisp.

They are really worth considering. I have found that the longer I use them, the wiser the decisions they encode were.

> The condition system seems pretty cool.

It really is. The more you get used to it, the more intolerable it is that other languages don’t provide something similar out of the box.

Common Lisp isn’t perfect, of course. It has some very real flaws! But at the end of the day, nothing else matches it, even though over the past three decades the baseline language has gotten closer.

Garbage collection was a huge win on the past. Gradual typing, too. But the baseline language of today still doesn’t have macros. It still doesn’t have something approaching CLOS, let alone the MOP. Almost no languages have reasonable condition handling.

In a ton of ways, Common Lisp is an improvement on its successors.

> What’s the problem with that? Emacs is, on balance, the best editor and ecosystem. Nothing else comes close, and time spent on anything else is ultimately wasted.

That is a pretty strong statement. Every time I try Emacs I revert to VS Code. Is it as immediately programmable? No. But it doesn't have to be because the extensions are plenty and quite good and work without mucking about with .emacs. One can go quite deep with it when developing an extension, though. VS Code has the remote server extensions as well, which are seamless. I use VS Code on three or four different computers (as in running it as a GUI as a desktop app) every day all the while developing locally, in a Docker container, in a Linux distribution through WSL, or SSHed into several Linux machines. My settings and extensions automatically sync, the extensions just work on any machine including while remoting, and its all seamless from use case to use case. No other editor has this experience. Plus, it supports rich graphics like Markdown preview, HTML preview, notebooks like Polyglot Notebooks that can share data between cells in different languages and display images and animations, and more. Emacs does not have that support to the same degree.

> Almost no languages have reasonable condition handling.

That's true. But Erlang and Elixir are major contenders. Erlang actually probably got there first.

> VS Code has the remote server extensions as well, which are seamless … No other editor has this experience.

Emacs has TRAMP.

> it supports rich graphics

Emacs supports graphics in a GUI. I view PNGs and PDFs in it all the time.

> HTML preview

Emacs has eww, w3m-el, emacs-webkit & xwidgets.

> notebooks like Polyglot Notebooks that can share data between cells in different languages and display images and animations

Have you heard of Org Mode? It’s rather more than that.

> Emacs does not have that support to the same degree.

While I am certain that is true of at least a few things, for many things it has more impressive support.

Emacs is not perfect. For one thing, it’s not written in Common Lisp (blame rms for a startlingly bad misjudgment there). And it’s certainly acquired some cruft over the course of its lifetime. But there is a very real sense in which time spent using or extending any other editor is ultimately wasted. I am fairly certain that in 40 years there will be no VisualStudio; there will be no Vimscript; there will be probably be no Lua; if we are very, very lucky there will be no JavaScript. But there will be Emacs, and there will be Lisp. Each investment made in either today will continue to pay dividends for the rest of one’s life, no matter how young one is today.

> Erlang actually probably got [condition handling] first.

Erlang was first released in 1986, according to Wikipedia. Guy Steele’s Common Lisp the Language was released in 1984 and documented a language which had been in use for quite some time. The condition system was reported on in 1983’s Signaling and Handling Conditions.

I think that Smalltalk’s condition-handling is similar to Lisp’s. Smalltalk is a really awesome language. So’s Erlang, from what I remember!

> The condition system was reported on in 1983’s Signaling and Handling Conditions.

Symbolics had the New Error System in 1983. Which was inspired by the condition system of Multics.

Yes, those are always mentioned. Do you have a video or documentation showcasing that they can do these things and in an integrated fashion and don't take more than 5 minutes to setup? Do all those packages work seamlessly when remoted into another machine and across operating systems?

In VS Code, I open VS Code and install the extensions. I don't need to install an extension manager. I don't need to configure an extension manager. I don't need to go and find which extension manager the extension I want is in. And I don't need to configure the extensions once installed (but can if I want).

I guess the point is, Emacs is not the only editor with a vast amount of workflows that can be merged together. It's powerful, for sure, but it's not the objectively best editor.

It doesn't give you good tools for networking, for writing concurrent or asynchronous code, for graphics, it doesn't give you a good package manager or means of distributing code, its data structures are unwieldy, it takes work to make it performant, etc.

The obstacles add up, and the time you spend on these issues by dealing with hopefully supported community libraries or rolling your own is time spent not dealing with the real problem you're trying to solve.

Racket and Clojure are better suited for pragmatic use in my opinion/experience.

Perhaps you can get some of the advantages that you cite as missing by using an embedded LISP or Scheme implemented in Rust, so you can piggy-back on Rust's nice ecosystem and still have a dynamic "LISP sub-system". For instance:

https://github.com/isamert/scheme.rs (a Scheme in 3853 lines of Rust)

That's really what Clojure is, except based on the JVM, with access to the Java stdlib.

> obstacles add up

I actually agree. It wasn't smooth for me to ship my first CL app. It's all better now (more tools, more documentation, more blog posts from several people, more SO questions and answers!).

> performant

SBCL is in the same ballpack of C, Rust or Java in many benchmarks.

In this article series, the author writes the same program in CL, Rust and Java. In fact, he copy-pastes a PG snippet from 30 years ago. This snippet beats Rust and Java in LOC and speed. But, yeah, he wasn't writing super efficient Rust code, so after many discussions, pull requests and sweating, the Rust code became the most performant. https://renato.athaydes.com/posts/revisiting-prechelt-paper-... It didn't take work to make the CL code performant, more so for the Rust one ;)

a benchmark after sb-simd vectorization: https://preview.redd.it/vn5juu36v2681.png?width=715&format=p... (https://www.reddit.com/r/Common_Lisp/comments/riedio/quite_a...)

> good tools for networking, for writing concurrent or asynchronous code, for graphics,

I refer the reader to https://github.com/CodyReichert/awesome-cl but yes, CL won't have the best libraries in some scenarii (GUI? Tk libs are good, we have Gtk4, a Qt5 library used in production© by a big player but difficult to install etc)

> it doesn't give you a good package manager or means of distributing code

Quicklisp is neat, with limitations, that can be addressed with Qlot, ql-https, or CLPM or the newest ocicl.

These types of comparisons aren't "fair". As in most fields of engineering, programming languages/databases/etc. are about tradeoffs, and benchmarks aren't even more tricky.

From the blog:

> If you don’t have time: Rust can run much faster, but so can the other languages, and it turns out that the Java implementation might run faster than the Rust fastest implementation, according to the new benchmarks I’ve run after many Rust developers came to assist in making Rust faster. Common Lisp may have fallen behind, but that’s likely just because it was not nearly as optimized as the Java and Rust implementations were.

The reason PHP, Ruby, Python, bash, etc. are popular are because of developer productivity, not performance.

Also, it's a false dichotomy. Java uses `zlib` for its GZIP*Streams for performance reasons, and if there was a nice Rust library I wanted to use, I would just build a shared library and call it from Java, just like Java internally uses for `zlib`.

Additionally, there's a fairly large difference in performance between C compilers. See clang, gcc, icc, etc., and the 100+ different options you can use when compiling something.

It's not even close to "apples vs oranges", it's like more "apples vs fire trucks". :-P

Here's the follow-up with an optimized CL version: https://renato.athaydes.com/posts/revenge_of_lisp-part-2.htm... See the conclusion:

- the CL version is the fastest of ALL… in certain conditions ;)

> obstacles add up well put, this is my experience as well. Still an awesome language so I did not mind learning it but I had better luck using clojure for my projects. Especially for web stuff Clojure is just excellent. It also doesn't hurt that you can actually get a job coding Clojure if you're experienced (or so I hear)

> but there doesn't seem to be one that really stands out as pragmatic, industrial

disagree ;) This industrial language is Common Lisp.

Some industrial uses:

- http://www.lispworks.com/success-stories/index.html

- https://github.com/azzamsa/awesome-lisp-companies/

- https://lisp-lang.org/success/

Example companies: Intel's programmable chips, the ACL2 theorem prover (https://royalsocietypublishing.org/doi/10.1098/rsta.2015.039...), urban transportation planning systems (SISCOG), Quantum Computing (HRL Labs, Rigetti…), big data financial analysis (Ravenpack, they might be hiring), Google, Boeing, the NASA, etc.

ps: Python competing? strong disagree^^

(edit) on using a CL library that adds Haskell-like type checking on top of CL to develop an industrial quantum compiler: https://coalton-lang.github.io/20220906-quantum-compiler/

Those industrial use lists are way out of date though. Many of the companies don't exist anymore, last I checked. And the presence of companies using Common Lisp doesn't really negate my comment. It certainly has its niches, which is probably what it is best at. But I highly doubt one could convince, at the outset of the project, a company that chose Elixir to instead use Common Lisp, for example. There are lots of companies actively using Elixir.

> ps: Python competing? strong disagree^^

I pretty strongly dislike Python, but we have to admit that any language today is competing against it in terms of mindshare, despite several languages being much better designed.

An example: I bought a slick application for macOS & Windows: OpusModus ( https://opusmodus.com ). It's a Lisp IDE tailored for algorithmic Music composition.

The Mac version is just a normal desktop app in the Applications folder. It automatically runs on Intel and Apple Silicon, natively. Upon download it is expanded to a Mac application. It has a native user interface. For sound generation it can use a Lisp library which compiles sound generators to C, which then get compiled by Xcode and loaded into Lisp all at runtime. Typically it talks via Midi to other applications (Logic, Mainstage, Ableton, ...). It creates nice looking scores...

There is also a Windows application, with the same features, native code & native UI, ...

I find that pretty cool for a language & application which is super niche.

OpusModus 3 is written in LispWorks (first released end 80s on Unix machines), a commercial Common Lisp, which can be used for such Desktop applications.

That is indeed an impressive application. Thank you for sharing. (I'm having trouble with the website on a tablet though.) Is there any talk where the author describes it?

I would propose to watch the linked Youtube video: https://www.youtube.com/watch?v=BMoHE96rh7w

It's actually several people working on it. Yes, one can develop Lisp software with more than one person. ;-)

During the 80's C was hardly the primary alternative to anything outside UNIX.

Reply on news.ycombinator.com