Learn Lisp the Hard Way
llthw.common-lisp.dev
llthw.common-lisp.dev
Granted, sometimes the mixing of paradigms can kind of clash, and that’s its own can of worms, but at least I never get that “ugh I could have done this with a library in <language X>”
The part that I do find annoying is when I have to break out macros to make my wrapper usable. When I was using the Java bindings for Apache Spark a few years ago, I felt like I was constantly reaching for macros to do some clever stuff, which is no fun.
Take a look at the CL libraries PY4CL, PY4CL2, and CL4PY. I use the first one for some things. It can be useful to bridge over to Python for something and come back with the answer.
I couldn't find it, but is there a "Rosetta Stone" of language ecosystems, where in a "libraries" section it lists Python/Go/Rust/Ruby/Perl/Lisp/etc. library near-equivalents?
When I say "learning" I mean in-depth application over, perhaps, a year, not something superficial. The question is how. I don't really have an answer for that. You have to find reasonably complex projects to be able to spend that much time with a language. That's the problem. I was getting paid to use these languages back then. Who wants to do non-trivial work in these languages today. Not many.
Learn Lisp the Hard Way - https://news.ycombinator.com/item?id=8573389 - Nov 2014 (32 comments)
Learn Lisp the Hard Way - https://news.ycombinator.com/item?id=8016950 - July 2014 (141 comments)
What would be a good project to showcase the strengths of Lisp, versus other well-known languages?
How/why do people end up learning Lisp, what is the usual route to entry?
Genuine questions, not trying to start a language battle here. I've never used Lisp before so am curious about all of this. To me it feels like one of those exotic old-school languages like Erlang or Perl.
If you consider Clojure a Lisp then I think before the tech stocks crashed there were seven unicorns built with Clojure. Banking, healthcare, analytics, ...
I'd say Clojure can be used anywhere Java can for sure. And Clojure runs on top of ClojureScript too.
Then Clojure is only one of the Lisp dialects and there are "lispier lisps" out there which also their success stories.
I'm holding onto some things that make common lisp look exciting and useful (static typing[0], APL DSL[1], speed [2,3,4]) and really want to get familiar with structural editing [5]
[0] https://github.com/phantomics/april [1] https://github.com/coalton-lang/coalton/ [2] https://renato.athaydes.com/posts/revisiting-prechelt-paper-... [3] https://github.com/fukamachi/woo/blob/master/benchmark.md [4] https://tapoueh.org/blog/2014/05/why-is-pgloader-so-much-fas... [5] https://github.com/drym-org/symex.el
I started learning CL starting around July/Aug 2022. How I got here is close to 2 years ago I read a blog post (I think) that said learning a niche language can help land a high paying part-time job, or, if applying to a position where said niche language is used you don't have to jump through all the Leet code, typical interview circus. Just knowing the language and having an interest was a huge factor in getting hired. It seemed fairly plausible to me.
Initially I started learning Clojure but kept reading stuff about how awesome CL was/is. The JVM kind of turned me off and I like compiled, stand alone applications which can be had in CL.
So here I am really enjoying learning CL. I am in a job/organization that does not use CL and probably never will. The benefits are awesome so I have no plans on leaving anytime soon. Maybe one day I may try to land a part time job using CL but I'm no where ready for that (both in my lack of CL knowledge and lack of time to commit to a part time job).
Now I have my interactive programming with the nicety of having SBCL warn me when I'm passing an integer to a function that expects a string or something like that. Not only does it help me make fewer mistakes while programming, the type declarations also help SBCL generate more efficient code. It's a win-win all around.
But please don't assume this is an exhaustive list, and please don't assume Lisp is only useful for Animation and Graphics, AI, Bioinformatics, B2B and Ecommerce, Data Mining, EDA/Semiconductor applications, Expert Systems, Finance, Intelligent Agents, Knowledge Management, Mechanical CAD, Modeling and Simulation, Natural Language, Optimization, Research, Risk Analysis, Scheduling, Telecom, and Web Authoring just because these are the only things they happened to list. Common Lisp really is a general language capable of a lot more than these few incidental application areas, even if this web page doesn't totally bring that out.
Here's an overview for 2022: https://lisp-journey.gitlab.io/blog/these-years-in-common-li...
companies: https://github.com/azzamsa/awesome-lisp-companies/
pgloader went from Python to CL: https://tapoueh.org/blog/2014/05/why-is-pgloader-so-much-fas...
my take on why lisp vs python: https://lisp-journey.gitlab.io/pythonvslisp/
If you count Clojure and ClojureScript as kind of Lisp then for instance our whole product (https://orgpad.com) backend and frontend is written in it. Of course there is also some SQL, CSS etc. in between but basically all the meat is in CLJ or CLJS.
Maybe people these days can't handle anything beyond a browser?
He told the MIT people in charge of developing courses that he and Abelson were quitting, and they should figure out what to do next. The world had moved on to interfacing complex chips to sensors and actuators, and it wasn't easy, because the chips were imperfectly specified. To a certain degree, you tried different things to see what worked. I think he was rather disgusted with the situation, compared to the clarity he had a couple of decades earlier using Scheme.
So, the MIT professors that replaced him went with Python. It had great libraries, and it was amenable to fast turn-around when gluing software and hardware together in the lab.
I had a discussion with Sussman a decade or so ago about this, and what he told me then agrees with what he says in the video.
I did early Lisp development on Symbolics Lisp Machines back in the day. Wrote 3D graphics on them. I didn't realize how special the development environment was until I got my first job, writing 3D graphics in Fortran on VAX/VMS. Spent the rest of my career (Fortran, C, C++, IRIX, Linux) bemoaning all the stuff I missed from my Lisp days.
I did venture back into Common Lisp development (MCL/CCL on MacOS) for my personal projects. Even did my masters thesis in CL by writing a socket connection to Maya and using it as a graphics engine from CCL.
It took me a long time (decades) to get over the "everyone should be using CL" mentality.
Thinking back on things, I have come to realize that Lisp is not a magic bullet: it won't necessarily make you a better programmer, and it won't necessarily make your project more successful.
What Lisp will do is give your developers more power at their fingertips. For better or worse.
The philosophy behind Lisp (and Smalltalk) back in the 70's and 80's was that programmers should be given as much expressive power as possible so they can develop better, faster, etc. The reason why CL allows you to define macros, redefine classes and methods on the fly, use generic functions and multiple inheritance, and recover from errors, is simply that the language designers considered these import enough to do the hard work to make them happen.
I feel that philosophy, backed by the economics of software development, fell out of favor over time. Performance, safety (preventing developers from shooting themselves in the foot), code maintenance by large teams, cost, etc became more important.
Today, the main factors in language choice are (1) the availability of developers, and (2) the availability of libraries/tooling. Going against these currents requires awareness and a strong commitment. To paraphrase an old computer industry saying: no one got fired for choosing Java.
On a personal note, I continue to use CL for my own projects, where the pleasure of development is more important to me than commercial considerations. I have done so for kons-9, my 3D graphics system, and am happy that I did so. I simply would not have been interested in doing the project in C++. I find that CL is a language which supports me in my development, rather than fighting back.
As an added bonus, the features of CL have encouraged me to think differently about the architecture and design of the system. A system built in C++ would simply have converged towards being another Maya/Houdini/Blender (at best, after many years of development). This way, I have a chance to do something different, which is a big part of the fun.
Interesting. Symbolics Lisp Machines seem almost mythical to me. Are there things about 3D graphics development that lent themselves particularly well to that hardware and Lisp in general? Did Symbolics Lisp Machines target certain types of development or industries?
This was in 1985. The Symbolics machines did not have any special 3D hardware. The SGI boxes were the first to do so.
The Symbolics did have a "high resolution" (around 1K pixels across) b&w bit mapped monitor with a windowing system and reasonably fast bitblt (copying of pixel areas) which made it possible to do simple wireframe animations.
In addition my machine had a second color frame buffer with 24-bit color, a rarity at the time which allowed me to develop my first 3D renderer.
Lisp machines were marketed as productivity enhancers with an emphasis on (old fashioned) AI. Expert systems and such. They were more complex pieces of hardware design than the emerging Sun workstations which ran Unix, and consequently more expensive.
A while after I graduated, Symbolics released their own suite of 3D applications (S-Geometry & co) which for a while were the best commercial packages out there, probably until around 1988 when Softimage was released on SGI machines (I developed their animation code).
I wrote about some of this in a series of blog postings:
https://medium.com/@kaveh808/5-zetalisp-and-the-talking-head...
I was curious about your statement:
>"Lisp machines were marketed as productivity enhancers with an emphasis on (old fashioned) AI."
Could you say what exactly would be considered "old fashioned" AI now from that era?
Recently, I've seen the acronym GOFAI used for "good old-fashioned AI".
that's been a thing in ruby since probably forever and it's one of the defining things about it
Not wrong that ruby took a lot from perl, but it was heavily influenced by lisp and smalltalk and I'd personally argue those are the two largest influences.
It does seem that ruby was specifically designed to grab Perl programmers though. Several features match exactly.
It seems like the longer I am in software development, I find out that more and more things came from Lisp.
the OP was talking about dynamic languages... haskell and scala are not too surprising that they have implicit returns since they are functional paradigms foremost.
To me it's more of a programmer's swiss army knife than a language I'd use in a big project that needs to be accessible to a larger team.
Common Lisp's greatest strength is that it's "the programmable programming language." You can customize everything from the lexer to the syntax, and you can extensively customize or even replace the object model. It's trivial to write code that writes code.
There are some limitations. Haskell, for example, is less customizable but it works very well at high levels of mathematical abstraction.
Now, in practice, I don't often need the specific kinds of power that Lisp offers. Or rather, when I do need it, I don't need that power instantly at my fingertips. Completely customizing your programming language is something you should think long and hard before doing, and the median team will do it very badly. So I'm happy with the tradeoff offered by Rust macros, for example: They're harder to use than Common Lisp macros, so you only use them when you need them badly enough. Even for most above-average teams, I think it's a net win for macros to be a bit obnoxious to use.
But if you ever find yourself on a small team of incredibly talented programming language designers, and you're working in a problem space where no off-the-shelf language can do what you need, then Common Lisp is well worth a hard look.
Paul Graham's "On Lisp" is actually an excellent overview of the kinds of things you can do with Lisp. Norvig's first AI book is also a good demonstration of Lisp, even if the "AI" part is hopelessly obsolete at this point.
Same goes for computer languages. I use LISP when I have computer science-y things I want to explore. For example, I was working on a path finding algorithm for fun years ago and I went to LISP because it is much more natural at conceptual programming. But I wouldn't use LISP to create a GUI, even though it can, nor would I try to write performant 3D graphics. But I would use it to craft a game engine (I wrote a frame language in the early 2000's as part of a game engine that I never finished because once i got the agents decision making working I felt satisfied enough).
The right tool for the right job...
Need to munge a file? PERL
Need to write an embedded application? C
Need to write a GUI? A JS framework (or Qt, or Motif/X if I'm feeling nostalgic)
Need to write a fast loop? ASM
Need to write a something stats-related? Python, R, JMP
Need to explore a computer science problem? LISP, SmallTalk (I don't know Haskell)
Need to work on a project that you're forced to? C++ :)
Let GCC/Clang solve that for you.
>Need to explore a computer science problem? LISP, SmallTalk (I don't know Haskell)
Scheme it's far easier.
Lisp is simply a different way of thinking, in S-expressions. It's hard to explain but easier to show you, via the above link.
That's a Catch-22, of course. How will you know if it's a good fit for you unless you learn it?
So that reduces the decision to: if you are curious whether Lisp is a good fit for you, learn enough of it to find out. If you're not curious about that, don't bother.
But you might want a preview of what things make it a good fit for some people. Lisp in general, and Common Lisp in particular, is a good fit for me because its default mode of operation is to start the Lisp running and teach it by interacting with it expression-by-expression how to be the program I want to build. Common Lisp is consciously designed to work this way, and provides a host of well-designed conveniences for it. It turns out that working that way makes me happier and more productive than working any other way, so it's my favorite programming language.
Not very many other programming languages work that way, but to be fair, some do. Examples of some that do are Smalltalk, FORTH, and Factor.
Some languages and implementations provide more support than others for that style of work. The most comprehensive support for it is to be found in Common Lisp and Smalltalk implementations.
If that sounds interesting, then it's probably worth your time to learn Lisp (or one of the other languages I mentioned). If not, then I wouldn't worry about it.
The way Lisp is structured means that almost certainly if you're missing language feature X you can bolt it on, stacking the new features on top of each other to make whatever abstraction you see fit.
It's the kind of thing that makes a Lisp head amused when someone talks about the genius of C# and LINQ (as an example). If you need an abstraction in Lisp, you can almost certainly build it in an ergonomic fashion that just doesn't have a parallel in other languages.
It's one of those powers you don't fully appreciate until you have it and then every other language seems hilariously inadequate in comparison. With great power cones great responsibility and it can all crumble with a lack of discipline (hence why I'm also a fan of ridiculously restrictive languages like Go), but honestly the sort of meta programming possible in Lisp has no equal I know of.
The simplest app I have in prod reads data from an existing DB, formats it and sends it to a FTP. No UI, it's a CLI. Runs daily with cron. Was easy to build, to deploy (standalone binary), to maintain. Build and forget.
Haven't you ever thought "hmm, I could do this job faster if I had my own plugin/extension here?"
Lisp is really good for that, when used within Emacs. Extensions are very very simple, basically just the simplest functions inserted into your .emacs file.
For ex I wrote about one such check here https://tech.perpetua.io/2022/01/generating-sqlite-bindings-...
This macro checks SQLite table/column references. Aka, if you try to access user_id when the table doesn’t have that column, the “user_id” text in your code will appear as a syntax error in your IDE and when you run the code.
The typed racket project is a bigger example. They added a typechecking system to the language, including a new typescript like variant.
I’m not aware of any other language where 10 lines of idiomatic code right in your codebase can add a new IDE warning.
Besides, if you're studying CS at a halfway decent school you'll learn a half dozen languages before graduating anyways (if not more), what's one more to you?
Also, we did not even learn tail call optimization :) so while the argument is valid (I guess it's easier to learn scheme than erlang, ocaml, etc.) it wasn't in my case.
You would have to build the layers of your stack from first-principals, or perhaps second-hand mimicry of third hand descriptions. Understanding your problem, and building exactly the tool needed is a very different approach from picking the tool with the momentum, growing developer base, checklist of features, and knobs to twiddle to somehow get something approaching a solution to your problem.
In that setting, Common Lisp has a very powerful feature set; macros, CLOS, interactive debugging with restarts, dynamic redefinition and a fast runtime made to be interactively modified. It is a few days work to make a minimalist ORM, a HTML template language, or a functional query language and engine and proceed to tune it for purpose.
I think we are approaching an inflection point. The software ecosystem complexity, and the specialization of developers into frontend, backend, or specific frameworks and the absolutely unfathomable and delicate chains of transitive trust in the layers and layers of abstraction will become a liability .
Doing "more" of the same, throwing more bodies at it, rewriting the world because your UI or ORM needs a reboot, trying to attach checksums and technical proxies for dependencies as a substitute for the basic fact that you need to know what code you are running and why to be a responsible software vendor; none of this will work.
Truly powerful languages and runtimes with appropriate technology fit for purpose are the way out of this conundrum and the best path to being a responsible and effective computer programmer.
Keep your parens sharp.
Doing "more" of the same, throwing more bodies at it, rewriting the world
because your UI or ORM needs a reboot, trying to attach checksums and
technical proxies for dependencies as a substitute for the basic fact that
you need to know what code you are running and why to be a responsible
software vendor; none of this will work.
I love Lisp. All Lisps. Common Lisp. Scheme. Clojure. Even elisp. And, for a long time I thought Lisp was the future. I thought, like you, that the ever expanding size of codebases written in more conventional programming languages would necessitate a switch to something more functional and more amenable to building non-leaky layers of abstraction because that would be the only way to continue to build without introducing at least one bug for every new feature.
However, the advent of LLM powered coding assistants (like OpenAI's Codex, or Github's Copilot) has made me reassess my view. LLMs benefit from having more training data. Now, the thing that I thought would kill Java/Python/Go codebases, i.e. the sheer number of lines of code required to do even the simplest thing, turns out to be a strength. Because each line (and all its variants) now represents a datum point in the training set for the AI.
Can Lisp be enhanced with AI support. Sure it can, in theory. But in practice, there's far less Lisp code out there. Primarily this is because far fewer people use Lisp on a regular basis. But it's also because, for any given task, far less code is required to accomplish it. Just like Java's greatest weakness turned out to be a strength in a world where we have AI assistance, Lisp's greatest strength turns out to be a weakness.
I'm going to continue to use Lisp wherever I can. If not at work, then in my projects at home. But I'm far less sanguine today about the eventual dominance of Lisp (and Lisp-like languages) than I have been at any time in the past.
It might be ironic, but I don't see any particular contradiction in saying that worse-is-better AI magnifies the advantages of worse-is-better programming languages.
Before that, I did some java and ruby, which was fun and helped pay rent. But before that I helped pay for many houses and even apartment buildings with Common Lisp. That project has paid for houses and school and living for many dozens of people over the last 20 years. It's still going.
I am not interesting in conquering anything, or winning anything, or what momentum or the power law, or economies of scale, or momentum or other efficiencies of industrial software production. People trot out those arguments all the time to explain why something can't be done, or why they can't do what they want to do, or why something they hate is winning or inevitable.
Believe in yourself and your compatriots, and if you wanna do it with lisp, find some better compatriots.
Cheers.
That was my first I/O device at home. (Heavy, slow and noisy!) But the paper-tape sub-system was fun to use.
One of my teachers: "Oh, you've never done any programming? Cool. Your first assignment is creating a game of 圍棋 in LISP. Go."
Awesome, thanks.
This books reads to me like this observation turned into a practical joke. The draft of this book suggests years of work to populate all the exercises and it has unfortunately hardly been edited since 2015.
I just looked at the author's github page and it fits this stereotype almost too well. I am like this too and I would really like to know why this is and why people like us flock to lisps.
This is clearly a very opinionated book. Personally I find Haskell, while perhaps not as simple, both more expressive and more elegant. But I don't imagine it is favored by the worlds best programmers.
Such a missed opportunity.
If audience is whom who doesn't know anything about LISP, you've lost time to go around to find things you want.
Other languages have playground, or at least INSTALLING at the chapter 1. It should answer some IMPORTANT questions:
- How to install the compiler
- Which IDE best to use, or which extension.
- How to install packages, how to build it.
Literally everything you're asking for is in that link. The author has the opinion that you don't need fancy IDE features which is an arguable point but they do list what editors for each OS work in their experience.
But for those who like neither Emacs nor Vim ... I don't know.
I see that in your professional capacity, you're an engineer on VS Code, so I bet that you know it well enough to make it work for you for Common Lisp, too. For all the oft-cited reasons, I cannot use VS Code. But it is great if it works for you.
[1] https://www.gnu.org/software/emacs/manual/html_node/emacs/CU...
As for CL itself, I dabbled in it in my time, and I found it more interesting to broaden the mind than useful for practical applications. The dearth of quality, well-maintained libraries is the main stumbling block for serious use. It's not that it's impossible - it's just that the effort involved in putting everything together exceeds whatever benefits accrue from the language itself being more powerful than usual.
That aside, though, the discussion was about how getting started with CL for someone new is more complicated than it needs to be in part because the ecosystem is so centered around Emacs for code editing. In general, people who get interested in CL are already devs - but Emacs isn't all that popular even among devs these days; I see Sublime far more often, for example. But all CL tutorials actively push Emacs - and for good reason, since it really is the best option available! - which makes the barrier of entry that much higher for people coming from, say, JS or Python.
doom emacs was a game changer for me and allowed me to pretty quickly switch from neovim. Since investing more in the ecosystem, I wont be letting go of emacs anytime soon
In general though, I'd say switch your focus to Scheme, and use Dr.Racket, if you really really don't want to use Emacs.
...SLIMA may work with Pulsar, or if you prefer Eclipse there's Dandelion (https://github.com/Ragnaroek/dandelion).
VSCodium may be able to use Alive, but as it can't fully turn off MS's spyware I can't in good faith recommend it.
Chapter 2 https://www.braveclojure.com/basic-emacs/ helped me get started with Emacs with just knowledge to be somewhat productive and start learning more.