Ask HN: Learning Lisp, and I have some questions

Lisp Programmer here,

- quicklisp is the default for package management. If you need to load a different version of a package you can use the local projects dir. I've never used Qi before, so I really can't say anything on it.

- For a framework I would recommend either Clack[1] (relatively small but featureful) or Ningle[2] (micro-framework).Both support multiple backends including Huntchentoot.

- There are object store databases and serializers like cl-store and cl-prevalence, but if you really want to take full advantage of Lisp I would recommend you use hash tables and reader macros. There's also plenty of mature (by lisp standards anyway) SQL APIs and ORMs like postmodern[3].

- For hosting I use AWS, haven't used anything else so can't say much on it.

[1] https://github.com/fukamachi/clack

[2] https://github.com/fukamachi/ningle

[3] https://github.com/marijnh/Postmodern

I would say that Racket is by far the best Lisp to learn. It has the basically been founded to teach programming and it has been fleshed out into a very strong general purpose language.

Watch these videos on a quick overview of Racket https://www.youtube.com/watch?v=vKjOTdVi99A&list=PLNm0vh7FGM...

You could program your pet project right afterwards. Also the documentation in Racket is better than any other community I have ever found. Clear, concise and uniform.

If I was to direct anyone into learning Racket I would say that 1) Realm of Racket is a great start for the language 2) How to Design Programs (Free online and a great resource)

Racket is just a great ecosystem that makes Deployment a piece of cake. I am a big polyglot but Racket has been my favorite find the past decade. Last time I got excited about a language was Python.

Just a disclaimer because the parent didn't say it: Racket is a Lisp, but it is not Common Lisp. It is far more similar to Scheme, and you will have to make some adjustments coming from Common Lisp. That said, it is indeed one of the best Lisps today, especially for learning

> What web frameworks are recommended

If you're intent on using Common Lisp, check out: https://github.com/fukamachi/clack

If you're willing to consider Clojure (widely used, actively developed, target platforms: JVM, browsers, CLR), there are _lots_ of options - I've had success using Ring/Compojure in small, web service projects. Check out Clojure Toolbox's "web frameworks" section: https://www.clojure-toolbox.com

Otherwise, if you're interested in Scheme (Guile, in this case), Artanis looks promising: http://web-artanis.com/

- the global folder of Quicklisp works for most purposes. It is built on top of ASDF, which you can configure too. But I would not worry about that first.

- for a URL shortener, Hunchentoot is already enough. Hunchentoot comes with an "easy-acceptor" scheme, which requires only (defparameter server (make-instance 'easy-acceptor :port 8080)) followed by (start server). But maybe you will need to define you own subclass of "acceptor" instead, I don't know.

- For storage, you can use hash-tables and serialize them with cl-conspack.

    (let ((hash (make-hash-table)))
      (setf (gethash :name hash) "YourName")
      (conspack:encode-to-file hash #P"/path/to/hash"))

Note that conspack:decode-file returns a list of values.

You can also interact with databases.

If you’re hoping to use a lispy language professionally, I would suggest looking at Clojure. It’s a much smaller language than Common Lisp and has a very active ecosystem.

Leiningen is the standard build tool: install dependencies, run tests etc.

http-kit gives you the minimum needed for implementing a web server.

There are several key-value stores written in Clojure, but I would stick with SQLite (with the Korma library for nice composable syntax).

Practical Common Lisp [0] is a really great (free) book that will help you understand the language better quickly. I read through that with three other guys at Linode in an informal book club and it went well. All of Zach Beane's blog posts are great for starting out with quicklisp, but here is a good one [1].

[0] http://www.gigamonkeys.com/book/

[1] http://xach.livejournal.com/278047.html

- If I'm not mistaken Quicklisp does allow different versions of a package coexist.

- Hutchentoot is very popular. Haven't used though.

- If you want common lisp only, IDK, but Clojure has Datomic. Though wrt databases, I'd rather care about quality, not implementation language, because it's mission critical software.

- NearlyFreeSpeech.net has SBCL and GNU Guile. But it's shared hosting. It should be easy to put an interpreter binary on a VPS kind-of service though. For Heroku there's a pack on github, though I don't have the link.

The only Lisp project that I could further is my emacs.d, because that's what I really use. I think that's key to learning something, having a real use for it. I wish you luck in your lisp adventure!

QL doesn't allow different versions of a system to coexist, but you can have multiple QL installs on a system for the rare cases in which you need two different versions of a system for two different projects.

Qlot does allow more finer grained control, but I've never had a need for it.

I think all of that is covered in Loving Common Lisp, by Mark Watson [0]. It's probably what you're looking for. Enjoy

[0] https://leanpub.com/lovinglisp

If I may ask a related question, how do lispers deal with the absence of a real boolean type in CL? My mind absolutely boggles at the prospect of not being able to tell the difference between null, false, and the empty list.

Am I missing something?

I find in most cases, I just don't care about the difference.

That said, pretty sure you can find the specific answer if you need it.

Most of the time there just isn't any need to make a difference between the three. There are basically just two common situations where you need to be able to tell the difference:

1. Telling apart optional arguments to a function that weren't given from arguments that were given and were NIL. Typically giving a default value for the argument is sufficient to avoid the problem, but you can also add an extra parameter that tells whether the argument was given.

    (defun foo (&optional (x :default-value xp))
      ;; X holds the argument given by the caller, or the keyword
      ;; :DEFAULT-VALUE if nothing was given.
      ;; XP is T if the argument was given, or NIL if it was not given.
      )

2. Finding an item that may be NIL. Functions such as GETHASH (which looks up a key in a hash table) usually allow you to specify a default value, and they will return a second value to indicate whether something was found.

    ;; Increment the value of KEY in TABLE, or if the key doesn't 
    ;; exist, use 10 as a default, increment that and set the 
    ;; result to the table.
    (incf (gethash key table 10))

    (multiple-value-bind (value existsp) (gethash key table)
      ;; VALUE holds the value if it exists, or NIL if it doesn't.
      ;; EXISTSP is T if the key exists, or NIL if it doesn't.
      )

Notice that in CL additional return values can be ignored if you're not interested in them, so you only need to use MULTIPLE-VALUE-BIND when you actually need it. You could also wrap it in a trivial macro if you're using it a lot.

    (defmacro if-some-let ((var form) &body (then &optional else))
      (let ((somep (gensym "SOMEP")))
        `(multiple-value-bind (,var ,somep) ,form
           (if ,somep ,then ,else))))
    
    (if-some-let (val (gethash key table))
      (format t "Found: ~a~%" val)
      (format t "Didn't find.~%"))

You can't make a difference; there is no three: it's all one object.

If you want an empty list that isn't nil, you have to implement your own list type.

If you want a symbol called "NIL" that isn't self-evaluating and Boolean false, you have to use your own package.

It can be a problem in advanced Lisp programming where you'll end up reinventing things like maybe monads. But it's never a problem in simple Lisp programs. (After all, C has the same issue with booleans.) Can you be more specific about your concerns?

I'm most concerned about interacting with other systems, like databases, http services, whatever, where the data that's being worked on includes such values as `false`, `null`, and `[]`.

In that context it's hard not to simply conclude that CL would be at a disadvantage.

In that situation you could simply represent `null` as a keyword :NULL, `false` as NIL and `[]` as a vector (or some other non-list datastructure).

Exactly. Symbols are first-class runtime objects in CL, so just make up a symbol name for the various null-like values. Keywords are the easiest symbols for this purpose since they're package-agnostic.

This is a legitimate concern - I've built some production lisp services that handled JSON back and forth, and you need to address this specifically. My solution was to hook custom encoders and decoders to `cl-json`, converting null to :null, false to :false, and [] to nil.

Other conversions are possible of course. This scheme has the advantage of allowing you to naturally iterate over JSON arrays as lists in application code, but you have to take care of the fact that :false evaluates to T, by for example writing your own macros for "if".

This might read like a nightmare, but Lisp makes it really easy to customize everything about it self, which can be beautiful, if you're into that.

It's rarely an issue for me and more often a convenience.

Perhaps because of the surrounding language?

>Perhaps because of the surrounding language?

Probably not, since C also has the same issue, and it's not very important there either, while at the same time being totally different as a language from Lisp.

>My mind absolutely boggles at the prospect of not being able to tell the difference between null, false, and the empty list.

What's so strange about it? That's how it has always been in C for example, Python didn't have a boolean type either for a good while, and lots of languages are like that.

Firstly, we have to consider that there are programming languages (some heavily advocated to the death) in which you simply cannot have a variable, such as a function argument, which at different times during the execution of the program can sometimes hold an empty list, sometimes a false value and sometimes a symbol.

It is actually the same in most dynamic programs also. A given expression, variable or object slot is understood to be a list, boolean flag or a symbol.

The fact that the domains of lists, Booleans and symbols happen to intersect is not relevant and doesn't really even come up. "Oh, this value a symbol too; nice, but we're not using it as one".

The empty list being false keeps a fair bit of unnecessary verbiage out of list manipulating code.

Variables and object slots being initialized to a default value of nil means that they serve as empty lists or false booleans, which is handy. A block of code which requires an initially stack can just be wrapped with (let (stack) ...). stack is ready to be pushed upon.

Really, when we consider everything, there is actually one item that is sometimes a bother: in a nested list structure, there is ambiguity between nil being just a symbol and a nested list. These two objects are equivalent: (a nil b) and (a () b).

If you're dealing with such data in which you need every possible word to map to a symbol that isn't an empty list, use a separate package, so that the "nil" token in the external data goes to mypackage:nil and not common-lisp:nil. (Treating such external data in a separate package is a good idea for other reasons, too).

Traditionally, in cases where it matters it is common to return two values, e.g.

  (values value present-p)

For example, `get-hash`: http://clhs.lisp.se/Body/f_gethas.htm

I'm using ClojureScript. I would definitely argue that ClojureScript is the right language you should pick up. It's based on JavaScript ecosystem and all pieces from npm are available. It's the largest ecosystem currently. If you argue JVM is the largest one, then just choose Clojure.

For ClojureScript, what package manager? npm. Also you may need Clojars and Maven for some Clojure(Script) packages. But you can use shadow-cljs to abstract the Clojars part. What framework? search in Node.js community, you can use it in ClojureScript. Which database? check out Node.js community. How to run on VPS? it's Node.js question.

For such simple application all you need is LISP implementation.

1. Package manager is not required. 2. Framework is not required, that is simple application, you can have running as CGI under any server. 3. Storage database? Just use the LISP date stored in files, and you can even keep username/password there.

focus on syntax / macros / CL packages - maybe make a 2-layer set of macros/functions to translate urls, deal with terminal IO (you can hook up a TTY to a web server simply), and store the results to a simple assq-on-disk affair, the rest will come later. get this, then asdf and gen executable; you can run multiple lib copies in cl packages and don't need them in production anyway, so global folder isn't important; that said you can customize quicklisp install.. write a macro to do this for you for pratice lol :)

the power is in the lisp language & syntax extension, not reproducing 'npm install foo' using native calls and getting frustrated that the ecosystem is smaller

excuse snarky tone, simply typing quickly

also, if you aren't using commercial lisp, use emacs. really. though it's the editor war, lisp-based-emacsen were written simultaneously with the lisp machines by the same people .. lisp+emacs grew together like c+unix; which is also a good analogy since it is really another world, best to throw out all baggage if you can..

Regarding Quicklisp, when I want to have all the dependencies in the project directory, I run sbcl with the option --eval '(setf asdf:central-registry (list #p"." #p"./lib"))'

You can also use ABCL in order to get access to JVM frameworks.

I would just store any data in the Lisp system itself, you have hashtables built in.

Why do you need to host it somewhere else ? Just get it running on your development machine.

Web applications are typically stateless, with state being managed by an external service such as a database.

It’s not a great architectural choice for most web apps though. I think the reason we’re stuck with it being the default is simply historical: the CGI script execution model, rise of PHP, etc.

The upside of an external service (even an embedded library if you want to go small, like sqlite) is that it handles the hairy persistence problem for you.

If you keep your data in memory, you may eventually want to store it somewhere so you can restart the service, reboot the machine, restore from older state, etc.

If you start managing the disk writing yourself and care about the edge cases (like ungraceful termination of your process or the os) you have quite a bit of work ahead of you. A service or library takes care of all of the right fsync calls on the right entities in the right orders for you.

Databases are fine. I don’t like the rigid idea of a stateless web application server that needs to hit the database for everything.

In my view, the web application server’s primary function is to be a smart cache that responds to HTTP requests. The developer’s goal should be to prevent most requests from having to wait on a socket to return data.

I would encourage you to consider the odds that a web app developer will write an application that caches better than a database, and why we don't see that pattern happen in practice more?

What do you mean by "the odds"? It's quite hard to write an application that doesn't cache better than the overhead involved in hitting a database on every request (building SQL queries, waiting on the socket, marshalling the result data...)

The app developer has a tremendous advantage over the database simply because she knows what the data is. Personally I find that it makes a lot of sense to think of in-memory objects and their layout first, then treat the database as a persistence solution. This was not doable in the '90s because of memory constraints, but today the cheapest cloud VM instances have more memory than what you'd get on a high-end box back then.

You asked why this doesn't happen more often in practice -- I guess there is simply a lot of inertia involved. Many popular frameworks like Rails are designed around the broken stateless model and can't be reset easily. There are also large platform vendors who benefit from the low-performance solution: companies like Heroku are very happy to pretend that scaling means buying more crappy stateless server instances from them.

The odds are low. The premise that you will write a write-through cache that is faster and less buggy than say, PostgreSQL, is one I would bet my enterprise against. It is foolhardy to dismiss the labor-years invested in databases.

Dude what is the point of state if it isn't persistent? You get non-persistent memory in your web application as a GIVEN. It's part of the language, just declare a variable.

I think databases are a great architectural choice for just about anything. It makes persistence and querying of complex data essentially a solved problem, especially considering "most" web apps have their foundations in some CRUD-flavored thing.

This is the ideal architectural choice for web application architecture. It allows for multithreading and multiprocessing in your web application without the need to reason about state across threads and processes in the web application. This is why developing CRUD apps are so easy, the database abstracts all the race condition problems that could potentially occur away from you so you don't have to think about it.

What’s the point of multithreading if your app is going to spend 99% of its time waiting for data to arrive over a socket?

For most web applications it’s a terrible architecture if you need any kind of performance. The real advantage is that there is a huge amount of investment made in supporting this type of app servers, so you can build on something like Rails.

>What’s the point of multithreading if your app is going to spend 99% of its time waiting for data to arrive over a socket?

What's the point of building an app if this is what it's doing. Obviously this is not the case for the majority of web services. The minimum viable requirement for a robust website is that it can serve requests without blocking itself. You don't get this with a single threaded process.

The reactor model popularized by nodejs and nginx is the alternative exception to the concurrency model of traditional servers. At best the only type of memory you can use the app itself for is caching.

Please note when I say stateless, I mean the state does not persist across requests to the server. Within the time delta of serving the request the server is, of course, stateful.

I've built apps which load the entire database into memory at boot. It's a great way to speed things up :)

They have something that does this, it's called a cache. Please note if you restart your computer, you lose everything.

I am also a beginner. I don’t have much experience with web development (I am an undergraduate Applied Math student with a deep interest on computer science). If you don’t mind, I would you suggest you to change your approach to learning Lisp. Get the classic freely available SICP book (from MIT professors) and do some exercises. The classes are also available at MIT OCW. Another good book is “How to Design Computer Programs”, from the guys who created Racket, a Lisp dialect. Maybe, these books will give you a purer sense of the language. Web frameworks are a different thing. In conclusion, if you are trying to understand Lisp, have a go on pure Lisp. Btw, Racket is great, I prefer Racket an Dr. Racket than Common Lisp.

I don't entirely agree with your reply, but I think it's unreasonable that it's being downvoted. I've upvoted it to try to balance that, although it's not clear my vote will count for much, if anything.

I think what you've said is reasonable, although it doesn't really address the poster's questions.