Rigetti Computing is a company that builds quantum computers.
We have used Common Lisp in production at Rigetti Computing for 4 years both in cloud deployment as well as for a downloadable SDK. Common Lisp is used to build what is currently the best free optimizing compiler for quantum computing [1, 2] as well as one of the most flexible quantum computer simulators [3]. They’re both open source, so you can actually see for yourself how we version, deploy as binaries, and deploy in Docker containers.
> If you are using lisp in production for non-trivial cases, do these issues still exist?
Before getting into the specifics, any large, non-trivial product that customers use will need tweaking. Especially languages that have runtimes, including languages like C. This is usually a result of leaky abstractions that just have to be dealt with in The Real World (TM).
> * poor ecosystem of libraries - few gems, most other half-baked
You have libraries for most things you need. Some are lacking, like in scientific computing. But all your usual stuff to glue things together is there and works.
Some people confuse “half-baked” with “no commits for 5 years”. Lisp libraries are older than many in vogue languages, and Lisp doesn’t change, so “5 years no commits” shouldn’t be worrisome.
At Rigetti, the biggest weight we had to pull that we wouldn’t have needed to do in, say, Python, is bind to a bunch of FORTRAN code, specifically LAPACK. That was a pain because you need to do FFI with complex double floats.
* poor community coordination
The community all agrees on one thing: Lisp is standardized.
Lisp lacks a sort of “centralized community” like Go or Clojure. I like to think that Common Lisp is more “democratic”. We have a “constitution” (the Common Lisp standard) that we all agree on and we all make our own contributions atop that.
With that said, #lisp on Freenode is attentive. SBCL, for example, responds to bug reports almost immediately. Several library ecosystems are kept up to date, etc.
> * Dependency management limitations with quicklisp
With Quicklisp, I personally realized a lot of other dependency management solutions are overblown and don’t actually solve problems. Maybe QL gets away with it because the ecosystem is smaller than others.
Xach, the creator of QL, does a lot of community diligence for free to ensure things work.
For production code, I’ve had 0 issues.
> * poor support for json
We use YASON [4] and it works fine.
> * poor support for async
This is true now but wasn’t true years ago. Lisp implementers have not been interested in implementing a runtime supporting async programming. Little in the Common Lisp language itself prohibits it. Before the world of threads and SMP, Lisp implementers did implement async runtimes.
Why aren’t Lisp implementers interested in async? I’m not sure. Maybe they see it as a passing fad?
> * have to restart the server every 20 days because of some memory leak
Memory leaks can happen but they’re usually the programmer’s fault for not understanding symbol interning, weak pointers, or finalizers. Writing long-running applications does require know-how.
> * hack to tune GC
I don’t consider it hacking. In SBCL we tune the GC to our application. This seems reasonable; different programs have different allocation patterns. But you wouldn’t need to do this out of the box.
> is there a way you can quantify effort is resolving them, and if yes, what is it?
Big applications that are customer products require care and feeding. The benefits of writing in Lisp in the first place largely outweigh some of the extra work needed to productionize it.
I will say that writing production Common Lisp code is greatly benefited by having at least one experienced Lisper on the team. If a team would, for some reason, write an application in Lisp but not really have a good understanding of the language, then productionizing and hardening will be difficult.
> and, finally, if you had to re-do your project, would you chose lisp or something else?
Common Lisp allows something that never existed before (a completely automatic, optimizing quantum compiler) to exist. If Lisp weren’t there, that wouldn’t exist.
My parting advice is this: The biggest difficulty in writing an application is to actually write it. It takes a lot of hard work to get requirements, listen to feedback, articulate a direction, etc. In Common Lisp, SBCL or LispWorks or whatever else, dumping an executable is so easy and you can have a deployable “production app” in 5 minutes. So the good news is that you won’t be held up trying to figure out how to dockerize your Python app.
I definitely agree that before embarking, doing an objective and relevant comparative analysis would be good (we did this at Rigetti), but ultimately you just need to sit down and write.
Almost every programming problem is solvable these days with enough glue and elbow grease. I wouldn’t be too worried.
[1] https://arxiv.org/abs/2003.13961
[2] https://github.com/rigetti/quilc
[3] https://github.com/rigetti/qvm
[4] https://github.com/phmarek/yason