Running Lisp in Production (2015) | Hacker News Reader

Running Lisp in Production (2015)

tech.grammarly.com

167 pointswglb60 comments

Previous discussions from last year: https://news.ycombinator.com/item?id=16860646 https://news.ycombinator.com/item?id=16679963

(Common) Lisp usually gets more use than exposure, so it's nice to see large-scale Lisp success stories. Makes me feel warm and fuzzy inside, as it's one of my two favourite languages (the other one being, of course, C).

Edit: APL is a close third, but let's be realistic.

APL is the ITS of the programming world: Both are immensely powerful and quirky, and were designed by people with a real allergy to typing. It would be nice if there were an Open Source APL which was the equivalent of SBCL in terms of being the default and feature-complete implementation; it would also be nice if APL were elegant as opposed to merely terse.

>(Common) Lisp usually gets more use than exposure

What? It gets exposure on the Internet all the time, while seemingly nearly no one uses it in production.

Edit: Just look at the end of [1]. They have exactly three examples.

Edit 2: They have more example in [2], but it is still pretty little.

[1] https://lisp-lang.org/

[2] https://lisp-lang.org/success/

This is getting downvoted and I don't see why. CL is easily in the top 10 programming languages I see mentioned on HN and it also comes up regularly outside of HN, yet the amount of people actually using it in production seems miniscule.

Also 2015: https://news.ycombinator.com/item?id=9785944

I did not realize common lisp was still in use and was therefore planning on learning Clojure instead. Which is better to learn then?

What made you think it was “no longer in use?” It’s a very big world. Just because something isn’t mentioned regularly in headlines doesn’t mean it’s not out there.

http://www.paulgraham.com/onlisp.html

"On Lisp" is a good book but perhaps too advanced for an introduction, whereas his "ANSI Common Lisp"[0] is better suited to the purpose. Barski's "Land of Lisp" has also received positive feedback.[1]

I'd be interested to see what others consider a suitable general learning path for Lisp.

[0] http://www.paulgraham.com/acl.html

[1] http://shop.oreilly.com/product/9781593272814.do

I think this "A Road to Common Lisp" is a very good introductory guide

http://stevelosh.com/blog/2018/08/a-road-to-common-lisp/

The usual starting book is 'Practical Common Lisp' by Peter Seibel. Some advanced stuff then in 'Common Lisp Recipes' by Edi Weitz. Some older advanced stuff with examples: Paradigms of Artificial Intelligence Programming, Cases Studies in Common Lisp (PAIP) by Peter Norvig.

I would imagine Clojure has more use than Lisp. It's a valid question which is better to learn, not one I'm qualified to answer, though. All I would say is that the Clojure community seems more active, and less prickly than the Lisp one.

“but we value choice and freedom over rules and processes.” Who could argue with that? It can sound pretty defensive though, like we know we have a bunch of nearly random stuff that few people can really understand and manage but, hey, freedom. Lisp has always been interesting and there’s always one or two companies in any time interval that are successful, and wildly so, using it. It could also be as much skill, great execution and a market.

It's easy to argue against, and many companies' higher ups successfully do and thus can mandate a monolanguage policy. For Grammarly, I still see them posting occasionally in the monthly who's-hiring threads for Lisp roles, I think they're doing fine.

I wonder if they've since run into more implementation specific issues and have shelled out for one of the proprietary Lisps. SBCL is always improving, though.

My experience is that mono-language policies (or similar tight, top-down controlling of the language stack) are simply a matter of time. All companies enforce that as they mature.

I have been working for two companies that adopted Clojure and Scala early on. They all have, within a period of two to three years, enforced a Java only monoculture with no new projects being written in the aforementioned languages.

The reasons were key engineers leaving for greener pastures and experiencing difficulties with staffing: Not being able to find (cheap) experienced developers, or failing to do in house training due to the lack of interest.

This might be hard to believe, considering that the average HN user would jump of excitement at the chance to learn something new, but then again, those languages were introduced with the JVM as the main selling point, meaning working mostly with Java developers, who are notorious for their resistance to learn _anything_.

> mono-language policies (or similar tight, top-down controlling of the language stack) are simply a matter of time. All companies enforce that as they mature.

Yea, that would be my cue to leave.

Indeed. Pick the right tool for the job.

Picking the tool just because it gets you lots of cheap disposable developers simply means you end up with lots of cheap disposable code.

Good for the current quarter’s C-suite bonuses, no doubt; maybe not so much for long-term business continuity when that code is critical to continuing business operations.

Something something price of everything and value of nothing.

That has not been my experience. Companies can and should develop a set of technologies that they as an organization feel comfortable with, but this list should be dynamic and should have some diversity in it. Don’t allow every technology under the sun without any vetting, but also don’t force everyone to use the one true programming language, Java.

But that's what I meant by top-down controlling of the language stack: Which in the context of the blog post, would be the contrary of "valuing choice and freedom over rules and processes."

It doesn't have to be a single language, of course. The typical "use / ask / forbidden" matrix for new projects would suffice.

I don't have a problem with that, personally. It is only when programming languages are _exclusively_ being chosen out of non technical reasons that the fun ends and I'm out. Sadly, it happens too often.

I don't really agree that it's a matter of time. There's a pressure, sure, but a stronger counter-pressure is simply having wider product categories and to a lesser extent enough employees to allow for some specialization.

The moment you do web application development, chances are you're using at least two languages if not more. There are some all-JS web companies, there are also some all-Clojure/ClojureScript companies, but neither are very common. Add a database, you might have another language (or not). Add a mobile application, and you're looking at potentially 6 languages just for that product category depending on how you set it up. DevOps? Take your pick, probably at least two used at the typical company that has such specific roles. Survive long enough (many companies don't even last a decade) and maybe make some acquisitions, now you've got acquired product X in some other language -- do you do a rewrite? Another aspect of surviving long enough is the language you picked might have changed a lot. Modern Java, JS, Python, and C++ are all significantly different now than they were not too long ago. Companies with big code bases can't afford a rewrite or mass update, so they have a mix, and developers need to be aware of the conventions of the old versions that may as well be called old languages. Of course some companies will insist on banning the modern styles and idioms even if they begrudgingly update the tooling, for much the same reasons they ban different languages.

Some companies do end up being long-lived and monolanguage though. And where I would agree with you is that individual products will tend to single language policies even if technically they can intermingle (i.e. all the JVM mingling). (Edit: and seeing your other clarifying comment, yeah, top-down control over letting teams make most of their own decisions probably is inevitable with time, this expands far beyond just language choice though.) Sometimes that language is Common Lisp! There are things that can be done to influence what it's likely to be, if it is to be. The "simplest" is probably just having a big enough, complex enough, and important enough product that no one's going to authorize a rewrite for even if the will and expertise to do so appeared. Then the monolanguage will be that language -- whatever a company starts with has a good chance of becoming The Language, but if you're starting with a not so popular language you're in danger of having it changed at some point and will need to protect against that if you care. (This to me explains Reddit's rewrite off of Lisp early on, the code that was written wasn't that big or complex yet, and also explains Facebook's lack of move off PHP and herculean efforts with Hack to compensate. But as companies Reddit seems much closer to the monolanguage endpoint than Facebook -- Facebook simply has way more engineers and a lot more projects going on.)

Another driver is what you highlighted, lack of interest, or indifference, and is probably the main one until you reach a "too big to change" tipping point. You need to have someone who loves the particular stack, or (also?) who hates the alternative stacks, because if you don't someone else will eventually come along with enough ambition and motivation (either from hatred of current stack or love for a new/alternate/more familiar stack) and the stronger passions will win the day with the indifferent employees just going along with it.

I don't know what they are doing currently, but the article from a few years ago said, "We run them on stock Linux images deployed to AWS. We use SBCL for production deployment and CCL on most of the developers’ machines."

Once a large application works in some environment, you usually need a good reason (such as lower overall cost or inability to keep it working) to switch. They hit a few unlikely bugs and have already developed solutions for them. If they switched, the odds are excellent that they'd hit new unlikely bugs that they haven't found solutions for, and it's hard to have a smaller cost than SBCL or Linux. So while they may have changed, there's a reasonable chance they haven't.

Right, but you at least need to experiment with the other runtimes first before you can even determine whether there might be a good reason (like not having to mess with tuning a bunch of GC flags anymore, one of the sells for Azul's proprietary JVM) apart from a forcing situation (like Oracle shedding updates and support contracts on their Java8 VM, forcing people to OpenJDK / Azul's OpenJDK build + cheaper support contract). In many cases it doesn't take much effort to spin up a mirror environment and see if the alternative "just works" and what its characteristics are apart from that.

So my curiosity is really whether they found further issues with SBCL later, or if it's been a champ since, and if at any time they investigated developing with SBCL/CCL and deploying with (maybe developing sometimes with) Allegro or LispWorks (which at least if they hit new bugs with, they have a support contract to help). The thread from last year includes one report from another individual about swapping deploying with SBCL to deploy with CCL because of resource savings. I'm just curious in hearing about what trajectories people have taken.

There are more companies being more successful using PHP.

And no doubt many more companies being more unsuccessful using PHP too.

If you really must go comparing apples to pears, a marginally less worthless metric might be the percentage of each that has worms in.

Grammarly is essentially an opt-in keylogger for your browser. To put it generously, it is hilariously insecure by design. Furthermore, I don't believe that most of their users understand that the plugin "phones home" continuously to function.

Hey, at least the developers get to tinker with fun languages while spying on people!

How long before Google acquires Grammarly for the data?

While true, I believe the whole intent is to use machine learning on real text to improve the customer's experience. For the customer, this is a feature rather than the bug. It is not how I would prefer to interface with grammar checking, mind you, but I think they have been transparent.

At Amazon I’ve seen Clojure used successfully as a DSL in a larger service. The scaffolding is Java and the DSL parts are Clojure.

Is grammarly still using Lisp? How's it going?

Want to know as well.

Does anyone in the HN crowd know of any other brand-name orgs using Lisp?

The lispwork page http://www.lispworks.com/success-stories/ has some that are specific to lispworks. Some also (with possible overlap) at https://lisp-lang.org/success/. Allegro Common Lisp success stories: https://franz.com/success/. ITA (now part of Google) https://www.itasoftware.com/

Thanks for the lists. Aside from ITA and Northwestern University, most of those were new to me. I was kinda hoping for some big commonly-known brands.

I'm trying to convince my boss to let me learn Lisp on company time.

It would seem as too much of a shot in the dark with questionable ROI for a non-technical (and probably even a technical) manager. From their perspective it would seem as the equivalent of introducing an inexperienced programmer into the team (as your Lisp experience will initially be understandably zero), and there wouldn't be anyone in the immediate vicinity to help with peer review or mentoring. Perhaps it would be more effective to learn Lisp on your own through a few hobby projects, and then gradually (and gently) introduce it as a potential solution for work projects? This does depend on your team's structure, though,

If anyone here has introduced Lisp into their workplace, I too would love to know their approach, be it successful or not. Although I'd be more interested in proper Lisp (OK, and Scheme) than Clojure, the latter wouldn't hurt either.

I think it's worth a try. The company is organized into small teams, and the HR department places a big emphasis on people learning new skills. We have a big catalog of in-house classes that are scheduled and taught every month (or quarter, depending on the complexity) with the purpose of promoting personal and professional growth.

While Lisp is not core to what I do, neither is cooking, yet the company gives us classes in that, too. So I think I have a shot.

Definitely go for it, then, and try to do Common Lisp rather than anything else. You can always expand later to JVM-based Clojure, or indeed contract to Scheme, but it's good to start with the thing itself.

The world always needs more good Lisp programmers.

In order to better solve what business problem, exactly?

If I was your boss, I’d demand to know your projected ROI calculations up-front too. That includes the near-term risks and costs of change management when phasing in your new technologies and practices, and long-term business continuity implications after you’ve personally buggered off to pastures new.

Apple, eBay, Intuit, Oracle, and Red Hat (among many large and smaller-but-still-largish companies) use Clojure according to the Clojure site.

https://clojure.org/community/companies

Another related link - https://clojure.org/community/success_stories

OSX sandboxing uses Scheme48 under the hood.

When I was there, the A/B framework developed at LinkedIn was in Closure, and you you could even download your segment as a sexpr.

It has been said that HN itself runs on Arc, based on Racket, a scheme/lisp. Paul Graham is a big lisp head.

I believe Cisco has used Scheme, having bought the Chez Scheme company (then in 2016 open-sourcing Chez). I've also seen a blog/twitter post from a Cisco guy about Clojure.

> Being a client-server application, it allows to run its backend on the remote machine and connect to it from your local Emacs (or Vim, if you must, with SLIMV).

I have used Slimv and found it quite enjoyable on Vim. Has anyone here tried Vlime? Has anyone tried both? Which one of the two do you recommend?

They still do not provide a per-subscription API for integration, which is plain stupid for 2019. It would be awesome to have it integrated with Vim, Emacs, whatever you like.

You could probably just crack open the browser extension, prettify it, and send a few sample requests to get the hang of whatever API is used pretty quickly. I doubt it will provide one, as there aren't many users demanding it.

This sounds kind of like an absolutely miserable experience. Can someone point out why a company would choose lisp over some other language which is just as fast performance wise?

There aren’t many dynamic languages with that level of performance and maturity.

Their problem wasn’t choosing Lisp, but not paying for ACL :)

Allegro is nice. It’s fast, and has a great development environment. I used it all the time years ago.

I recently decided to screw around with Lisp again and have been using SBCL with Atom via the Slime plugin, but it’s not very good. I can’t evaluate functions from the editor, nor can I copy-paste multiline functions into the repl. It’s very frustrating.

Assuming you don't want emacs, so...

If you don't mind branching out Racket w/Dr Racket is zero setup and great fun w/great docs. If you go Clojure, many options that don't require setup (LightTable, maybe, e.g). If you want to stay common lisp, Lisp Works has a "personal" and "hobby" version of their IDE and that's free, zero setup, and a mature IDE.

Oh I guess my "assuming you don't want emacs, so..." was interpreted as a ding against emacs so I was downvoted...

I don't care about the downvote, but ftr, I a have been using emacs daily for about 25 years, spacemacs now, and love it. So there.

All I meant was that the OP I assumed knew about emacs and it's lisp history and was choosing Atom instead. The reason to do that would be learning curve, which I respect. Fair enough; I judge not. Just... you, know, for clarity.

FTR the reason why i don't use emacs is because it frustratingly has too many degrees of freedom, and thus is perpetually misconfigured, and I find it has atrocious UX.

The last emacs I used with any regularity was XEmacs.

Are you using the right slime plugin? I haven't tried it yet (I'm happy with vim+slimv) but I saw a thread on the CL subreddit last month that stated "atom-slime is now replaced by Slima which is more complete".

I've given Allegro a spin, and LispWorks too. I liked Allegro's UI better, but both just give off the "We designed our UI in the 90s, updated to the WinXP plastic chrome at one point in the early 2000s, and haven't touched it since!" vibe. It's just not very pleasant, especially with a big screen... I use Eclipse at work for my Java day job (IntelliJ is also an option but Eclipse is fine for me) and it makes me sad that its UX with all its clunkiness and compromises to be a mega plugin-extendable open source IDE still feels better than the IDEs that just need to support Lisp. If either Lisp company would offer a subscription license model similar to IntelliJ's along with even vague promises of modernizing the UX (like it's not a good sign when your product landing page literally still has a Windows XP screenshot http://www.lispworks.com/products/lispworks.html ) I'd start paying immediately, but I suspect most of their revenue is from the core Lisp image, their side products (I've heard AllegroGraph is amazing if you need that sort of thing), and various customer support contracts for not just the products but bespoke extensions / feature requests. That is to say, very little from the editor itself. Yeah you can always use their Lisp with emacs/slime but then part of the point is lost. ;)

I personally cannot stand Allegro's IDE, but LispWorks looks like your regular native application on macOS and Windows. And since v7 it looks good on hi-res screens (sadly, the free version is stuck on the former major release, which is bad marketing IMHO).

It might not look as trendy as IntelliJ or VScode, but compared to stuff like Eclipse I'd say it does quite well.

Here's a screenshot on macOS https://i.imgur.com/ml4jlYB.png browsing some example code.

EDIT: A similar screenshot on Windows https://i.imgur.com/qkgIXhE.png

> Windows XP screenshot

There are a bunch of oldish things in it, but the backend is relatively native - so the application has a newer look on new systems.

They also improve the CAPI GUI layer with releases with new features.

http://www.lispworks.com/documentation/lw71/RNIG/html/readme...

http://www.lispworks.com/documentation/lw70/RNIG/html/readme...

There are a bunch of new IDE features, like visual profiler output, Code Coverage tool, support for remote debugging, ...

You'd have to read the release notes to find all that stuff. All in all it is a very feature-rich system...

I am definitely a Lisp noob, but portacle has been a pretty great "intro to lisp programming" environment. (I'm in a Windows environment, as the context to this comment.) Portacle is basically emacs with SLIME and a Common Lisp REPL integrated. What I just wrote might actually be nonsense because, again, I am noob af.

Dealing with a bug in the language implementation is almost always a miserable experience, and all implementations have them; SBCL isn't unique or even unusual in this respect. Where it, and other Common Lisp implementations with similarly mature tooling, are indeed unusual and exceptional is in the unparalleled flexibility they offer to inspect, introspect, and instrument running code in real time, without having to so much as restart the system under inspection, and also without the constant headaches of something like gdb and its often flaky translation of byte slaw into symbolic structures.

There's a goodly number of reasons not to choose Common Lisp for production, not least of which is that engineers skilled and familiar with the language can be hard to find. Debuggability, though, is not among them.

The long and the short of it is that lisp, for all its faults, is superior to commonly used programming languages in most cases.

It's almost as fast and way, way more expressive than C, C++ and Java.

It's still more expressive and 100x faster than Perl, Python, Ruby.

It's the expressivity that hooks you in, in my experience. Once you've gotten used to programming with macros (code rewriting programs), the thought of programming without them seems almost as bad as programming without functions.

Because languages with the same speed are generally a lot less flexible. If you have access to allegro CL or LispWorks and you and your team know CL you must have some very good reasons not to use it.

SBCL (the implementation they use) was(is?) known to have a pretty bad GC situation.

Reply on news.ycombinator.com