Hacker News Reader

Or the simpler reason is that they just knew LISP better than other languages and chose to use it.

Once they needed to grow by adding developers, they decided to rewrite into a language that other people already knew.

The only "Super Power" is the complete understanding of what your endusers want so that you can grow.

When you are writing a tiny protoype 15 years ago, any language you know is fine. But over time -- both the life of the product, and the world overall, software grows far more complex and the ecosystem matters much more.

Knowledgeable developers are part of every language's ecosystem.

Yes but ease of attracting developers is harder in some languages versus others.

No need to include the "knowledgeable" modifier.

Interestingly I have encountered voices that using "elitist" language like Common Lisp, OCaml or Haskell combined with widening recruitment pool (by going globally distributed) actually increased some companies abilities to hire experienced developers - because it filtered popular languages that suffered from higher number of people who had higher opinion of their skills than warranted - or who just wanted to "fake it till you make it", especially after boot camps became popular.

When Alexis Ohanian gave a talk at Google in 2013 I got to talk with him about he and Steve dropping Lisp for Python. He said the Lisp version just kept falling down.

In production using SBCL on a customer project we had good results hiring a SBCL maintainer/developer to work past a few issues. From personal experience, I can assure you that LispWorks and Franz both provide incredibly good support for their customers.

That said, if I had to develop Reddit right now, I would not use Common Lisp either.

I’ve looked a bit at the original reddit code (it is/was published on GitHub) it does a lot of things in unusual ways for a lisp project and I’m not too surprised that they abandoned it: I’m pretty sure that if they had rewritten it in lisp, taking advantage of what they learned, the second version would have worked as well as the Python rewrite.

Also, quicklisp didn’t exist/wasn’t as widely used then: I’ve done all sorts of Lisp projects now and never had serious issues finding good libraries for standard web development tasks.

Didn't they also had to deal with wanting to use threaded architecture while simultaneously deciding to run on less supported platform (FreeBSD)?

What would you use instead if you had to choose a Lisp based language today?

Clojure?

Or a non-lisp language instead?

> Blub Paradox") -- but other languages also have other superpowers (ecosystem/libraries)

Indeed. The "blub paradox" is profoundly arrogant - it presumes not only that people are too stupid to see the advantages of Lisp, but that Lisp advocates are smart enough to know that other languages have nothing unique to offer that Lisp doesn't.

PG calls it a Paradox because no one (not even the Lisper) can know whether their language is Blub.

You are right, though, that the Lisp community read the essay and internalized that all other languages are Blub. In my opinion this led them to collectively dismiss, for far too long, Dictionary Literals in every dynamic language which has eaten Lisp's lunch in the marketplace in the last two decades. Objections were raised ("you need homoiconicity for macros!") and everyone assumed they were valid and stuck with Common Lisp's woeful hash table interface and the loop macro keywords, insisting all the while "oh, that's a DSL, you could implement that with Macros"[0].

And then Clojure implemented Map literals and - what do you know - Clojure still has Lisp Macros!

[0] Efforts in this direction thankfully forthcoming: https://github.com/inaimathi/clj

This is all surface, little substance.

Map literals are 20 lines of Common Lisp [1], via the programmable in CL reader (that Clojure doesn't have).

Richard Gabriel's "Worse is Better" is a lot more relevant about "Lisp in the marketplace" than anything you've mentioned here.

[1] https://gist.github.com/rmoritz/1044553/b2d2d8e4b933cb0f6c3f...

Unfortunately:

- that code doesn't provide printing for hash tables. Oops!

- breaks tooling: - doesn't provide editing or syntax highlighting support. - confuses any tool that reads all of the files of a project using the Lisp reader.

- it gratuitously introduces a vector syntax that is unnecessary since #(1 2 3) is perfectly fine and standard across more than one Lisp dialect.

- hard-codes the #'equal equality

- doesn't play with the newer ways of registering Lisp read syntaxes, like https://github.com/melisgl/named-readtables

> and stuck with Common Lisp's woeful hash table interface and the loop macro keywords

You are not stuck with anything in Common Lisp.

Instead of loop you have lots of options, from the built-in constructs (do, dolist, map...) to very powerful libraries like SERIES and ITERATE.

You are not stuck with the default hash table interface either. Many libraries for data structure are available, you just pick the one that suits you best.

>And then Clojure implemented Map literals and - what do you know - Clojure still has Lisp Macros!

Most of those Clojure features are already available in CL through the use of libraries. Even the ability to easily call Java libraries, by using the Armed Bear Common Lisp implementation.

I think that Beating the Averages demonstrated that Lisp was PG's Blub, and maybe he wasn't using a language with strong static typing or logic programming or something like that for the same reasons that Java programmers wouldn't choose Lisp.

I don't think that the fundamental flaw with that essay is the arrogance so much as the assumption that "language power" lies on a continuum in the first place.

Please reconsider your assumptions. PG has written two Lisp books in the nineties, one of them containing a Prolog interpreter. In fact, many Lisp programs utilize some form of logic programming. With regards to "strong static typing", types are but one way to think about things. Since Lisp affords powerful ways to shape the language, there is a culture of implementing and thinking about various paradigms, typing included. For example, the first Haskell implementation was written in Lisp, and there are other explorations of that particular paradigm. All programmers are blub programmers to some extent, but Lisp has the tools and the culture to fight it.

I apologize that my tone came across as dismissive. Chapter 24 of my paper copy of On Lisp does contain a "sketch of [Prolog's] implementation". I can certainly see the value in Lisp and the culture of bottom-up programming (there's a reason I own a physical copy of the book).

I shouldn't have brought up logic and statically-typed programming as specific examples; my point was more that it's misguided to consider the "power continuum" as one-dimensional, where languages lower on the continuum are "dumb" and languages higher on the continuum are "weird". As you said, a language with good types is not necessarily more or less powerful than a highly dynamic language.

There was nothing special about the language chosen for viaweb. It was more the ability to understand what their endusers wanted better than their competitors.

If their competitors were writing their applications in C++ or Perl, it probably would have made a noticeable difference to their iteration speed. Writing a web application in C++ (especially in 1995) would not have been too efficient.

I'm sure that they had a good understanding of their customers, but the language had to have made a difference as well. These days, we're writing web apps in languages like JavaScript and Ruby, not C++, so the difference would be a lot less significant.

Knowledge of the customer and programming ability is the most important driver of success.

The idea that LISP was a secret sauce for viaweb does not hold up under scrutiny.

If you have access to lexisnexis to lookup news articles from the era, it’s easy to see that.

Google is known to write a lot of their web apps in Java, which is very similar to C++.

>I think Reddit's decision to move from Common Lisp

That original Reddit system was very simple, yet the Lisp source code was of poor quality (it is currently available on the web). When it was released, the lispers at comp.lang.lisp said they could write a clone in a few days. One of them did it in about 2 or 3 hours (true story).

So it wasn't good code. Moreover they, for some strange reason, chose to develop using a different implementation than the one in the server, and things like that.

Then Reddit got a famous name recommended by PG, who was a Python expert. So he rewrote the thing in Python and obviously, being a Python expert, did a good job.

My understanding was that Steve mostly wrote it in Lisp to impress PG, and then Aaron convinced Steve to rewrite it (although Aaron was kind of a Python fanboy and it's telling that the rewrite only took a weekend).

Do you know of any of this 'superpower'code released somewhere? Like is it possible to study the original viaweb code?

https://github.com/reddit-archive/reddit1.0

Thanks. Looks neat. Will see a bit more of it. Of course the next question is, why they abandoned it, and reading this https://news.ycombinator.com/item?id=3815491 My translation of this is they had bug which came from the libs which they couldnt fix.

Maybe thid is not a showcase of 'superpower'. From what I read it took a whole summer to write in lisp and the rewrite in python took a weekend. (But will try to double check this)

>From what I read it took a whole summer to write in lisp and the rewrite in python took a weekend.

And one guy in comp.lang.lisp wrote a working Reddit 1.0 clone in Lisp in 4 hours:

https://groups.google.com/d/msg/comp.lang.lisp/9WD9-yEaqMs/H...

It didnt seem to be online for a significant amount of time? How would one know it worked (and was mantainable)?

I would like to consider Python a common lisp alternative. But it's one or two orders of magnitude slower and is not really standardized. I think there is a lot more that's attractive than just macros.

...and Python doesn't have it. Image-based programming is the other thing besides macros that gives Lisp an edge. Python might as well be batch-compiled.

> and is not really standardized

What do you mean by this ?

My understanding is the Python standard is essentially whatever C Python chooses to do. They do write the changes to a specification separately, but essentially its one moving implementation, with alternatives that have compatibility limitations.

Indeed. I'm curious what Python 4 will do to Python 3.

"standard" is "whatever CPython managed to miss a segfault for today".

And PyEvalFrameEx is the stuff of nightmares that can't be really fixed because it hardcoded spaghetti that, if rewritten, could subtly break python in myriad ways that would make 2->3 migration a cakewalk.

Oh, and there's no real test suite to verify if you didn't break something subtle in implementation.

While parsing Ruby code is well known to be problematic, there is a very good reason why Ruby can have interchangeable implementation while Python pretty much can't.

these days many newer languages have macros: lisp is not alone

We haven't figured out macros yet.

There's a perceptual litmus test here. Some people see life on earth and see God. Others see the faintest exploration of the design space, an impoverished code base that made it impossibly far. A miracle, perhaps, but not what we'd see in a billion runs of the simulation.

Same with Lisp macros. Their few variants look bolted on. We haven't begun to explore the design space, yet the best macro systems in other languages are copies of what lisp has figured out so far.

There's an 80:20 rule for language emergence. Ecosystem questions aside, a Turing-complete language needs to start by getting one thing right. APL leveraged multidimensional array handling into a general purpose language. Scripting languages gain much of their leverage from nested hash tables.

Concise versions of monadic parsing are exhilarating in their power and expressiveness. If a language was designed so parsing and manipulating itself was its sweet spot, generalizing this parsing to typed trees would be easy, and we'd invent many new programming paradigms. A general purpose language would follow.

Lisp, an implementation of the lambda calculus, is not this language. Lisp is a machine language precursor to this idea, with various macro systems bolted on.

Haskell, an implementation of typed category theory, is not this language either. At the expression level, Haskell is stuck at a few hard-wired pattern matching primitives. Haskell does not make the assumption that everyone has learned to compose arbitrary parsing operators.

> We haven't figured out macros yet.

the racket lang team has investigated this heavily.

https://www.youtube.com/watch?v=ABWLveMNdzg

https://www.cs.utah.edu/plt/publications/popl16-f.pdf

https://www.cs.utah.edu/plt/scope-sets/

> Haskell, an implementation of typed category theory

http://math.andrej.com/2016/08/06/hask-is-not-a-category/

> yet the best macro systems in other languages are copies of what lisp has figured out so far.

These are going to be controversial assertions in a Lisp thread, but I believe

1. C++ Templates + Concepts is a good Macro facility

2. Most of what makes it good is not borrowed from Common Lisp

On the other hand, I agree strongly with this assertion:

> Scripting languages gain much of their leverage from nested hash tables.

I've been pondering this comment, and I'm not sure I agree that C++ Templates + Concepts are really equivalent to Lisp macros. They're about enabling generic programming with type constraints (when you add concepts, in particular). This seems more akin to Common Lisp's defgeneric and defmethod, where you can specialize your implementation based on the type. Like, assuming some class hierarchy of foo (base) -> bar -> baz:

  (defgeneric do-something (object))
  (defmethod do-something ((object foo))
     (something every foo can do))
  ;; bar is the same as foo
  (defmethod do-something ((object baz))
     (something only bazzes should do))

Templates and concepts are certainly a powerful tool, and I personally like them (as I've come back to C++ after a long break). But macros let you do something else entirely. You can create entirely new syntactic structures, or modify existing functions (see Norvig's PAIP, chapter 9, where macros are used to memoize recursive functions), or any number of other things.

> I'm not sure I agree that C++ Templates + Concepts are really equivalent to Lisp macros.

I think I said as much in my comment.

> This seems more akin to Common Lisp's defgeneric and defmethod

Except that Templates are statically dispatched and type-checked.

But also one can perform meaningful compile-time computation with them - I have used this in anger to implement stencils and iteration logic for partial differential equation solvers in 1-3 dimensions so I can say from experience that it is not a mere curiosity. And I don't think that CLOS really has any analog for the techniques used to implement std::tuple.

> But macros let you do something else entirely. You can create entirely new syntactic structures, or modify existing functions (see Norvig's PAIP, chapter 9, where macros are used to memoize recursive functions), or any number of other things.

I think the thrust of my comment is that Lisp programmers identify this type-blind AST-mapping functionality as the core of "a Macro System," while users of other languages (with more syntax to begin with) see this as somewhat secondary to the goal of type-checked compile-time computation.

If C++ had a good macro system, the Qt library wouldn't need the "moc" preprocessor to implement its "slots" abstraction. Templates aren't powerful enough to fill that role, even when you add Concepts to them.

My understanding is that moc is no longer necessary, but sticks around for legacy compatibility, but I may be wrong.

I don't think another language's macros can match lisp macros unless the other language is also homoiconic.

Latter-day Dylan is infix with a Scheme-y macro system: https://opendylan.org/books/dpg/macros.html

Cue arguments, I guess, about whether Scheme is a Lisp, whatever their standards say.

Clojure is weakly heteroiconic and seems to have rough parity

Clojure is “homoiconic”: Clojure source code is represented in terms of the data structures Clojure applications manipulate. There’s nothing preventing Common Lisp from having a similar surface syntax either: a couple read macros on [ and {, and a library of normal macros would be sufficient and could probably be written over a weekend.

In fact, since Common Lisp "source code" is NOT defined in terms of text but data structures (that happen to have standard serialization/deserialization) and that's the level that macros work on, you could theoretically even make semi-algolish syntax and still keep CL's macro system.

It would just look weird as fuck.

Like https://readable.sourceforge.io ?

That's one example, not as crazy as what's possible. Though I have to say I dislike the indent based blocks :/

i am not sure: you can have "quoting" without this property

C has macros. They are nothing like lisp macros. Please cite a non-lisp language with macros a powerful as lisp's.

The major advantage of Lisp macros over macros in other languages aren't their power but the fact that using Lisp's macro system is virtually the same as using the rest of Lisp.

Lisp macros look like Lisp and in manipulating code via macros you can leverage the rest of Lisp and Lisp's ecosystem.

So writing Lisp macros is entirely natural for someone who already knows Lisp. You don't have to learn or use what is effectively a separate language, as you do with every other language that's not homoiconic.

As Alan Perlis once said, "Beware the Turing tarpit, where everything is possible and nothing is easy."

C macros and lisp macros (In particular, I'm thinking of hygienic macros à la Scheme) are two wildly different extremes. In between them lies a huge continuum of interesting things. There are Scala's macros. There are Haskell's language extensions. There are code generators like the ones used to implement much of gRPC, and there are compiler plugins like the ones used to implement aspect-oriented programming in C# and Java.

Most of them arguably qualify as being somewhere between an 80% solution and a 90% solution for the practical problems that lisp might solve with macros.

They're frequently not anywhere near as convenient to use or understand. But that's arguably a good thing in the long run, because it encourages communities to pool resources and work on a coordinated solution that can be shared by the community. My take on the curse of lisp, for what it's worth, is economies of scale. Lisp makes it too easy (and fun) to just keep your head down and work on your own thing, which hinders pooling of resources.

Elixir.

https://elixir-lang.org/getting-started/meta/macros.html

Scala, Nim, Clojure, Haxe

Clojure is a Lisp so it doesn't count.

Julia

Julia is lisp in drag so it doesn't really count, but I'll give you the point.

In what sense is Julia a lisp in drag? I'm just starting in on it it doesn't seem any more lispy than any other dynamic language. In particular I've yet to see a car/cdr.

Haskell, OCaml

Rust?